Finding similar items (opens in new tab)

This post describes an approach to the problem of finding near-duplicates among crawled items and how this was implemented at Zyte. Near duplicate content is everywhere on the web and needs to be considered in any web crawling project. Web pages might differ only in a small portion of their content, such as advertising, timestamps, […]