Identifying AI Web Scrapers Using Canary Tokens

arXiv:2605.13706v1 Announce Type: cross Abstract: From pre-training to query-time augmentation, web-scraped data helps to improve the quality and contextual relevancy of content generated by large language models (LLMs). However, large-scale web scraping to feed LLMs can affect site stability and...