A blog owner explains blocking old browser user agents to combat high-volume crawlers gathering data for LLM training. The blocking affects legitimate users with outdated browsers and archive services like archive.today that use old Chrome user agents and unidentifiable IP addresses. The author recommends archive.org as a

2m read timeFrom utcc.utoronto.ca
Post cover image
1 Comment

Sort: