I got tired with all the AI scrapers that were bullying my git server, so I made a tool to stop them for good.

Lobsters is a community-driven platform for sharing and discussing links to articles, tutorials, and projects related to technology and programming. Readers can learn about a wide range of topics, from software development and system administration to cybersecurity and artificial intelligence. With user submissions, comments, and voting, Lobsters provides a platform for collaborative learning and knowledge sharing among technology enthusiasts.

Lobsters

Anubis is a reverse proxy tool designed to protect servers from AI scrapers by implementing a proof-of-work challenge. This method ensures that only browsers or bots capable of solving the challenge can access the site, effectively blocking most AI scrapers that can't execute modern JavaScript. While it demands certain resources from legitimate users and may lock out those with outdated browsers or disabled JavaScript, it offers significant protection and reduces the server load from aggressive scrapers. Anubis can be integrated with Docker or Kubernetes setups and offers commercial support options for customization.

Block AI scrapers with Anubis