SubINaclS/Lyre
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
d3ac3f1925
Lyre/techniques/anubis/technical_anubis.md

2.4 KiB
Raw Blame History

Anubis - Proof-of-Work Wall for Content Protection

Anubis is the recommended first-line defense for individual creators against aggressive AI scrapers. Developed by Xe Iaso and the Techaro team, it is a lightweight, self-hosted reverse proxy that forces suspicious clients to complete a JavaScript proof-of-work challenge before they can reach your origin server.

Why Anubis Matters

Most AI crawlers use headless browsers or high-concurrency scripts that cannot efficiently solve computational challenges. Real human visitors using modern browsers experience only a one-time, sub-second delay. The result is a dramatic (90–95%) reduction in bot traffic within the first week of deployment, as documented across multiple production sites in 2025.

Anubis directly addresses the economic asymmetry described in the primary dissertation: scrapers pay a high per-request cost they did not budget for, while content creators pay almost nothing.

Key Features for Individuals

  • Hardened policy mode - Optimized for maximum bot deterrence with minimal human impact.
  • robots.txt awareness - Can serve a compliant robots.txt on behalf of the origin.
  • UA and IP reputation integration - Works seamlessly with the aggressive-bot list maintained in known-aggressive-bot-user-agents.md.
  • Docker and binary deployment - Runs on a single VPS or behind Cloudflare with zero ongoing maintenance.
  • Open source (MIT) - Full transparency and community support.

How It Fits the Defense Stack

  1. Anubis (this document) - First filter (PoW challenge).
  2. Tarpits (howto-tarpit-deployment.md) - Second filter for violators who bypass or ignore the challenge.
  3. Active techniques (decompression bombs, malformed content, slowloris) - Third layer for persistent offenders.
  4. UA reference list (known-aggressive-bot-user-agents.md) - Shared intelligence used by all layers.

Official Resources

Recommended Starting Point

Deploy Anubis in front of your origin today. It is the single highest-impact, lowest-effort action an individual creator can take to begin reclaiming control of their content.

Anubis is the cornerstone of the passive defense layer. All other techniques in this repository are designed to complement it.