Skip to content
AI News HubLIVE
More
In-site rewrite1 min read

AI has lost all meaning

Summary

The article discusses the problem of AI companies aggressively scraping websites causing downtime, and introduces Anubis, a Proof-of-Work solution to make mass scraping more expensive, as a temporary measure until better headless browser fingerprinting is developed.

SourceHacker News AIAuthor: spzb
AI has lost all meaning
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

Making sure you're not a bot!

Loading...

You are seeing this because the administrator of this website has set up Anubis to protect the server against the scourge of AI companies aggressively scraping websites. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.

Anubis is a compromise. Anubis uses a Proof-of-Work scheme in the vein of Hashcash, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.

Ultimately, this is a placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.

Please note that Anubis requires the use of modern JavaScript features that plugins like JShelter will disable. Please disable JShelter or other such plugins for this domain.

Key points and analysis

Article intelligence

EngineersIntermediate

Key points

  • Mass scraping by AI companies causes website downtime and resource inaccessibility.
  • Anubis uses a Proof-of-Work scheme to increase the cost of scraping at scale.
  • It is a placeholder until more advanced headless browser fingerprinting is implemented.

Highlights and analysis are generated automatically and may contain errors. Check the original source.