That ain't clickbait, Xe Iaso's personally-made anti-scraping measure (originally made to deal with Amazon overloading their git server) has ended up being used by the literal United Nations.
I recommend reading the full blogpost, its a wild ride.
this post was submitted on 16 Apr 2025
110 points (100.0% liked)
NotAwfulTech
443 readers
1 users here now
a community for posting cool tech news you don’t want to sneer at
non-awfulness of tech is not required or else we wouldn’t have any posts
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Are you talking about anubis? Because you’re very clearly wrong.
And now I think about it, regardless of which approach you were talking about, that’s some impressive arrogance to assume that everyone involved other than you was a complete idiot.
Eta:
Ahh, looking at your post history, I see you misunderstand why scrapers use a common user agent, and are confused about what a general increase in cost-per-page means to people who do bulk scraping.
I see we have another primo technology understander driveby, so enlightened