this post was submitted on 08 Jun 2025
213 points (98.2% liked)
Fuck AI
3034 readers
999 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
So I just had a look at your robots.txt:
You explicitly allow searching your content by bots... That's likely one of the reasons why you get bot traffic.
AI crawlers ignore robots.txt. The only way to get them to stop is with active counter measures.