AI companies are violating a basic social contract of the web and and ignoring robots.txt

Andy Reid@lemmy.world to Technology@lemmy.world – 933 points –
The rise and fall of robots.txt
theverge.com
198

You are viewing a single comment

hide a link no one would ever click. if an ip requests the link, it's a ban

Except that it'd also catch out people who use accessibility devices might see the link anyways, or use the keyboard to navigate a site instead of a mouse.

i don't know, maybe there's a canvas trick. i'm not a webdev so i am a bit out of my depth and mostly guessing and remembering 20-year-old technology

If it weren't so difficult and require so much effort, I'd rather clicking the link cause the server to switch to serving up poisoned data -- stuff that will ruin a LLM.

Visiting /enter_spoopmode.html will choose a theme and mangle the text for any page you next go to accordingly (think search&replace with swear words or santa clause)

It will also show a banner letting the user know they are in spoop mode, with a javascript button to exit the mode, where the AJAX request URL is ofuscated (think base64) The banner is at the bottom of the html document (not nesisarly the screen itself) and/or inside unusual/normally ignored tags. `

Would that be effective? A lot of poisoning seems targeted to a specific version of an LLM, rather than being general.

Like how the image poisoning programs only work for some image generators and not others.

1 more...
1 more...
1 more...