Disallow GPTBot to access your siteIgnacio@lemmy.ml to Technology@lemmy.ml – 114 points – 1 years agoplatform.openai.com13Post a CommentPreviewYou are viewing a single commentView all commentsOr don't do anything. There are plenty of crawlers out there and disallowing won't stop the unethical ones.Just because some people might break into my house doesn't mean I'll stop locking my doors.that doesn't lock anything, it's not a security feature.A house door lock isn’t that much about security either.It's a deterrent. Which is a pretty apt comparison for robots.txt and user agent blocking.2 more...2 more...
Or don't do anything. There are plenty of crawlers out there and disallowing won't stop the unethical ones.Just because some people might break into my house doesn't mean I'll stop locking my doors.that doesn't lock anything, it's not a security feature.A house door lock isn’t that much about security either.It's a deterrent. Which is a pretty apt comparison for robots.txt and user agent blocking.2 more...2 more...
Just because some people might break into my house doesn't mean I'll stop locking my doors.that doesn't lock anything, it's not a security feature.A house door lock isn’t that much about security either.It's a deterrent. Which is a pretty apt comparison for robots.txt and user agent blocking.2 more...
that doesn't lock anything, it's not a security feature.A house door lock isn’t that much about security either.It's a deterrent. Which is a pretty apt comparison for robots.txt and user agent blocking.
A house door lock isn’t that much about security either.It's a deterrent. Which is a pretty apt comparison for robots.txt and user agent blocking.
Or don't do anything. There are plenty of crawlers out there and disallowing won't stop the unethical ones.
Just because some people might break into my house doesn't mean I'll stop locking my doors.
that doesn't lock anything, it's not a security feature.
A house door lock isn’t that much about security either.
It's a deterrent. Which is a pretty apt comparison for robots.txt and user agent blocking.