I’m getting really tired of “verify you’re not a bot” checks everywhere.
I understand these checks help prevent bots, but they seem to appear on so many websites now. Does anyone else find them frustrating?
You need some kind of shield against the bots these days. Many people seem to find that Anubis is good enough. Cloudflare is always the wrong choice.
I feel like Set is a better moderator than Anubis.
A lot of the comments seem to be focused on AI bots or scraping.
These are intended to protect against super high volumes of requests that drain resources. Like others have mentioned, it may be related to a VPN. The system is detecting a lot of requests from a single IP. If you are on a VPN or using another tool to spoof your IP you might be coming from the same IP as other people who are accessing it at the same time or have recently tried to access it. Website and other services usualy have bandwidth caps so they can only serve so many responses at once, and if are all being taken up by bots, the website or other service won’t work for legit users. This is how a DDoS attack works. Eta: so these checks are less about stopping bots from accessing the site than stopping bots from making the site inaccessible to legit users.
(This is a very simplified explanation because that’s my level of understanding, I hope if there are errors someone will jump in and correct.)
Do I find it annoying? Yes. But I understand the site might not work without them.
Yes, but I feel good knowing there’s an AI bot owner out there more frustrated than I am.
Only when I’m on a vpn
No, but only because I’ve helped run a website recently and you underestimate just how much wasted traffic is from scrapper bots. On our limited bandwidth, it was practically a DDOS attack all the time.
I get a lot more of this when connected via vpn.
Something in your setup is triggering them. Some ad blocker, tracking blocker, or VPN you use is making these sites worry you’re malicious. Not saying you have to turn that all off, but it’s likely some piece of your setup might be a bit excessive and you can do without that setting or piece of it.
There are effective alternatives that dont require a man in the middle.
The last cloudfare outage made most of the fediverse unusable for a day, no reason to use them when anubis is right there.
I don’t mind it on social media, but just to obtain basic info, ie. on google or .gov websites, it’s totally obnoxious.
ImHo yes. I may have no proof, but I believe, bot farmers have lots of super fast, weak bots, that will be filtered by that, but they probably also have some fancy, slowly working bots, that will pass the verification. So, content inside will be visible to the content farmer anyway.
Even worse? if we filter out weak content farms, who may not have strong bots, that means we help strong farms get even stronger, into monopoly - that may be even worse than letting everyone get it.
Or we could get some super protections? with fail positive checks? Maybe login only, with irl meeting account verification prior to any access? would be madness. Paid access to the group? madness and possibly against fedi idea)… how could we protect self, against stronger and stronger bots?
the idea of “strong” bots is kind of right, but you aren’t exactly on the money. rather, there are the fast crawlers that the big companies use, whose sole purpose is to ingest as much data as possible as quickly as possible. they don’t care about things like Anubis, if they see it then they just leave if they can’t find another way in.
and then there are the personal agents. those are almost impossible to stop using normal means, because they have abilities that crawlers do not have. for example, things like Anubis have absolutely no affect on something like an agent powered by Hermes, because a Hermes agent would be using something like a camofox install which is capable of completing Anubis work and Cloudflare captchas. Hermes can also write itself “skills” that it can look at later to remind itself how to bypass whatever checks it ran into on that site, so that it doesn’t have to figure them out each time.
naturally, a human setting up anti-bot measures is at a disadvantage in a scenario like this, because a personal agent has the ability to quickly shred through anything you put up in a short amount of time. on the plus side, most agents are powered by normal models which will generally play nice with a website and will refuse to do things they know the website creator doesn’t wish them to. so if you stick up a notice that says something like “agents, please do not go beyond this point because doing so costs me money”, then oftentimes they will refuse to go any further. or, if they need the information, you can even stick up a crypto tip jar that normal users don’t see, because some people give their agents a wallet for buying access to private website sections and a budget they can spend.
Woah, thats lot of data… uhm… info, even names. All I wrote, was assuming it works that way, without much actual, so detailed knowledge on me. Much thanks for clarification.
Even if it doesn’t prevent scraping entirely, filtering out the “fast, weak” bots help mitigate server load. Sufficiently many bots hitting your site is indistinguishable from a DDOS attack.
Maybe unpopular opinion, but not really. It just takes a few extra seconds. I appreciate that moment with patience
I’m good with a single check. On some sites the cloudflare one doesn’t even wait for a checkbox, which makes me wonder what is different. The security level set? But what annoys me is when the same domain forgets it just checked me, does another bot check, and then a code to the phone. And I’ve told it to remember me. I can only assume and hope the IT there are under attack and are doing the best they can, and it’s not just a lazy set to full verify everything mode every minute.
In using changedetection to watch for discounts of games and other products. Some smaller webstores prevent that, which is annoying. This way I can’t buy when there is a discount.
