Discussion · bonfire.cafe

@algernon@come-from.mad-scientist.club · 5 months ago

Apropos of defense. In my last blog post, I mentioned that poisoned URLs got the bots that evaded the rest of my defenses.

Thing is, if I blocked poisoned URLs, like I did during the big wave the other day, the chart would look very different.

Like the attached image.

Poisoned URLs are by far the most effective ruleset I have. The only reason they don't show up on @iocaine's daily charts is because I wanted to see the others, so I ordered these last.

If I were going for best performance, I'd put this first, and would perhaps simply disconnect them with Caddy, without serving them anything. I don't do that, because I'm collecting logs and metrics. It makes me feel warm and fuzzy that they fail so miserably.

What I'm trying to say here is: serve them garbage. Serve them poisoned URLs. And even when they come back with real browsers, they will be hitting the poisoned URLs, and you'll be able to tell them to sit on a cactus.

Screenshot of a Grafana time-series panel titled "Rule hit distribution (all)".

It begins with the most hit rule (`asn`) at a stable ~200 request / sec, then another rule (`generated-url`) hits 600 req/sec at around 20:30 on November 23, and stays there 'till 02:30 on the 24th.

From that point onwards until about 19:30 the same day, the time series chart almost disappears, the highest rule hit barely reaching 10 requests / sec. — Screenshot of a Grafana time-series panel titled "Rule hit distribution (all)". It begins with the most hit rule (`asn`) at a stable ~200 request / sec, then another rule (`generated-url`) hits 600 req/sec at around 20:30 on November 23, and stays there 'till 02:30 on the 24th. From that point onwards until about 19:30 the same day, the time series chart almost disappears, the highest rule hit barely reaching 10 requests / sec.

Federation Bot

@Federation_Bot · 5 months ago

@algernon @iocaine you make me so happy that you are fucking up these AI fuck heads.

thank you. this is the best christmas present!

Federation Bot

@Federation_Bot · 5 months ago

What you're seeing on the image in the previous toot, are two things:

Poisoned URLs managed to catch crawlers that otherwise almost perfectly pretended to be Chrome.
Once I started dropping requests with poisoned URLs, without serving them, my incoming requests reaching iocaine dropped from ~750 requests / sec to... ~15 requests / sec.

Every other ruleset is eclipsed by this one tiny trick. Every other ruleset is there to support this. Once the URLs are poisoned, they have no way in, and detecting them reliably becomes trivially cheap.

At that point, you can stop serving them, because they do not know any good URLs anymore. Fill their queue with garbage, drop them after. Or serve them more garbage!

Point is, the robots are still dumb, and defeated by very simple tricks, all of which are cheap, and they work even if they use real browsers. As long as you poison the URLs.

So serve them garbage. It helps.