Bot detection methods, honestly ranked

Every bot-detection vendor claims "advanced AI-powered detection." Here's the unglamorous truth: there are about five real techniques, each with known strengths and failure modes, and the interesting choices are which ones you refuse to use. This is the ranking we'd give a technical friend — including where our own approach sits and what it can't do.

1. User-agent matching — necessary, laughably insufficient alone

Reading the UA string catches every honest bot: reputable crawlers identify themselves (the current list). Cost: near zero. Failure mode: total, against anyone who lies. A UA is a self-reported name tag. On our own site, roughly two-thirds of checkable claimed-agent visits turned out to be impostors wearing GPTBot's name. UA matching is the index, never the verdict.

2. IP verification — the highest-value upgrade almost nobody does

Major vendors (OpenAI, Google, Microsoft, Perplexity, Amazon…) publish official IP ranges for their agents. Checking a claimed identity against its vendor's ranges turns the name tag into a checkable credential: inside the ranges = genuine; outside = impostor, downgrade it. Cost: keeping ranges fresh (they rotate — automate the refresh) and a fast lookup per request. Failure mode: only covers agents whose vendors publish ranges, and says nothing about bots that don't claim an identity. Still: highest signal-per-effort of any technique here. It's the core of our engine, and you can try it by hand with the free checker.

3. Behavioral / environment signals — the only way to catch agentic browsers

AI agents driving real browsers execute JavaScript and carry human-looking UAs — the first two methods barely see them. What does: environment tells. navigator.webdriver set, headless markers, zero plugins, no languages, a Chrome UA with no window.chrome, inhuman timing patterns. Individually weak, jointly strong. Failure mode: an arms race by nature — signals rot as automation frameworks patch them, so treat scores as classifications, not verdicts, and expect maintenance forever.

4. Fingerprinting — effective, and we won't use it

Canvas hashes, font enumeration, audio-context quirks — building a persistent device identity to track visitors across sessions. It genuinely catches sophisticated bots. It's also privacy-hostile by construction: the same identifier that tracks a bot tracks every human, consent frameworks exist precisely because of it, and browsers actively fight it. We ruled it out on principle — no fingerprinting, no cookies, GPC honored — and accept the detection gap that choice implies. Vendors using it should say so plainly; most bury it in "advanced signals."

5. CAPTCHAs and challenges — punishing humans to inconvenience software

Challenges stop unsophisticated bots and delay sophisticated ones, at the cost of friction for every legitimate visitor — and in the agentic era, they specifically break the AI assistants acting for real customers. Solving services have made them a $2-per-thousand speed bump for determined actors. Legitimate uses remain (signup abuse, credential stuffing), but as traffic classification they're the wrong tool: they don't identify anything, they just gatekeep.

The honest architecture

No single method survives contact with reality. The stack that works: UA matching to index the honest, IP verification to convict the liars, behavioral signals to catch the browser-shaped agents — layered, with each claim carrying its evidence (verified: true/false/unverifiable) instead of a black-box score. And equally important, the refusals: no fingerprinting, no challenges, fail-open when unsure — because a detection layer that breaks real visitors to catch bots has inverted its own purpose.

That's the whole secret. Anyone promising more is selling the word "AI," not detection.