The Gatekeeper Audit: Scoring UK Bot Checks on Friction, Failure and Fairness

There is a familiar moment in UK product reviews when someone asks whether the bot check can be removed. The answer is usually no, followed by a vague reference to security. What rarely follows is evidence. The check stays, the form grows a little slower, and a small but significant group of users discovers that the last step is the one they cannot take.

This is not an argument against bot mitigation. Fraud, scraping and credential stuffing are real. It is an argument for measuring the gate before it becomes permanent. The Gatekeeper Audit is a practical scoring method for UK teams who need to separate genuine security need from inherited friction, and to do so with enough rigour to survive a procurement conversation.

Why a scorecard beats a strong opinion

Most debates about CAPTCHA and similar checks collapse into taste. One colleague finds a puzzle easy; another finds it irritating. Neither observation is data. A scorecard replaces anecdote with a small set of observable dimensions, each scored consistently, so that teams can compare one gate against another and against the risk it is meant to address.

The method below is deliberately lightweight. It can be run in a fortnight by a small product team, with or without an external researcher, and it produces a number that is useful in a release gate rather than a report that is filed and forgotten.

The five dimensions

Score each dimension from 0 to 3, where 0 means the check fails the criterion outright and 3 means it is handled well. Record the evidence beside each score: timings, session recordings, assistive technology notes, and verbatim comments.

1. Time to pass. Measure median and 90th percentile completion time for the check alone, not the whole form. A gate that adds twenty seconds for most users but three minutes for a meaningful minority is not a uniform cost. Test on a mid-range Android phone over a congested connection as well as on a desktop.

2. Recovery after failure. Deliberately fail the check. Count the attempts allowed, the clarity of the error message, whether the challenge resets to something harder, and whether the user is returned to their entered data or forced to start again. A gate that punishes a mistyped character by discarding a completed checkout is a design failure, not a security feature.

3. Assistive technology parity. Test with a screen reader, keyboard only, switch access, and at 200 per cent zoom. Note whether instructions are announced, whether focus is managed when the challenge changes, whether audio alternatives exist and work, and whether time limits can be extended. Parity is not a nice-to-have bolted on after launch; it is the difference between a gate and a wall.

4. Dead-end exposure. Identify what happens when a user genuinely cannot pass. Is there a documented alternative channel, such as a telephone line or an in-person route, and is it reachable within the same service hours? If the only fallback is an email address that replies in five working days, the gate has created an exclusion with no proportionate remedy.

5. Proportionality. Compare the strength of the check with the risk of the task. A bot check before a password reset on a high-value account is easier to defend than one before a newsletter sign-up or a low-value basket. Ask what specifically is being protected and whether a less intrusive signal, such as rate limiting or passive risk scoring, would achieve the same outcome.

Weighting and thresholds

Not every dimension carries equal weight. For public-sector and account-access services, dead-end exposure and assistive technology parity should be weighted double. For retail checkouts, time to pass and recovery after failure matter more, because abandonment is immediate and measurable.

A simple weighting produces a score out of a possible total. Teams should agree a threshold in advance: below it, the gate cannot ship without a named owner accepting the exclusion risk. Above it, the check is retained but reviewed at the next scheduled audit. The number is not the point; the prior agreement is.

Running the audit in practice

Recruit five to eight participants, including at least two people who use assistive technology regularly. Set realistic tasks rather than asking people to 'test the CAPTCHA'. A participant signing in to check an appointment, or completing a purchase, will encounter the gate in context, which is where friction becomes visible.

Record the session with consent. Time the gate specifically. Ask participants to describe what they expect to happen before they interact with it, and what they would do if it failed twice. The second question often reveals more about dead ends than any heuristic review.

Run the same protocol on the supplier's staging environment before procurement. Ask the vendor for their own accessibility evidence, but do not accept a generic statement of conformance in place of a demonstration. If the check cannot be tested before contract, that is itself a finding.

What good looks like

Good bot mitigation is often quiet. It uses passive signals, applies friction only when risk is elevated, and offers a clear, accessible alternative when a challenge is presented. It does not ask a user to identify traffic lights for the fourth time in a single journey, and it does not treat a failed challenge as evidence of bad faith.

In the UK context, teams working on public services should also consider the expectations set by accessibility regulations and the government's own service standards. A check that cannot be completed by a switch user is difficult to reconcile with a commitment to inclusive service delivery, whatever the security rationale.

Making the gate a release criterion

Add the Gatekeeper Audit to the definition of done for any feature that introduces a bot check, and to the due diligence checklist for procurement. Name an owner for the score, review it annually, and publish the fallback route internally so that support staff know what to offer when a user is stuck.

The aim is not to remove every gate. It is to make sure that each one is earning its place, and that the people who cannot pass it are not discovered after launch. In 2026, with fraud pressure rising and accessibility expectations hardening, that balance is too important to leave to a strong opinion.