In vulnerability research, most of the work is ruling things out

Between client engagements we hunt on public vulnerability disclosure programs: companies that have published a policy inviting outside testing of named assets, on stated terms. It is deliberately unlike a scoped engagement. Nobody has written the rules of engagement around what we happen to be good at, nobody has handed us credentials, and nobody has any obligation to agree with us. A claim is worth nothing until it reproduces in front of a stranger who would rather it did not.

That is a useful place to keep a practice honest, and this piece is about what the work actually consists of, because the popular picture is wrong in a specific way. The popular picture is discovery: finding the thing nobody else found. In practice, discovery is the cheap half and getting cheaper every quarter. The expensive half is discarding the candidates that look identical to a finding right up until you try to prove one.

The gate before the first packet¶

Everything starts with a scope document, and the runner treats it as a technical control rather than as a piece of paperwork filed nearby. With no authorization document naming the hosts, the tool does not run in a reduced mode or print a warning. It writes out a template to be filled from the program's policy page, and refuses. Parsing an unfilled template yields zero in-scope entries, so the refusal persists until somebody has actually pasted in the program's own words.

This is not caution theatre. Authorization in this work is a legal boundary with a person's name on the wrong side of it if it is crossed, and the moment you find out whether your discipline is real is the moment an interesting hostname resolves at the edge of the scope, late, when you are tired. A rule enforced by the tool is available at that moment. A rule enforced by intention is not.

# This script only performs reconnaissance, crawling, templated known-issue checks,# and GET-based content discovery. It does not submit forms, upload files, brute# force credentials, exploit findings, or mutate application state. With --verify it# additionally runs read-only active probes (canary reflection, open-redirect# sentinel, hidden-param discovery, SSRF OOB probe prep) plus, like the GraphQL# stage, a non-mutating introspection query — still no state-changing requests.

Then the arithmetic of ruling out¶

A recon pass over a mid-sized estate produces candidates in the hundreds: hostnames pointing at deprovisioned cloud resources, exposed source maps, forgotten staging origins, endpoints that answer without a session, dependency versions with a published advisory. Each one is a finding-shaped object. Almost none of them are findings.

A dangling DNS record is only a takeover if the service it points at will actually let you claim that name today. An exposed source map matters if it reveals an internal API surface or a key, and not if it reveals that the front end is written in TypeScript. An unauthenticated endpoint is a vulnerability if the data behind it is not already public, and a vulnerability with no impact is a paragraph you are asking an engineer to read for no reason. A dependency advisory applies if your code reaches the vulnerable path, which for most advisories it does not.

So each candidate gets checked against the target's actual threat model before anything is written up, and the overwhelming majority die there. Our reference implementation entry for the harness records one of these passes plainly: two hundred and fifty-three takeover candidates surfaced, zero real. That ratio is not a failure of the tooling. It is the tooling doing its job, and it is the number a client should ask any testing firm for.

What a validator can and cannot enforce¶

There are two gates in front of a write-up, and conflating them is how automated security tooling ends up producing confident nonsense. The first is mechanical. Before a draft can leave the machine it is linted against the report anatomy: title, summary, severity with a real CVSS vector, affected asset, numbered reproduction steps, a proof-of-concept section that carries an actual captured request or response rather than a description of one, impact, remediation. Leftover template placeholders fail it. An unredacted secret fails it. A proof section that is only prose is flagged for a human rather than waved through.

The second gate is judgement, and it is not a tool. Is the triggering party untrusted across a real boundary? Does the attack need a configuration nobody runs? Has the affected code shipped? Who owns the root cause: the application, or a dependency the vendor also has to fix? Is this asset in scope for a reward, or merely in scope for reporting? Every one of those questions has closed a technically correct finding in our corpus, and none of them can be answered by a linter.

We use language models heavily for the first half of the work and never for the claim itself. They are genuinely good at breadth: enumerating a surface, correlating one host's behaviour against another, reading more source in an afternoon than a person reads in a week. They are also, structurally, excellent at producing a fluent report about a vulnerability that does not exist. The counter-rule has to be mechanical, because a discipline that depends on remembering to be sceptical at the end of a long day is not a discipline.

The same problem, seen from the other side of the inbox¶

Everything above describes the reporting side. The reason it belongs on a page a buyer might read is that the receiving side of the same pipe is now the harder job, and the projects with published numbers have made that unusually legible.

curl ran a bounty from April 2019, paid out more than $100,000, and confirmed 87 real vulnerabilities through it. Its founder's own account of ending it gives the shape of the collapse: a confirmed-report rate that had run above 15% for years fell below 5% during 2025, against a rising volume of AI-generated submissions that took real hours to debunk. The bounty stopped on 31 January 2026. Reports went to another platform, that move was judged a mistake within a month, and the project returned to accepting reports without any money attached from 1 March 2026. Nextcloud suspended its own payouts in April 2026 for the same stated reason.

Read that as an operations problem rather than as a story about artificial intelligence. The reward was purchasing volume. Triage capacity was the constraint the whole time, and paying for more submissions without adding triage is a way of finding out where your constraint is. Note also what neither project abandoned: both still accept reports and still handle them. Disclosure survived. The bounty is what got cut.

If you are standing one up¶

Four things carry most of the weight, and none of them is the reward table. Publish the policy where a researcher looks for it: RFC 9116 specifies /.well-known/security.txt, with a contact and an expiry date, and it takes an afternoon. State scope precisely, in terms of assets and permitted techniques, because a vague scope is resolved in the reporter's favour and you will argue about it later. Say what you will do and when: acknowledgement inside a stated window, a decision, and a disclosure position. And include safe harbour, in writing, for work that stays inside the scope you published.

That last one is not optional in practice. A policy with no safe-harbour statement asks a researcher to absorb legal risk on your behalf, and most will decline, which selects for exactly the reporters you would rather not hear from. Federal civilian agencies have been required to publish such a policy since CISA's Binding Operational Directive 20-01 in September 2020, and its language is a defensible starting point for a private program. For the internal side of it, the handling path from report to fix to advisory, the CERT/CC guide to coordinated vulnerability disclosure remains the reference, and its most useful chapter is the one on what to do when the process goes wrong.

We publish our own policy under the same rules, including the part most policies leave out: what we do not offer. There is no reward attached to it, and saying so is more useful to a researcher deciding where to spend an evening than a vague hint that there might be.

Then write the rubric that lets you close a report fast and defend the closure. Most programs die of politeness: a report that should have been closed on the threat model in ten minutes instead sits open for six weeks because nobody wants to be the person who said no to a researcher. The rubric is what makes saying no a process rather than a personality trait.