Table of contents
19
By Salman Khan, OSCP+, Founder of XHack, SRT (Synack Red Team member)
Read this in 30 seconds: Nobody has published a fair head-to-head of any LLM for bug bounty work on ordinary in-scope targets, so any ranking you read is an opinion, a vendor self-report or a test on someone else’s tasks. I read the evidence that does exist on 8 November 2026 and turned it into a test you can run in an afternoon.
- The only refusal study measured a different question. A May 2026 Gray Swan and Carnegie Mellon paper tested 8 models on exploit tasks aimed at critical-infrastructure targets, and 6 of 8 showed near-zero refusal rates on its medium and hard tasks. It says nothing about how they treat an in-scope bug bounty target.
- Hunters who pick an LLM for bug bounty hunting name different ones. One YesWeHack interviewee picks Opus 4.6 and 4.7 plus GPT 5.5, another says “Claude by a long shot”, and neither mentions refusals, cost or long sessions.
- The model matters less than the proof step. YesWeHack’s own guide shows an agent finding a real DOM XSS and then handing over a payload that did not work, which is a failure every model can have.
- Price spreads are 5x, not 2x. Anthropic’s list prices run from $2 to $10 per million input tokens between Sonnet 5.5 and Fable 5.1, so cost per verified finding is the number to track, not the monthly bill.
- Reduced-refusal programs exist but are gated. OpenAI’s Daybreak and Anthropic’s Cyber Verification Program both require applications, and neither promises a bug bounty target is covered.
If you searched for the best LLM for bug bounty, you probably want a name. I can give you the names hunters use, but a name goes stale within a quarter, and the evidence behind it is thinner than the confidence it is delivered with. A better answer for choosing an LLM for bug bounty work is a short list of what to measure and a way to measure it.
This guide covers four things that decide whether an LLM for bug bounty work is any good, and what the evidence says about each: how often it refuses, whether long sessions stall, whether its proof steps hold up, and what a verified finding costs. I did not benchmark models myself, so every number below comes from a source I name.
The public evidence for choosing an LLM for bug bounty work is a handful of practitioner interviews, three academic or vendor benchmarks and the vendors’ own documentation, and none of them test ordinary in-scope bounty hunting end to end. The table says what each source measures, so you can see why no ranking falls out of them.
| Source | What it measures | What it leaves out |
|---|---|---|
| YesWeHack interviews, June 2026 | Which models two full-time hunters say they prefer | Any measurement; refusals, cost and long sessions are not discussed |
| YesWeHack agentic CLI guide, 18 June 2026 | A worked example of a correct finding with a failing proof | Comparison between models |
| Gray Swan refusal paper, 31 May 2026 | Refusal and utility for 8 models on 29 web-exploit challenges themed on critical infrastructure | Ordinary bounty targets; newer models |
| BountyBench, Stanford, v3 December 2025 | Detect, exploit and patch rates on 25 systems and 40 real bounties for 5 agents | 2026 models; refusals |
| Cybench, 40 CTF tasks | Share of CTF tasks solved, mostly from vendor system cards on 35 to 39 task subsets | Real targets; cost; refusals |
| Alias Robotics benchmarking page | Cybench pass@3 from a vendor that also sells a competing model | Methodology, which it sells as a service |
The pattern across every LLM for bug bounty source is that capability is measured on puzzles or on 2025-era agents, refusal is measured on a different target class, and cost is not measured at all. Treat each source as one input and trust none of them as a ranking.
Refusal behaviour is the least-measured property of any LLM for bug bounty hunting, and the one study that exists tested targets you will never be allowed to hunt. The Gray Swan paper built 29 challenges themed on four red-zone domains, which are government, healthcare, power grid and transportation systems, and proposed that agents should always refuse those.
Its results show how uneven refusal is. On the easy tier, which used dummy servers on localhost, refusal rates ranged from 3 percent for Nemotron Super 120B and 13 percent for Gemini 3.1 Pro to 100 percent for Claude Sonnet 4.5, GPT-5.2 and GPT-5.1 Codex. The paper’s headline is that 6 of 8 models showed near-zero refusal on its medium and hard tasks, which used real production sites with injected test pages.
Two cautions keep that from becoming a ranking. First, those challenges were designed so that refusing was the correct behaviour, and an ordinary program with a published scope is the opposite case. Second, the models were Claude Sonnet and Opus 4.5, GPT-5.2, GPT-5.1 Codex, Gemini 3.1 Pro and 3 Flash, Nova 2 Lite and Nemotron Super 120B, which are a generation behind what you are likely to run today.
The useful lesson is narrower: the paper reports that refusals depended on how the target and task were framed, and that models often acted on a target after noticing it looked like a real system. So refusal rates for an LLM for bug bounty work measured with one prompt on one target are weak evidence for a different prompt on a different target.
Both big vendors now have gated access routes, and the details are in their own documentation.
| Vendor | What happens to a flagged security request | Route to fewer blocks |
|---|---|---|
| Anthropic | Claude Code’s model page says Fable models’ safety classifiers most often flag cybersecurity and biology content, which triggers automatic fallback to another model | Cyber Verification Program; individuals can apply for the Defense tier only, per press coverage of the October 2026 relaunch |
| OpenAI | Standard safeguards on mainline models | Daybreak Blue (reduced refusals on supported models) and Daybreak Red (cyber models), both by application |
On OpenAI’s side, the Daybreak help page says individual access needs an eligible paid plan, Advanced Account Security, at least one physical FIDO2 hardware key and identity verification, and that approval is not automatic. It also says Daybreak does not remove all refusals or grant zero data retention by default, and that it is meant for systems you own or are explicitly authorized to test. I could not tell from the page whether a public bug bounty scope counts as explicit authorization, so ask before you rely on it.
For Anthropic, I covered the fallback table and the Cyber Verification Program in the Claude Code bug bounty guide. The short version is that your hardest requests may silently run on an older model unless you set Claude Code to ask first.
Long-session stalls are mostly a harness and context problem, and no public benchmark isolates them by model. Irregular’s research on offensive-security capability names long-horizon planning, state management and error recovery as the places current systems still fall short, without a per-model breakdown.
The practitioners show the same pattern in how they work. Rhynorater says he runs four tmux panes of Claude Code beside manual hacking, plus an autonomous setup that attacks predefined targets, and that he built a validator agent to cut false positives. The YesWeHack guide argues that how you drive the model, what context you give it and how carefully you validate matter more than which model you choose. Those are workflow answers to a stall problem.
Vendor claims point the same way but should be read as claims. Anthropic’s documentation says its Fable models sustain long autonomous sessions and verify their own work more often. That is a vendor description with no number attached.
BountyBench shows why the harness matters. It ran five agents on 40 real bounties, and Claude Code and OpenAI Codex CLI had high patch scores of 87.5 and 90 percent but exploit scores of 57.5 and 32.5 percent, while custom agents were more balanced. Detecting an unknown bug was the weakest task. Those are 2025 agents, so read it as a lesson about task types, not a model verdict.
A stall is a stretch where the agent keeps calling tools without new information. Log the minutes between the last useful observation and your intervention, and the number of times you had to restart or summarise the context. If one model needs three nudges per lab and another needs none, you have your answer without a benchmark.
A correct finding with a broken proof of concept is the most common failure, and the YesWeHack guide documents it with an example. Claude Code found a real DOM-based XSS in the search parameter of PortSwigger’s practice shop, then produced a payload that failed because the application blocks injected HTML tags. The guide’s summary is that the vulnerability was real but the proof was not, and that a model can name the right sink while still giving a PoC that does not work.
That matters for money. Intigriti and Bugcrowd both warn about unverified AI reports, and our AI bug bounty policy guide covers what each platform says. A model that is right about the bug and wrong about the proof costs you a report that gets closed.
So the verification rule is the same whichever LLM for bug bounty work you pick: you reproduce every finding by hand before you write it up. Rhynorater’s validator agent is a reasonable second layer, but he does not say how well it works, so do not treat it as a replacement. The bug bounty report writing guide covers what a report needs after that.
The price gap between models is large enough to dominate your budget, so measure cost per verified finding and not the headline price per token. Anthropic’s pricing page, read on 8 November 2026, lists these base prices per million tokens.
| Model | Input | Output |
|---|---|---|
| Claude Fable 5.1 | $10 | $50 |
| Claude Opus 5.5 | $4 | $20 |
| Claude Sonnet 5.5 | $2 | $10 |
| Claude Haiku 5.5 | from $0.10 | from $0.50 |
I could not read OpenAI’s or Google’s API price tables on that date, so I have not listed them. Check each vendor’s page, because the figures change.
Here is illustrative arithmetic, not a measurement. A session that sends 2 million input tokens and receives 200,000 output tokens costs $6 on Sonnet 5.5 ($4 input plus $2 output), $12 on Opus 5.5 and $30 on Fable 5.1. Real sessions vary widely, and prompt caching lowers the input side, so the only trustworthy figure is the one from your own usage page.
Subscriptions add a wrinkle to any LLM for bug bounty budget. Claude Code’s documentation says that depending on your plan and seat tier, Fable usage can bill to usage credits instead of drawing on your plan’s included limits. In non-interactive mode it bills without asking. If you run an agent unattended, check that setting first, and if you are weighing the cost of Burp’s AI features too, the Burp AI pricing guide works through that bill.
You can rank models for your own workflow in an afternoon by running the same lab set through each one and logging four numbers. Use targets you are allowed to attack when you test an LLM for bug bounty work, such as PortSwigger’s free labs or your own test application, and keep the harness, prompt and tools identical between models.
| Test | How to run it | What to record |
|---|---|---|
| Refusals | Same scoped, honest prompt for each lab | Refused or not, and the refusal text |
| Stalls | Let it run unattended for a fixed time | Minutes stalled, restarts needed |
| Proof | Reproduce every claimed finding by hand | Verified or not |
| Cost | Read tokens from the provider’s usage page | Input and output tokens per run |
Run each lab at least three times, because a single run tells you little about an LLM for bug bounty setup. Then compute cost per verified finding, which is total spend divided by findings you reproduced. This script reads a CSV with one row per run and prints that number for each model. Replace the prices with the ones from each vendor’s page.
import csv
from collections import defaultdict
# USD per million tokens (input, output). Copy these from each vendor's price page.
PRICES = {
"sonnet-5.5": (2.00, 10.00),
"opus-5.5": (4.00, 20.00),
"fable-5.1": (10.00, 50.00),
}
stats = defaultdict(lambda: dict(runs=0, refused=0, verified=0, cost=0.0, stall=0.0))
with open("runs.csv", newline="") as f:
for r in csv.DictReader(f):
s = stats[r["model"]]
p_in, p_out = PRICES[r["model"]]
s["runs"] += 1
s["refused"] += int(r["refused"])
s["verified"] += int(r["verified"])
s["stall"] += float(r["minutes_stalled"])
s["cost"] += (int(r["input_tokens"]) * p_in + int(r["output_tokens"]) * p_out) / 1_000_000
print(f"{'model':<12}{'runs':>5}{'refused':>9}{'verified':>10}{'cost $':>9}{'$/verified':>12}{'stall min':>11}")
for m, s in stats.items():
per = f"{s['cost'] / s['verified']:.2f}" if s["verified"] else "n/a"
print(f"{m:<12}{s['runs']:>5}{s['refused']:>9}{s['verified']:>10}{s['cost']:>9.2f}{per:>12}{s['stall'] / s['runs']:>11.1f}")The CSV needs the columns model,lab,refused,verified,input_tokens,output_tokens,minutes_stalled, with refused and verified as 0 or 1. I tested the script on a four-row sample file, and the sample numbers are not results, so none are published here.
These LLM for bug bounty pairings are my judgment from the evidence above, not benchmark results. Treat them as hypotheses for your own scorecard to confirm or overturn.
| Job | Reasonable starting point | Why |
|---|---|---|
| Summarising recon output and triage | A cheaper model | The task is reading, not exploiting, and the price spread is 5x |
| Chaining findings and building a proof | The strongest model you can afford | Proof failures cost the most here |
| Second-opinion validation | A different vendor’s model | A different model is less likely to repeat the same mistake |
| Local or open-weight work | Only if a refusal-free setup matters more than reliability | In the Gray Swan paper, Nemotron Super 120B refused rarely but solved almost nothing on the easy tier, which the authors tie to malformed tool calls and early stops |
The last row is the useful counter-example to the idea that fewer refusals means a better LLM for bug bounty work. An agent that never refuses but cannot finish a tool call is no use.
So yeah, here is where we talk about our own product. I am biased, it is my company, and a coding agent gives you control over the model that XHack does not.
If you want to pick and swap models yourself, a general coding agent wired to your proxy is the flexible route, and the Claude Code bug bounty guide shows how to set it up safely. The cost is that you own the model choice, the refusal handling, the context management and the verification, which is exactly the list this article just walked through.
XHack takes that work off you. It combines three things most providers sell separately: an autonomous AI agent, a continuous security platform and human testers when you want them. The agent runs up to 12 sub-agents by default and controls three browsers for live testing, and the plans include multi-model support, so you are not running the comparison above before every hunt. If you would rather not run the LLM for bug bounty comparison yourself, that is the gap it fills. It does not store your agent chats, which stay on your own computer and can be deleted at any time.
The honest trade is that exploitation unlocks after identity verification, similar in spirit to the vendor programs above. Individual plans are $20, $49 and $150 a month, with a 7-day free trial, one per person. Details are on the pricing page, and the agent for bug bounty guide shows the workflow. If you would rather have people run the testing, managed engagements start at $2,500, and you can book a free consultation even if it turns out we are not the right fit. Brutal honesty is kind of our thing.
There is no evidence-backed single winner. Two YesWeHack interviewees name Claude, with one adding GPT 5.5, but neither backs it with measurements. The best LLM for bug bounty work for you is the one with the lowest cost per verified finding on your own targets, which the scorecard above measures.
Nobody has measured that on ordinary in-scope targets. The one refusal study, from Gray Swan and Carnegie Mellon, found wide variation by model and by task framing on critical-infrastructure-themed challenges, and it tested models that are now a generation old. Run your own refusal test with an honest, scoped prompt instead of relying on a forum thread.
Only if privacy outweighs reliability. In the Gray Swan paper, the open-weight Nemotron Super 120B almost never refused but solved almost no easy tasks, mainly because of tool-call failures and early stopping. A local model can work for summarising output, but chaining and proof steps are where weaker agents fail.
The cost of an LLM for bug bounty work depends on usage, and published prices differ by 5x between tiers. Anthropic lists $2 to $10 per million input tokens from Sonnet 5.5 to Fable 5.1, and a session of 2 million input and 200,000 output tokens works out to $6 to $30 on those two. Measure your own token use, because caching and session length change the total.
Both big vendors have gated programs. OpenAI’s Daybreak Blue and Red reduce refusals after approval and identity checks, and Anthropic’s Cyber Verification Program offers a Defense tier for individuals. Neither page clearly confirms that a public bug bounty scope counts as authorization, so ask the vendor before you rely on it, and never use a jailbreak.
When you choose an LLM for bug bounty work, the evidence leans toward the harness. YesWeHack’s guide says how you drive the model and validate its output matters more than the choice, and BountyBench found large gaps between agents. Both are limited, since BountyBench used 2025 agents. Test both together as one LLM for bug bounty setup, not the model alone.
The best LLM for bug bounty work is a measurement, not a name. Pick two or three LLM for bug bounty candidates, run the same labs through each, reproduce every finding by hand, and divide spend by verified findings.
Re-run the test when a new model ships, because the vendors change their models, prices and safeguards every few months. Until someone publishes a fair LLM for bug bounty benchmark on real in-scope work, your own scorecard is better evidence than any ranking, including this one.
Categories
Related articles
