Find the weaknesses in your AI before attackers do
AI Probe runs thousands of adversarial attack payloads against your chatbots, assistants, RAG pipelines and agents, then an AI judge scores every response. You see exactly where your AI gives way, across every known category of LLM vulnerability, inside the XHack platform you already use.
app.xhack.io / ai-probe
3500+
Adversarial attack payloads
OWASP LLM Top 10
Full category coverage
AI judge
Scores every response
5
Severity levels, critical to informational
What it does
Adversarial testing for AI, end to end
Connect a target, choose your attack categories, and let AI Probe try to break it the way a real attacker would. Here is the whole product at a glance.
Any AI target
Connect any system with an HTTP API. Presets cover OpenAI, Anthropic Claude, Google Gemini and OpenAI-compatible endpoints.
3500+ attack payloads
A library spanning every OWASP LLM Top 10 category, ready to run against your target out of the box.
Multi-turn attacks
Crescendo, persistence, context manipulation and confusion strategies that attack across a full conversation.
AI judge
A dedicated security judge reads intent and context, scoring every response across five severity levels.
Background scans
Launch and walk away. Scans run in the background with real-time progress and automatic rate-limit pacing.
Escalate findings
Take any finding into an interactive chat, generate bypasses, and push the attack deeper by hand.
Professional reports
Export HTML or PDF assessments with an executive summary, a grade and per-finding detail.
Tenant-isolated
Scan data stays inside your tenant. Keys and auth headers are encrypted at rest and never returned.
Full coverage
Every category in the OWASP LLM Top 10
AI Probe ships with 3500+ pre-built payloads organised across every category in the OWASP LLM Top 10, the industry standard for AI security assessment.
Prompt Injection
Attacker instructions smuggled in through overrides, delimiter confusion, encoding and payload splitting.
Sensitive Data Disclosure
Attempts to coax out personal data, API keys, internal configuration or training data.
Insecure Output Handling
Responses crafted to inject XSS, SQL or other code into the systems that consume them.
Excessive Agency
Pushing the model to take unauthorised actions, escalate privileges or work beyond its scope.
System Prompt Leakage
Dozens of techniques aimed at extracting the hidden system instructions from your AI.
RAG Poisoning
Document injection and context abuse that hijack behaviour through the retrieval pipeline.
Misinformation
False citations, manufactured consensus and plausible-sounding disinformation on demand.
Denial of Service
Token bombs, recursive expansion and context abuse that exhaust resources or billing.
Jailbreaks
DAN variants, developer mode, roleplay escalation, skeleton key and many-shot bypasses.
Bias and Toxicity
Framing designed to lead the model into hateful, discriminatory or stereotyped output.
Connect in minutes
Point AI Probe at any AI target
AI Probe works with any AI system that exposes an HTTP API. Configure a target by setting the endpoint URL, request body template, response JSONPath, authentication headers and a timeout. Presets cover the providers you are most likely to be running.
Presets for OpenAI, Anthropic Claude, Google Gemini and any OpenAI-compatible API.
The {{payload}} placeholder carries a single-turn attack prompt into your body template.
The {{messages}} placeholder carries a full conversation array for multi-turn tests.
REST APIs, chat completions and custom LLM interfaces all work without modification.
ai-probe / add target
ai-probe / scan
Hands-off scanning
Launch a scan and walk away
Select your target, choose which attack categories to include, and launch. The scan runs in the background, so you can close the browser, get on with other work, and check progress whenever you like.
Each payload is sent, the response captured, and the AI judge rules on whether the target held or gave way.
Progress updates in real time, so you can watch findings appear or come back to the full results later.
Scans pace themselves with configurable delays so rate-limited targets never trigger throttling.
Multi-turn attacks
Real attackers use conversation, so does AI Probe
One-shot attacks only test the surface. Multi-turn payloads attack across a whole conversation using four distinct strategies. Every turn is logged and evaluated, so a model that refuses on turn one but complies on turn five is caught.
Crescendo
Starts with innocent questions and escalates across turns, testing whether defences weaken as rapport builds.
Persistence
Repeats the same request with different framings until the target gives in.
Context manipulation
Shifts topics mid-conversation and circles back, testing whether safety context survives across turns.
Confusion
Sends contradictory instructions to destabilise the model and break its coherence.
AI at the core
An AI judge on every single response
AI Probe does not grade with keyword lists. A dedicated AI judge, built for security assessment, reads the intent behind each attack and the substance of each response, then scores it by real impact.
AI judged
Intent and context
The judge reads what the attack is trying to do and what the response actually gave up, rather than scanning for a keyword.
AI judged
Refusal, not theatre
It tells a genuine refusal apart from a soft refusal that leaks information, and apart from real compliance with the attack.
AI judged
Impact-based scoring
Severity reflects real impact. A leaked system prompt is critical. A clean refusal with no leakage is informational and safe.
AI judged
Bypass generator
In Escalate, the AI proposes fresh attack payloads based on everything it has observed in the conversation so far.
Scored across five severity levels
Severity reflects the actual impact of the response, not just whether the AI said yes or no. A reply that reveals the full system prompt is critical. A reply that merely hints at hidden tools is medium. A clean refusal with no leakage is informational and marked safe.
Critical
High
Medium
Low
Informational
See where it failed
A findings dashboard that reads like a report
Results are organised into three tabs: vulnerable findings, safe findings, and other for errors, timeouts and inconclusive runs. Each finding shows its severity, title, attack category, security score and response time.
Expand any finding to see the full prompt sent, the complete target response, and the judge reasoning behind its classification.
A summary panel shows the overall security score, from A through F, at a glance.
Totals by severity and a pass rate across every tested category sit alongside the score.
ai-probe / findings
ai-probe / escalate
Go deeper by hand
Escalate any finding into a live attack
When AI Probe discovers a vulnerability, Escalate lets you continue the attack by hand. It opens an interactive chat pre-loaded with the original context: the prompt that was sent, the target response, and the judge assessment.
Send your own follow-up messages, test variations of the attack, and try to push the impact further.
Ask the AI bypass generator to suggest a new payload from what it has seen in the conversation so far.
Reset the conversation to the original finding state to explore a different path without losing your start point.
Retest after you fix
Retest re-sends the same payload on any finding and has the judge score the new response.
The result updates in place, so you watch the status move from vulnerable to safe.
Retest All runs every finding again in one pass for a full before-and-after across the scan.
Build your own payloads
Custom payloads support every field a system payload does: category, severity, attack mode, strategy and prompt template.
Set your own expected behaviour and custom judge criteria so scoring fits your system.
Test organisation-specific risks: proprietary data leakage, business logic abuse and industry compliance.
Share the results
Professional reports in HTML or PDF
Generate a complete security assessment at any point during or after a scan. Reports are formatted for professional use, ready to hand to engineering teams, security leads, compliance officers or clients.
Executive summary, overall security score and grade, and a severity breakdown chart up front.
Detailed findings with the prompt, the response, the judge reasoning, and category-by-category results.
A confidential banner, scan metadata and clear status indicators for any incomplete scan.
ai-probe / report
Who it is for
Built for security teams and researchers alike
AI Probe is available on the Elite company plan and the Elite researcher plan, so organisations and individual researchers both get the full attack library and AI judge.
Elite company plan
For organisations. Manage targets, configure payloads, view scan history and monitor scans across the whole team from the platform dashboard, with the role-based access you already use everywhere in XHack.
Elite researcher plan
For individual researchers. The same full attack library, multi-turn strategies and AI judge, in a workspace built for solo, deep-dive testing of the AI systems you care about.
Tenant-isolated and encrypted by default
All scan data is processed inside your own tenant and is never shared across organisations. API keys and authentication headers are encrypted at rest and are never returned in API responses, so the credentials you connect stay yours.
A closer look
AI Probe across the platform
The overview and per-scan detail, inside the same multi-tenant workspace as the rest of XHack.
AI Probe overview
Targets, scan history and payload library in one place for the whole organisation.
Scan detail
Drill into any scan to follow its findings, progress and results end to end.
Questions
AI Probe, answered
Anything that exposes an HTTP API. You configure a target with an endpoint URL, a request body template, a response JSONPath, auth headers and a timeout, and presets are ready for OpenAI, Anthropic Claude, Google Gemini and any OpenAI-compatible API. The body template uses placeholders: {{payload}} is swapped in for single-turn attacks and {{messages}} carries the full conversation for multi-turn tests, so chatbots, assistants, RAG pipelines and tool-calling agents all work without modification.
It runs 3500+ pre-built payloads organised across the full OWASP LLM Top 10: prompt injection, sensitive data disclosure, insecure output handling, excessive agency, system prompt leakage, RAG poisoning, misinformation, denial of service, jailbreaks, and bias and toxicity. You choose which categories to include on each scan.
A dedicated judge built for security assessment reads both the attack and the response in context. It does not rely on keyword matching, so it can tell a genuine refusal apart from a soft refusal that leaks information and from real compliance. Findings are scored across five severity levels, from critical down to informational, based on the actual impact of the response.
Yes. AI Probe ships multi-turn payloads that use four strategies: crescendo escalation, persistence, context manipulation and confusion. Every turn is logged and evaluated, so if your AI refuses on turn one but complies on turn five, the judge catches it.
Use Retest on any finding to re-send the same payload and have the judge score the new response. The result updates in place, so you can see the status move from vulnerable to safe. Retest All runs every finding again in one pass, giving you a full before-and-after across the whole scan.
AI Probe is available on the Elite company plan and the Elite researcher plan, so it serves both security teams and individual researchers. All scan data is tenant-isolated and never shared across organisations. API keys and authentication headers are encrypted at rest and are never returned in API responses. It runs inside the same XHack platform and account you already use.
Probe your AI before someone else does
Connect a target, pick your attack categories, and let AI Probe and its AI judge show you exactly where your AI gives way. Start in the dashboard or book a walkthrough with our team.