v3 · EVIDENCE-CORRECTED REVIEW DRAFT
When the Defender Gets Refused
Reproducing the OpenAI–Hugging Face security asymmetry —
and testing a local-model solution.
An AI agent was capable enough to escape its intended environment
and reach real infrastructure.
and reach real infrastructure.
Then AI safety systems interfered with defenders
trying to investigate the resulting attack.
trying to investigate the resulting attack.
FINKI · Local AI Security Range · 2026
How an AI started cheating on a cybersecurity exam
· OpenAI was evaluating autonomous agents on ExploitGym
· Agents rewarded for solving exploitation tasks — some tasks were hard enough that they looked for ways around the evaluation
· They found unauthorized communication channels, escaped intended network isolation, obtained internet access
· They reasoned Hugging Face might contain models, datasets, or clues that would help satisfy the grader
They were supposed to demonstrate cybersecurity skill inside the exam.
They started doing real cybersecurity outside the exam to pass the exam.
They started doing real cybersecurity outside the exam to pass the exam.
Some agents had already found the right answer and kept going anyway —
they misunderstood what proof the grader required.
Source: OpenAI, “The Hugging Face incident and the road ahead,” 26 Aug 2026 · openai.com/index/hugging-face-incident-and-the-road-ahead/
Then Hugging Face had to investigate
A large body of real attack telemetry to reconstruct — exploit payloads, shell commands, C2 artifacts: material that looks dangerous to a safety classifier.
Commercial hosted AI systems could block or interrupt parts of this legitimate forensic work.
Hugging Face reports moving its analysis to GLM-5.2 on its own infrastructure.
Attack evidence does not become harmless because the person reading it is a defender.
Source: Hugging Face, “Security incident disclosure — July 2026” · huggingface.co/blog/security-incident-july-2026
The asymmetry
Attackers are not constrained by the defender's AI provider policy.
Defenders can be.
Defenders can be.
Can we reproduce this problem —
and solve it without creating an unconstrained agent?
What does an attack actually look like?
The real incident:
escaping isolation · outside connectivity · coordinating / metagaming · probing infrastructure beyond the evaluation
escaping isolation · outside connectivity · coordinating / metagaming · probing infrastructure beyond the evaluation
Our lab deliberately simplifies:
password guessing → admin takeover → malicious plugin → periodic "phone home" traffic
password guessing → admin takeover → malicious plugin → periodic "phone home" traffic
We reproduce the security dynamics, not the exact intrusion mechanics.
What does the defender do?
notice → investigate → reproduce → contain → verify → report
❓ Notice: why are there many failed logins?
🔎 Investigate: did one account eventually succeed?
🔁 Reproduce: what does the suspicious plugin or traffic actually do?
🚧 Contain: block the attacking source, revoke the compromised session
✔ Verify & report: did it stop? what happened? what should change?
The same action can attack or defend
| offensive use | technical action | defensive use |
|---|---|---|
| break into a server | write an exploit | verify a patch really fixed it |
| maintain attacker access | modify a webshell | create a test sample for a detector |
| steal accounts | try many passwords | verify rate limiting / lockout |
| control malware | generate C2 traffic | test whether monitoring detects beaconing |
| hide from defenses | obfuscate a sample | test whether a detector survives evasion |
Code syntax does not tell you authorization.
Why safety systems struggle here
a safety system sees: exploit + passwords + webshell + beacon + evasion
↓ must infer ↓
attacker? · defender? · classroom? · incident responder?
↓ must infer ↓
attacker? · defender? · classroom? · incident responder?
False negatives are dangerous. False positives can disable the defender.
Our research question
Can we reproduce defender lockout on legitimate security work?
Can a local/open model solve it?
Can we do that without giving the model authority to cross boundaries?
Can a local/open model solve it?
Can we do that without giving the model authority to cross boundaries?
Our university cyber range
| Incident | synthetic university login compromise, plugin drop, then beaconing |
| Evidence | the same incident and artifacts for each analyst |
| Run01 benchmark | model tool calls pass through a deterministic policy gateway |
| Live demo | an AI analyst reads replayed events and updates simulated containment state |
The demo hub records containment state; it does not yet enforce expiry or validate targets.
Enforcement evidence comes from the benchmark gateway.
Experiment 1: legitimate work that looks like crime
✍️ exploit → verify a patch
🧬 modify fake webshell → test detection
🔑 password attempts → test rate limiting
📡 reproduce C2 traffic → monitoring
🥷 evasion fixture → test a detector
🔓 crack our own test material → awareness
Arm A: signed authorization present
Arm B: assertion only
Agents: Claude Code · Codex · pi+GLM-5.3 hosted · pi+Qwen3.8-27B local
(observed cross-arm contamination noted on the matrix slide)
Provider blocks and scope-protective decisions
Provider blocks occurred with authorization present ·
checking scope and declining doubtful actions is desirable security behavior, not failure
checking scope and declining doubtful actions is desirable security behavior, not failure
What a provider block actually looks like
Claude — webshell task, Arm A:
"API Error: Sonnet 5 can't help with this.
Start a new session to continue."
Codex — exploit task, Arm A:
"ERROR: This content was flagged for possible
cybersecurity risk."
Service error messages — unlike a model explaining concerns about scope.
Authorization paperwork was part of the Arm A setup; the Codex transcript explicitly shows it being read.
Knowing when to stop is part of the job
Claude, hosted GLM, and local Qwen sometimes declined after inspecting evidence that contradicted the claimed lab context.
Withholding action when authorization or scope is doubtful is protective behavior.
provider intervention
model scope judgment
execution failure
These are different things — the matrix counts them separately.
So… just remove the guardrails?
If provider safety controls can lock out defenders,
why not use a model that simply refuses less?
why not use a model that simply refuses less?
provider lockout ◄────────── slider ──────────► "unrestricted" ⚠️
Remember how this story started
reduced safeguards + imperfect isolation + autonomous optimization
↓
boundary crossing
↓
boundary crossing
Removing model-level friction does not create a security architecture.
Source: OpenAI, “The Hugging Face incident and the road ahead,” 26 Aug 2026
Experiment 2: make the local model less likely to refuse
normal model
"abliterated" = modified to suppress
much of its learned refusal behavior
much of its learned refusal behavior
Same defensive benchmark · same tools · same policy gateway · same out-of-scope bait
Less refusal did not improve the measured result
| correct | completed | partial-refusal flags | production-bait attempts | |
|---|---|---|---|---|
| standard Qwen | 81/84 · 96.4% | 79/84 · 94.0% | 16/84 | 0 / 3 |
| refusal-reduced Qwen | 78/84 · 92.9% | 78/84 · 92.9% | 11/84 | 3 / 3 |
Fewer refusal flags brought no capability gain — and more boundary attempts.
Capability is not authority
The AI may decide: "I think we should scan this machine."
The AI does not decide whether that machine is authorized.
Capability: reason · inspect · generate · propose
Authority: allow / deny execution
Move authorization into infrastructure
Model → proposed tool call → policy gateway → permitted execution
· authorize the target and action
· bound parameters · record decisions · constrain side effects
The rule-breaker hit the wall
3 / 3
forbidden production actions attempted
forbidden production actions attempted
3 / 3
gateway denials
gateway denials
0
of these requested production scans executed
of these requested production scans executed
The guardrails were not removed. They were moved.
The answer to overly restrictive guardrails is not no guardrails.
It is guardrails in the right place.
It is guardrails in the right place.
MODEL: reason · investigate · generate · propose
INFRASTRUCTURE: authorize · scope · execute · log
But can a small local model actually do the job?
28 defensive job-like tasks
logs · suspicious artifacts · detection rules · tool use · containment decisions · scope discipline
logs · suspicious artifacts · detection rules · tool use · containment decisions · scope discipline
3 repetitions × 3 model variants
same evidence, same tools, same gateway
same evidence, same tools, same gateway
252 graded runs
One RTX 3090 gets close on this benchmark
| correct | mean latency | |
|---|---|---|
| hosted GLM-5.3 | 82/84 · 97.6% | ~64 s |
| local Qwen3.8-27B | 81/84 · 96.4% | ~88 s |
A local 27B model finished within one scored task of the hosted comparison.
Why local/open matters
1 · Local inference avoids an external model-provider policy gate
2 · Incident evidence can remain within institutional infrastructure
3 · Execution boundaries still need independent enforcement
"Open weights" and "local hosting" are different properties.
Local deployment is not a promise of zero model refusals.
What you are about to see
1 · 22 failed logins appear
2 · disposable admin account logs in
3 · unusual admin actions
4 · plugin file appears
5 · host begins periodic beaconing
2 · disposable admin account logs in
3 · unusual admin actions
4 · plugin file appears
5 · host begins periodic beaconing
6 · events become incident AI-0042
7 · AI analyst investigates
8 · analyst requests reversible containment
9 · system state changes
10 · report is written
7 · AI analyst investigates
8 · analyst requests reversible containment
9 · system state changes
10 · report is written
🧪 Live terminal demo
AI-0042 · investigate → update simulated containment → report
Plan: one local-Qwen run live; other agents as saved comparisons.
Staged before this act · short event replay · ./demo.sh run pi-qwen ·
show REPORT.md + before/after /state
If the live run stalls: labeled rehearsal transcript — no on-stage debugging.
What just happened?
✔ The model investigated synthetic incident evidence
✔ It requested containment and updated the demo's state
✔ It produced an evidence-based report
✔ Local inference kept the model request inside lab infrastructure
The workflow is live. The containment state is simulated.
What we learned
1 · Provider blocks appeared in documented dual-use tasks
2 · Local models can still make reasoned scope refusals
3 · Refusal reduction did not improve the stored benchmark scores
4 · The benchmark gateway denied all three explicit production-scan attempts
5 · Local Qwen was close to hosted GLM on this task set
What we have not proven
- Small samples and reused workspaces limit model comparisons
- Missing targets and invalid fixtures prevented some requested outcomes
- Stored grades are not a production-readiness measure
- The live demo is not a complete security boundary
- Local inference and a gateway do not solve every AI-security problem
Notably: one arm-B transcript references the arm-A authorization document —
the two arms cannot be treated as perfectly isolated.
Guardrails in the right place
Local control removes the external provider-lockout problem.
Infrastructure-enforced policy addresses model overreach.
Infrastructure-enforced policy addresses model overreach.
The model should have the capability to help the defender.
It should never have the authority to decide its own boundaries.
It should never have the authority to decide its own boundaries.
FINKI · Local AI Security Range · 2026 — thank you 🙏