How the Adversarial Patch Loop decides when a vulnerability is actually sealed

A rescan is not a proof

The standard static-analysis workflow ends like this: find the flaw, propose a fix, let a developer merge it, confirm on the next scan that the finding is gone, close the ticket. The problem is that the last step proves nothing. The judgement that created the finding and the judgement that cleared it come from the same engine, the same rules, the same point of view. The person who wrote the exam is grading their own answers.

Across customer repositories we kept seeing three failure modes. First, the signature dodge: the patch changes only the string pattern the rule matched, so the scanner goes quiet while the attack path stays open. Second, partial mitigation: the most obvious entry point is closed and a second handler sharing the same weakness is left untouched. Third, displacement: validation moves up a layer and opens a new bypass on the way. In all three cases the rescan comes back clean.

A scanner going quiet and an attack failing are two different events. Most pipelines measure the first one and report the second.

The five phases of a loop

The Adversarial Patch Loop puts two agents against each other inside CI. The AI red team takes one confirmed finding and tries to exploit it for real; the AI blue team writes the smallest patch that defeats that attempt. One round runs through five phases, and each phase leaves behind an artifact the next one consumes.

1. 01

   Attack — the red team plans an exploit

   It receives the finding location, the call graph and the list of entry points, then builds a multi-step attack sequence. The artifact is an executable request sequence plus the observable conditions that define success: response body, status code, side effects.

2. 02

   Breach detection — run it in isolation

   The repository is built into a disposable container and the sequence is executed. If the observable conditions are met, the finding is promoted from theoretical risk to reproduced exploit and the trace is stored.

3. 03

   Patch — the blue team writes the minimum change

   The blue team's input is the exploit trace, not the flawed code. The objective is the smallest change that makes this trace fail; refactors and style edits are explicitly out of scope.

4. 04

   Re-attack — replay the original and the mutations

   The original sequence is replayed against the patched tree, then the mutation suite is thrown at it. Anything that still gets through becomes the next round's input for the blue team.

5. 05

   Seal verification — confirm nothing else broke

   A patch that clears the security gates still has to pass the existing test suite and a response-parity check on normal input. A patch that stops the attack by breaking the feature is not a seal.

The Adversarial Patch Loop's three seal gates

"Sealed" is not a marketing word here — it is a label applied only when three boolean conditions are all true. If any one is false the state is not sealed, and the verdict log records which gate failed and why.

How mutations are generated

The mutation suite has exactly one job: tell whether the blue team blocked a string or blocked a path. Using the successful exploit trace as a seed, three families of variants are derived.

• Encoding family — double URL encoding, Unicode normalisation differences, null-byte insertion, mixed casing: the same meaning expressed in different bytes. Regex-based filters usually collapse here.
• Context shift family — the same payload delivered through a different entry point: another route, a batch API, a webhook receiver, an admin-only handler. This is the gate that catches partial mitigation.
• Chain reorder family — steps of a multi-stage attack are reordered, or an intermediate step is replaced by a different means. An auth-bypass → file-read chain becomes auth-bypass → path-traversal → file-read.

The default is 12 mutations, weighted by finding type: injection findings lean toward encoding variants, while authorisation findings are more than half context shifts. Generation is deterministic under a fixed seed, so the same finding always regenerates the same suite. During an audit you have to be able to show exactly what was thrown.

When it does not seal — mitigation and escalation

The loop does not run forever. The default ceiling is four rounds, and each round the blue team inherits the full context of the previous failure. At the ceiling, the finding settles into one of three terminal states.

1. 01sealed — all three gates cleared. The patch diff and the verdict log open as a pull request.
2. 02mitigated — the original attack fails but some mutations still land. The reproduction steps for the surviving mutations are attached verbatim.
3. 03escalated — a structural flaw that a minimal patch cannot close. It goes to a human with an explicit statement that a design change is required. A broken authorisation model is the classic example.

Reading a verdict log

Every loop terminates in a single verdict object. The same object drives the dashboard, the merge gate and the audit report.

{
  "finding": "FD-2026-07-1183",
  "cwe": "CWE-89",
  "severity": "critical",
  "verdict": "sealed",
  "rounds": 2,
  "gates": {
    "original_exploit": { "reproduced": false, "attempts": 3 },
    "mutation_suite": {
      "passed": 12,
      "failed": 0,
      "families": ["encoding", "context-shift", "chain-reorder"],
      "seed": "fd-1183-a"
    },
    "regression": { "tests": 412, "failed": 0, "behavioural_diff": "none" }
  },
  "patch": { "commit": "a1f7c02", "files": 1, "added": 6, "removed": 2 },
  "sealed_at": "2026-07-09T04:12:38Z"
}

The field that matters most is not verdict but gates. The verdict tells you pass or fail; the gates tell you what that judgement was based on. Because mutation_suite.seed is recorded, an auditor six months later can regenerate the identical suite and confirm the same result.

Wiring it into CI as a gate

The loop earns its keep as a merge gate. The usual configuration requires different terminal states per severity: sealed for critical and high, mitigated acceptable for medium and below.

name: security-gate
on: [pull_request]

jobs:
  flawdetector:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: floatfactory/flawdetector-action@v1
        with:
          api-key: ${{ secrets.FLAWDETECTOR_API_KEY }}
          mode: loop                # scan, then run red team vs blue team
          max-rounds: 4
          seal-gate: critical,high  # these severities must reach "sealed"
          allow-mitigated: medium,low
          comment-pr: true          # post the verdict log on the PR

# exit codes
#   0  gate passed
#  12  mitigated findings remain at a seal-gate severity
#  13  escalated findings present — human review required

When you first switch the gate on, start with seal-gate: critical. Adding high in the same step usually means existing debt blocks every merge on day one, and the decision that follows is to turn the gate off. Splitting debt into a separate backlog and gating only new code had a far higher survival rate.

What we measure, and what we distrust

The internal metric we track is not "number of issues auto-fixed." That number grows with how messy a codebase is, so it correlates weakly with quality. We watch three others instead.

• Seal lead time — from confirmed finding to sealed verdict. The only speed metric a team actually feels.
• Reopen rate — the share of sealed findings that reproduce again within 90 days. It is the first thing to move when verdict criteria drift.
• Mutation pass-through — how often a first-round patch is caught by the mutation suite. A proxy for whether the blue team is blocking strings or blocking paths.

The metrics we distrust are just as explicit. Patch acceptance rate goes up when developers approve diffs without reading them, and average severity reduction improves if you only change your labelling policy. For any security tool, the first question about a metric is whether the tool itself can move it.

In summary

The design principle behind the loop fits in one sentence: separate the party that judges from the party that fixes, and leave the basis of the judgement reproducible. Splitting red team from blue team, fixing the mutation seed, keeping sealed and mitigated as distinct labels — all of it follows from that.

The hard part of security automation was never writing the fix. It is refusing to grade your own fix, and saying so out loud when the fix is not enough.