00The thesis
An AI model is not a harness. A prompt is not a harness. The model generates candidate actions. The harness decides which of those actions are authorised, executes them safely, records what happened, validates the evidence, and feeds the results back into the next planning cycle. If you conflate the model with the orchestration layer around it, you end up with an unscoped, unauditable system that can hurt the target, the operator, or itself.
This is the architecture for separating the three concerns cleanly: the model (inference, planning, interpretation), the policy (what is and is not permitted), and the execution engine (how permitted actions are carried out, contained, recorded, and validated). The entire design rests on one principle: the model never touches the network, the filesystem, or the target directly. Every action passes through a deterministic policy gate and a sandboxed executor before anything leaves the harness.
01Architecture overview
The harness is organised into five modules, each with a distinct responsibility. They communicate through typed interfaces and share state through a structured memory layer, never through the model's context window alone.
| Module | Responsibility | Core principle |
|---|---|---|
| Inference & routing | Model interaction, multi-provider routing, budget control | The model proposes; it never executes |
| Scope policy | Versioned authorisation, target boundaries, risk gating | Default-deny on every axis |
| Execution safety | Sandboxed execution, egress control, credential isolation | Every action is contained and reversible until proven safe |
| State & memory | Planning, attack graph, persistent memory, checkpoint recovery | State lives outside the context window |
| Evidence & validation | Append-only evidence store, reproducibility, multi-agent verification | Every claim is independently verifiable |
The model is one component. The harness is everything else: the policy, the sandbox, the memory, the evidence chain, and the orchestration that ties them together.
02The control loop
Every cycle of the harness follows the same six-stage loop. The loop is deterministic at the policy and execution layers, even though the planning layer is stochastic. That separation is the point: the model's uncertainty never leaks into what the harness actually does.
03Module 1: Inference and model routing
The inference layer treats models as interchangeable backends behind a unified interface. The harness does not depend on any single provider. Models are registered in a provider registry, each with its transport adapter, authentication, rate limits, and capability profile.
Multi-provider routing
Not every task needs the same model. Reconnaissance planning, vulnerability analysis, exploit generation, and report writing have different requirements for reasoning depth, speed, and cost. The routing layer assigns tasks to models along five axes:
- Capability. Does the model handle the task class? Code generation, natural-language reasoning, structured extraction, and tool-use each map to different model strengths.
- Cost. Budgets are first-class controls. Every inference call has a cost, and the harness tracks spend per engagement, per phase, and per task. When the budget tightens, the router falls back to cheaper models for lower-stakes tasks.
- Latency. Time-sensitive tasks (live exploitation windows, race conditions) route to faster models. Planning tasks that can wait route to deeper reasoners.
- Reliability. The router tracks success rates per model per task class. Models that consistently fail at a task class are deprioritised automatically.
- Compliance. Some engagement scopes restrict which providers can see target data. The router respects these constraints before anything else.
Failover and concurrency
The model pool runs behind a semaphore-controlled concurrency layer. If a provider goes down, rate-limits, or returns degraded results, the harness fails over to the next model in the priority chain. This is deterministic: the fallback order is defined in the engagement configuration, not decided by the model at runtime. Pause and resume controls let the operator freeze inference without losing state.
Design principle
Why orchestration is not function-calling
Function-calling is a model feature: the model emits a structured request, and the runtime executes it. That works for simple tool use. It does not work for offensive security, because the model should never decide what gets executed. The harness decides. The model proposes. The policy evaluates. The executor runs. Function-calling collapses all four roles into one, and the result is a system where a hallucinated tool call can reach the network. The harness architecture keeps them separate by design.
04Module 2: Scope policy
Scope is the hardest problem in automated offensive security. A human tester internalises the rules of engagement and applies judgment continuously. An AI system needs those rules encoded as machine-readable policy that can be evaluated deterministically on every action, every cycle, without exception.
The scope manifest
Every engagement begins with a versioned scope manifest: a structured document that defines exactly what is in scope, what is out, and what conditions must be met before certain actions are permitted. The manifest is signed, timestamped, and immutable once the engagement starts. Changes require a new version with an explicit diff and approval chain.
Default-deny across six dimensions
The policy engine evaluates every proposed action against six independent dimensions. An action must be explicitly permitted on all six to proceed. Failing any one dimension blocks it.
| Dimension | What it gates | Example constraint |
|---|---|---|
| Target | Which hosts, networks, applications, and endpoints may be touched | Only 10.0.0.0/8 and the three named web apps |
| Action | What classes of operation are permitted (scan, exploit, exfiltrate, persist) | No persistence, no data exfiltration beyond proof-of-concept |
| Protocol | Which protocols, ports, and methods may be used | HTTP/HTTPS only; no raw TCP to database ports |
| Identity | Which credentials, accounts, and privilege levels are authorised | Only the three provisioned test accounts |
| Time | When actions may be performed | Business hours only; no weekend testing |
| Risk | What destructive potential is acceptable | No actions above a deterministic risk score of D2 |
SSRF and resolution safety
Cloud metadata endpoints, localhost addresses, internal network ranges, and link-local addresses are blocked at the policy layer, not at the network layer. The policy engine resolves DNS before evaluating the target dimension, which defeats DNS rebinding: if a hostname resolves to an internal address, the action is denied regardless of what the original target string looked like. Redirect chains are followed and re-evaluated at every hop.
Capability tokens
For actions that require temporary elevated authorisation, the harness issues capability tokens: signed, time-limited, revocable grants that authorise a specific action against a specific target for a specific window. The token is consumed on use. It cannot be replayed, extended, or transferred. This is how the harness handles escalation within an engagement without permanently widening the scope.
Architecture note
The two gateways
The policy engine operates as two independent gateways. The pre-execution gateway evaluates the proposed action against the scope manifest before any network traffic leaves the harness. The post-execution gateway evaluates the result: did the action stay within its authorised impact? Did it touch anything outside scope? Did it produce unexpected side effects? Both gates must pass. The model is not involved in either evaluation. Policy is deterministic, not probabilistic.
05Module 3: Execution safety
Once an action passes the policy gate, the execution engine runs it inside a controlled environment. The design principle is simple: every action is assumed to be dangerous until it proves otherwise. The executor provides containment, not trust.
The sandbox
Every task runs in an ephemeral container with no network access by default. Egress is granted per-action by the policy engine, only to the specific targets authorised for that action. The filesystem is temporary and destroyed after execution. Resource limits (CPU, memory, wall-clock time) are enforced at the container level. A task that exceeds its limits is killed, not warned.
Destructive limits
The harness classifies actions on a five-level destructive scale:
- D0: Read-only. Observation, scanning, enumeration. No state change on the target.
- D1: Reversible state change. Authentication attempts, configuration reads that create log entries. Minimal footprint.
- D2: Significant but recoverable. Exploitation that changes target state but can be rolled back. File writes, service restarts.
- D3: Potentially destructive. Actions that may cause data loss or service disruption. Requires explicit human approval before execution.
- D4: Destructive. Denied by default. Only permitted with engagement-level override and real-time human supervision.
The scope manifest sets the maximum permitted destructive level. The policy engine enforces it. The model cannot override it.
Credential isolation
Credentials are stored in a vault that the model cannot read. When a task requires authentication, the executor retrieves the credential from the vault, uses it inside the sandbox, and destroys the local copy after execution. The model sees the result of the authenticated action, never the credential itself. This prevents prompt injection attacks from extracting secrets through the model's context window.
Every action passes through these controls in order. Failing any one stops the action.
- Dry-run mode: simulate the action without network traffic, verify the command is well-formed
- Approval gate: actions above the configured risk threshold require human sign-off
- Circuit breaker: if the error rate for a target or action class exceeds a threshold, the entire class is paused
- Kill switch: the operator can halt all execution instantly, from any interface, at any time
- Fail-closed: if any component of the harness fails, all actions are denied until the component recovers
06Module 4: State, memory, and the attack graph
The model's context window is finite. An engagement is not. The state module solves this by maintaining structured, persistent memory outside the context window, recalling only what is relevant to the current task, and discarding nothing.
Hierarchical planning
The harness decomposes work into three levels: objectives (engagement-level goals like "achieve domain admin"), phases (reconnaissance, initial access, lateral movement, privilege escalation), and tasks (individual actions within a phase). The planner operates at all three levels, but the model only sees the task level at any given moment. The harness maintains the higher-level context and feeds the model the relevant slice.
The scheduler
Task ordering is not left to the model. The scheduler prioritises tasks by a weighted combination of risk (lower risk first), coverage (untested areas first), and entropy (areas of highest uncertainty first). It also enforces loop prevention: if the harness has attempted the same approach three times without progress, the scheduler forces a different strategy or escalates to human review.
Four-tier memory
The memory system operates in four layers, each with different retention and recall characteristics:
- Working memory. The current task context. Fits in the model's context window. Refreshed every cycle.
- Short-term memory. Recent observations and results from the current phase. Summarised and compressed as the phase progresses.
- Long-term memory. Engagement-level facts: discovered hosts, credentials, network topology, vulnerability findings. Persists for the entire engagement.
- Failure memory. A dedicated store of approaches that were tried and failed, with the reason for failure. This prevents the harness from repeating unsuccessful strategies, a failure mode that plagues naive agent loops.
The attack graph
Every discovered fact, every successful action, and every relationship between them is recorded in a typed, directed graph. Nodes represent assets (hosts, services, credentials, vulnerabilities). Edges represent relationships (runs-on, authenticates-to, exploitable-via). Every node and edge carries provenance metadata: when it was discovered, by which tool, with what confidence, and what evidence supports it.
Graph operations
Attack trees and path prioritisation
The attack graph supports two critical operations. Attack trees enumerate all known paths from the current position to a target objective, scored by feasibility, risk, and evidence strength. Path prioritisation ranks these paths using a Bayesian confidence model: paths with more validated steps are scored higher than paths with untested assumptions. The model uses these ranked paths to propose its next action, but the graph, not the model, maintains the ground truth.
Evidence-state ladder
Every node in the graph has an evidence state: hypothesised (the model believes it exists), observed (a tool returned data suggesting it), confirmed (independently validated), or exploited (successfully used in an attack path). The harness refuses to build attack paths through hypothesised nodes. Only confirmed or exploited nodes carry weight in path scoring.
07Module 5: Evidence, validation, and orchestration
A finding without reproducible evidence is a claim. The evidence module ensures that every finding the harness reports can be independently verified, reproduced, and traced back to the exact request, response, and tool output that produced it.
The append-only evidence store
Every action the harness executes is recorded in full: the proposed action, the policy evaluation, the execution environment, the raw request, the raw response, any artefacts produced, and the validation result. Records are append-only and cryptographically chained. Nothing is deleted or overwritten. The evidence store is the engagement's audit trail and the source of truth for all reporting.
Reproducibility and validation
For every confirmed vulnerability, the harness records three types of test:
- Positive test. The exploit succeeds: the vulnerability is confirmed.
- Negative test. The exploit is run against a patched or non-vulnerable variant to confirm it fails. This proves the finding is specific, not a false positive caused by a permissive response.
- Control test. A benign version of the same request is sent to establish the baseline behaviour. This distinguishes the vulnerability from normal application behaviour.
Independent verification
Findings are not validated by the same agent that discovered them. The harness runs a separate verifier agent, with its own model instance and its own context, that receives only the finding claim and the evidence, then attempts to reproduce it independently. The verifier has no access to the discoverer's reasoning or intermediate steps. This prevents confirmation bias: the verifier is trying to break the finding, not support it.
Orchestration
The multi-agent roster
The harness supports multiple specialised agents, each with a defined role, a scoped set of tools, and its own policy constraints. Agents communicate through a structured blackboard, not through shared context windows. The roster includes:
- Reconnaissance agent. Discovery and enumeration only. No exploitation capability.
- Exploitation agent. Authorised to attempt exploitation within scope. Cannot persist or exfiltrate.
- Validation agent. Reproduces findings independently. Cannot discover new vulnerabilities.
- Reporting agent. Consumes validated evidence and produces deliverables. No target access.
Each agent sees only the state relevant to its role. The reconnaissance agent cannot see exploitation results. The reporting agent cannot see raw credentials. Role isolation is enforced at the harness level, not by the model.
08Operational controls
The harness itself is a security-critical system. It needs the same rigour applied to its own operations that it applies to the targets it tests.
Telemetry and observability
Every component of the harness emits structured telemetry: inference calls, policy evaluations, execution events, memory operations, and evidence commits. The telemetry pipeline is separate from the evidence store. It feeds dashboards and alerting, not findings. An operator can see, at any moment, what the harness is doing, what it is planning, and what it has spent.
Versioning and provenance
The harness, the policy manifests, the tool definitions, and the agent configurations are all versioned and signed. Every engagement records the exact versions of every component used. This supports audit, reproducibility, and rollback. If a finding is disputed six months later, the exact environment that produced it can be reconstructed.
Privacy and data handling
A privacy gateway sits between the model and any data that flows through the harness. It enforces data-loss prevention rules, strips personally identifiable information from model context when the engagement scope requires it, and ensures that target data is never sent to a model provider whose data-handling terms conflict with the engagement agreement. Tenant isolation guarantees that data from one engagement never contaminates another.
Continuous evaluation
The harness includes a separate evaluation framework that measures its own performance: coverage (what percentage of the attack surface was tested), accuracy (what percentage of findings were confirmed on manual review), efficiency (cost and time per validated finding), and safety (how many out-of-scope actions were attempted and caught). These metrics are tracked per engagement and across engagements, and they feed back into the harness's own development cycle.
09Why this architecture
| Without the harness | With the harness |
|---|---|
| The model decides what to execute | The model proposes; deterministic policy decides |
| Scope is a prompt instruction the model may ignore | Scope is a machine-readable manifest enforced on every action |
| Credentials pass through the model's context | Credentials stay in a vault the model cannot read |
| Actions run in the same environment as the harness | Every action runs in an ephemeral, network-isolated sandbox |
| Memory is the context window; it fills up and forgets | Structured memory persists outside the context window with selective recall |
| Findings are whatever the model says they are | Findings are independently reproduced by a separate verifier agent |
| The system has no audit trail | Every action is append-only recorded with full provenance |
| Failure loops: the model retries the same approach endlessly | Failure memory and loop prevention force strategy changes |
| One agent with all permissions | Role-isolated agents on a structured blackboard with scoped access |
The harness is the difference between an AI system that is useful for offensive security and one that is safe for it. The model provides intelligence. The harness provides control. Neither works without the other, and collapsing them into a single layer is the fastest way to build something that nobody should trust with access to a production network.