In one sentence: when an AI (or a person working with an AI) does something that matters, most teams keep a story — chat history, green CI checks, screenshots. What you need instead is proof you can re-check later, offline: fingerprints of the files involved, a cryptographic signature from the system (not the model), and a check that stops the pipeline if anything does not match.
This article explains the problem, the solution pattern, who may do what, where it shows up in practice, and how the verification pipeline works — in technical language, without assuming you know a particular product or repo. Screenshots are from a live demo — click an image for full screen.
Engineering already knows how to ship signed software artefacts: build a binary or image, hash it, sign it, verify the signature in CI before release. That works because you have a stable pile of bytes.
AI-era “consequential moments” are different. Often what you need to trust is not a binary in a registry, but a decision or an answer that claims to be grounded in documents. Today that evidence usually lives as narrative:
- The bot cites sources in the reply — but were those the exact document chunks it used?
- A person exports a spreadsheet; the model drafts a memo — the trail is split across Drive, chat, and email
- Someone types “looks good” in Slack before a refund or a hard-to-undo change
- Staging goes out because CI was green — months later: who allowed that, and on what evidence?
- An auditor asks what the system did last Tuesday — and the answer is “log into the vendor dashboard”
Stories answer: something happened.
They do not answer: can we still prove this claim matches these files and these keys, without trusting the chat UI?
A warning in a log is not a control.
A control fails closed: the job exits non-zero and nothing ships.
Solution pattern: for each consequential run, produce a portable, signed record:
- Write down what the run intended (e.g. summarise this ticket, deploy if checks pass)
- Store fingerprints (cryptographic hashes) of the inputs and outputs that mattered
- Put those fingerprints into one JSON body, normalise that body so every verifier sees the same bytes, and sign it with a system key — the language model never holds that key
- For high-impact actions, require a second signature from a human, locked to that same run — not a casual chat message
- Before merge, deploy, or audit sign-off, re-hash the files and check the signatures. Pass → allow. Fail → stop, with a clear machine-readable reason
flowchart LR
subgraph today["What teams keep today"]
N[Chat · CI ticks · vendor UI · screenshots]
end
subgraph target["What a gate should check"]
A[Hash files into a store] --> B[Normalise + system signs]
B --> C[Optional human co-sign]
C --> V[Verify offline]
V --> G{Pass?}
G -->|yes| OK[Allow]
G -->|no| FC[Stop + named reason]
end
N -.->|replace as gate input| V

Figure 0 — Auditor path: you receive a folder of artefacts, run verify on your machine, and get pass or a named failure. You do not need the vendor’s password.
Most of the security property is role separation. If the model can invent hashes or hold signing keys, the “proof” is meaningless.
flowchart TB
Human -->|ask / export / co-sign| Host
Model[Language model] -->|proposes text · asks for tools| Host
Host -->|hash files · build record · sign| Record[Signed record + file store]
Host --- Keys[(Signing keys)]
Model -.->|no access| Keys
Verifier -->|re-hash + check signatures| Record
Verifier -->|pass or stop + reason| Gate[CI · deploy · audit]
Attacker -.->|tamper · forge · omit| Record
Verifier -->|detect| Attacker
Host (the runtime around the model). This is the only component that should call tools, write files into the hash store, build the record, and sign. Think of it like the CI runner that signs a release — not like the compiler inventing its own checksum.
Language model. It proposes wording and may request tool use. It must not choose the hashes, must not write the human-approval fields, and must never see private keys. Otherwise you are signing whatever the model claims.
Human. May be the subject of the run (analyst, operator) and/or the co-signer when policy says the action is high-impact. Co-sign means a second signed file tied to that run — not “LGTM” in Slack.
Verifier. A program you can run without the chat product. It re-hashes stored files, checks signatures against a small trust list of public keys, and exits successfully only if everything matches. On failure it exits non-zero and prints a stable reason code (so CI can assert on it).
Attacker (design lens). Useful questions while designing: what if someone edits a stored file after hashing? forges the signature? ships with no record? skips human co-sign? drops a fake approval file? Each should be a hard failure — or the control has a hole.
Same pattern everywhere: fingerprint what mattered, sign, verify before you trust the outcome. Only the story changes.
Problem. The reply lists sources. Those strings are easy to edit later. Nothing proves the answer was bound to particular retrieved bytes.
What to do. When the host retrieves document chunks, hash those exact bytes into the store and put the hashes on the signed record. Later, verify re-hashes them. If the file was changed under the same name → digest mismatch. If the file is missing → fail. “Grounded” becomes a check, not a marketing footnote.

Problem. Export in one place, draft in chat, send in email. “A human was involved” is a feeling, not a field on one signed object.
What to do. One record for the session: human as subject, export and draft hashed, host signs. You can still say a person was in the loop — and prove which artefacts that loop produced — without logging into the chat vendor later.

Problem. “Looks good” in chat is not locked to a specific run. It can be copied, spoofed, or left next to a different action than the one that mattered.
What to do. Two keys, like a joint bank account. The system signs the decision record. A human signs a separate approval file that points at that same record. The gate checks both. No approval file → stop. Bad approval signature → stop. Chat remains for conversation; it is not the control.

Problem. “Checks were green” lives inside a CI vendor. It is weak as a long-lived, portable explanation of why deploy was allowed.
What to do. Hash the check inputs/outputs, emit a signed decision record, and make the deploy (or merge) job require a successful offline verify. No record or failed verify → do not ship. Green ticks can still help operators; they stop being the sole gate.

Problem. “Proof” that requires signing into a SaaS console is not portable and ages badly when UIs and retention policies change.
What to do. Hand the auditor the signed record, the hashed file store, and the public keys needed to check signatures. They run verify on their laptop. Pass or named failure. No vendor session. (Figure 0.)
You can implement this with different libraries. The shape should stay: hash → normalise → sign → verify offline → fail closed.
flowchart LR
subgraph host["Host runtime"]
I[Tool / file I/O] --> Store[(Hash store)]
Store --> Body[Normalised JSON body]
Body --> Sig[System signature]
end
Sig --> Rec[Signed record]
Store --> Bundle[Folder you can hand over]
Rec --> Bundle
HumanSig[Human approval file] -.-> Bundle
Bundle --> Ver[Verifier]
Ver --> Q{OK?}
Q -->|yes| Allow[Allow]
Q -->|no| RC[Named failure reason]
On emit (host runtime): every tool result or document chunk that matters is written into a content-addressed store (filename = hash of contents). The record lists those hashes, plus intent and metadata. Before signing, the JSON is put into a canonical form so two verifiers always see the same byte string (random key order or whitespace must not break signatures). The host signs with Ed25519 (or similar) under a key id that appears in a local trust list. The model never performs these steps.
On verify: parse the record; reject unknown schema versions; look up the public key; verify the system signature; for every listed hash, load the file and re-compute the hash; if a human co-sign was required, check that second signature and that it points at this same run. Any failure → non-zero exit and a stable reason string (digest mismatch, bad signature, missing approval, and so on). Integrity faults must not be “warnings.”
tools/files → hash into store → normalise JSON → system signs
→ optional human approval signature
→ verify: re-hash + check sigs → pass | stop + reason
| Need | Building block | Plain reason |
|---|---|---|
| Same signature everywhere | Canonical JSON (e.g. JCS / RFC 8785) | Verifiers must hash/sign identical bytes |
| Bind real files | SHA-256 + content-addressed store | You can re-hash; missing file = failure |
| Prove who attested | Public-key signature (e.g. Ed25519) + trust list | Unknown key = reject |
| High-impact | Second key on a separate approval file | Compromising the system key alone should not invent human OK |
| Bind “which policy” | Hash of the policy file on the record | Stops claiming policy A while checking policy B |
| Detect gaps over time | Hash link to the previous run | Dropped or reordered history becomes visible |
| Automate in CI | CLI exit code + structured reason | Jobs assert numbers/codes, not log poetry |
Honest limit: this proves the record is intact — signatures and hashes still match. It does not prove the host told the truth about the world. If a compromised host hashes the wrong file and signs it, verify can still pass. That is “signed garbage.” Fixing it needs a harder host (sealed tools, stronger key custody) — a separate layer. Saying so is part of a serious design.
Give failures stable names, the way HTTP status classes are stable. Examples:
- Stored file no longer matches its hash
- Hash listed but file missing
- System signature invalid or key unknown
- Policy file does not match the hash on the record
- Human approval required but missing, forged, or bound to another run
- History link to the previous run does not match
verify record.json --store ./cas --trust ./trust --policies ./policies
# exit 0 → proceed
# exit ≠ 0 → stop; print the named reason
| Someone tries… | Gate should… |
|---|---|
| Edit a stored file after the fact | Fail — hash mismatch |
| Ship with no signed record | Fail — nothing to verify |
| Forge the system signature | Fail — bad signature |
| Skip required human approval | Fail — approval missing |
| Drop a fake approval file | Fail — bad approval signature |
| Swap the policy file | Fail — policy hash mismatch |
A demo can draw those attacks for teaching. Production means: a failing test case per reason, and a CI job that goes red when verify fails. Pictures without failing tests are documentation, not a control.
This work sits closer to supply-chain and code-signing engineering than to prompt writing. The mindmap is the map; the subsections are what “good” looks like in practice.
mindmap
root((Attested AI runs))
Evidence design
What to hash
Intent vs files
Enough for later questions
Crypto pipeline
Canonical JSON
Signatures
Trust lists
Isolation
Host vs model
System vs human keys
No key access for the model
Automation
Fail closed
Named reasons in CI
Negative fixtures
Honesty
Intact record ≠ truth
Hardened host next
Say the limit out loud
What it is. Choosing which bytes belong on the signed record so a future reader can answer real questions — not dumping entire disks, not hashing a vague summary of a summary.
Why it matters. If you hash the wrong thing, verify still passes and you have learned nothing. If you hash everything, the store becomes unusable and people bypass the gate. Evidence design is the product judgment layer above crypto.
What “good” looks like.
- Ask: six months later, what will security / audit / incident response actually ask?
- Hash raw tool outputs and retrieved chunks, not only the final chat sentence
- Record intent (what we meant to do) separately from I/O fingerprints (what we touched)
- Prefer fewer, consequential artefacts over a noisy firehose
- When in doubt, hash the bytes that would settle a dispute — not the bytes that look nice in a demo
What it is. The mechanical path from files → hashes → normalised body → signature → verify, with no ambiguity about which bytes were signed.
Why it matters. Most “it verified on my machine” bugs are not exotic crypto breaks. They are unstable serialisation, missing trust entries, or soft-failing checks. A boring, deterministic pipeline is the skill.
What “good” looks like.
- Canonical JSON before sign — same logical document ⇒ same bytes for every verifier
- Content-addressed store — re-hash is the check; missing object is failure, not a skip
- Public-key signatures (e.g. Ed25519) with an explicit trust list of allowed key ids
- Unknown key ⇒ reject; do not “try the next key until something works”
- Document the algorithm and encoding once; freeze them for the schema version
What it is. Enforcing who may compute hashes, who may sign, and who may only propose text — in code and in review, not only in a slide.
Why it matters. If the language model can invent hashes or hold private keys, attestation collapses into “the bot said so, and we signed it.” Isolation is what makes the signature mean the host runtime attested, not the model narrated.
What “good” looks like.
- Host mediates tools, writes the hash store, builds and signs the record
- Model proposes; never chooses digests; never writes approval fields; never sees private keys
- System signing key ≠ human approval key — compromise of one should not mint the other
- Code review treats “model touched the signing path” as a severity-one smell
- Tests that the model path cannot call sign or write the approval artefact
What it is. Making verify a machine contract: exit codes, named reasons, fixtures — so merge/deploy jobs can enforce the control without a human reading logs.
Why it matters. A control that only fails in a demo UI will not survive Friday afternoon. CI only trusts numbers and stable strings. If integrity failures are warnings, they will be ignored.
What “good” looks like.
- Verify exits 0 only on full pass; any integrity fault ⇒ non-zero
- Named reasons stay stable across releases (like HTTP status classes)
- One negative fixture per reason — tamper, missing file, bad sig, missing approval, …
- Merge/deploy jobs call verify and fail closed; no “continue on error”
- Optional JSON report for dashboards — but the exit code remains the gate
What it is. Stating clearly what attestation proves — and what it does not — so leadership and auditors do not confuse an intact signature with ground truth.
Why it matters. Overclaiming (“this proves the AI did the right thing”) destroys trust the first time signed garbage appears. Under-claiming (“it’s just a log”) wastes the control. Precision is a competency.
What “good” looks like.
- Say in docs and reviews: intact record ≠ proven world-truth
- Name signed garbage: wrong bytes hashed on purpose or by bug, then signed → verify still passes
- Treat hardened host (sealed tools, stronger key custody, identity-bound approve) as an explicit next layer — not magic inside the signature
- Prefer demos that show both PASS and catch paths over demos that only celebrate green
- Never market verify-pass as legal certification or “the model was correct”
Together these five competencies are the difference between a slide about “AI governance” and an engineering control you can put on a merge or deploy job.
- Demos reward fluent chat; attestation rewards audit and hard gates
- “Human in the loop” sounds like process without producing a signed artefact
- Vendor dashboards keep “proof” behind login
- Agents started doing consequential work faster than portable evidence became a default merge/deploy check
Hashes and signatures are not new. What is late is treating AI decisions and assisted runs with the same seriousness we already apply to signed software releases — and refusing to treat the chat transcript as the system of record.
Problem: consequential AI runs leave stories.
Solution: fingerprint what was used, system-sign the record, optionally human-co-sign, verify offline, stop on failure with a named reason.
If you cannot re-check why an action was allowed without opening the chat product, you do not have a control. You have a diary.
Stories describe. Attestation re-checks. Controls fail closed.
