Mission-critical AI, Guardrails, Auditby Michael Wybraniec

Anatomy of a Mission-Critical AI System

How we built a public-warning assistant where models propose, policy decides what is allowed, and humans alone authorize the send — contracts, plan_hash, faithfulness, and sealed audit.
Read in:

How we built a public-warning assistant where models propose, policy decides what is allowed, and humans alone authorize the send.

This is the anatomy of control around an LLM for France-style evacuate / FR-Alert channels — not a chatbot with a send button. Demo below is stub/sim (no live cell broadcast). Figures are full width with alt text — click an image for full screen.

Animated preview of the intentEvacuate operator demo — chat, architecture graph, and systems console

Demo walkthrough (~1:55). Preview above is the social cut (2×) — Play demo or Play demo with music (Bensound).

Imagine an operator under time pressure. Flood waters are rising. They type, in French:

Évacuer la zone inondable autour de cette commune maintenant via cell broadcast et app.

If the system is a typical chatbot with tools, the next step is terrifyingly easy: invent a population number, pick a vaguely related polygon, draft a confident message, and “send.” Under stress, a fluent answer feels like competence.

In public warning, fluency without control is a failure mode.

This article is about preparing an AI system for that world—not by chasing a bigger model, but by building the anatomy of control around it. The product is an operator-facing Mission Assistant: natural-language crisis intent in, governed plan out, human confirmation before any irreversible alert, and a reconstructable audit trail afterward.

We did not set out to prove that AI “understands” a crisis. We set out to prove something harder: that you can put language models next to high-blast channels (cell broadcast, app push, location-based SMS in simulation) without ever letting the model press send.

Figure 1 — Full-width operator home: MikeGPT chat, live architecture graph, and systems telemetry during an evacuate mission
Figure 1 — Operator surface is not a naked chatbot.

Figure 1 — The operator surface is not a naked chatbot. Chat (left), live architecture graph (center), and systems telemetry (right) make control visible while a mission runs.

In a fast-onset flood, the authority’s goal is simple: get people out of a dangerous zone, now, through the right channels.

What software usually offers instead is a pile of separate tools—map, population estimate, risk signals, alert composer, channel gateways, procedures. The mission is clear in the operator’s head; the product still demands expert knowledge of which algorithm, parameter, and channel to use. Under stress, people click the wrong zone, use stale density, pick the wrong mix of channels, or skip checks. Afterward, it is hard to prove who decided what, on which evidence.

So the gap is not “we need another dashboard” or “we need a cleverer model.” The gap is intent → safe, audited activation of existing location and alerting capabilities.

Customers buy outcomes under crisis. Competitors can claim “AI.” What régalien (government / safety-critical) buyers actually require is:

  • Speed under stress
  • Traceability
  • Fail-safe behaviour (prefer refuse / confirm over a silent wrong action)
  • Integration with proven engines—not a greenfield rewrite of geo and alerting

That is the industrial bar. A demo chatbot does not clear it.

If this system has a spine, it is this split:

LayerDecides
AIHow to ask, how to structure the mission, how to explain, how to draft
Algorithms + policy (code)What is true enough, what is allowed, what must stop
HumanWhether to act—whether to send

In one sentence we use internally:

AI decides how to ask; algorithms decide what is true; humans decide whether to act.

That sentence is the product. Everything else is plumbing.

What the assistant is (and is not)

It is:

  1. Accept an intent (plus an explicit zone from the UI when possible).
  2. Clarify missing slots (zone, channels, urgency, authority).
  3. Call underlying capabilities (population-in-zone, risk signals, channel eligibility, alert drafting).
  4. Show a preview grounded in tool results (reach, freshness, confidence, risks).
  5. Require human confirmation before any irreversible send.
  6. Write a full audit trail suitable for after-action and régalien review.

It is not:

  • A new flood-prediction research model
  • An unconstrained LLM that “decides” to alert alone
  • Production cell-broadcast on day one of a POC (we simulate and stub dispatch; live networks need sandbox credentials)
  • A free-form agent with a tool belt that includes dispatch_alert without a confirm gate

Models are replaceable proposers. The gate stays.

Figure 2 — Full-width architecture graph with EDGE CORE AI TOOL DATA INFRA lanes and yellow Confirm and Dispatch mutate gates
Figure 2 — Anatomy in the UI: mutate nodes stay gated.

Figure 2 — Anatomy in the UI: EDGE → CORE / AI / TOOL → Confirm and Dispatch (mutate). Faithfulness sits in CORE—claims must match tool evidence before confirm is allowed. The model never owns the yellow nodes alone.

The orchestration loop is intentionally boring. Boring is a feature when the blast radius is a city.

1. Receive intent (+ zone from UI)
2. Clarify → structured mission object until schema-complete
3. Plan → ordered capabilities; validate against a registry allowlist
4. Execute read tools (population, risk, channels, RAG, draft)
5. Policy evaluate → REFUSE | CLARIFY | PREVIEW | CONFIRM
6. Show preview (claims grounded in tool call IDs + SOP citations)
7. On confirm: verify confirmation_id + plan_hash + authority
8. Only then execute mutate: dispatch_alert
9. Append audit chain; return summary

Not one “AI model” — five roles behind one gateway

We do not ship a single mega-agent with every tool and every personality. We ship an LLM gateway that agents call with a role. The UI never imports OpenRouter or Ollama SDKs.

RoleJobWhy it is separate
clarifyAsk for missing slots (zone, channels, …)Cheap, structured; can be a small/static path
planPropose ordered capabilities from the registryMust emit JSON the orchestrator can validate
draftWrite FR public-warning copy per channelTone + length limits; editable in Preview
judgeFaithfulness / draft critique when sampledAdversarial pair to the drafter
coachOperator-facing guidance (Request Agent)Soft coaching; never mutate

They propose structured outputs. Code validates plans, runs tools, evaluates policy, and binds confirmation. Policy and confirm are never delegated to the model.

Figure 3 — Full-width Settings LLM gateway: Ollama or OpenRouter with per-role models clarify plan draft judge coach
Figure 3 — Models are swappable; roles are the architecture.

Figure 3 — Gateway + per-role model IDs. POC often points every role at llama3.2 (Ollama) or gpt-4o-mini (OpenRouter). That is fine—the roles are the architecture; the weights are a dial.

Entire model vs more models?

  • Entire / same weights for all roles — valid for a POC and for on-prem demos: one Ollama tag, five roles, same model_id. Cost and ops stay simple. Safety still comes from the orchestrator, not from model diversity.
  • More models (per role) — useful when you scale: a fast small model for clarify / coach, a stronger model for plan / draft, a separate judge so the drafter does not grade its own homework. Settings already store ollama_models[role] and openrouter_models[role].
  • What we never do — give any role a free tool belt that includes dispatch_alert. Roles propose; the registry + confirm gate execute.

Plan validation rejects the obvious disasters: unknown capabilities, mutate steps before confirm, zone or channel edits after preview that would silently reuse an old confirmation. Edits re-hash the plan; prior confirmation dies.

Every planned action resolves to exactly one outcome, evaluated in a fixed order:

  1. REFUSE — cannot proceed (policy violation, stale evidence, injection, partial failure mid-plan)
  2. CLARIFY — missing or ambiguous slot (especially zone)
  3. PREVIEW — read-only plan shown; numbers must come from tools
  4. CONFIRM — mutate-class action; human confirmation at the required authority
  5. EXECUTE — only with a valid, unexpired confirmation_id bound to the exact plan_hash

Production readiness is not “the model is accurate.” It is whether mutate actions are impossible without that binding.

Policy as data, not scattered ifs

Rules live in versioned packs (for France: fr-default, fr-strict)—freshness windows, channel allowlists, authority escalation. Example shape:

  • Refuse send if evidence is older than the pack allows (e.g. 600s / 300s).
  • Require an explicit polygon or admin code—no silent “Paris” guess.
  • Mutate always requires confirm; confirmation TTL (e.g. 300s); plan hash must match.
  • Low confidence + high-blast channel (cell broadcast) → escalate to supervisor.
  • Hard refuse on injection patterns (“ignore policy”, “skip confirm”, “send now without…”).

Customer or préfecture packs can override defaults later. The engine around the LLM stays deterministic.

Refusals worth showing the public

A wide audience learns more from a refusal than from a successful send:

TriggerOutcomeOperator sees
Stale population / risk evidenceREFUSEWhich tool, how old, invite to refresh
Zone missing or multiple matchesCLARIFYCandidates or “draw the zone” — won’t auto-pick
Low confidence + cell broadcastCONFIRM at supervisorWhy escalated
Prompt injection / out-of-policy askREFUSE + auditNo partial execution
Zone conflict with another active missionCONFIRM / escalateConflict shown; nothing on the wire
Partial tool failure mid-planREFUSE further mutateWhich step failed

Prefer refuse or confirm over a silent wrong action. That is the culture encoded in software.

Figure 4 — Full-width chat clarify for Evacuate south of Paris: ambiguous gazetteer matches and bind-one fixture chips on an inline map
Figure 4 — Best demo: ambiguity forces a human bind.

Figure 4 — Best demo in the product: place name is ambiguous, gazetteer returns fixtures, MikeGPT says bind one—or click the map. No silent polygon invention.

Every number in the preview must come from a tool call. We render freshness and confidence next to estimates. We show a source badge (stub / sim / real)—never hide the ground truth of the backend. The draft message is editable; edits invalidate prior confirmation.

Faithfulness is a first-class CORE check: if a UI claim does not match tool JSON, confirm must not proceed. SOP / doctrine citations from RAG ground draft language—not invented procedure. Conflict with another active mission on the same zone must be visible, not papered over.

This is how you keep the assistant useful without letting it invent a blast radius.

Zone binding is part of that contract. Geometry is never invented from a place name—operators bind an admin fixture or paste GeoJSON; ambiguous entries stay blocked.

Figure 5 — Full-width Map Zone gate with Paris FR-75-12 bound on the satellite map and ambiguous fixtures blocked
Figure 5 — Bind exactly one target before tools run.

Figure 5 — The zone gate: bind exactly one target. Ambiguous codes cannot be auto-picked. Population, risk, and channels stay offline until a zone is bound.

A confirmation object is not a checkbox. It is a cryptographic-ish commitment to a specific plan:

  • confirmation_id
  • mission_id
  • plan_hash (canonical hash of mission steps, channels, zone)
  • actor + required authority
  • expiry
  • which preview evidence was shown

dispatch_alert must verify: id exists, not expired, hash matches current plan, authority is sufficient. Any mismatch → denied. No code path reaches a mutate adapter without that check.

If you only remember one production invariant from this article, remember that one.

Dispatch is not the end of the story. The Summary surface is a sealed after-action record: tabs for map, process diagram, channels & output, and audit. Nothing here sends again.

Process lanes (SIDE / EDGE / CAPABILITY / DATA / MUTATE)

The process diagram is a swimlane over time—not a marketing architecture poster. Continuous SIDE lanes (Mission API, Audit, Policy) run across the mission; CAPABILITY tools fire when needed; MUTATE (Confirm, Dispatch) appears only at the end. That visual is the anatomy of control in one glance.

Figure 6 — Full-width sealed summary process swimlanes from intent through policy tools confirm and dispatch
Figure 6 — Continuous policy/audit; late mutate steps.

Figure 6 — Sealed process diagram. Policy and audit are continuous; Confirm and Dispatch are late mutate steps. Ambiguous places still show up as “I won’t auto-pick” earlier in the chain.

Channels, hashes, and proposal-only follow-ups

After a successful (stub/sim) send, Channels & output shows:

  • Who confirmed (YOU · OPERATOR) and which confirmation_id
  • What System dispatched (channels + delivery ids)
  • Immutable audit count for régalien review
  • Binding IDs: plan_hash, confirmation_id, mission_id, delivery_ids

And critically: optional next steps (monitor remaining population, widen zone) are marked PROPOSAL ONLY—not started until a human confirms a new mission. The system may think ahead; it does not auto-chain sends.

Figure 7 — Full-width Channels and output with plan_hash confirmation id delivery ids and proposal-only follow-ups
Figure 7 — Paper trail + proposal-only next steps.

Figure 7 — Paper trail after dispatch: hashes and delivery ids, plus follow-ups that stay proposals until someone starts a new confirm.

After a crisis, someone will ask: who approved what, on which evidence, when?

Observability for this product is reconstructability. We treat audit as an append-only event chain correlated by mission_id: intent received and parsed, clarifications, plan proposed, every tool call and return, preview shown, policy evaluated, confirm requested/granted/denied, dispatch sent, refuse issued, summary.

For send-class actions we aim at a completeness invariant: you can walk
intent → plan → tool evidence → preview → policy(CONFIRM) → confirm.granted → dispatch.sent
with matching plan_hash. Incomplete chains are not “ops debt”—they are a product defect.

The Summary UI makes that checklist visible: Plan · Tools · Preview · Confirm · Dispatch all checked for a send-class mission—including policy violations that happened mid-flight (e.g. ZONE_CONFLICT) before a supervisor/operator path cleared the send.

Figure 8 — Full-width Audit trail send-class completeness checklist and append-only events including ZONE_CONFLICT
Figure 8 — Reconstruct who decided what, on which evidence.

Figure 8 — Append-only proof log. Confirm and Dispatch are labeled OPERATOR. A red policy.violation can still appear in a completed chain—evidence that friction was real, not skipped.

Measure audit completeness the way you measure uptime.

In mission-critical AI, the dangerous mistake is treating the chat transcript as the system of record. We treat typed contracts as truth and Postgres as the durable proof.

There are three layers of “model” that matter—and only one of them is an LLM:

LayerWhat it isCritical rule
LLM rolesGenerative proposers behind the gatewayNever the source of population numbers or send authority
Runtime contractsJSON Schema / Zod: mission object, capabilities, confirm, audit eventsInvalid shape → clarify / refuse, never dispatch
PersistencePostgres tables + fixture filesAppend-only audit; confirmations bind plan_hash

1. Mission object (the structured intent)

Natural language is parsed into an EvacuateMissionObject before tools run. Locked fields for the POC:

  • action: always evacuate
  • locale: always fr
  • zone: admin_code XOR GeoJSON polygon (never both ambiguous, never neither)
  • channels: non-empty subset of cell_broadcast | app_push | lb_sms
  • urgency: immediate | planned
  • optional authority_hint, time_window, notes

Critical: if the object is incomplete, the loop stays in clarify. The LLM does not get to invent a zone to “be helpful.”

2. Capability contracts (tools with side effects)

Every tool in the registry declares a versioned contract: name, version, side_effect: read | mutate, input/output schemas, typed errors, optional SLA (p95_ms, max_data_age_s).

Capabilityside_effectWhy it exists
population_in_zonereadCAMARA-shaped estimate + evidence (data_age_s, confidence, source)
risk_scorereadRisk signal for the bound zone
channel_eligibilityreadWhich FR-Alert-style channels can fire
rag_retrievereadSOP / doctrine citations for grounding
draft_alertreadChannel drafts (still not a send)
mobility_remainingreadOptional mobility / remaining population
dispatch_alertmutateRequires mission_id, plan_hash, confirmation_id, channels

Critical on read tools: evidence freshness and confidence feed policy. Stale data_age_s → refuse. Low confidence + cell broadcast → supervisor.

Critical on mutate: dispatch_alert input without a valid confirmation is POLICY_DENIED. There is no “almost send.”

3. Plan hash (the binding between preview and send)

plan_hash = sha256(canonical JSON of { mission_id, steps, channels, zone }).

Canonical means sorted keys, stable structure. Any edit to zone, channels, or steps after preview produces a new hash and invalidates the prior confirmation. That is how we stop “confirm plan A, silently dispatch plan B.”

4. Confirmation and dispatch records

Field that must be rightWhy
confirmation_idOpaque grant ticket
plan_hashMust equal current plan at dispatch
actor_role / required_authorityoperator vs supervisor escalation
expires_atTTL (e.g. 300s)—expired grant is dead
statusgranted → consumed on successful dispatch
delivery_idsStub/sim fake ids today; real PWS ids later—always auditable

5. Audit event envelope

Append-only rows: event_id, mission_id, monotonic seq, type, actor, policy_pack, payload.

Critical: payloads must not store secrets or raw injection text in the clear (hash patterns instead). Send-class missions must reconstruct the completeness chain from these rows alone.

6. Postgres entities (durable anatomy)

Mission 1──* Plan
Mission 1──* ToolCall 1──1 ToolResult
Mission 1──* Confirmation 1──0..1 Dispatch
Mission 1──* AuditEvent
SopChunk (RAG corpus) · SimRun · MetricsSample (ops / game-day)
TableHoldsCritical invariant
missionszone, channels, policy_pack, backend, statusZone present; channels allowlisted; locale fr
plansplan_hash, steps JSONSteps only name registry capabilities
tool_calls / tool_resultscapability, side_effect, evidencePreview numbers ⊆ returned tool JSON
confirmationshash + authority + TTLDispatch only if granted, unexpired, hash match
dispatchesconfirmation_id, delivery_idsNo row without a consumed confirmation
audit_eventstyped event streamAppend-only; completeness on send-class
sop_chunksFR SOP text (+ embeddings later)Citations are real ids, not free prose
sim_runs / metrics_samplesGame-day + KPIsSim never silently labeled as real

Population fixtures live primarily as files under adapters (admin_code → CAMARA-shaped payload); the DB mirrors mission life, not the entire geo catalog.

erDiagram
  missions ||--o{ plans : has
  missions ||--o{ tool_calls : has
  missions ||--o{ confirmations : has
  missions ||--o{ dispatches : has
  missions ||--o{ audit_events : has
  tool_calls ||--|| tool_results : returns
  confirmations ||--o| dispatches : grants

  missions {
    text id PK
    text locale
    text zone_admin_code
    jsonb zone_polygon
    jsonb channels
    text policy_pack
    text backend
    text status
  }
  plans {
    uuid id PK
    text mission_id FK
    text plan_hash
    jsonb steps
  }
  tool_calls {
    text tool_call_id PK
    text mission_id FK
    text capability
    text side_effect
    text backend
  }
  tool_results {
    text tool_call_id PK
    jsonb evidence
    text status
    int latency_ms
  }
  confirmations {
    text id PK
    text mission_id FK
    text plan_hash
    text actor_role
    timestamptz expires_at
    text status
  }
  dispatches {
    uuid id PK
    text mission_id FK
    text confirmation_id FK
    jsonb delivery_ids
    jsonb channels
  }
  audit_events {
    text event_id PK
    text mission_id FK
    int seq
    text type
    jsonb payload
  }

Figure 9 — Postgres ER (Mermaid): missions through audit_events. Satellite tables sop_chunks / sim_runs / metrics_samples are listed above. Population fixtures stay in adapter files.

What is not in the data model (on purpose)

  • No “LLM decided to send” flag—only operator/supervisor confirm + dispatch rows
  • No free-form channel strings—enum only
  • No silent backend promotion (stub / sim / real is explicit on every tool call)
  • No multi-country locale packs in the POC mission object

If a field cannot be defended in an after-action review, it does not belong in the contract.

Mission-critical AI in this domain is not a neutral chatbot with a map widget. Geography, locale, channels, and tone are product law:

  • Locale: French intents, drafts, SOPs, golden evals
  • Channels (POC allowlist): cell_broadcast, app_push, lb_sms—never invent channels in the model
  • Tone: préfecture-style / official public-warning voice
  • Policy packs: fr-default / fr-strict with explicit freshness and escalation

Live public-warning system (PWS) send is out of scope for the POC; adapters speak stub and sim, shaped for real contracts (including CAMARA-aligned population-in-zone) when sandboxes exist. That honesty belongs in the article as much as in the architecture: production readiness for AI orchestration starts with contractual truth about what you can and cannot fire.

“Ready” is not a feeling after a happy-path demo. It is a checklist you can fail closed against.

1. Golden paths and adversarial paths

Happy-path orchestration is necessary and insufficient. Safety suites attack injection, ambiguous zones, stale data, and confirm bypass fantasies. Golden evals cover plan shape and policy outcome—not just “the draft sounded good.”

2. Simulation before live networks

Stub and sim backends always exist. Real backends appear only with credentials. Fallback rule we locked early: refuse on stale or unavailable data; no silent backend swap that would make a stub look like live telecom truth. Sim control and fault-inject logs exist so game-days can break things on purpose.

3. Scripted game-days

Before claiming scale, run scripted national-tier load and mission scenarios—not informal “we clicked around.” Scripted personas beat hoping live operators will improvise your test plan.

4. Capability registry and tool contracts

Plans may only name allowlisted capabilities with versions and side-effect classes (read vs mutate). Unknown capability → reject. Mutate before confirm → reject. Tool typed errors feed policy (refuse/clarify), not improvisation.

5. Operator UX that makes the gate tangible

Chat and map are not decoration. Zone must be explicit. Preview must show evidence. Confirm must show authority. Summary must seal the after-action record. Systems/audit surfaces must make the chain inspectable while the mission is live—not only in a log file after the fact.

6. What we still will not claim

  • The model does not “understand” crisis the way a trained operator does.
  • AI does not replace geo engines or invent headcount.
  • A successful simulated dispatch is not a production broadcast.
  • Multi-country export packs and live FR-Alert wiring are product phases, not slideware.

Humility is part of régalien credibility.

Be careful what promises you make. Do not sell “the AI evacuates the city,” “fully autonomous public warning,” or “production FR-Alert” when what you shipped is a governed assistant on stub/sim with a human confirm gate. Promise what you can defend in an audit: refuse over silent wrong action, hash-bound confirm, reconstructable send-class chains, replaceable models behind fixed roles. Over-claiming is not marketing—it is a régalien liability.

  1. Production readiness ≠ model accuracy. It is whether irreversible actions are unreachable without a hash-bound, time-bound, authority-checked confirmation.
  2. The product is the decision ladder, not the chat skin.
  3. Contracts outrank transcripts. Mission object + capability schemas + plan_hash are the system of record.
  4. Policy belongs in data + tests, not in prompt paragraphs the model might ignore.
  5. Your best demo is often a refusal—stale data, ambiguous zone, injection.
  6. Ground every claim shown to a human in tool evidence; treat inventing numbers as a safety bug.
  7. Faithfulness is a product feature, not a nice-to-have validator.
  8. Audit completeness is an SLO. If you cannot reconstruct a send, you were not ready to send.
  9. Follow-ups stay proposals until a new confirm—never silent mission chaining.
  10. Prefer refuse over silent wrong action. Design the culture into the orchestrator.
  11. One model for all roles is OK for a POC; per-role models are an ops dial—roles and gates matter more than weight variety.
  12. Keep models replaceable. Gateway roles, registry, and audit vocabulary outlive any single provider.
  13. Say stub/sim when it is stub/sim. A green “DISPATCHED” in demo is not a live broadcast—label the backend so nobody confuses them.

We titled this Anatomy of a Mission-Critical AI System because the interesting part is not the neural tissue—it is the skeleton: orchestrator, policy engine, confirm gate, faithfulness, tool contracts, sealed summary, and audit chain.

Put a language model next to public warning if you must. Put control between intent and broadcast if you care about the people on the receiving end.

The model proposes.
Policy decides what is allowed.
Humans alone authorize the send.

That is the anatomy. Everything else is implementation detail—and implementation detail is exactly what you write tests for before you call anything “production.”

Built as a France-only public-warning Mission Assistant POC (intentEvacuate): GenAI proposes; policy + confirm gate decide; humans alone authorize the send. Stub/sim backends — not live FR-Alert.

Michael Wybraniec

Michael Wybraniec

System Design, GenAI Automation & Architecture