How we built a public-warning assistant where models propose, policy decides what is allowed, and humans alone authorize the send.
This is the anatomy of control around an LLM for France-style evacuate / FR-Alert channels — not a chatbot with a send button. Demo below is stub/sim (no live cell broadcast). Figures are full width with alt text — click an image for full screen.

Demo walkthrough (~1:55). Preview above is the social cut (2×) — Play demo or Play demo with music (Bensound).
Imagine an operator under time pressure. Flood waters are rising. They type, in French:
Évacuer la zone inondable autour de cette commune maintenant via cell broadcast et app.
If the system is a typical chatbot with tools, the next step is terrifyingly easy: invent a population number, pick a vaguely related polygon, draft a confident message, and “send.” Under stress, a fluent answer feels like competence.
In public warning, fluency without control is a failure mode.
This article is about preparing an AI system for that world—not by chasing a bigger model, but by building the anatomy of control around it. The product is an operator-facing Mission Assistant: natural-language crisis intent in, governed plan out, human confirmation before any irreversible alert, and a reconstructable audit trail afterward.
We did not set out to prove that AI “understands” a crisis. We set out to prove something harder: that you can put language models next to high-blast channels (cell broadcast, app push, location-based SMS in simulation) without ever letting the model press send.

Figure 1 — The operator surface is not a naked chatbot. Chat (left), live architecture graph (center), and systems telemetry (right) make control visible while a mission runs.
In a fast-onset flood, the authority’s goal is simple: get people out of a dangerous zone, now, through the right channels.
What software usually offers instead is a pile of separate tools—map, population estimate, risk signals, alert composer, channel gateways, procedures. The mission is clear in the operator’s head; the product still demands expert knowledge of which algorithm, parameter, and channel to use. Under stress, people click the wrong zone, use stale density, pick the wrong mix of channels, or skip checks. Afterward, it is hard to prove who decided what, on which evidence.
So the gap is not “we need another dashboard” or “we need a cleverer model.” The gap is intent → safe, audited activation of existing location and alerting capabilities.
Customers buy outcomes under crisis. Competitors can claim “AI.” What régalien (government / safety-critical) buyers actually require is:
- Speed under stress
- Traceability
- Fail-safe behaviour (prefer refuse / confirm over a silent wrong action)
- Integration with proven engines—not a greenfield rewrite of geo and alerting
That is the industrial bar. A demo chatbot does not clear it.
If this system has a spine, it is this split:
| Layer | Decides |
|---|---|
| AI | How to ask, how to structure the mission, how to explain, how to draft |
| Algorithms + policy (code) | What is true enough, what is allowed, what must stop |
| Human | Whether to act—whether to send |
In one sentence we use internally:
AI decides how to ask; algorithms decide what is true; humans decide whether to act.
That sentence is the product. Everything else is plumbing.
What the assistant is (and is not)
It is:
- Accept an intent (plus an explicit zone from the UI when possible).
- Clarify missing slots (zone, channels, urgency, authority).
- Call underlying capabilities (population-in-zone, risk signals, channel eligibility, alert drafting).
- Show a preview grounded in tool results (reach, freshness, confidence, risks).
- Require human confirmation before any irreversible send.
- Write a full audit trail suitable for after-action and régalien review.
It is not:
- A new flood-prediction research model
- An unconstrained LLM that “decides” to alert alone
- Production cell-broadcast on day one of a POC (we simulate and stub dispatch; live networks need sandbox credentials)
- A free-form agent with a tool belt that includes
dispatch_alertwithout a confirm gate
Models are replaceable proposers. The gate stays.

Figure 2 — Anatomy in the UI: EDGE → CORE / AI / TOOL → Confirm and Dispatch (mutate). Faithfulness sits in CORE—claims must match tool evidence before confirm is allowed. The model never owns the yellow nodes alone.
The orchestration loop is intentionally boring. Boring is a feature when the blast radius is a city.
1. Receive intent (+ zone from UI)
2. Clarify → structured mission object until schema-complete
3. Plan → ordered capabilities; validate against a registry allowlist
4. Execute read tools (population, risk, channels, RAG, draft)
5. Policy evaluate → REFUSE | CLARIFY | PREVIEW | CONFIRM
6. Show preview (claims grounded in tool call IDs + SOP citations)
7. On confirm: verify confirmation_id + plan_hash + authority
8. Only then execute mutate: dispatch_alert
9. Append audit chain; return summary
Not one “AI model” — five roles behind one gateway
We do not ship a single mega-agent with every tool and every personality. We ship an LLM gateway that agents call with a role. The UI never imports OpenRouter or Ollama SDKs.
| Role | Job | Why it is separate |
|---|---|---|
clarify | Ask for missing slots (zone, channels, …) | Cheap, structured; can be a small/static path |
plan | Propose ordered capabilities from the registry | Must emit JSON the orchestrator can validate |
draft | Write FR public-warning copy per channel | Tone + length limits; editable in Preview |
judge | Faithfulness / draft critique when sampled | Adversarial pair to the drafter |
coach | Operator-facing guidance (Request Agent) | Soft coaching; never mutate |
They propose structured outputs. Code validates plans, runs tools, evaluates policy, and binds confirmation. Policy and confirm are never delegated to the model.

Figure 3 — Gateway + per-role model IDs. POC often points every role at llama3.2 (Ollama) or gpt-4o-mini (OpenRouter). That is fine—the roles are the architecture; the weights are a dial.
Entire model vs more models?
- Entire / same weights for all roles — valid for a POC and for on-prem demos: one Ollama tag, five roles, same
model_id. Cost and ops stay simple. Safety still comes from the orchestrator, not from model diversity. - More models (per role) — useful when you scale: a fast small model for
clarify/coach, a stronger model forplan/draft, a separatejudgeso the drafter does not grade its own homework. Settings already storeollama_models[role]andopenrouter_models[role]. - What we never do — give any role a free tool belt that includes
dispatch_alert. Roles propose; the registry + confirm gate execute.
Plan validation rejects the obvious disasters: unknown capabilities, mutate steps before confirm, zone or channel edits after preview that would silently reuse an old confirmation. Edits re-hash the plan; prior confirmation dies.
Every planned action resolves to exactly one outcome, evaluated in a fixed order:
- REFUSE — cannot proceed (policy violation, stale evidence, injection, partial failure mid-plan)
- CLARIFY — missing or ambiguous slot (especially zone)
- PREVIEW — read-only plan shown; numbers must come from tools
- CONFIRM — mutate-class action; human confirmation at the required authority
- EXECUTE — only with a valid, unexpired
confirmation_idbound to the exactplan_hash
Production readiness is not “the model is accurate.” It is whether mutate actions are impossible without that binding.
Policy as data, not scattered ifs
Rules live in versioned packs (for France: fr-default, fr-strict)—freshness windows, channel allowlists, authority escalation. Example shape:
- Refuse send if evidence is older than the pack allows (e.g. 600s / 300s).
- Require an explicit polygon or admin code—no silent “Paris” guess.
- Mutate always requires confirm; confirmation TTL (e.g. 300s); plan hash must match.
- Low confidence + high-blast channel (cell broadcast) → escalate to supervisor.
- Hard refuse on injection patterns (“ignore policy”, “skip confirm”, “send now without…”).
Customer or préfecture packs can override defaults later. The engine around the LLM stays deterministic.
Refusals worth showing the public
A wide audience learns more from a refusal than from a successful send:
| Trigger | Outcome | Operator sees |
|---|---|---|
| Stale population / risk evidence | REFUSE | Which tool, how old, invite to refresh |
| Zone missing or multiple matches | CLARIFY | Candidates or “draw the zone” — won’t auto-pick |
| Low confidence + cell broadcast | CONFIRM at supervisor | Why escalated |
| Prompt injection / out-of-policy ask | REFUSE + audit | No partial execution |
| Zone conflict with another active mission | CONFIRM / escalate | Conflict shown; nothing on the wire |
| Partial tool failure mid-plan | REFUSE further mutate | Which step failed |
Prefer refuse or confirm over a silent wrong action. That is the culture encoded in software.

Figure 4 — Best demo in the product: place name is ambiguous, gazetteer returns fixtures, MikeGPT says bind one—or click the map. No silent polygon invention.
Every number in the preview must come from a tool call. We render freshness and confidence next to estimates. We show a source badge (stub / sim / real)—never hide the ground truth of the backend. The draft message is editable; edits invalidate prior confirmation.
Faithfulness is a first-class CORE check: if a UI claim does not match tool JSON, confirm must not proceed. SOP / doctrine citations from RAG ground draft language—not invented procedure. Conflict with another active mission on the same zone must be visible, not papered over.
This is how you keep the assistant useful without letting it invent a blast radius.
Zone binding is part of that contract. Geometry is never invented from a place name—operators bind an admin fixture or paste GeoJSON; ambiguous entries stay blocked.

Figure 5 — The zone gate: bind exactly one target. Ambiguous codes cannot be auto-picked. Population, risk, and channels stay offline until a zone is bound.
A confirmation object is not a checkbox. It is a cryptographic-ish commitment to a specific plan:
confirmation_idmission_idplan_hash(canonical hash of mission steps, channels, zone)- actor + required authority
- expiry
- which preview evidence was shown
dispatch_alert must verify: id exists, not expired, hash matches current plan, authority is sufficient. Any mismatch → denied. No code path reaches a mutate adapter without that check.
If you only remember one production invariant from this article, remember that one.
Dispatch is not the end of the story. The Summary surface is a sealed after-action record: tabs for map, process diagram, channels & output, and audit. Nothing here sends again.
Process lanes (SIDE / EDGE / CAPABILITY / DATA / MUTATE)
The process diagram is a swimlane over time—not a marketing architecture poster. Continuous SIDE lanes (Mission API, Audit, Policy) run across the mission; CAPABILITY tools fire when needed; MUTATE (Confirm, Dispatch) appears only at the end. That visual is the anatomy of control in one glance.

Figure 6 — Sealed process diagram. Policy and audit are continuous; Confirm and Dispatch are late mutate steps. Ambiguous places still show up as “I won’t auto-pick” earlier in the chain.
Channels, hashes, and proposal-only follow-ups
After a successful (stub/sim) send, Channels & output shows:
- Who confirmed (
YOU · OPERATOR) and whichconfirmation_id - What System dispatched (channels + delivery ids)
- Immutable audit count for régalien review
- Binding IDs:
plan_hash,confirmation_id,mission_id,delivery_ids
And critically: optional next steps (monitor remaining population, widen zone) are marked PROPOSAL ONLY—not started until a human confirms a new mission. The system may think ahead; it does not auto-chain sends.

Figure 7 — Paper trail after dispatch: hashes and delivery ids, plus follow-ups that stay proposals until someone starts a new confirm.
After a crisis, someone will ask: who approved what, on which evidence, when?
Observability for this product is reconstructability. We treat audit as an append-only event chain correlated by mission_id: intent received and parsed, clarifications, plan proposed, every tool call and return, preview shown, policy evaluated, confirm requested/granted/denied, dispatch sent, refuse issued, summary.
For send-class actions we aim at a completeness invariant: you can walkintent → plan → tool evidence → preview → policy(CONFIRM) → confirm.granted → dispatch.sent
with matching plan_hash. Incomplete chains are not “ops debt”—they are a product defect.
The Summary UI makes that checklist visible: Plan · Tools · Preview · Confirm · Dispatch all checked for a send-class mission—including policy violations that happened mid-flight (e.g. ZONE_CONFLICT) before a supervisor/operator path cleared the send.

Figure 8 — Append-only proof log. Confirm and Dispatch are labeled OPERATOR. A red policy.violation can still appear in a completed chain—evidence that friction was real, not skipped.
Measure audit completeness the way you measure uptime.
In mission-critical AI, the dangerous mistake is treating the chat transcript as the system of record. We treat typed contracts as truth and Postgres as the durable proof.
There are three layers of “model” that matter—and only one of them is an LLM:
| Layer | What it is | Critical rule |
|---|---|---|
| LLM roles | Generative proposers behind the gateway | Never the source of population numbers or send authority |
| Runtime contracts | JSON Schema / Zod: mission object, capabilities, confirm, audit events | Invalid shape → clarify / refuse, never dispatch |
| Persistence | Postgres tables + fixture files | Append-only audit; confirmations bind plan_hash |
1. Mission object (the structured intent)
Natural language is parsed into an EvacuateMissionObject before tools run. Locked fields for the POC:
action: alwaysevacuatelocale: alwaysfrzone:admin_codeXOR GeoJSONpolygon(never both ambiguous, never neither)channels: non-empty subset ofcell_broadcast|app_push|lb_smsurgency:immediate|planned- optional
authority_hint,time_window,notes
Critical: if the object is incomplete, the loop stays in clarify. The LLM does not get to invent a zone to “be helpful.”
2. Capability contracts (tools with side effects)
Every tool in the registry declares a versioned contract: name, version, side_effect: read | mutate, input/output schemas, typed errors, optional SLA (p95_ms, max_data_age_s).
| Capability | side_effect | Why it exists |
|---|---|---|
population_in_zone | read | CAMARA-shaped estimate + evidence (data_age_s, confidence, source) |
risk_score | read | Risk signal for the bound zone |
channel_eligibility | read | Which FR-Alert-style channels can fire |
rag_retrieve | read | SOP / doctrine citations for grounding |
draft_alert | read | Channel drafts (still not a send) |
mobility_remaining | read | Optional mobility / remaining population |
dispatch_alert | mutate | Requires mission_id, plan_hash, confirmation_id, channels |
Critical on read tools: evidence freshness and confidence feed policy. Stale data_age_s → refuse. Low confidence + cell broadcast → supervisor.
Critical on mutate: dispatch_alert input without a valid confirmation is POLICY_DENIED. There is no “almost send.”
3. Plan hash (the binding between preview and send)
plan_hash = sha256(canonical JSON of { mission_id, steps, channels, zone }).
Canonical means sorted keys, stable structure. Any edit to zone, channels, or steps after preview produces a new hash and invalidates the prior confirmation. That is how we stop “confirm plan A, silently dispatch plan B.”
4. Confirmation and dispatch records
| Field that must be right | Why |
|---|---|
confirmation_id | Opaque grant ticket |
plan_hash | Must equal current plan at dispatch |
actor_role / required_authority | operator vs supervisor escalation |
expires_at | TTL (e.g. 300s)—expired grant is dead |
status | granted → consumed on successful dispatch |
delivery_ids | Stub/sim fake ids today; real PWS ids later—always auditable |
5. Audit event envelope
Append-only rows: event_id, mission_id, monotonic seq, type, actor, policy_pack, payload.
Critical: payloads must not store secrets or raw injection text in the clear (hash patterns instead). Send-class missions must reconstruct the completeness chain from these rows alone.
6. Postgres entities (durable anatomy)
Mission 1──* Plan
Mission 1──* ToolCall 1──1 ToolResult
Mission 1──* Confirmation 1──0..1 Dispatch
Mission 1──* AuditEvent
SopChunk (RAG corpus) · SimRun · MetricsSample (ops / game-day)
| Table | Holds | Critical invariant |
|---|---|---|
missions | zone, channels, policy_pack, backend, status | Zone present; channels allowlisted; locale fr |
plans | plan_hash, steps JSON | Steps only name registry capabilities |
tool_calls / tool_results | capability, side_effect, evidence | Preview numbers ⊆ returned tool JSON |
confirmations | hash + authority + TTL | Dispatch only if granted, unexpired, hash match |
dispatches | confirmation_id, delivery_ids | No row without a consumed confirmation |
audit_events | typed event stream | Append-only; completeness on send-class |
sop_chunks | FR SOP text (+ embeddings later) | Citations are real ids, not free prose |
sim_runs / metrics_samples | Game-day + KPIs | Sim never silently labeled as real |
Population fixtures live primarily as files under adapters (admin_code → CAMARA-shaped payload); the DB mirrors mission life, not the entire geo catalog.
erDiagram
missions ||--o{ plans : has
missions ||--o{ tool_calls : has
missions ||--o{ confirmations : has
missions ||--o{ dispatches : has
missions ||--o{ audit_events : has
tool_calls ||--|| tool_results : returns
confirmations ||--o| dispatches : grants
missions {
text id PK
text locale
text zone_admin_code
jsonb zone_polygon
jsonb channels
text policy_pack
text backend
text status
}
plans {
uuid id PK
text mission_id FK
text plan_hash
jsonb steps
}
tool_calls {
text tool_call_id PK
text mission_id FK
text capability
text side_effect
text backend
}
tool_results {
text tool_call_id PK
jsonb evidence
text status
int latency_ms
}
confirmations {
text id PK
text mission_id FK
text plan_hash
text actor_role
timestamptz expires_at
text status
}
dispatches {
uuid id PK
text mission_id FK
text confirmation_id FK
jsonb delivery_ids
jsonb channels
}
audit_events {
text event_id PK
text mission_id FK
int seq
text type
jsonb payload
}
Figure 9 — Postgres ER (Mermaid): missions through audit_events. Satellite tables sop_chunks / sim_runs / metrics_samples are listed above. Population fixtures stay in adapter files.
What is not in the data model (on purpose)
- No “LLM decided to send” flag—only operator/supervisor confirm + dispatch rows
- No free-form channel strings—enum only
- No silent backend promotion (
stub/sim/realis explicit on every tool call) - No multi-country locale packs in the POC mission object
If a field cannot be defended in an after-action review, it does not belong in the contract.
Mission-critical AI in this domain is not a neutral chatbot with a map widget. Geography, locale, channels, and tone are product law:
- Locale: French intents, drafts, SOPs, golden evals
- Channels (POC allowlist):
cell_broadcast,app_push,lb_sms—never invent channels in the model - Tone: préfecture-style / official public-warning voice
- Policy packs:
fr-default/fr-strictwith explicit freshness and escalation
Live public-warning system (PWS) send is out of scope for the POC; adapters speak stub and sim, shaped for real contracts (including CAMARA-aligned population-in-zone) when sandboxes exist. That honesty belongs in the article as much as in the architecture: production readiness for AI orchestration starts with contractual truth about what you can and cannot fire.
“Ready” is not a feeling after a happy-path demo. It is a checklist you can fail closed against.
1. Golden paths and adversarial paths
Happy-path orchestration is necessary and insufficient. Safety suites attack injection, ambiguous zones, stale data, and confirm bypass fantasies. Golden evals cover plan shape and policy outcome—not just “the draft sounded good.”
2. Simulation before live networks
Stub and sim backends always exist. Real backends appear only with credentials. Fallback rule we locked early: refuse on stale or unavailable data; no silent backend swap that would make a stub look like live telecom truth. Sim control and fault-inject logs exist so game-days can break things on purpose.
3. Scripted game-days
Before claiming scale, run scripted national-tier load and mission scenarios—not informal “we clicked around.” Scripted personas beat hoping live operators will improvise your test plan.
4. Capability registry and tool contracts
Plans may only name allowlisted capabilities with versions and side-effect classes (read vs mutate). Unknown capability → reject. Mutate before confirm → reject. Tool typed errors feed policy (refuse/clarify), not improvisation.
5. Operator UX that makes the gate tangible
Chat and map are not decoration. Zone must be explicit. Preview must show evidence. Confirm must show authority. Summary must seal the after-action record. Systems/audit surfaces must make the chain inspectable while the mission is live—not only in a log file after the fact.
6. What we still will not claim
- The model does not “understand” crisis the way a trained operator does.
- AI does not replace geo engines or invent headcount.
- A successful simulated dispatch is not a production broadcast.
- Multi-country export packs and live FR-Alert wiring are product phases, not slideware.
Humility is part of régalien credibility.
Be careful what promises you make. Do not sell “the AI evacuates the city,” “fully autonomous public warning,” or “production FR-Alert” when what you shipped is a governed assistant on stub/sim with a human confirm gate. Promise what you can defend in an audit: refuse over silent wrong action, hash-bound confirm, reconstructable send-class chains, replaceable models behind fixed roles. Over-claiming is not marketing—it is a régalien liability.
- Production readiness ≠ model accuracy. It is whether irreversible actions are unreachable without a hash-bound, time-bound, authority-checked confirmation.
- The product is the decision ladder, not the chat skin.
- Contracts outrank transcripts. Mission object + capability schemas +
plan_hashare the system of record. - Policy belongs in data + tests, not in prompt paragraphs the model might ignore.
- Your best demo is often a refusal—stale data, ambiguous zone, injection.
- Ground every claim shown to a human in tool evidence; treat inventing numbers as a safety bug.
- Faithfulness is a product feature, not a nice-to-have validator.
- Audit completeness is an SLO. If you cannot reconstruct a send, you were not ready to send.
- Follow-ups stay proposals until a new confirm—never silent mission chaining.
- Prefer refuse over silent wrong action. Design the culture into the orchestrator.
- One model for all roles is OK for a POC; per-role models are an ops dial—roles and gates matter more than weight variety.
- Keep models replaceable. Gateway roles, registry, and audit vocabulary outlive any single provider.
- Say stub/sim when it is stub/sim. A green “DISPATCHED” in demo is not a live broadcast—label the backend so nobody confuses them.
We titled this Anatomy of a Mission-Critical AI System because the interesting part is not the neural tissue—it is the skeleton: orchestrator, policy engine, confirm gate, faithfulness, tool contracts, sealed summary, and audit chain.
Put a language model next to public warning if you must. Put control between intent and broadcast if you care about the people on the receiving end.
The model proposes.
Policy decides what is allowed.
Humans alone authorize the send.
That is the anatomy. Everything else is implementation detail—and implementation detail is exactly what you write tests for before you call anything “production.”
Built as a France-only public-warning Mission Assistant POC (intentEvacuate): GenAI proposes; policy + confirm gate decide; humans alone authorize the send. Stub/sim backends — not live FR-Alert.
