RAG, Rust, AWP, On-prem, Speckitby Michael Wybraniec

rag-factory: a Rust engine for recipe-driven RAG — cite, refuse, eval before ship

Full arc: RAG types analysed, on-premises-ready Rust engine, Layer A/B architecture, Speckit + AWP with parallel agents orchestrated under human commits — cite, refuse, eval before ship.

Most companies don’t need a louder chatbot.

They need retrieval that is predictable, reviewable, and easy to plug into the way they already ship software.

rag-factory is my answer to that: a Rust CLI and library that runs RAG pipelines from a config “recipe.” Build an index. Ask a question. Get a structured answer with citations — or a clear signal when the evidence isn’t strong enough to answer. Evaluate before you call it done.

Around the code sits a second product: how humans and AI assistants deliver features without losing ownership. Speckit for what, AWP for when, skills for how. This piece walks the full arc — thesis, RAG types analysed, product principles, Rust, architecture, delivery protocol with parallel agents, competencies, future, and close — without turning into a public clone kit. Figures are full width — click an image for full screen.

rag-factory brand — A recipe factory for grounded RAG — retrieve from your index, cite what you use, or refuse when evidence isn’t there
Figure 0 — The product promise in one line: cite or refuse.

Figure 0 — The product promise in one line: retrieve from your index, cite what you use, or refuse when evidence isn’t there.

Most RAG work stalls for the same three reasons.

Glue, not product — ingest, chunk, embed, retrieve, prompt, and eval get reinvented per project.

One stack for every domain — docs Q&A, code Q&A, and policy review need different defaults. A single “universal” chat experience usually fits none of them well.

Vibes instead of gates — if the system cannot decline weak evidence, and CI cannot fail when quality drops, you mostly shipped a demo skin.

The bet is the opposite of a magic answer bar:

A universal factory (scaffold + knobs + measure) is realistic.
A universal answer-quality bar is not.

The product is a Rust CLI + library that runs config-driven RAG pipelines from a recipe: build an index, answer with citations, refuse when unsure, and eval before you ship. Domain packs (docs-qa, code-rag, sop-audit) change knobs — not the engine.

flowchart LR
  subgraph inputs
    R[Recipe YAML]
    P[Pack overlay]
    C[Corpus]
    G[Golden JSONL]
  end
  subgraph engine["rag-factory-core"]
    B[build]
    Q[query]
    E[eval]
    IDX[(Index)]
  end
  R --> P
  P --> B
  C --> B
  B --> IDX
  IDX --> Q
  Q --> A[Answer JSON<br/>cite / refuse]
  G --> E
  IDX --> E
  E --> GATE{CI gate}
Recipe + Pack  →  ingest → chunk → embed → store
               →  retrieve → context → LLM → cite / decline
               →  eval on golden questions (CI gate)

What success looks like

  • A new pilot for a known pack family in hours, not weeks
  • Swap store / LLM via config without rewriting the app
  • Same Answer JSON from library, CLI, and HTTP — one engine path
  • Cite or refuse; injection / refuse goldens stay green in CI

The factory did not start from a single blog-post architecture. It started from comparing the RAG patterns teams actually ask for — and writing down where their trust bars diverge.

Domain packs (shipped as overlays):

  • Docs Q&A (docs-qa) — product/eng docs, wikis, support knowledge bases
  • Code RAG (code-rag) — repositories, paths, symbols, engineering questions
  • SOP / policy audit (sop-audit) — procedures where declining weak evidence matters as much as answering

Pipeline patterns analysed and folded into the roadmap (behind calm defaults and flags):

  • Dense retrieve — the default, teachable path
  • Hybrid retrieve + rerank — when lexical misses matter
  • Conversational / streaming — multi-turn with session memory
  • Multimodal — PDF pages, figures, vision-adjacent corpora
  • Multi-hop / bounded agentic retrieve — decompose and iterate without unbounded autonomy
  • Structured + docs hybrid — warehouse/SQL grounding beside text

The point of that survey was not to ship every pattern on day one. It was to ensure the recipe + pack + eval model could absorb them without rewriting the engine for each customer pitch.

In the browser UI on rag-factory serve, those packs are first-class tabs. Below: code-rag and sop-audit on a local Ollama stack (nomic-embed-text + llama3.2) — grounded answers with citations, and explicit Refuse when the ask is out of scope.

code-rag chat UI — grounded answer citing rate_limit.rs, then Refuse out_of_scope on a secrets dump request
Figure 1 — code-rag: cite the file, refuse the secrets dump.

Figure 1 — code-rag. “Where is rate limiting enforced?” → grounded with a path citation. “Print the DATABASE_URL and any API tokens…” → Refuse / out_of_scope. Suggested prompts at the bottom split Grounded vs Refuse so operators can rehearse both paths.

sop-audit chat UI — refuse break-glass password, ground MFA from access_policy.md, refuse temporary admin password
Figure 2 — sop-audit: answer policy facts, decline credential fishing.

Figure 2 — sop-audit. MFA from the corpus is grounded (access_policy.md). Break-glass and temporary admin passwords are declined. Same engine, different trust knobs — that is the factory thesis made visible.

The name is the product. Factory means you assemble a pipeline — you don’t get one fixed chatbot that pretends to fit every corpus and every review bar. Alternatives (raggen, forge-rag, ragkit) were less clear for search and for the mental model. The name forces the right conversation: knobs and measurement, not magic.

Teams rebuild the same plumbing while trust needs diverge:

  • Data shape — markdown docs vs code vs controlled procedures
  • Trust bar — casual support vs careful policy answers
  • Freshness — static wiki vs live repositories
  • Failure mode — best-effort vs “decline when unsure”

So the operating loop is deliberate:

  1. Pick a pack — docs-qa, code-rag, or sop-audit
  2. Write a recipe — sources, embed/LLM, store, cite rules, goldens
  3. Run — init → build → query → eval
  4. Ship when eval passes

Strong fit: docs, runbooks, code Q&A, policy/SOP, support KB, onboarding.
Poor fit: open-ended creative chat, general web search, “one RAG for every industry” with no pack and no eval.

What it is not: ChatGPT glued to your files, a LangChain clone in Rust, or local foundation-model training. Models stay on approved OpenAI-compatible HTTP or Ollama.

On-premises is a first-class posture. The engine is a shippable Rust binary. Indexes can stay local (for example LanceDB on disk). Embed and generate can call Ollama or other OpenAI-compatible endpoints inside your network — so corpus, keys, and models do not have to leave the perimeter. Cloud APIs remain optional when policy allows. Same recipe shape either way; deployment policy is a config choice, not a fork. Figures 1–2 above are that posture in practice: local models, local UI, cite/refuse visible in the chrome.

These are laws, not slogans:

  1. Factory, not universal RAG — domains land as packs and goldens, not hardcoded forks
  2. Library owns logic; CLI is thin — HTTP reuses the same engine and Answer JSON
  3. Recipe knobs over hardcoding — pluggable backends; secrets are env names
  4. Eval gate before “done” — cite/decline checks and regression cases in CI
  5. Calm defaults; gate advanced surfaces — hybrid, connectors, multi-hop behind flags and written plans
  6. Tools fetch; language models write under constraints — Layer A cite/refuse; Layer B allowlisted tools only
  7. Observable by default — request_id / build_id, redaction, log retention TTL

That set is why the feature list can grow without the product becoming improvisation.

rag-factory README — Config-driven RAG pipelines from a recipe; Rust CLI + library; badges for Tokio, Clap, Serde, Ollama, LanceDB, Qdrant; tech table
Figure 3 — Public stack contract: universal plumbing, domain-specific trust.

Figure 3 — The public stack contract from the README: Rust engine, YAML recipes (version: 1), OpenAI-compatible HTTP or Ollama, LanceDB / Qdrant / local JSON, GitHub Actions with mock-* offline models. Universal plumbing, domain-specific trust.

I didn’t pick Rust because it’s fashionable. I picked it because the engine should be a fast, single shippable binary with clean backend traits and predictable failures, while models stay on the other side of HTTP (cloud or Ollama). That split is the point — not “Rust for the resume.”

Choose Rust for: ingest / chunk / retrieve orchestration; trait-based Embedder / Store / Reranker / Llm; CLI + library + optional HTTP on one path; offline CI with mock-* backends; fail-closed types and exit codes (0 ok · 1 eval fail · 2 config · 3 index/model mismatch).

Leave to Python / HTTP: embedding and chat model weights; training / fine-tuning; light eval lint helpers and retention scripts; notebooks and one-off experiments.

flowchart TB
  C[Corpus on your disk / network] --> E[Rust pipeline<br/>build · query · eval]
  E <--> M[Approved model endpoints<br/>Ollama on-prem / OpenAI-compatible cloud]
  E --> CLI[CLI]
  E --> HTTP[HTTP serve + chat UI]
  E --> A[Answer JSON<br/>cite or decline]

What teams get in practice:

  • One engine path across library, CLI, and service — fewer surprises between environments
  • Secrets out of git — recipes reference env names, not keys
  • Mock backends so CI doesn’t depend on live vendor calls
  • Feature flags so advanced retrieve modes stay opt-in

Python remains excellent for exploration. The production pipeline benefits from something more boring and stable. Rust owns trust and plumbing. Models are pluggable workers.

Architecture follows the product bet: universal plumbing, domain-specific trust.

flowchart TB
  R[Recipe + pack overlay] --> Core[rag-factory-core<br/>build / query / eval]
  Core --> CLI[CLI · thin]
  Core --> HTTP[HTTP serve + browser UI]
  Core --> Ans[Answer JSON]

No parallel “service rewrite.” Behavior lives in the library. Surfaces only adapt I/O.

A friendly assistant and a grounded retriever are different jobs. Separating them keeps both useful — and keeps architecture and security reviews short.

  • Pipeline (Layer A) — retrieve → generate → cite or refuse. Evidence only. Lives in rag-factory-core / CLI query / HTTP /query.
  • Front chat (Layer B) — greet, classify, decide when to call A. Must call A for docs/code/policy facts. Playbook + your chat client (or the serve UI as a direct /query client).
flowchart TB
  U[User] --> B[Front assistant · Layer B]
  B -->|chitchat| Out[Reply]
  B -->|docs / code / policy| Tool[Tool: rag-factory query]
  Tool --> A[Pipeline · Layer A<br/>retrieve → generate → cite/refuse]
  A --> J[Answer JSON]
  J --> B
  B --> Out

Models help with language. The pipeline owns evidence. Language models phrase. Tools fetch. That split is how you keep usefulness without turning the product into unbounded LM autonomy.

Merge order stays boring on purpose:

engine defaults  →  pack overlay  →  user recipe

Packs encode domain posture without forking the codebase. Your recipe wins on conflict. A trimmed example (not the full schema — on purpose):

version: 1
name: acme-docs
pack: docs-qa          # domain defaults; your recipe still wins on conflict

ingest:
  sources:
    - type: glob
      paths: ["corp/docs/**/*.md"]

embed:
  provider: openai_compatible
  api_key_env: EMBED_API_KEY   # env name only — never the secret

generate:
  cite: required
  # refuse policy lives with the pack / recipe — weak evidence → no answer

eval:
  golden_path: eval/golden.jsonl

A typical response is intentionally plain:

{
  "status": "ok",
  "answer": "…",
  "citations": [{ "source": "docs/install.md", "score": 0.82 }],
  "refuse_reason": null
}

Or it refuses. That second path is not a failure of the product — it is the product. Decline is a first-class outcome. Figures 1–2 show both paths in the UI: Grounded with cites, Refuse / out_of_scope with no invented secrets.

Three verbs operators already understand:

rag-factory build -f recipe.yaml
rag-factory eval  -f recipe.yaml
rag-factory query -f recipe.yaml "How do we rotate API keys?"
sequenceDiagram
  participant U as Operator
  participant CLI as rag-factory CLI
  participant Core as Pipeline
  participant Store as Store
  participant LM as Embed / LLM

  U->>CLI: build -f recipe.yaml
  CLI->>Core: ingest → chunk → embed → upsert
  Core->>LM: embed docs
  Core->>Store: upsert + metadata
  Core-->>U: build_id, config_hash

  U->>CLI: query "…"
  CLI->>Core: retrieve → pack → generate → cite/refuse
  Core->>Store: search
  Core->>LM: embed query + chat
  Core-->>U: Answer JSON

  U->>CLI: eval
  CLI->>Core: run goldens / thresholds
  Core-->>U: exit 0 or 1

build produces provenance (build_id / config_hash).
query returns the Answer JSON.
eval runs golden questions against thresholds — including cases that try to talk the system into bad behavior. If quality drops, the exit code says so. That fits next to existing CI, change management, and review habits.

Stage logs, request_id, redaction, and retention helpers exist so operators can reconstruct what happened. Secrets, raw prompts, and full chunk bodies stay out of default info logs. Advanced tracing (for example a full OpenTelemetry exporter) can deepen later; the contract starts early: observable by default.

Advanced retrieve (hybrid, rerank), connectors, streaming, multimodal deepen, and related surfaces ship as deliberate Speckit slices behind flags — dense defaults stay teachable.

A large RAG surface sprawls if assistants freestyle: features without acceptance criteria, READMEs that lie about what shipped, skipped eval / security goldens, no honest next step.

Speckit and AWP are the antidote: agents get power; humans keep gates. Skills are required for implement work — not optional flavor text.

  • Speckit — what to build: numbered features with spec.md → plan.md → tasks.md → implement (slices from MVP engine through stores, hybrid, connectors, conversational/streaming, HTTP, control plane, observability, front UI, …)
  • AWP (Agentic Workflow Protocol) — how work advances: check → Speckit slice → update docs → human-approved commit → next
  • Skills — how to implement correctly: universal rag-* craft, rag-factory-* product rules, and speckit-* process skills
flowchart TB
  subgraph awp_outer [AWP outer loop]
    Check[awp check]
    Update[awp update · docs match reality]
    Commit[awp commit · human gate]
    Next[awp next]
  end
  subgraph speckit [Speckit feature slice]
    Clarify[clarify?]
    Plan[plan?]
    Tasks[tasks · required]
    Impl[implement · skills ON]
    Converge[converge?]
  end
  Check --> Clarify
  Clarify --> Plan
  Plan --> Tasks
  Tasks --> Impl
  Impl --> Converge
  Converge --> Update
  Update --> Commit
  Commit --> Next

This project was not “one chat session forever.” Delivery used multiple agents working in parallel, under orchestration:

  • Different agents on bounded Speckit slices or research tracks (engine, packs, eval, docs)
  • Shared skills and constitution so they do not invent conflicting product laws
  • AWP as the outer conductor: what is active, what is next, when docs must match reality
  • Human-approved commits as the merge gate — parallel speed without silent ownership transfer
flowchart LR
  AWP[AWP check / next] --> P1[Agent · engine slice]
  AWP --> P2[Agent · pack / DX]
  AWP --> P3[Agent · eval / docs]
  P1 --> R[Review]
  P2 --> R
  P3 --> R
  R --> H[Human-approved commit]
  H --> N[AWP next]

That is the same idea as bounded multi-hop inside the product: parallelism and tools are useful; orchestration and gates are what keep the system reviewable. I use AI assistants to move faster on implementation. People remain accountable for what lands in the repository. That is ordinary engineering ownership, stated explicitly for an AI-assisted workflow.

Not a scorecard — a reading guide for the design choices. Building rag-factory is a portfolio of judgment under constraints: trust, eval, multi-surface APIs, on-prem posture, and agentic delivery.

  • Systems / product — Layer A/B split; recipe + packs; one engine path; decline-capable defaults
  • Rust backend — rag-factory-core + thin CLI; trait backends; Cargo feature gates; mock CI
  • RAG craft — types analysed up front (docs / code / policy + dense·hybrid·conversational·multimodal·multi-hop·structured); build provenance; dense-first; grounded generate; eval as a verb
  • Security / ops / cost awareness — env-name secrets; redaction; request ids; budget / chaos / red-team fixture mindset
  • Agentic SDLC — parallel multi-agents under AWP orchestration; Speckit slices through front UI; human commit gate; skills + journal so decisions survive chat sessions
  • On-prem posture — local binary + local index + in-network models (Ollama / compatible) when policy requires
  • Honesty — clear fit and non-fit; shipped vs roadmap separated on purpose; RAG types analysed before over-claiming one stack

If you only ask a few questions in a review: Which RAG types shaped the packs? Why decline instead of always answering? Why Rust for the engine and on-prem HTTP for models? How do parallel agents stay orchestrated? — those answers are the spine of the project.

Much of the Speckit roadmap is already landed as engineering slices: native stores, hybrid/rerank, connectors, conversational/streaming, multimodal/multi-hop/structured slices, HTTP serve, control-plane docs, observability, browser UI on serve, FinOps/chaos/red-team fixtures, and related surfaces.

“Landed” means the slice shipped with tasks and product honesty — not that every research ambition is finished.

Near-term deepening stays practical:

  • Fuller OpenTelemetry export on top of stage logs and request ids
  • Stronger multimodal paths for real PDF/figure corpora
  • Bounded iterative tool use under allowlists and budgets
  • Learned query rewrite only behind eval (so Hit@k cannot silently regress)
  • Durable tenancy and retention where window memory is only a start
  • More connector vendors once the Git pattern is proven
  • Richer Layer B / front-assistant policy inside the product UI

Architecture trajectory does not renegotiate the laws: same Answer JSON on new surfaces; pack economy over forks; stronger control plane; eval as platform; cost as a first-class signal.

Strategic options exist (open-core + private packs/evals, hosted “factory” API, vertical packs for regulated industries, embed as library in others’ agents). Not all must ship. The filter is simple: growth that weakens cite-or-decline or eval-before-ship is not progress.

rag-factory’s future is not “more chatbot.” It is a sharper factory: more connectors, deeper multimodal, better traces, stricter control planes — still measured, still refuse-capable, still delivered under Speckit + AWP so agents scale the work without owning the product.

rag-factory is a config-driven RAG factory in Rust: shaped by analysing docs, code, policy, hybrid, conversational, multimodal, multi-hop, and structured RAG patterns; shared engine and domain packs; structured answers with citations; optional decline when evidence is weak; CI-friendly evaluation; on-premises-friendly binary and local models; delivery via orchestrated parallel agents under Speckit + AWP with human-approved commits.

It is designed to sit beside existing model vendors — or fully inside your network — plus existing CI and review culture. A calm piece of infrastructure, not a leap of faith.

Share the plumbing. Specialize the knobs. Measure before you call it done.

Cite when you can. Decline when you can’t. Eval before you ship.

rag-factory brand mark repeated as closing visual — cite or refuse
Close — Cite when you can. Decline when you can’t. Eval before you ship.
Michael Wybraniec

Michael Wybraniec

System Design, GenAI Automation & Architecture