Quote-Matching Pipeline — Step-by-Step Tutorial¶
A guided, chapter-by-chapter walkthrough that takes you from an empty checkout to a working system: your externally-run Postgres (catalog graph), Ollama (local models), and n8n (orchestration), plus the deterministic Python core that does the matching.
By the end you will understand every moving part, have the pieces connected, and know exactly what to configure to point it at your own catalog and mailbox.
Infrastructure runs separately. Postgres, Ollama, and n8n are not managed by this repo's Docker Compose — you run them your own way (existing instances, separate compose files, managed services, etc.). This project connects to them via configuration and provides a helper to initialize the database. The only optional container here is a small tools image for running the Python core.
What this tutorial covers: connecting to your infrastructure, initializing the schema, and the deterministic core. The two LLM steps (email → facts, facts → draft prose) and the hybrid-search service are wired on top — the tutorial shows exactly where they plug in, and flags the pieces that depend on your own data/infra decisions.
Table of contents¶
- Chapter 0 — The big picture
- Chapter 1 — Prerequisites
- Chapter 2 — Get the code and tour the layout
- Chapter 3 — Configuration (
.env) - Chapter 4 — Connect to your infrastructure
- Chapter 5 — Initialize the database
- Chapter 6 — Pull the models into your Ollama
- Chapter 7 — Run the deterministic core
- Chapter 8 — Generate and embed system cards (hybrid search)
- Chapter 9 — Wire the n8n workflow
- Chapter 10 — Model placement and privacy
- Chapter 11 — Load your own catalog
- Chapter 12 — Build the eval set and tune
- Chapter 13 — Going to production
- Chapter 14 — Troubleshooting
- Appendix A — Configuration reference
- Appendix B — Open questions
- Appendix C — Serve these docs (deploy to a VPS)
Chapter 0 — The big picture¶
The system turns a customer email ("I need a fire-rated partition for an office stairwell, ~24 m², 2.8 m tall") into a draft reply a salesperson reviews: the right system, its bill of materials, and any clarifying questions.
Two decisions shape everything (full rationale in
architecture.md):
- Vector search finds the entry point; the graph delivers the relationships. Embeddings pick which system the email is about; the exact bill of materials is read from SQL relation tables, never guessed by a model.
- Thin model, thick scaffold. The LLM is used only for the two genuinely hard/quality-sensitive steps. Everything else — requirements, matching, BOM math, validation — is deterministic, auditable code.
Gmail trigger → preprocess → EXTRACT(LLM) → rules → match → validate → DRAFT(LLM) → Gmail draft + log
│ └──────── deterministic core ────────┘
└─ observable facts only (constrained JSON)
The three infrastructure services — which you run separately — and their usual ports:
| Service | Role | Usual port |
|---|---|---|
| Postgres | catalog graph + card embeddings (pgvector) | 5432 |
| Ollama | local LLM + embedding backend (dev) | 11434 |
| n8n | orchestration (the arrow diagram above) | 5678 |
This repo does not start these; it connects to them. If you don't already have them, spin them up however you prefer (each ships an official image/binary).
Chapter 1 — Prerequisites¶
You provide the infrastructure (running separately):
- Postgres 16+, ideally with the pgvector extension available (for semantic search; lexical search works without it).
- Ollama (dev LLM + embeddings), or a frontier API for the LLM steps.
- n8n (orchestration).
- ~8 GB free RAM on the Ollama host for the dev models (a 7B quantized model fits an 8 GB Mac).
On your workstation:
- Git.
- Python 3.11+ to run the deterministic core, or Docker if you prefer the optional tools container.
psqlon PATH to initialize the database (or use the tools container).
Check:
Chapter 2 — Get the code and tour the layout¶
The parts you will touch:
docker-compose.yml deploy the docs site (Dokploy + Traefik: domain + TLS)
docker-compose.local.yml local docs site (host port) + tools container
.env.example configuration template → copy to .env
db/
apply.sh initialize an external Postgres (schema + pgvector + seed)
migrations/ Postgres schema (the catalog graph)
docker/ pgvector enablement (optional step)
seed/ illustrative sample catalog
src/quote_match/ the deterministic core (rules, matching, validation, cards)
eval/ extraction eval harness (gold set, predictor, scorer)
examples/demo.py runnable end-to-end demo (no infra needed)
docs/ architecture, pipeline wiring, this tutorial
If you just want to see it work with zero setup, jump to Chapter 7 — the core runs without any services. Otherwise continue in order.
Chapter 3 — Configuration (.env)¶
Copy the template and edit it:
Point it at your running services:
| Variable | What it is | Do this |
|---|---|---|
DATABASE_URL |
connection to your external Postgres | set host/port/user/pass/db |
OLLAMA_URL |
your Ollama endpoint | e.g. http://localhost:11434 |
EMBEDDING_MODEL |
Ollama embedding model | bge-m3 (multilingual default) |
EXTRACTION_MODEL |
model for email → facts (dev) | qwen2.5:7b-instruct |
DRAFT_MODEL |
model for the customer draft (dev) | qwen2.5:7b-instruct |
ANTHROPIC_API_KEY |
optional frontier key for hard steps | leave blank to run fully local |
APP_DATABASE_URL / APP_OLLAMA_URL |
same, as seen from inside the tools container | only if you use the app container |
DATABASE_URL / OLLAMA_URL are for commands you run on your host (migrations,
the core). The APP_* variants exist because inside a container localhost
means the container itself, so they default to host.docker.internal — change
them if your services live on another host.
Chapter 4 — Connect to your infrastructure¶
You run Postgres, Ollama, and n8n separately. This chapter just confirms the tooling can reach them; make sure all three are up first.
Create the database (once) on your Postgres server, if it doesn't exist:
createdb -h <pg-host> -U <pg-user> quotematch
# or: psql -h <pg-host> -U <pg-user> -c 'CREATE DATABASE quotematch;'
Check connectivity using the values from your .env:
set -a; source .env; set +a # load DATABASE_URL, OLLAMA_URL, ...
psql "$DATABASE_URL" -c "SELECT version();" # Postgres reachable?
curl -s "$OLLAMA_URL/api/tags" | head -c 200 # Ollama reachable?
Open your n8n instance in a browser (commonly http://localhost:5678) and make sure you can log in. Note the hostnames n8n will use to reach Postgres and Ollama on your network — you'll need them in Chapter 9.
Chapter 5 — Initialize the database¶
Because Postgres runs separately, initialization is a one-time manual step. The helper applies everything in the right order:
Options:
bash db/apply.sh --no-vector— skip pgvector (lexical search still works).bash db/apply.sh --no-seed— schema only, no sample data.- No
psqllocally? Run it through the tools container:docker compose -f docker-compose.local.yml run --rm app bash db/apply.sh.
What it applies:
db/migrations/0001_init.sql— the catalog graph (products, systems, procedures, edges).db/migrations/0002_cards_and_search.sql— system cards + lexical search.db/docker/03_pgvector.sql— enables pgvector and adds theembeddingcolumn (skipped with--no-vector).db/seed/sample_catalog.sql— the illustrative W111/W112 data (skipped with--no-seed).
Verify it loaded:
psql "$DATABASE_URL" -c \
"SELECT code, attributes->>'fire_rating' AS fire FROM system ORDER BY code;"
Expected:
pgvector not installed on your server?
CREATE EXTENSION vectorwill fail. Either install pgvector, or run with--no-vectorand use lexical search only until it's available.Re-applying: the scripts are additive. To start clean, drop and recreate the database (or
TRUNCATEthe tables) before re-running.
Chapter 6 — Pull the models into your Ollama¶
On your Ollama host, pull the dev models (a few GB the first time):
(If Ollama runs in its own container, docker exec <ollama> ollama pull ....)
List what you have and smoke-test embeddings against your endpoint:
curl -s "$OLLAMA_URL/api/tags" | grep -o '"name":"[^"]*"'
curl -s "$OLLAMA_URL/api/embeddings" \
-d '{"model":"bge-m3","prompt":"fire-rated office partition"}' | head -c 120
You should get a JSON array of floats back. That confirms the embedding backend the hybrid-search layer will use (Chapter 8).
Why these models:
bge-m3is multilingual (matches mixed-language inboxes) and CPU-friendly.qwen2.5:7b-instructsupports structured/JSON output for constrained extraction and is small enough for an 8 GB dev box. Swap either via.env. In production the hard extraction and the customer draft can move to a frontier API — see Chapter 10.
Chapter 7 — Run the deterministic core¶
This is the heart of the system, and it needs no database and no models.
With Docker (using the tools profile):
docker compose -f docker-compose.local.yml run --rm app pytest # 24 tests
docker compose -f docker-compose.local.yml run --rm app python examples/demo.py
Or directly on your machine:
The demo feeds three hand-written "extracted facts" through the whole non-LLM path and prints what each stage decides. For example, a bathroom request:
Derived requirements: {'moisture_resistant': True}
- Wet room -> impregnated (moisture-resistant) board. (wet_room_moisture)
Chosen: W112-MR
Draft payload:
BOM:
- 52.8 m2 Moisture-resistant plasterboard 12.5 mm (GB-MR-12.5)
- 225.0 piece Drywall screw TN 25 mm (SC-TN-25)
...
Read examples/demo.py alongside the output — it is the clearest map of
rules → match → validate → draft. The same logic is one call:
quote_match.pipeline.run_deterministic(facts, candidates).
Chapter 8 — Generate and embed system cards (hybrid search)¶
Retrieval is two moves: hybrid search narrows the whole catalog to the top 5–10 candidate systems, then the graph expands those into exact BOMs.
The card is what search operates on. Generate one for any system:
docker compose -f docker-compose.local.yml run --rm app python -c \
"from quote_match.catalog.cards import render_card; \
from quote_match.catalog.sample_data import W112; print(render_card(W112))"
To make cards searchable you populate the system_card table:
- Lexical half (works today): insert the card
body; thebody_tsvgenerated column + GIN index give you BM25-style ranking viats_rankwith no extra service. - Semantic half: embed the card body with
bge-m3(Chapter 6) and store the vector insystem_card.embedding(the pgvector column added in Chapter 5). Query withembedding <=> $query_vector(cosine).
A minimal indexing loop (pseudocode for the service you'll add):
for system in catalog:
body = render_card(system)
if content_hash(body) unchanged: continue
vec = ollama.embed(EMBEDDING_MODEL, body)
upsert system_card(system_id, body, embedding=vec, content_hash=...)
Status: the schema, card generator, and lexical index ship now. The thin embedding+search service (a loop like above plus a
search_catalog(query)endpoint) is the next component to build; its exact shape depends on the open questions in Appendix B, so it is intentionally left for you to wire once those are answered.
Chapter 9 — Wire the n8n workflow¶
Open your n8n instance and build the pipeline as nodes. Use whatever hostnames
reach Postgres and Ollama on your network (the ones you confirmed in
Chapter 4) — e.g. http://<ollama-host>:11434 and <pg-host>:5432. If n8n and
the other services share a Docker network, use their service names; if n8n is on
the host and they're elsewhere, use those addresses.
The flow (each arrow = one or a few nodes):
- Gmail/IMAP trigger — filter to your quotes label/alias.
- Preprocess (Code node) — strip signatures, quoted history, HTML → clean fresh-message text.
- Extract (HTTP → Ollama, or a frontier API) — call the model with the
constrained JSON schema. Get the schema:
Pass it as Ollama's
docker compose -f docker-compose.local.yml run --rm app python -c \ "from quote_match.extraction.prompts import guided_json_schema; \ import json; print(json.dumps(guided_json_schema()))"format(structured output) so only legal enum values come back. System/user prompts live inquote_match.extraction.prompts. - Deterministic core — run rules → match → validate → draft assembly. Wrap
pipeline.run_deterministicin a small HTTP service and call it from an n8n HTTP node, or run it via the tools container. This is the boxed core from Chapter 0; it needs theDATABASE_URLto your external Postgres. - Draft (HTTP → model) — feed the assembled
DraftContextto the draft prompt (quote_match.draft.draft_prompt). The prompt forbids inventing products, quantities, or standards. - Gmail: create draft in-thread — never auto-send. The salesperson edits and sends.
- Log (Postgres/Sheets node) — sender, matched system, confidence, thread link.
Configure Gmail/IMAP and any API keys as n8n credentials (n8n → Credentials),
not in this repo's .env. Export your finished workflow to n8n/workflow.json
so it's version-controlled.
Chapter 10 — Model placement and privacy¶
Where each task runs (decided in planning):
| Task | Difficulty | Dev (this stack) | Production |
|---|---|---|---|
| Email → observable facts | hard | Ollama Qwen 7B | frontier API / Qwen 14B LoRA |
| Rules derivation | trivial | code | code |
| Graph search + BOM | easy | SQL/code | SQL/code |
| Candidate ranking | easy | code | code / Qwen 14B–32B |
| Customer draft | quality-sensitive | Ollama Qwen 7B | frontier API |
| Embeddings | trivial | bge-m3 (Ollama) | bge-m3 (self-host) |
To move a step to a frontier API, point that n8n node at the hosted endpoint and
set ANTHROPIC_API_KEY (or your provider's key) in the node credentials.
Privacy: emails contain PII and flow to whatever runs extraction/draft. The
catalog stays local regardless. If you use a hosted model but must keep PII
local, strip PII before the API call and re-insert it locally when rendering the
draft — the deterministic core only ever sees ExtractedFacts, never raw PII.
Chapter 11 — Load your own catalog¶
The sample data is illustrative placeholder data. To use your own:
- Model it as products (SKUs), systems (solutions), procedures, and the
edges between them (BOM with
quantity_per_unit,requires,compatible). The schema (db/migrations/0001_init.sql) is the contract. - Write an importer from your source format (ERP export, manufacturer data, flat SKU list) into those tables. This is a thin adapter — the schema is ready; only the parsing differs by source. (Source format is open question #1 — see Appendix B.)
- Tune the rules table (
src/quote_match/rules/rules.yaml) with a domain expert. The placeholder fire/acoustic values must be replaced with the national standards you actually adhere to. - Regenerate and re-embed cards (Chapter 8) after loading.
Extraction enums (src/quote_match/extraction/schema.py) should be widened to
your catalog's real vocabulary — every enum value should be something the
matcher can act on.
Chapter 12 — Build the eval set and tune¶
Measure before you tune. The whole design is "thin model, thick scaffold":
rules, matching, BOM math, and validation are deterministic and already covered
by unit tests (pytest). That leaves only two LLM steps to evaluate, and
only one of them is objectively scorable:
| Step | How you evaluate it | Risk if wrong |
|---|---|---|
Extraction (email → ExtractedFacts) |
field-by-field against a gold set | drives everything downstream |
| Draft (payload → prose) | grounding check + rubric / human review | low — salesperson reviews every draft |
So the bulk of this chapter is about extraction, because that is where a number tells you the truth and where errors propagate.
12.1 Assemble the eval set¶
Collect 30–50 real emails that a salesperson would recognize as
representative. Redact PII first (names, addresses, phone numbers) — you only
need the substance. Then, for each, write down the correct ExtractedFacts —
the gold label. A domain-aware person should do this; it is the same judgement
they apply when reading the email.
Deliberately cover variety, not just easy cases:
- every
application/building_type/room_typevalue you care about; - emails where the right answer is
unknown/unclear(this is the most important class — see the hallucination metric below); - missing-information emails (vague one-liners);
- multilingual emails, if your inbox is;
- terse vs. rambling, single-ask vs. multiple-asks, awkward phrasing.
Store it as JSONL under eval/ and version it in git. The repo ships a
runnable starter harness in eval/
— eval/gold.jsonl (sample annotations you replace with your own),
eval/predict.py (runs extraction via Ollama), and eval/score.py (the scorer
below). Keep a dev split you iterate against and a held-out test split
you look at rarely — otherwise you overfit your prompts to the test set and the
score lies to you.
{"id": "e001",
"email": "Hi, we're fitting out a bathroom on the ground floor, about 12 m2, need a stud wall. Can you quote?",
"gold": {
"application": "partition_wall",
"room_type": "bathroom",
"building_type": "unknown",
"area_m2": {"value": 12, "unit": "m2"},
"height_mm": {"value": null},
"is_wet_room": "yes",
"mentions_fire_protection": "unclear",
"missing_info": ["wall height", "building type"]
}}
12.2 Score extraction¶
Generate predictions, then score them:
# 1) run extraction over every email (needs Ollama — Chapter 6)
OLLAMA_URL=http://localhost:11434 EXTRACTION_MODEL=qwen2.5:7b-instruct \
python eval/predict.py eval/gold.jsonl > predictions.jsonl
# 2) score predictions against gold
python eval/score.py eval/gold.jsonl predictions.jsonl
score.py prints per-field accuracy (worst first), overall accuracy,
missing_info F1, and the hallucination rate. It compares to gold per field,
because different field types need different rules:
- Closed enums (
application,room_type,building_type): exact match. - Ternary micro-classifications (
is_wet_room,mentions_*): exact match, but treat a confident-wrong answer (goldunclear, predictedyes/no) as the worst kind of error. - Measurements (
area_m2,height_mm): comparevaluewithin a tolerance (e.g. ±1%) and whetherNone/known agree. missing_info[]: it's a set — score precision/recall/F1, not equality.
The single metric that matters most for this system is the hallucination
rate: how often the model asserted a concrete value where gold says unknown
/ unclear. The design prefers "I don't know" over a confident guess, so track
this separately and never let it climb, even if overall accuracy rises.
The core of eval/score.py — comparing a predictions JSONL to the gold JSONL:
import json, sys
ENUMS = ["application", "room_type", "building_type"]
TERN = ["is_wet_room", "is_escape_route",
"mentions_moisture", "mentions_fire_protection", "mentions_acoustic"]
UNKNOWN = {"unknown", "unclear", None}
def measure(v): # pull the numeric value out of a Measurement
return (v or {}).get("value") if isinstance(v, dict) else v
def score(gold, pred):
hits, total, halluc, halluc_total = 0, 0, 0, 0
for f in ENUMS + TERN:
g, p = gold.get(f), pred.get(f)
total += 1
hits += (g == p)
if g in UNKNOWN: # hallucination = asserted a value where gold is unknown
halluc_total += 1
halluc += (p not in UNKNOWN)
# measurements: value agreement within 1%
for f in ["area_m2", "height_mm"]:
g, p = measure(gold.get(f)), measure(pred.get(f))
total += 1
if g is None or p is None:
hits += (g is None) == (p is None)
else:
hits += abs(g - p) <= 0.01 * abs(g)
# missing_info F1
gi, pi = set(gold.get("missing_info", [])), set(pred.get("missing_info", []))
tp = len(gi & pi)
f1 = (2*tp / (len(gi)+len(pi))) if (gi or pi) else 1.0
return {"field_acc": hits/total,
"missing_info_f1": f1,
"hallucination_rate": halluc/halluc_total if halluc_total else 0.0}
golds = {json.loads(l)["id"]: json.loads(l)["gold"] for l in open(sys.argv[1])}
preds = {json.loads(l)["id"]: json.loads(l)["pred"] for l in open(sys.argv[2])}
rows = [score(golds[i], preds[i]) for i in golds if i in preds]
n = len(rows)
for k in ["field_acc", "missing_info_f1", "hallucination_rate"]:
print(f"{k:20} {sum(r[k] for r in rows)/n:.3f}")
Break the score down by field too — the worst fields tell you exactly where to spend effort next.
12.3 Improve cheaply before fine-tuning¶
Work from cheapest to most expensive, re-running the eval after each and keeping a log of the scores:
- Fix the schema first. A missing enum value forces wrong answers — if
"plant room" keeps arriving and there's no enum for it, add it
(
extraction/schema.py). No enum change is cheaper than any prompt change. - Add few-shot examples. Populate the 5–8 example slots in
extraction/prompts.py(EXAMPLES) with real failures from the eval set. - Decompose weak fields into micro-classifications (
MICRO_QUESTIONS) — small yes/no/unclear questions are far more reliable than one mega-extraction. - Try a bigger off-the-shelf model (Qwen 14B/32B, or a frontier API for this step only) through the same interface. Often beats fine-tuning a small one, with no training pipeline to maintain.
Escalate only when the cheap options plateau.
12.4 The correction flywheel (free labels)¶
Every time a salesperson edits a drafted reply or corrects the extracted facts,
that correction is a labeled example at zero labeling cost. Capture it: the
n8n log step (Chapter 9) should record both the model's ExtractedFacts and the
corrected version. Over weeks this accumulates into hundreds of gold pairs.
Feed corrections into three places: (a) the eval set as new hard cases, (b) the few-shot pool, and (c) eventually the fine-tuning set.
12.5 When — and how — to fine-tune¶
Only fine-tune when all of these hold: you have a few hundred corrected examples, prompt-level gains have plateaued, and a measured gap remains that actually matters for quotes.
Then LoRA-fine-tune Qwen 14B — parameter-efficient, cheap on a rented GPU, and it keeps the base model swappable. Format the data as instruction → JSON pairs that match the extraction schema exactly. Rules of the road:
- Keep the held-out test split untouched so the improvement you measure is real.
- Watch for regressions: fine-tuning on your distribution can hurt rare cases and, worst of all, raise the hallucination rate. Gate on the safety metric, not just average accuracy.
- Serve the adapter with vLLM (Chapter 13).
12.6 Evaluating the draft step¶
The draft is open text, so it's judged, not diffed — but the most important
property is automatable: the draft must not invent any SKU, quantity, or
standard that isn't in the DraftContext payload. Check it programmatically —
every SKU and number in the prose should appear in the payload; flag anything
that doesn't as a grounding failure. For tone, completeness, and whether it asks
the right clarifying questions, use an LLM-as-judge over a sample plus a human
spot-check. Because a salesperson reviews every draft before it goes out, this
step is lower-risk — grounding failures are the thing to catch.
12.7 Keep a scores log¶
Track results over time in a simple table (date, model, prompt version, field accuracy, hallucination rate, draft-grounding pass rate). Make it a rule that a change never ships if it lowers a safety metric, even when overall accuracy goes up. That log is what turns "the model feels better" into evidence.
Chapter 13 — Going to production¶
- Serving: replace dev Ollama with vLLM on a rented GPU (Hetzner GX /
RunPod) for throughput and structured-output (
guided_json) support, then pointOLLAMA_URL/ the n8n model nodes at it. - Postgres: use a managed/backed-up instance; keep pgvector. Revisit Postgres-vs-Neo4j only if relationship traversal gets deep (open question #4).
- n8n: run the self-hosted or cloud instance your ops prefers (open question #3); secure it behind auth and TLS.
- Secrets: never commit
.env; use your platform's secret store. Rotate the DB password and any API keys. - Safety: the draft is always a Gmail draft — keep the human in the loop.
Chapter 14 — Troubleshooting¶
| Symptom | Cause / fix |
|---|---|
psql: could not connect |
Wrong host/port/creds in DATABASE_URL, or Postgres not reachable from your machine. Confirm with psql "$DATABASE_URL" -c 'select 1'. |
db/apply.sh: type "..." already exists |
Schema already applied. Drop/recreate the DB (or TRUNCATE) to re-apply cleanly. |
CREATE EXTENSION vector fails |
pgvector isn't installed on your server. Install it, or run bash db/apply.sh --no-vector. |
Ollama model not found |
Pull it on the Ollama host: ollama pull <model> (Chapter 6). |
| Ollama endpoint refused | Check OLLAMA_URL; if Ollama binds only to localhost, set OLLAMA_HOST=0.0.0.0 on its host so other machines/containers can reach it. |
| n8n can't reach Ollama/Postgres | Use hostnames valid on n8n's network, not localhost (which is n8n itself). Same-network containers → service names; otherwise the real host/IP. |
app container can't reach services |
Inside a container localhost is the container. Set APP_DATABASE_URL / APP_OLLAMA_URL (default host.docker.internal). |
docker compose -f docker-compose.local.yml run app fails to build |
First build compiles deps; check docker compose -f docker-compose.local.yml build app. Needs network for the base image. |
| Tests pass but demo differs from real | Sample data is illustrative; results reflect sample_data.py, not a real catalog. |
Useful commands:
set -a; source .env; set +a
psql "$DATABASE_URL" # inspect the catalog DB
curl -s "$OLLAMA_URL/api/tags" # what models are loaded
docker compose -f docker-compose.local.yml run --rm app pytest # run the core in-container
Appendix A — Configuration reference¶
All configuration lives in .env (copied from .env.example). Postgres,
Ollama, and n8n run separately, so these are connection settings, not
service definitions.
| Variable | Default | Used by | Meaning |
|---|---|---|---|
DATABASE_URL |
postgresql://quote:change-me@localhost:5432/quotematch |
db/apply.sh, host scripts |
connection to your external Postgres |
OLLAMA_URL |
http://localhost:11434 |
host scripts, indexing | your Ollama endpoint |
EMBEDDING_MODEL |
bge-m3 |
indexing/search | embedding model |
EXTRACTION_MODEL |
qwen2.5:7b-instruct |
extract node | email → facts model |
DRAFT_MODEL |
qwen2.5:7b-instruct |
draft node | draft prose model |
ANTHROPIC_API_KEY |
— | optional frontier steps | hosted-model key |
APP_DATABASE_URL |
...@host.docker.internal:5432/... |
app container |
Postgres as seen from inside the tools container |
APP_OLLAMA_URL |
http://host.docker.internal:11434 |
app container |
Ollama as seen from inside the tools container |
n8n's own configuration (its database, model endpoints, Gmail/IMAP credentials) is set inside n8n, since you run it separately.
Appendix B — Open questions¶
These were carried over from planning and gate specific pieces of the build. Answer them before the corresponding step:
- Catalog source format (flat SKU list? ERP export? manufacturer data?) → determines the importer (Chapter 11).
- Languages of incoming emails → confirms the embedding model (
bge-m3is the multilingual default) and the extraction prompt language(s). - Where n8n runs (self-hosted vs cloud) → deployment (Chapter 13).
- Postgres vs Neo4j → defaulted to Postgres; revisit only if traversal depth demands it. The node/edge model ports either way.
See pipeline.md for how these map onto the remaining
components.
Appendix C — Serve these docs (deploy to a VPS)¶
This tutorial can be published as a browsable website (MkDocs Material built into an nginx container). There are two compose files:
| File | For | How it's exposed |
|---|---|---|
docker-compose.yml |
Dokploy + Traefik on a VPS | by domain, automatic TLS, no host port |
docker-compose.local.yml |
local browsing / running the core | host port DOCS_PORT (default 8090) |
Locally¶
docker compose -f docker-compose.local.yml up -d --build docs
# browse http://localhost:8090 (change with DOCS_PORT in .env)
On a VPS with Dokploy (domain + automatic HTTPS)¶
The default docker-compose.yml is written for Dokploy's Traefik: it routes by
domain over the shared dokploy-network and gets a Let's Encrypt certificate —
no published host port, so it never collides with other apps.
- DNS: create an A record for your subdomain (e.g.
quote-match.previewrun.de) pointing at the VPS IP. - Dokploy app: create a Compose app from this repo (it uses
./docker-compose.yml). - Environment: set
DOMAIN_URLto that subdomain — it's interpolated into the TraefikHost(...)rules: - Deploy. Traefik picks up the labels, requests the certificate, and serves
the site at
https://<DOMAIN_URL>.
To use a different subdomain, just change DOMAIN_URL (and its DNS record) —
nothing in the compose file needs editing.
Updating content¶
Edit the Markdown under docs/ and redeploy (Dokploy) or rebuild locally:
The image is a static build served by nginx — small, with no runtime dependency on Python, the database, or the models. It's just the docs.
Editing live while writing: for a fast edit loop run the dev server instead of the container:
pip install mkdocs-material && mkdocs serve→ http://localhost:8000 with live reload.