Known gaps
This page lists what Canon does not handle, and what its numbers can and cannot show. Read it before you trust a figure in the README.
The data is synthetic
Every store, order, incident and person is generated from a seed (python -m canon.world.generate --seed 23 --out data/canon.duckdb). Nothing comes from a real platform.
What that gives:
- The generator records every incident it injects, so precision and recall can be checked against an answer key. Production data has no such key.
- A run is repeatable. The same seed gives the same world.
- The controller, the budgets and the approval queue run at realistic volume (about 4,700 stores, a week of full orders) without touching anyone's systems.
What it cannot show:
- Real traffic. Volumes, timing and failure patterns follow assumptions about delivery operations. If an assumption is wrong, the numbers are wrong with it.
- Incidents the world does not contain. The detectors were written against the incident kinds the generator injects. Recall says how many injected incidents were found. It says nothing about kinds nobody injected.
- Right thresholds. "Late after N minutes", "stuck after N seconds" and the rest are settings to tune on real data, not findings.
- Real false-alarm rates. The generated baseline is cleaner than a live one, so false alarms are probably undercounted.
- A hold-out in the strict sense. The detectors, thresholds and world parameters were tuned on seeds 7 and 11. The README numbers come from seed 23, which was never used for tuning. A new seed is a new draw from the same generator. It guards against fitting one draw. It does not guard against a generator that is wrong about the world. The changes made while looking at the answer key are listed in
canon/dictionary/TUNING.md.
Who decides: rules first, the model only for a real choice
Most incidents are decided by rules. The policy file lists the actions allowed for each incident kind. When only one action is possible at tier 0 or 1, a fixed rule fills it in and the model is not called. The model is asked when there is a real choice between actions, or when the decision may need a person's approval and a written reason helps the person who reviews it.
Either way the controller checks the proposal against the written policy: authority tier, budgets, cooldowns. If the model's answer is missing or invalid, the rule proposes instead, and the ledger row says which path was used.
The model is pluggable and off by default. Canon talks to any OpenAI-compatible chat endpoint. Until CANON_LLM_BASE_URL is set, everything runs on the rules, and the ledger says so. Runs that use a model are not deterministic, so two runs on the same seed can differ a little. Runs on the rules alone are identical.
Not handled
Each item says what happens today and what a pilot would need.
Nothing is sent or paged. Every message Canon writes (to a store, a delivery partner, the on-call) is a draft in the outbox; a person reads and sends it. The queue's "on-call notified (draft)" means exactly that. Paging and sending are production work (milestone 2 in the full design).
Acceptance targets per aggregator. Canon uses one line: 98% of orders accepted over seven days, set in the acceptance_risk definition. Aggregators each define acceptance and its target in their own way, and the target can change by market and by contract. Canon holds no per-aggregator targets. A pilot needs the real ones, read from the partner agreements.
Scheduled and pre-orders. The world has no order placed for a later time. The detectors measure from placement or acceptance, so an order placed in the morning for 19:00 would be flagged as stuck or late at the wrong moment. A pilot needs the requested time on every order.
Totals that differ by tax rounding. Prices in the world include the local sales tax (VAT in the Gulf and Europe, GST in India, state and city sales tax in the US world). In practice line items and the order total can differ by a rounding unit, and a POS may refuse the order for it. The world does not generate this and Canon does not compare line sums to totals. Such a rejection would look like a failing integration, with no cause named.
Store-hours sync. World stores have no opening hours. Canon does not compare the hours held in the POS with the hours held on each channel, and an order that arrives outside hours is not an incident.
Cancel after injection. A guest or an aggregator can cancel an order after it has reached the POS. The world records cancellations and the definitions leave cancelled orders out of late and stuck. Canon has no action that cancels in the POS or tells the kitchen to stop, so the kitchen may still make an order that no longer exists.
Webhook redelivery. Aggregators resend a webhook when they do not get an answer in time. Canon reads orders after the fact. It detects duplicates, but it does not sit in the webhook path and cannot stop the same delivery arriving twice. Before it re-sends an order it checks whether the POS already acknowledged it, so its own repush should not create a duplicate.
Hardware. Rerouting to a backup printer, or showing a ticket on a KDS or tablet, only works where that hardware is installed. The world carries one flag for a backup printer. Canon knows nothing else about what a real store has. Where the hardware is missing it drafts a message to the store instead.
Busy mode differs per aggregator. Canon treats busy mode as one action that behaves the same on every channel. In practice some aggregators expose it through an API, some only through their own tablet or portal, and some not at all. What "busy" does to new orders also differs. A pilot needs a list of what each aggregator supports.
Which system owns the menu. Canon assumes the platform's menu and the POS menu can be compared and the platform side pushed (save_menu is a full overwrite, so it needs a person). Some customers treat the POS as the master and the platform as a mirror, others the reverse. Canon does not know which is true for a given customer. It flags a mismatch that causes rejections and does not decide which side is right. A price that differs between channel and till is not a mismatch: channel markup is deliberate.
Labels in shadow mode. In the world every incident has a label because the generator wrote it. In production nothing does. In shadow mode, precision and recall can only be estimated from what people record: support tickets, an ops review of the "would do" rows, replies from stores. Those labels are partial and arrive late, and tickets exist only for problems someone noticed. Recall measured that way will read higher than the truth. A pilot should plan a regular review of a random sample of rows and treat the first numbers as rough.