Docs / The method / Monitoring

Monitoring

Monitoring is where a failure mode stops being a label on some traces you read once and becomes a number measured every day against traces nobody read.

What monitoring does

When you monitor an error, Zoroval scores new traces against it every day and records a verdict per trace. Those verdicts produce the rate on the dashboard, the rows in the logs, and the sections of the report.

Monitoring is the only part of Zoroval that runs continuously and the only part with a recurring cost, which is why it is opt-in per code and capped.

Setting it up

An error reaches Monitoring setup two ways, and the screen shows them as two tabs.

• Defined Monitoring. On Defined errors you name a failure you already know about and describe what it looks like. You do not need to have coded a trace first. This is usually the faster way to start, because it begins with something you can already say.
• Axial Monitoring. A pattern that came out of open coding and was approved as an axial code. This is the slower way in and the one that finds failures nobody thought to name.

Both end at the same object and are judged the same way. What differs is how the error got its name, and that each tab carries its own cap: filling one never takes a slot from the other.

1. Give the error one real exampleWhichever way it arrived, an error needs one approved example from a real trace before it can be watched. That example is what the retrieval specification is designed against, and what gets replayed to check the specification before it goes live. A named error with no example is listed and says so rather than being hidden.
2. Select the codes to watchPick the failure modes whose rate you expect to move when your team ships.
3. Watch the first completed dayThe dashboard reads Awaiting first completed trace day until a full day of traces has been evaluated. Monitoring scores completed days, not partial ones, so a rate is never computed from half a day of traffic.
4. Pause and swap as priorities changeA monitored code can be paused. Its history is preserved and its row stays visible marked No longer sampled, so old numbers stay readable without implying they are current.

What runs each day

For each monitored code and each sampled trace of a completed day, the judge returns a structured verdict: yes, no, or insufficient, with a confidence and the evidence behind it. The verdict is stored against the trace day, so rates are attributed to when the traffic happened rather than to when it was scored.

Limits, sampling and cost

Cost predictability is a design constraint, not a setting. Three rules hold on every plan:

• Every trace is ingested, only a sample is judged. Your full trace history stays available for open coding and drill-down. Sampling applies to paid judging alone.
• The number of active codes is capped. That forces prioritisation and bounds the daily spend. To add one beyond the cap, pause another: the UI says Limit reached. Pause one to add another.
• Scoring history never happens by default. Backfilling months of traces is a large one-off cost, and it needs explicit approval every time.

A trial runs on a fixed daily allowance, so the ceiling is a flat number whatever your traffic. On a paid plan, sampling is a small percentage of your daily traces instead, so it tracks your volume rather than sitting at a number that stops meaning anything as you grow.

The cap on active codes and the sampling rate are set by your plan. We agree them with you before anything paid runs, so ask us for the figures that apply to you.

Choosing what to monitor

The cap is a real constraint, and treating it as one improves the taxonomy. Some heuristics:

• Monitor what you would act on. If a rate doubling would not change anyone's week, the slot is better spent elsewhere.
• Monitor what your fixes target. The point of the rate is the before and after around a release. A code nobody is working on produces a flat line.
• Monitor the expensive failure, not the frequent one. A 4% rate on a failure that loses trust beats a 40% rate on a cosmetic one.
• Start with one to three. A first pass with two well-defined codes teaches you more than a full slate of loosely defined ones.
• Prefer codes with a clear boundary. A code whose scope is fuzzy will produce a noisy line that erodes trust in the whole dashboard.

When a code changes

What to expect in the first weeks

• Day one gives you a baseline, not a verdict on your agent.
• The first week gives you the shape of the line, and usually one surprise where a rate is far higher or lower than the team guessed.
• The first fix is the real test. If a targeted change does not move the rate, either the fix did not work or the code is not measuring what you think, and both are worth knowing.