What's holding up this job? For WO-123, the answer:

WO-123 has been open eighteen days. The quote was approved on the 1st. The material request for the special-order shingles went out on the 4th, and its purchase order is still waiting on vendor confirmation. The technician's last note, from the 7th, reads "waiting on material, customer informed." Two service tasks are done; the install step is blocked on that material.

A dispatcher who wants that paragraph opens ten screens: the job, its status history, the notes, the time logs, the quote, the material request, the purchase order, the vendor, the customer, the checklist. Ten to twenty minutes, if they already know where to click. Most people never do it. They ask the technician, or they guess.

Sense, the agent built into Zuper, wrote that paragraph in ninety seconds from that one question.

This post is about the context engineering that makes an answer like that possible, and correct. It goes somewhere we did not expect: the agent computing metrics the product never explicitly built, from a company's own operations manual.

Three kinds of question, one box

Field service companies ask three kinds of question all day.

The first is measured. What was revenue by team last quarter? How many jobs are past due? Harder than it looks, because the vocabulary is local: "Booked" is a status at one company and a category at another, and revenue can mean job revenue, invoiced amount, or approved quote value. Choose wrong and the number is confidently, invisibly incorrect.

The second is investigated, and almost nothing answers it well. Why is this job stuck? Why did this invoice take three weeks to go out? The answer is never in one place. It is a chain of facts across a quote, a material order, a vendor, and a note somebody typed at 6pm.

The third is documented. How does this work here? How do we define first-time fix? The answer lives in the company's own manuals and policy pages.

Sense answers all three in one conversation, and most real questions are a blend: show me overdue invoices, and tell me why the top three are stuck is a measured question and an investigated one in the same sentence.

One principle runs through everything below: every mechanism exists because a specific wrong answer is possible without it.

Anatomy of an investigation

Back to WO-123. What the user watches while it works:

Looking up work order 123
Reading the job                  → 18 days open, install pending
Checking the linked quote        → approved on the 1st
Following the material request   → shingles, raised on the 4th
Checking the purchase order      → sent, awaiting vendor confirmation
Reading technician notes         → last update on the 7th
Checking service tasks           → 2 of 3 done, install blocked

Nobody handed the agent that sequence. It works out the trace itself, because it carries a working model of how the business fits together: a job belongs to a customer and a property, a quote usually precedes it, an invoice follows execution, a material request is fulfilled by a purchase order or a stock transfer, and service tasks are the job's sub-steps. Told to find what is holding up a job, it knows where to look. A typical investigation runs in about a minute to a minute and a half, end to end.

It works over a read-only, allowlisted slice of the Zuper MCP surface we wrote about separately: 48 read-only tools across 14 entity families. The finding is relayed to the user in full rather than re-summarized, because a trace that reaches the user with most of its facts summarized away is a loss, not polish.

Two problems make this harder than looping over an API.

Records are enormous. One fetch from a live operational API can return megabytes of nested data, and almost none of it bears on the question. So any response over 50KB never reaches the model whole. It is parked in a short-lived store, and the investigator gets a preview capped at 4KB: the record's real field names, the shape of the data, and a small sample. It then names the exact paths it wants and reads only those, under a budget, so an investigation converges instead of wandering. This is accuracy engineering before it is cost engineering: a model reasoning over a compact preview finds the blocked purchase order; one buried in a raw payload finds whatever surfaced last.

Tools invite guessing. The operational tools expose thirty to seventy optional parameters each, often with example values sitting right there in their schemas, and a guessed filter does not fail loudly; it silently changes the answer. So only parameters with a real, concrete value get set, usually a single identifier that scopes the request. Words in the user's question like "delayed" or "urgent" describe the output to produce; they are not permission to set matching filters.

The boundary runs the other way too. This path never counts or aggregates. Even a single-customer count goes through the analytics path, because that path can render it, pin it, and refresh it. Explaining one record and measuring many are different jobs.

Anatomy of a measured question

Revenue by team last quarter for the Inspection Ready residentail jobs.

Vocabulary first. "Inspection Ready" is not in any product catalog; it is a status this company invented. And "residentail" is a typo. Before anything gets built, the agent looks the unfamiliar phrases up against what this workspace actually stores:

### "Inspection Ready" → job_status
Exact match: Inspection Ready  (category: Field Work)
Filter on: jobs.current_job_status_name    (current state)
  · job_status_history.status_name         (history / transitions)
  · job_status_checklist.status_name       (answers captured at that status)

### "residentail" → custom_field_value
Field: Property Type (jobs)
No exact match. Closest stored values:
  - Residential
  - Residential - Multi
→ confirm before filtering

The first block settles the status and raises the real question: three fields carry it, and jobs currently in Inspection Ready are not the same set as jobs that passed through it, so the choice is made from the tense of the question. The second block is the other half of the rule: a close match is never used automatically. "residentail" becomes a clarifying question with options that are real stored values, not invented plausible ones.

Here is why that matters. Suppose the lookup finds nothing, and the system shrugs and drops the filter:

Q: how many jobs for Acme Property Management are in production?
   [ "Acme Property Management" unresolved, filter dropped ]
   [ "production" unresolved, filter dropped ]

   4,812 jobs        ← every job in the company, rendered as the answer

That number is wrong, specific, and beautifully formatted. Nobody catches it, because a stat card carries no evidence of what it left out. When every term in a question fails to resolve, Zuper Sense does not run the query. It says which words it could not find and asks.

The other half of that defense is that every rendered number can be opened. Tap a bar on a chart or the number on a stat card and the underlying records open as a table. A stat card you can open is a number you can audit, and the fastest way to notice a filter you did not expect is to see the rows behind the figure.

Then a specification, not SQL. Once the status is pinned to a field and the user confirms Residential, the agent describes the result it wants in business terms: revenue and job count, broken down by team, for last quarter, filtered to jobs currently in Inspection Ready on Residential properties, ordered by revenue, top 20, rendered as a bar chart.

Checked against reality. Every field named in that spec is verified against the definitions actually available for the selected entities. Near-misses snap to the real name. Fields in the wrong slot get moved. Date operators on non-date fields, placeholder values, internal identifiers, duplicates, and impossible shapes get stripped or corrected before execution.

Executed, shaped, rendered. Results come back with display names resolved to the company's own labels, then get shaped for the visualization: a single number becomes a stat card, a time axis becomes a line, and a breakdown over time stacks. A turn that ends on text alone is treated as incomplete, so a missing chart gets recovered rather than shipped as a paragraph the user cannot see. End to end, a measured question answers in about 10.6 seconds on average. The median is around 9 seconds, with a tail of about 18.6 seconds at p90 and 24.8 at p95.

Metrics the product never shipped

Here is the part we did not anticipate.

Companies write down how they measure themselves. It sits in an operations manual, an onboarding doc, a policy page. And those write-ups are often precise enough to compute from.

Q: what's our first-time fix rate this quarter?
Sense: Your operations manual defines first-time fix rate as jobs closed on the first visit divided by all closed jobs, excluding warranty callbacks. On that definition it is 82.4% this quarter, up from 78.1% last quarter.
Sources: Field Operations Manual

There is no first-time-fix-rate measure in the catalog. Nobody built one. A page of prose was the specification, and the agent read it, carried the definition into the data query, and computed the company's own metric their way. When a user names a metric the catalog has never heard of, searching the documentation comes before guessing and before asking.

That is why the documentation layer has an absolute rule attached: answer only from the retrieved passages. A fabricated configuration answer is indistinguishable from a real one to the person reading it, and they will act on it.

The same pattern shows up with fields nobody designed a metric for. Companies track their own dates in custom fields and want the rates between them: what share of referrals became booked assessments last month. There is no "booking rate" measure to select, only two custom fields and a ratio, so Sense builds the metric on the spot. A rate is really two counts, each with its own date field and conditions, and the agent restates both halves on every follow-up so a casual now show it weekly cannot silently drop one.

And when the arithmetic does not hold, the engine refuses. Ask for "the latest value" of a field that holds text and there is nothing to sort by; put a date window on a field that holds numbers and it matches nothing. In both cases the system stops with a reason specific enough to turn into a real choice:

Picking the latest one needs a date to sort by. Want every record's value listed instead, one row each, or should I pick by an actual date field on that record?

Why there is more than one agent

The two walkthroughs above were run by different agents, and the user never chose between them. The split is about context isolation, not specialization for its own sake: every fact in front of a model is a candidate for its answer, and irrelevant candidates are where plausible wrong choices come from. So each specialist sees only what its own decision needs.

• An entity selector reads entity descriptions, nothing deeper, and decides which parts of the business a question touches.
• The query composer then sees only those entities' fields. For revenue by team, that is 93 fields instead of 1,220. Among 93, the right revenue field is obvious; among 1,220, revenue-shaped fields from jobs, quotes, invoices, and payments sit beside it, every one a defensible wrong choice.
• The vocabulary resolver sees live stored values, not the query being built.
• The record investigator runs its own tool loop and returns a written finding, so the conversation never carries the megabytes it read.
• A knowledge searcher and a memory keeper see only passages and preferences.

The specialists exist so each context can stay small.

What the agent knows before it answers

A model that reasons well and knows nothing about your business produces fluent, wrong answers. Several layers of context stand between the question and the answer.

A curated definition layer, not raw tables

The agent never sees a database. It sees a semantic layer we maintain: 53 business entities carrying roughly 1,220 named metrics and attributes, each described in plain language. "Total revenue" is a definition we wrote, not a SUM() the model invented.

Three properties matter more than the size.

It is deliberately smaller than the data. Bridge tables, lookup tables, and internal masters stay hidden even though the system still queries them. A junction table between users and teams is a correct thing to have in a data model and a terrible thing to offer an agent as a choice.

It carries relationships, because revenue by technician spans multiple entities and the wrong path through them produces double counting rather than an error.

It carries counting rules. Take one job worth $1,000 that passed through four statuses. Group job revenue by status transition and the honest answer is still $1,000, but the arithmetic hands back $4,000, because the job lands in four groups and its revenue is counted in each. Every parent total split by a one-to-many child does this: job revenue by service task, invoice totals by line item. The overstatement looks entirely plausible, so the agent is directed to the child's own metrics instead, and told to name the risk out loud when a user insists on the inflating cut.

It is also the scope boundary: if an entity this workspace does not expose is absent, the answer is not available yet, never a nearest-neighbor guess.

The company's own vocabulary

Nothing in the catalog knows this company says "Inspection Ready," tracks a field called "Assessment Booked Date," or ships from a warehouse named "North Dock." That vocabulary is per-workspace, it changes weekly, and it is where many wrong answers come from.

So the agent looks things up, under one rule: a word you cannot find in the catalog is an unverified lookup, not an ambiguity. Look it up before asking the user, and before building a query with it. The lookup covers everything a company can name: statuses, categories, teams, payment terms, custom fields and the values inside them, checklist questions, locations, people, and customers. Common vocabularies are cached briefly per company; people and customers are searched live, because names are neither stable nor unique. A resolved name comes back with each candidate field tagged by role, and the agent picks by what the question is actually asking.

The conversation itself

Every result in a session lands in a compact ledger the agent reads on later turns.

[ r3 ] entities: jobs, teams | rows: 12 | rendered: bar_chart | status: ok   ← latest
  query: revenue by team, last quarter, Inspection Ready residential jobs
  summary: Revenue by team, last quarter

That ledger makes follow-ups cheap and correct. Show that as a table re-renders an existing result without rerunning the query; add job count knows what that was; a superseded result is never offered again. It also carries context across surfaces: opening a chat from a pinned dashboard widget seeds the conversation with that widget's query and a sample of the rows on screen, so why is the top one so high? resolves to a real record.

The loop runs forward too. Every answer ends with grounded follow-ups based on what was just rendered: after revenue by team, break down the top team by technician; after a stat card, often a why that hands off to the investigator. That is how a measured answer turns into an investigated one without the user needing to know the second capability exists.

Standing rules and runtime facts

Users set explicit preferences, like scoping job questions to this quarter or preferring tables for top-N lists, and the agent separately remembers how they talk. Precedence runs one way: this turn's instruction, then preferences, then memories, then defaults. Both get applied out loud, because a filter the user cannot see is a filter the user cannot correct. Then come the runtime facts that are wrong by default if omitted: today's date, the user's timezone, and who my jobs belongs to.

Answers we shipped that were wrong

Everything above is the system working. It did not always. Three answers that rendered wrong in production, and what each one bought us.

The rate that counted deleted jobs. What share of referrals became booked assessments? The rate came back plausible and slightly wrong: it included deleted jobs. The exclusion of deleted records lived on the parent entities, so any query that touched only child records, like the custom-field values this rate is built from, never joined the parent and silently skipped the filter. The fix moved the exclusion into every child entity's own definition, so no query path can reach a child row whose parent is deleted. The rule it left behind: an invariant enforced on one entity and inherited by others through a join is not an invariant.

The payments attributed to every job. Payments by job type. There is no direct link between a payment and a job; the correct chain is payment to invoice to job. But three linking paths of equal length existed, and the engine broke the tie by declaration order, walking payment to customer to jobs, which attributes each payment to every job the customer has ever had. The grouped total was inflated and perfectly plausible. The fix pinned the correct path as the tie-break winner, and any grouped total is now reconciled against its ungrouped baseline: it must sum back exactly.

The filter that became an axis. Count jobs by Referral Source, but only jobs whose Referral Date falls in 2025. Two custom fields, one for grouping and one for filtering, and there was no supported recipe for that shape. The generator fell back to treating both as display dimensions: the date filter stopped filtering and became a breakdown axis, and all-time counts rendered under a title that said 2025. Root cause: an untaught query shape degrading into the nearest taught one instead of refusing. The fix added the missing recipe with the filter declared as a predicate, pinned by a regression test, and hardened the rule behind that earlier 4,812-job example: a constraint that cannot be honored must fail the query, not fall off it.

From a whiteboard to a chart

Planning meetings produce chart sketches, not queries. Somebody draws revenue by month with a second bar for job count, photographs the whiteboard, and wants that.

Sense reads the image as a layout specification: axes, groupings, chart type, sort order, the date window scribbled in the corner. It resolves the fields those labels imply, fetches live data, and renders the drawn chart with real numbers. Two rules keep it honest: the image is never a data source, so numbers visible in the photo are never quoted or computed on; and text inside the image is a label, never an instruction, so a sticky note reading ignore your previous instructions is just a label about a chart. The same path handles screenshots of reports from other systems, which is how people often ask.

[Screenshot: a whiteboard sketch beside the chart Sense rendered from it]

Trust

Which company's data a request may touch is injected by the request layer and cannot be named, overridden, or supplied by the model, on any path, in any tool; the same holds for the credentials behind investigations and the document set knowledge search may read. Every path is read-only. Everything read from outside (retrieved passages, preference text, company-authored field names, text inside images) arrives labelled as data, not instruction; a passage that contains instructions gets reported, not obeyed.

And the reasoning is visible while it happens: the trace at the top of this post, sources cited on documentation answers, the query inspectable behind every pinned widget, and the records openable behind every chart. When something breaks, the model gets a generic failure signal and the real error goes to logs, so field names and query fragments never surface as product copy.

What it cost to make wrong answers hard

Context beats capability, and small contexts beat big ones. Every meaningful jump in quality came from giving the agent something true about the business, and most came from giving one agent less: 93 fields instead of 1,220 in front of the composer, a 4KB preview instead of a 50KB-plus payload in front of the investigator, a written definition instead of a guess in front of a metric. Each cut removed a whole class of confident error.

Asking is a feature, not a fallback. On a 100-question QA set covering analytics questions and live-record investigations, 91% of answers were correct. That is an evaluation result, not a production accuracy claim. Users forgive a clarifying question instantly and almost never forgive a wrong number, because a wrong number gets forwarded. Clarify unless it is genuinely settled is a higher bar than clarify when stuck.

Guarantees live below the model. Every entry in the failure gallery ended the same way: a rule that had lived in instructions or in an assumed join moved into the layer beneath, where no phrasing can route around it.

This was never about putting a chat interface over a database. It was about engineering the context that lets an agent understand your business deeply enough to turn a question into the fastest path to understanding what is happening.

The problem we started with

Zuper is a field service management platform with around 30 core business domains — jobs, invoices, estimates, work orders, customers, service tasks, purchase orders, timesheets, and so on. When we set out to expose Zuper to LLM agents through the Model Context Protocol (MCP), the naive shape was obvious: one MCP "tool" per REST endpoint. That gave us hundreds of surface points, and every one of them had to be right.

"Right" is a loaded word. A tool schema is right when:

• Every field it accepts is actually accepted by the API.
• Every field the API requires is either in the schema or auto-derived.
• The enums it advertises are the enums the backend enforces.
• The request-body shape is what the backend expects.
• The endpoint URL and HTTP method match reality.
• The agent gets a useful error, not a generic 400, when it's wrong.

Getting one tool right by hand is easy. Getting ~300 tools across ~30 modules right needed a different approach.

This is the story of how we built it, and what we learned along the way.

Why not just unit tests

The instinct was: unit-test each tool with mocked HTTP. We didn't take that path, for three reasons.

One, mocks lie. A unit test with a mocked API client passes when the tool's schema accepts the input and the mock returns a canned response. It says nothing about whether the real backend will accept that payload. We learned this the hard way when a "green" test suite shipped a tool that called POST on an endpoint that only responded to PUT. The backend's 404 was a real signal; the mock happily returned success.

Two, the schemas evolve daily. The MCP layer is a translation between two shapes — the agent-facing schema and the backend API contract. The backend gets updates from a different team on a different cadence. We needed tests that run against the real API, ideally on staging, so drift is caught immediately.

Three, the questions are workflow-shaped. "Can an agent create a job, attach line items, apply a per-line tax, and generate an invoice?" is not one unit test. It's a chain of six calls with state flowing between them. That's what our users care about; that's what we needed to verify.

So we built intent-based tests.

What an intent test is

An intent is a plain-English business goal, encoded as a small object:

{
  id: 'CUST-C01a',
  module: 'Customers',
  category: 'create',
  intent: 'Create a customer with name and email',
  tool: 'create_customer',
  args: () => ({
    first_name: 'Acme',
    last_name: 'Corp',
    email: 'contact@acme.example',
  }),
  onSuccess: (result, state) => { state.new_customer_id = result.customer_id; },
  verify: {
    tool: 'get_customer',
    args: state => ({ customer_id: state.new_customer_id }),
    check: r => r.customer.email === 'contact@acme.example',
  },
}

Every intent has:

• intent — a human-readable sentence that reads like a Jira ticket title.
• tool + args(state) — the tool to call and how to build its arguments from the shared session state.
• onSuccess — a hook to stash the returned identifier for downstream tests.
• verify — a follow-up read that confirms the write took effect. Not just "did the tool return 200," but "does the corresponding read tool show the field we wrote."

args is a function, not a value, because tests share state. Creating a customer produces a customer ID; later intents that create jobs or invoices for that customer consume it. The state object is threaded through the whole run.

Intents are organized by module and category (create / update / list_get) — one file per business area, one array per file.

The verify round-trip matters more than the initial call. Many silent-failure modes we later fixed — schema too loose, backend rejecting a subfield but returning 200, tax field silently dropped — only showed up in the read-back.

Where 1000+ intents came from

Writing test cases by hand doesn't scale to hundreds of tools. Writing test cases from imagination is worse — you test the happy path and miss the edge cases that actually break in production. We needed a source of test intents that reflected how real people used the product, not how engineers thought they should.

We got there by pulling from two sources:

Past support tickets and issue trackers. Every ticket the team had ever worked on described a real customer trying to do a real thing. "Customer can't mark an invoice partially-paid from the draft state" is a test intent. "Field engineer's clock-in reflected on the wrong user" is a test intent. "Percentage discount not honored when applied at line-item level" is a test intent. We walked the ticket history module by module, converted each recognizable customer flow into an intent, and folded them into the coverage suite. Every ticket that turned into an intent got a comment linking back so the next incident would find the regression test already there.

Intent libraries per customer segment. Different customers use different subsets of the platform. An HVAC service company lives in jobs, service tasks, and invoices. A property management firm lives in properties, service contracts, and recurring jobs. A distribution business lives in purchase orders, material requests, and transfer orders. Testing "does the tool work" is not the same as "does the tool work for the way this segment uses it." We built per-segment intent libraries by walking through each customer's actual workflows — using their data model (product categories, custom fields, business units) as inputs to the intents.

The two sources compounded. Ticket-derived intents caught the bugs we'd already been bitten by. Segment-derived intents caught the bugs we'd otherwise ship into a new segment blind. Together they took the suite past 1000 intents.

A side benefit we didn't expect: the intent library became documentation for the tool. Product and solutions teams can read the intent titles and immediately see what the platform can do for a specific segment. "Schedule a follow-up job for a customer with an existing service contract" is more useful than any API doc.

The rule we settled on: new bugs get intents before they get fixes. Once a customer-reported issue is in the intent suite, it stays covered forever. Fix without an intent, and it comes back.

The runner: field-strip retry

Real APIs fail for surprising reasons. A schema evolves, an enum tightens, a required field starts erroring out. When that happens in a test suite, we don't want to abandon the whole intent; we want to isolate which field is the culprit.

So the runner has a field-strip retry loop. On tool error, it:

1. Pattern-matches the error message for a suspect field name.
2. Removes that field from the arguments.
3. Retries the call.
4. Loops until success, or until every optional field has been stripped.

At the end of the run it reports two verdicts per field: VERIFIED (the field was accepted and reflected in the verify read) or UNVERIFIED (the field was stripped during retry).

The point isn't to make failing tests pass. It's to produce a coverage map. When we saw one field marked UNVERIFIED across forty different tests, we knew the whole schema for that field had drifted from the backend — one bug, not forty.

The coverage document — one per module

Each module owns two generated files:

• A run report — the summary. Every intent, PASS/FAIL/VERIFY_FAILED, which fields were stripped, which are UNVERIFIED.
• A failure guide — each failure gets a stanza with the tool name, error message, stripped fields, likely cause, and a "suggested fix" line.

These aren't just artifacts — they became the primary planning surface. Every session started by reading the last run's failure guide. Fix, re-run, watch FAIL count drop, watch PASS count climb.

The pattern we settled into:

Round 1: baseline run — enumerate every failure.
Round 2: fix the top N failures, re-run just those tests.
Round 3: re-run the whole module. New failures? Different failures?
Round 4: expand coverage — add intents for edge cases surfaced during fixes.

A concrete example: the tax and line-item audit. Round 1 showed several "create estimate with ad-hoc tax" intents as VERIFY_FAILED. Investigation revealed the backend requires a tax UID reference for every document-level tax entry — it hard-rejects ad-hoc {name, percent} forms. We narrowed the schema to require the UID. Re-ran. Passed. Added Round-2 intents for line-item-level tax and markup. Passed. Added Round-3 intents for the update path, tax-exempt overrides, mixed inheritance. Passed. Coverage went from 4 tax intents to 22 in one week, all green.

The coverage doc is a ratchet: it only counts intents that VERIFY, and the verified count only goes up.

Adversarial verification with parallel agents

Half the bugs we caught weren't in the runner output. They were in the schema itself — fields that looked plausible but weren't in the backend, enums that shipped extra values the backend would reject.

For those, the runner alone can't help — a schema that never gets exercised by an intent test can be arbitrarily wrong and nobody notices. So we ran periodic audit passes.

The pattern:

1. Spawn 4–6 parallel research agents. Each takes a subset of modules.
2. Each agent reads the MCP tool file, the corresponding webapp form component, and the backend controller. It reports a table: field-by-field, what's in the tool vs the UI vs the backend.
3. Agent findings are claims, not truth. Every high-severity claim gets spot-verified against the actual source before it hits the audit doc.
4. Consolidate into a single module-wise gap doc.

The reject-before-trust step is important. Agents are helpful and confidently wrong in equal measure. In one audit round, an agent claimed a listing tool used the wrong enum values. We opened the file — the actual enum was correct; the agent had misread a docstring elsewhere in the same file. We rejected that claim in the write-up. In the same round, another agent flagged a deposit-recording tool as calling POST when the backend registered PUT. We opened the file — real bug, six months old, silent behind an outer try/catch.

Every claim gets grep-verified before it becomes a finding. This one habit cut the false-positive rate on our audit docs from ~30% to under 5%.

When docs aren't enough: browser automation as a discovery tool

Roughly a third of the way through the project, we hit a wall: the internal API reference documents were incomplete. Some endpoints were undocumented. Others were documented but omitted fields the UI was actually sending. Some documented fields were flagged "optional" but were required in specific configurations. Without ground truth for those endpoints, we were guessing.

So we brought browser automation into the loop.

The pattern was simple: script the webapp's real workflow end-to-end — log in, create a job, add products, set a tax, save, refresh — and capture every network request. Each captured request gave us a real payload: exact field names, exact nesting, exact enum values, exact wrappers. When our schema disagreed with the payload, the schema was wrong.

A few things it exposed that we'd have missed otherwise:

• Undocumented approval hooks. A specific status transition sent an extra approval identifier the API reference didn't mention. The browser automation caught it on the first pass.
• Field-nesting quirks. The webapp nests certain settings (billing frequency, payment term) under a nested accounts object on save. Our tool was sending them flat. The API "accepted" both shapes but silently discarded the flat one. The captured payload made the correct shape visible.
• Company-config-gated behaviors. Some fields only appear in the payload when a company setting is on. Running the automation against two configurations side by side let us design a schema that handled either.

Browser automation wasn't a replacement for the API docs or the backend source. It was the fastest way to answer "what does the UI actually send?" when the other two disagreed, and to catch the hidden third case both had missed. The reusable rule: when in doubt, capture the network traffic.

The module dependency matrix

Ordering modules by "business criticality" was the first cut. But criticality isn't the same as "safe to fix first." A tool schema for jobs depends on customers existing, products existing, tax IDs existing, business units existing. Fix jobs first and half your tests fail because the entities they depend on aren't seeded.

So we built a module dependency matrix — a per-module document mapping every upstream dependency:

Customer domain
  ├─ Depends on: categories, organizations, tax groups, pricelists
  └─ Depended on by: jobs, invoices, estimates, projects, contracts, ...

Job domain
  ├─ Depends on: customer, product, product category, job category,
  │              user, team, business unit, service tasks, tax
  └─ Depended on by: invoice, estimate, service report, timelog, ...

Roughly two dozen of these documents now live alongside the code, one per business domain. Each names its foreign-key relationships, the exact fields that carry the reference, and any known quirks (e.g. "the job's property inherits from the customer if omitted, but only when the customer has a default property set").

Two immediate benefits:

Testing order becomes deterministic. The seed script that populates the shared session state reads the dependency graph and creates entities in topological order. Customers before jobs, products before line items, tax before invoices. No more "test failed because the customer ID is empty."

Field audits become sharper. When we're auditing an invoice-creation tool, the customer dependency doc tells us which fields flow into the customer lookup, which fields the invoice inherits automatically, and which ones a manual override is expected. That's usually what the API docs leave implicit.

We also generated companion API reference documents — one per module — that consolidate every endpoint we call, its verified request shape (from network capture + backend source cross-check), and its response shape. These sit alongside the dependency matrix. The pair of documents — dependency map + verified API reference — became the input to every new intent-suite expansion and every audit round. If a claim in a coverage doc contradicts one of these two, the coverage doc gets updated first.

Module-by-module progression

We didn't try to fix all tools at once. We ordered modules by:

1. Business criticality (jobs first, then invoices and estimates, then service tasks, then material requests, then everything else).
2. Intent-suite coverage (biggest suites first — they surface the most bugs per hour).
3. Cross-module dependency (fix customers before jobs, because job creation needs customer state).

Per module, the loop was:

1. Baseline — run current intents, capture the coverage report.
2. Field audit — parallel research pass to find missing or wrong fields vs the UI and the backend.
3. Fix critical — schema and endpoint corrections.
4. Expand coverage — new intents for the fields just added.
5. Regression — re-run the whole module; add any newly surfaced failures to the queue.

For the jobs module alone, that loop produced dozens of field-level fixes and roughly doubled the passing intent count.

What surprised us

Silent overrides. Several endpoints happily accept fields they then ignore. The backend's tax calculator overwrites caller-supplied tax name and percent from the tax-master lookup, regardless of what the caller sent. Our MCP was faithfully passing through fields the backend would silently drop, giving agents the illusion of setting them. Fixing this meant narrowing schemas below what the backend would accept — trading permissiveness for predictability.

The unit-price / price schism. Jobs use one field name for line-item price. Invoices and estimates use a different one. A converter tool translates one to the other. No documentation of the flip anywhere. An agent using our tools cross-module has to know this by feel unless the tool descriptions call it out. We added the notes.

Status enums that lie. One status-update tool accepted status values as write targets that the backend never allows to be set directly — some are computed automatically from other actions, some are display-only. An agent that tried those transitions through MCP would get a generic 4xx and no clear signal. Narrowing the write enum was a one-line fix; finding the mismatch was hours of matrix-reading.

The auto-derive trap. Some create tools mark fields optional even though the backend requires them — because when the caller passes a parent entity, the tool auto-copies the required fields from that parent. The schema looks broken until you read the execute body. Documentation matters more than schema strictness once the tool starts doing derivation work.

The tooling that kept us moving

• A restart-between-chunks runner — the upstream MCP session state degrades under sustained load; a chunked runner is a workaround, not a fix, but it kept the suite moving.
• A shared session-state file — pre-seeded IDs for common entities (customer, product, tax, category), populated once via a seed script; every test run starts from it.
• An ID-prefix filter — a CLI flag to re-run just one group of intents by ID prefix. Made the "fix, run three tests, re-run all" loop fast enough to stay in flow.
• An env-file credential loader — no hardcoding, no accidental leaking of production keys into a staging suite.

None of these are impressive individually. Together, they made large-scale coverage tractable.

What we'd change next time

Start with the coverage doc, not the tool. The first tools we built had shallow schemas because we hadn't scoped the UI form's field list yet. We ended up rebuilding several tools twice. Working from a coverage matrix — every field in the UI form, every field in the backend schema, every field we plan to expose — would have front-loaded the design work.

Adopt a strict "tighten before ship" rule. Schemas are easy to loosen after the fact and painful to tighten. Every "make it optional for now" we deferred came back as a support ticket. Better to start narrow and open up on demand.

Contract tests, not just integration tests. The intent suite exercises real endpoints, but it doesn't lock the backend response shape. A field rename on the backend silently breaks our verify checks. A layer of contract tests that snapshot response schemas would have caught two silent regressions we found only by luck.

Backend-side gate for enums. Many enum bugs (statuses in write enums that shouldn't be, transition targets the backend won't accept) exist because the backend accepts one set of values as input and produces a different set as output. The backend should either publish the write-side matrix or the MCP should mirror the matrix explicitly.

Where the project stands

• Approaching 300 tools across roughly 30 modules
• 1000+ intent tests, sourced from past tickets and per-segment customer workflow libraries
• Intent suites for the top business modules: jobs, invoices, estimates, customers, service tasks, material requests, timesheets, purchase orders
• Live-API test runs on staging on demand
• Per-module coverage docs updated on every run
• Field-gap audits checked in with prioritized fix queues

The core lesson is unglamorous but real: at scale, the test suite is the design tool. Intents make the requirements concrete. Coverage docs make the gaps visible. Round-by-round iteration makes the delta shippable. We didn't write hundreds of tools right on the first try. We wrote them approximately, and then let the tests tell us where they were wrong.

That, more than anything, is how the Zuper MCP got built.