Measuring agent quality beyond resolution rate
Written by
Aahan Sawhney
Category
Research
Date
Share this article
Revisions, not edits
When we shipped the first version of Workflows, the authoring model was simple: one canonical definition, edited in place, deployed on save. That simplicity carried teams surprisingly far, right up until two operators edited the same escalation path in the same afternoon and production traffic split between drafts neither of them had finished.
Today we're releasing a revision model that separates what your agents are running from what your team is writing. Every workflow now has a frozen active revision and any number of drafts. Edits fork a draft; publishing is an explicit, reviewable act. Nothing reaches a customer conversation until someone decides it should.
Compile-time gates
Drafts are cheap because the compiler is strict. Before a revision can be published, it passes a gate suite: unreachable steps, unbound context variables, tool calls without fallback paths, and handoff targets that no longer exist all fail the build. The gates run on every save, so authors see problems while the fix is a keystroke away, not during an incident review.
The three most common catches so far: stale knowledge-base references, orphaned approval steps, and context variables renamed on one branch but not the other.
What we measured in beta
Over four weeks of private beta across forty teams, gates caught an average of 3.2 would-be production defects per team per week while the full suite stayed fast enough to run on every keystroke-level save.
Defects caught: 3.2 per team, weekly.
Compile time: 412ms p95 for the full gate suite.
Rollbacks: 0 across 40 teams.
A graph you can actually read
The new editor renders workflows as a directed graph with the same density rules as the rest of the dashboard: hairline edges, status dots for gate results, and identity tags for team ownership. Long branches collapse to their entry and exit nodes until you need them. The goal is that a reviewer can approve a revision by reading the diff of the graph, not by replaying the author's edit history.
"Publishing became a decision instead of a side effect. That one change ended our Friday-afternoon freeze."
Platform lead, beta customer
What's next
Revision-aware simulations are in progress: every draft will run against your persona suites before the publish button lights up, so the gate suite grows from static analysis to behavioral regression testing.
Availability and rollout
Workflows 2.0 rolls out to all workspaces over the next two weeks. No migration required. Your current definitions become the first active revision automatically, and drafts are included on every plan.
Share Blog
![](data:image/svg+xml,<svg display="block" role="presentation" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path d="M 16 7.999 L 16 0 L 0 0 L 0 7.999 Z M 16 16 C 13.878 15.999 11.843 16.842 10.343 18.342 C 8.842 19.842 7.999 21.877 7.999 23.999 L 0.001 23.999 L 0.001 23.794 C 0.11 15.086 7.176 8.054 15.896 8 L 24 8 L 24 24 L 16 24 Z" fill="rgba(255, 255, 255, 0.03)" height="24px" id="lngUwvl9j" width="24.0000007312145px"/></svg>)