Agents don't fail loudly. They drift.
TraceHarness watches every step an AI agent takes: each tool call, MCP call, edit and test. It spots the behaviors that come before a failure and steers the agent back before it ships the wrong thing.
Same agent. Same task. A different run every time.
Agents are nondeterministic. Run one task fifty times and you get fifty trajectories: some efficient, some wasteful, some quietly wrong. A pass/fail score hides all of it. The trajectory shows you where things went wrong.
Swing in tokens per solved task for the same model, just from changing the harness around it.
Drop in pass@1 for every model family we tested when the context window was shortened.
Baseline failures that were submitted with every visible test green. The agent believed it was done.
Measured with HarnessLab on 432 live runs: three model families, six harness configurations, eight tasks, three repeats.
The behaviors that come before a failure.
Every detector is a named rule with a reason and a suggested fix, so a warning always says what the agent did and what it should do instead.
Submitting untested work
The agent edits code and hands it in without running the tests again.
Test tampering
It edits or deletes the tests so they pass, instead of fixing the code.
Destructive shell
It runs rm -rf, force-pushes, or other commands you can't undo.
Repeat loops
The same call with the same arguments, again and again, without making progress.
Persistent failing tests
The same test keeps failing across attempts while the agent keeps going.
Half the budget, no edit
It spends half its budget exploring and never touches the code.
Scope creep
It changes files that have nothing to do with the task.
Boundary probing
It tries paths, secrets or tools that are outside its sandbox.
Budget burn
Tokens or cost climb much faster than the task is getting closer to done.
Stopped using tools
The model stops acting and starts writing prose about what it would do.
From a raw trace to an intervention, one step at a time.
TraceHarness checks the partial trajectory at every step, before the agent's next tool calls run. It can let the call through, add a nudge, block it, or stop the run.
Record every step
Stream spans live from your harness, or import sessions your coding agents already wrote to disk.
Keep one ledger
Steps, tokens, edits, tests and cost for every run. You see outcome distributions, not a single score.
Score the trajectory so far
Named rules and a trained risk model estimate, at each step, how likely the run is to end badly.
Act before the call runs
Nudge, block or abort. Every decision is logged in the same ledger, so you can audit it later.
Pattern detectors
every stepNamed rules, each with a reason and a suggested fix. They cost nothing to run and are easy to read.
Risk model
every stepA lightweight model over 26 features of the partial trajectory. It's cheap enough to call on every step and trained on your own finished runs.
LLM sentinel
on escalationAn optional second model reads a compact version of the trajectory and suggests a fix. It runs only when the cheaper layers flag a problem.
of eventual failures flagged, on 500 real SWE-agent runs
false-alarm rate at that threshold
steps before the end of the run, on average
Your next user is an agent.
We help companies move into a new era where AI agents use websites and apps as first-class users, calling structured WebMCP tools instead of scraping the screen.
Expose
We turn your product's key actions into WebMCP tools, with clear names, schemas and descriptions agents can follow.
Test
We run real agents against your tools hundreds of times and show you which tools confuse them, where they loop, and what they misuse.
Observe
We trace agent traffic in production, so you can catch bad behavior before a customer does.
WebMCP is an emerging W3C community proposal, currently in a Chrome origin trial. We keep your integration current as the spec changes.
- search_flights JFK→SFO42 ms
- search_flights JFK→SFOrepeat
- filter_results nonstop18 ms
- select_seat aisle31 ms
- confirm_bookingneeds user ok
// your page, one tool document.modelContext.registerTool({ name: "search_flights", inputSchema: { /* from, to, date */ }, async execute(args) { /* … */ } });
See what your agents are really doing.
We're working with a small group of teams that run agents in production or are getting their products ready for agent users. Tell us what you're building.