Letting Jev make an AI agent more efficient
- The question
- Can a small, cheap model that only classifies decide what enters a large model's context, without making the agent worse?
- What came out
- Yes, and it pays: 29% cheaper on average, 61% on the best task, with 99% of quality checks passing in every arm.
What I wanted to find out
In July I argued that small models should decide what big models see, and ended it by saying the economics were a reason to experiment rather than proof of anything.
Jev shipped on 15 September. It writes no text at all, it returns calibrated probabilities over answers your code defines, and it costs $0.042 per million input tokens. That made two very concrete versions of the idea cheap enough to actually measure, so I tried them:
- Preload the skills. Let it read the request and the one-line description of every internal document, and inject what matters before the agent’s first turn, so the agent never spends a round trip fetching them.
- Filter the tool payload. Let it score every field of an API response and strip the ones the agent will not act on, before they enter the context window and start being re-sent every turn.
Both are classification problems. Neither needs a model that can write.
How it was tested
A fictional B2B SaaS with 17 tools whose payloads copy the real shape of Stripe, Zendesk, HubSpot and a help-centre CMS, 11 internal policy documents of which several bear on nothing, and 8 tasks spanning two axes: how many turns they take, and how much of their payload is noise.
Four arms, six repetitions each, 192 runs. Grading is deterministic: 59 checks, each one either a tool call with exact arguments or a fact that appears only inside a policy document. No LLM judge anywhere, so “did the agent actually have the policy” is measurable rather than arguable.
What came out
Mean cost of one agent run, priced on Claude Opus 5
The mean is 29% cheaper. The best task is 61%, the worst 13%. Turns fall from 4.44 to 3.71, and fresh input tokens, the expensive kind, fall by 49%.
The two mechanisms answer to completely different things, which matters more than the average if you are deciding whether to use either:
Preloading responds to removable turnsNot to payload size
- Best where a task needs one policy and would spend a turn fetching it
- Worth nothing where no policy applies
- Can go negative: a wrongly loaded document is re-sent every turn
Filtering responds to payload densityNot to task length
- Best where most of the bytes are text nobody reads
- Near zero where every field is load-bearing
- Compounds, because a removed field is not re-sent next turn
What I take from it
A field is not useful in itself. It is useful for what the agent is about to do with it, and that information is not in the payload. Telling the scorer which tools the agent can still call moves the field the task depends on from 0.18 to 0.73, while cutting the number of fields kept from 51 to 27. Richer context makes the filter safer and more aggressive at the same time.
The price of your main model decides whether filtering is worth it. It costs a fixed number of cheap tokens and saves a variable number of expensive ones, so the sign flips. On a $0.20 per million model it costs 21% more than it saves; on Opus 5 it saves 29%. The crossover is around $0.55 per million with prompt caching on.
A stripped field does not produce an error. It produces a confident wrong answer. An early version dropped an account id and its duplicate, and the agent acted on the only identifier left, which was the wrong object, then filed a clean report about it. Anything deployed needs declared critical paths and a threshold calibrated against them.
Two starting points were tested here. Choosing which files an agent opens, deciding when to compact a conversation, gating a write before it happens: none of that was measured, and I would not assume any of it works until it is.