A gate that checks every query before it's answered. One cheap call decides whether it's safe, how hard it is, and which model should handle it. Nothing reaches a model until the gate clears it.
A gateway that sits in front of a support team's LLMs. One parallel call scores each query for security risk, intent and complexity. Simple questions go to a fast, cheap model, hard ones go to Claude, and anything that looks like an attack or contains personal data is blocked before a model sees it.
Without a gate, “what are your hours?” costs as much to answer as a multi-step billing dispute, and both carry the same risk of leaking a system prompt or a card number. On a 250-query benchmark this cut cost by 68% and median response time by 64%, and blocked all five adversarial test queries. That makes an AI support queue affordable to run at volume.
One judgment instead of three chained models.
Security, intent and complexity are separate questions, but they don't need separate calls. Jev answers all three in parallel on the same input for a fraction of a cent. A slow or expensive gate would just become a new bottleneck. TypeSafe's published benchmarks put it at 20 to 200x faster and 40 to 400x cheaper than a general-purpose LLM on this kind of classification. Those are their numbers; I haven't reproduced them.
Jev answers three questions about each query, and those answers decide where it goes. The two kinds of mistake aren't weighted equally. Missing a jailbreak or a PII leak is far worse than an unnecessary block, so the security check has a much lower threshold than the routing check.
Taken from the test suite rather than written for this page. Each outcome is what the build returned when it ran.
The obvious first version is a small LLM with a classification prompt in front of everything. That works until you price the gate itself at volume. At that point the guardrail starts to cost as much as the spending it was meant to prevent.
That's TypeSafe's published pricing. On this build the gate call finishes in about 0.2 to 0.4 seconds, measured. Any more latency or cost and the guardrail becomes the bottleneck.
The security check has its own threshold. A missed jailbreak or PII leak is worse than an unnecessary block, so it trips earlier than the routing check does.
There's no prompt template to maintain and no JSON to parse and hope for. Three typed questions go in (a yes/no, a choice between options, and a score on a scale), and typed answers come back with confidence scores. That's the whole interface.
Verbatim from router.py. The highlighted security check runs first, on its own threshold, separate from the routing confidence check below it.
Scoring security and routing confidence against the same 0.50 line would be a decision nobody made on purpose. Missing a jailbreak costs more than an unnecessary block, so the two thresholds are different numbers, named as constants, with the reasoning written next to them.
The first draft of this page quoted a cost saving and a latency figure before any benchmark had run. I pulled both until a 250-query test set existed, weighted toward the mix of traffic a support queue really gets.
An org-scoped API key isn't the same as a workspace-scoped one. No Claude call worked until I added an explicit workspace header.
The first content block in a Claude response isn't always text. A leading reasoning block broke content[0].text until I filtered for the text block.
Two queries I'd labelled “ambiguous” came back from Jev with full confidence. On a second look, the labels were wrong and the router was right.
Local inference isn't automatically fast. A local model took 7 to 16 seconds a call on my hardware, slower than Claude, so I moved the cheap tier to hosted inference on Groq.
The routing logic passed its five unit tests on the first try. Every real failure happened one layer out, in the connections to Jev, Groq and Claude. That's where the field notes above come from, and a logic-only test suite would have missed all of them.