TypeSafe recently released Jev, a model for making fast, structured decisions. The company reports response times of 70 to 500 milliseconds, with its workflow benchmarks showing up to 193.6x faster execution and 444.6x lower cost than the LLM configurations it tested.
For a QA agent making decisions throughout a test run, that matters. Reducing that total cost and time needed for response means you can run more tests within the same budget and get feedback sooner.
The demo below pairs Jev with agent-device, our open-source tool for letting agents inspect and interact with apps. Jev chooses the next action, and agent-device carries it out.
The run above took 14 seconds and cost $0.0023 in model inference.
What Jev does
Jev is TypeSafe's first System One model. You give it context and questions with defined outputs. For a question such as "What should happen next?", those outputs can represent adding a product to the cart, opening the cart, or stopping the run.
The model selects an answer, and your code decides what to do with it. Jev also returns probabilities for the available choices, so the application can account for uncertainty.
This is where some of the speed comes from. TypeSafe evaluates questions in parallel and returns structured values directly. Jev doesn't generate a written explanation or a sequence of tool calls. The surrounding application supplies the actions and controls the loop.
For mobile QA, that means we need to turn the app into something Jev can read and give it a useful set of choices.
agent-device provides the app state
Apps expose information about their interface through accessibility APIs. That includes things such as button labels, text, and the values of form fields. agent-device reads this information and presents it as a snapshot, with references we can use to interact with individual controls.
A simplified product screen might look like this:
@e2 [text] "Canvas backpack"
@e3 [text] "Cart: 0 items"
@e4 [button] "Add to cart"
@e7 [button] "View cart"There is enough information here to make the next decision. The runner can tell Jev which product is open, what is in the cart, and which controls are available. If Jev chooses the Add to cart action, the runner presses the button at @e4.
Jev currently accepts text and structured data. It does not read screenshots or video, which makes this connection to the app's accessibility information useful.
Building the available actions
In the sample, we turn each snapshot into a set of actions. A button or switch becomes a press action. An editable field can become a fill action. Controls known to be hidden, disabled, or blocked are left out.
{
"id": "a10",
"kind": "press",
"ref": "@e4",
"target": "Add to cart",
"description": "Press button \"Add to cart\" at @e4."
}Each action has a description for Jev and the details the runner needs to execute it. Its kind identifies the operation, and ref points to the control.
For text entry, we include the value in the action and its description. Jev doesn't generate strings; it chooses an action with the text already provided. For a Search field at @e8, that could look like this:
{
"id": "a11",
"kind": "fill",
"ref": "@e8",
"target": "Search",
"text": "Canvas backpack",
"description": "Fill textfield \"Search\" with \"Canvas backpack\"."
}Jev selects the action, and agent-device types its text into the field identified by ref.
Other actions control how the run proceeds, such as scrolling to reveal content or waiting for loading to finish. These choices are available alongside the controls found on the screen.
Choosing the next action
We send Jev the current snapshot (screen), your task (prompt), and the previous screen and action (previousScreen and previousAction) so it can see what changed.
Using the TypeSafe JavaScript SDK, we turn the actions built above into choices and ask Jev which one to take:
import { TypeSafeClient, choice } from '@typesafe-ai/sdk';
const jev = new TypeSafeClient();
const options = Object.fromEntries(
actions.map(action => [action.id, action.description])
);
const { answers } = await jev.systemOne({
model: 'jev-latest',
state: { task: prompt, screen: snapshot, previousScreen, previousAction },
questions: {
nextAction: choice('Choose the next action for the task.', options),
},
});
const actionId = answers.nextAction.choice;actionId identifies the selected action, such as a10 for Add to cart. You can find the full request and runner in the repository.
If Jev selects Add to cart, agent-device presses the button. The runner reads the updated screen and repeats the process.
Jev can also choose pass, fail, or incomplete to end the run. It makes that decision from the current and previous screens and the previous action, based on what your task asks it to verify.
Try it with your app
We've packaged the proof of concept as a CLI so you can try it yourself. Clone the repository from GitHub and set TYPESAFE_API_KEY in your environment.
Here's an example using Settings on an iOS simulator:
npm run qa -- \
--app com.apple.Preferences \
--platform ios \
"Open Accessibility, then Display & Text Size. Scroll to Smart Invert and verify that its switch is visible. Do not change any settings."Replace the app ID and task to try it with your app!
PS. If you'd prefer to use an LLM instead of Jev, check out our QA demo built with Vercel Eve.
