Just now, the wildly popular Jev model has been fully opened to the public, and 120 million tokens are provided as a free benefit for all users.

Just now, the recently viral large model Jev announced that it is open to all users with no waitlist application required. All registered users will receive a $5 credit, which is equivalent to around 120 million Tokens.

It is worth noting that Jev consumes far fewer Tokens than ordinary large models. Some netizens have tested that processing 1 million input Tokens only costs 0.042 cents, so the 120 million free Tokens are more than enough for unrestricted use.

Experience link: console.typesafe.ai

No one expected that in September, when large models are fiercely competing, even GPT and Claude cannot firmly hold the top spot. The most popular model that has gone viral all over the internet recently is Jev, which cannot generate conversational text.

Diogo Almeida, the developer of Jev, is a researcher at OpenAI. He participated in the R&D of ChatGPT, and later proposed the Reinforcement Learning from Human Feedback (RLHF) algorithm.

To some extent, this has defined the development direction of large language models in the past few years. However, while this method has pushed the whole industry to prosperity, Almeida has become increasingly skeptical about it.

"We had a one-off successful innovation, but we didn't turn it into something truly useful."

It took him a long time to figure out where the problem lies: We have been optimizing the processing capability of human language... Over the past four years, we have performed extremely well in processing human language, but this is not useful for automation, because computers use a different "language".

Two years ago, Almeida left OpenAI and founded TypeSafe AI together with Erik Gafni and Sasha Sheng. The company has been operating in stealth mode until its official debut on September 15, bringing two things at the same time: one is a $40 million seed round financing led by DCVC, and the other is their first model, Jev.

Jev is still a Transformer-based model, but it is deliberately not a large language model, and will not output a complete sentence.

When you give it a piece of program state and a predefined question, it returns a typed answer, an option, a score, or a probability between 0 and 1, plus a confidence score.

TypeSafe calls this output "calibrated decision". This is also the reason why many people were confused at the first sight of the name Jev when it first came into the public view.

The screenshot shows the performance comparison between Jev and large language models when responding to the same request

The most popular model in the AI circle follows a completely different path from ChatGPT

To understand what Jev is doing, you may as well put aside the thinking habits of large language models first.

It is easy to understand when we take the customer service scenario as an example. A user sends a message: The payment service has been disconnected for several consecutive days, which has affected the business. At this point, several judgments need to be made: whether it should be handed over to the technical team or the finance team, how dissatisfied the other party is, and whether the matter is urgent.

According to TypeSafe's interface design, questions in the same request share a copy of input, but are evaluated independently and in parallel. Classification and urgency assessment can be processed at the same time.

Specifically, its output has three basic forms. The first one is called Choice, which selects one item from a pre-defined list, supports up to 255 options, and is suitable for routing and classification. The second one is called Score, which scores the input on a predefined scale to measure urgency, quality or risk. The third one is called Noul, which is essentially a yes-no judgment, and the answer is a number representing the probability that the matter is true.

Each response comes with a complete probability distribution and confidence level, which is a strongly typed result. There is no need to write JSON prompts, no additional parser is required, and you don't have to worry about the model suddenly wrapping the result in a Markdown code block.

Adopting this method, the most direct benefits are time saving, cost reduction, and fewer format errors caused by free generation.

Figures released by TypeSafe show that Jev's end-to-end latency ranges from 70 to 500 milliseconds, which is 20 to 200 times faster than similar large language models.

According to the current public price, the input cost is $42 per billion tokens, which translates to $0.042 per million tokens under the common statistical standard; the output is free of charge. Newly added questions and option descriptions will still occupy input tokens, but no extra fees will be charged according to the length of the generated answer.

Since users can pre-define the output results, the model will not generate hallucinations. (That Jev does not generate hallucinations means it will not give random answers beyond the pre-set option range, but it is still possible to give wrong answers within the option range.)

In a public Ably Pong demo, Jev made 47 operation decisions in 12 seconds, while Gemini, Claude and GPT only made two or three in the same period, even though the latter still gave correct judgments in most cases.

In the Ably Pong demo, the program directly passes the numbers in the game to Jev: the position of the ball, the movement direction, the position of the racket, and the vertical coordinate of the ball when it is expected to reach the racket's position. Jev can select one of the three options of "move up, move down, stay still" based on these numbers.

What are the usage scenarios of Jev

To build a playable product with Jev, you need game programs and real-time communication. The above project is promoted by the back end to run the game, call the model, and then send the new state to the browser through Ably. In other words, the game rules, graphics and networking are not generated by Jev, it is only inserted into the repeatedly executed three-option selection link.

The speed advantage expands the application scenarios. Jev can act as the judgment layer for computer operations, guiding agents to execute instructions quickly; it can also be used for context compression to judge which content is key information, so as to quickly narrow down the context of millions of tokens.

The public project jev-ultrafast of Browser Use specifically implements a browser agent. Every time it opens a web page, it first reads the currently operable page elements, and organizes them into a numbered list: which one is a button, which one is an input box, what is their name, and what content is filled in now.

Jev receives this structured state, user goals and operation history. The selection space given to it by the program is also dynamically generated by the elements actually existing on the current page.

Use Jev as the operation judgment layer to guide the model to quickly operate your computer https://x.com/gregpr07/status/2100411066966749359

Take booking flight tickets as an example, the goal can be "Find one-way flights from Zurich to London, set the specified date, number of passengers and cabin class, and stop when eligible results appear".

In each round, the program will ask several questions at the same time: Should I click, input, select a drop-down option, or wait next? If you need to click, which number should you click? If you need to input, which number should you fill in? These questions share the same web page state, but are answered separately.

Finally, the program only adopts the target matching the actual operation: if clicking is selected, the click target will be used, and other answers will not be used temporarily.

When it is necessary to input the city name, the program will call a small text generation model separately, and let it generate the content to be filled according to the target and the current input box. After the browser executes the operation, it reads the new state of the page.

Jev judges the key information of the context content to achieve near-instant context compression https://x.com/tamarajtran/status/2100694549362553153

Pranit Sharma, a software engineer at Vercel, said that his company used OpenAI's ChatGPT Luna 5.6 to run a classifier to check the security of commands. After Vercel replaced OpenAI's Luna with Jev, the processing speed increased by 5 to 18 times, and the accuracy was also greatly improved.

Nikhil Mudholkar, CTO of another developer Bryo AI, also tested Jev and Gemini to evaluate their performance in classifying business emails. In his test, Gemini had slightly higher accuracy, but the cost was 10 to 20 times higher.

We can also build it by ourselves according to this idea. We can first prepare a table, each row containing the email title, body, receiving time and necessary context.

Then split the business into several clear questions. For example, for an email that says "The order has been deducted repeatedly, and I hope it can be processed today", we can use Choice to judge whether it should enter the after-sales, sales, cooperation or other queue; use Noul to judge whether the sender clearly requests action; then use Score to judge the processing priority.

"In the final analysis, this approach shifts the responsibility for handling the hallucination problem to the user a little bit," explained Armin Ronacher, CTO of Earendil. "Users need to decide: if the probability of this situation is only 50%, then maybe it can be ignored. But if the probability reaches 95%, then I can make use of it."

Ronacher said that another potential application of Jev is model routing. It is very useful to predict whether a task requires a specific model, but using a large language model to complete this task will be very expensive. Jev is low-cost and runs fast, so it can realize this real-time routing function.

Automatically allocate the appropriate model to process tasks according to task complexity https://x.com/mdlahfir/status/2100314182201802811

The wisest decision

The name Jev is taken from William Stanley Jevons, an economist in the 19th century. The paradox put forward by Jevons points out that when the cost of a certain commodity decreases, this commodity will be used more and more widely. In this case, the decline in the cost of intelligence should lead to the widespread application of intelligence.

In other words, once the invocation becomes cheap, it will be used in more places that were not worth using before. Almeida is obviously very satisfied with this analogy. The future he envisions is not that several huge applications monopolize everything, but a large number of tiny intelligent judgments running scattered everywhere, "more like the early Internet, rather than the kind of large applications that people are trying to build now".

TypeSafe has not made public the specific architecture of Jev so far, and it is widely speculated that it is modified based on an open-source large language model.

The company itself calls it System One Models, which is taken from Kahneman's concept of intuitive thinking, emphasizing that it relies on intuitive judgment rather than reasoning chains, and is specially tuned for specific tasks.

The official suggestion is to split complex judgments into several clear questions and combine them with code, instead of letting the model handle everything in a rush.

Almeida revealed that, he predicted very early that he would embark on the path of processing synthetic data, which is perhaps the wisest decision he has ever made in his life, wiser than the company going public, and even wiser than relying on real human feedback.

At present, TypeSafe is the only company in the market that adopts this route, but Ronacher expects that as the practical value of this model is gradually verified