Both Codex and Claude Code have gone astray, a former OpenAI researcher claims that the AI world was a tragedy before the emergence of Jev.

Compiled by | Qibei Lin, Fangfang Cai

Planned by | Tina

"It will gradually recede into the software background, becoming a common capability that developers can call at will, just like databases, regular expressions or other infrastructure."

This is the vision of Diogo Almeida, CEO of TypeSafe and founder of Jev, for AI: when intelligence is truly integrated into software, users don't even have to be aware of its existence. But how can developers safely hand over judgments to AI without having to stay by and repeatedly check the results?

A few months ago, in a speech at the AI Engineer Conference, this researcher who once participated in the work of OpenAI InstructGPT proposed that "the next era will not be the Claude Code era". In his view, Claude Code and Codex essentially still belong to the same stage — the main role of AI is still to provide assistance around humans. What he cares about is why AI, which is so smart that it can solve the millennium problems in mathematics, cannot automate even the most basic tasks?

On September 22, in an interview on the Latent Space podcast, Diogo once again talked about this dissatisfaction, using more direct wording: "In my opinion, the AI world before Jev was simply a tragedy." In his opinion, the problem is: the powerful "intelligence engine" already exists, but the interface to connect it into actual business processes is still lacking.

This time, he brought his own answer — Jev. Through Reinforcement Learning for Calibrated Decision (RLCD), the team hopes to make the model output choices, scores and probabilities that can be directly used by code, so that developers can set thresholds based on uncertainty to decide when to execute and when to hand over to humans for processing.

From participating in training models that understand human instructions to trying to make code call intelligence directly, why did Diogo reselect the training target? How will he make AI a common software capability like a database? This two-hour interview tells his judgments and attempts.

Too Long Didn't Read Version:

Swyx: For those who may not know the situation well or just want exact answers, what exactly is Jev?

Diogo Almeida: The most accurate name for it at present is System 1 model, whose capability scope far exceeds decision-making itself. The goal of this type of model is to make code the direct user of the model's outputs.

Swyx: One of your core views on RLHF is that the model's responses tend to cater to what users want to hear, rather than reflecting its real internal confidence in a matter. Could you elaborate on this?

Diogo Almeida: Almost no one notices the shortcomings of RLHF, especially "mode dropping". The mode dropping of RLHF will make the model biased towards generating safer and more common answers at the expense of the real calibration of the probability distribution. This not only masks the accumulation of errors in long texts, but also explains why text models are not good at decision-making.

Swyx: What is the core difference between RLCD and RLHF?

Diogo Almeida: The first difference lies in the optimization objective: RLHF focuses on making the model follow human instructions and give recognized responses, while RLCD hopes that the model can become a capability that software can reliably call.

Swyx: Why doesn't Jev directly build the refusal mechanism into the bottom layer of the model?

Diogo Almeida: I don't object to safety itself, but I don't think specific value judgments should be directly written into the underlying capabilities of general models. Instead, we should draw a clear boundary between technical capabilities and specific usage responsibilities like what databases do.

Swyx: What do you mean by reliability? Can developers trust that the behavior of the same version will remain stable?

Diogo Almeida: I pay more attention to robustness: if the meaning of the problem does not change, the judgment should not be drastically changed just by adding irrelevant characters, instead of only pursuing the same output for the same input. We will not secretly modify deployed models, because APIs are dependencies in other people's programs. But we will release new versions quickly, and we cannot promise to permanently maintain every old version for now.

Swyx: Without relying on public leaderboards, how do you judge whether the model has truly achieved higher cost-effectiveness?

Diogo Almeida: We will compare cost and capabilities through internal evaluations, pursuing stronger capabilities at the same cost and lower cost at the same capability. I don't object to evaluations, what I oppose is optimizing around leaderboards to make scores divorced from real value. Developers ultimately need to put the model into their own workflows to measure actual performance, instead of only looking at price, speed or a total score.

Swyx: How should developers organize tasks to use Jev more reliably?

Diogo Almeida: I suggest splitting complex tasks into the smallest semantic units that can be judged independently, providing necessary information through structured input, and then letting the code control the final behavior. Choice corresponds to selection branches, Noul corresponds to conditional judgment, and Score corresponds to scoring, sorting and filtering. Each step can be evaluated separately and the threshold can be adjusted; when the capability is insufficient, it can be transferred to manual processing or not deployed temporarily.

Swyx: For enterprise developers, what application directions of Jev are worth trying?

Diogo Almeida: We have sorted out four categories: analyzing "dark data" that has been left idle due to too high processing cost; providing rapid judgment for real-time processes; checking calls and outputs of other models; embedding intelligent judgment into the core logic of software. I am particularly optimistic about dark data analysis and programming Agents, but the specific way to combine models still depends on whether it can reduce costs and solve problems that cannot be solved originally.

Swyx: In the face of the risks of cutting-edge AI, is slowing down the development speed the only choice?

Diogo Almeida: I think this kind of discussion often defaults that everyone must continue to increase investment along the existing RLVR route, but both research objectives and technical routes can be reselected. How much action space you give to the model to improve performance is a design decision in itself, which should not be regarded as an inevitable premise. I prefer to explore other directions to make existing intelligence reliably enter software and automate actual work.

Swyx: If researchers cannot get resources in cutting-edge laboratories, would you advise them to start their own businesses?

Diogo Almeida: It depends on why they leave. If they just want to freely try research topics, existing laboratories may still be the most suitable; if they find new tasks worthy of long-term investment, I will support starting a business. I don't think a beautiful research resume will automatically create value, and I don't favor the practice of repeating existing work after getting funds. First clarify your core goal, and then carry out research around problems that are really worth solving.

1 In the first week after Jev was released, the founder was in a terrible mood

Swyx: Welcome to the studio. Just this week, my good friend Diogo released Jev, which almost occupied the hot topics on social platforms. How do you feel right now?

Diogo Almeida: Emotionally, I have never been so bad. I'm like a exhausted zombie now, because so many things are happening at the same time, and there are problems waiting for me to deal with everywhere.

However, at the psychological level, it's completely different. I often talk about this, and I have repeated it in those large and small project activities over the past few years. The entire AI field is like a house of distorting mirrors in an amusement park, and everyone is like crazy. Everyone is saying all kinds of strange and unreasonable things.

But this week, I seem to be more in tune with reality, as if I suddenly felt: "Oh, everyone finally sees it." — AI can do far more than people imagined in the past. We may really be driving an AI-driven economic revolution, and this matter is back on the table again, which is absolutely amazing. One thing I am particularly excited about is that developers can really understand what we are doing, and this feeling is very strong. I also want to express my long-standing gratitude to these developers. I am very excited about the entire developer community and everything that is happening now, it's really great.

Swyx: You told me yesterday that you decided to prioritize doing town halls, that is, public community exchanges, instead of spending all your time on VIPs, investors and other people. Because you want to make sure that the people who get the most of your attention are engineers, developers, these people who actually use the product.

Diogo Almeida: Yes. At that time, I did have a feeling like "Oh my god, I'm talking to some very important people now." I probably shouldn't reveal who they are. But for me, if in my huge schedule full of people to talk to, the developer community is not included, it will be very uncomfortable for me. In fact, according to my ideal state, I will always communicate with the developer community. I was even thinking just now, "Should I walk to your studio while holding a community town hall?" Then I thought again, "No, that's too crazy."

Swyx: For those who may not know the situation well or just want exact answers, what exactly is Jev?

Diogo Almeida: This question is actually quite difficult to answer, but this is how I look at it: we need a brand new category of models. As for what this category is called, we are not obsessed with a certain name.

At present, the most accurate name we can think of is System 1 model. The reason we don't call it a "decision model" is that the capability scope of System 1 far exceeds decision-making itself. I can only say so much now. We didn't expect this release to attract so much attention, so we still have things that haven't been taken out.

Swyx: You should have said it was a "low-key research preview" at that time.

Diogo Almeida: To some extent, it really is. It's actually a bit like a research preview. Anyway, we have several internal words to describe this new type of model that has emerged. For example, machine-native model, System 1 model, large programmable model. In my understanding, the goal of this type of model is to make code the direct user of the model's outputs.

Pre-trained large language models were originally oriented towards internet text completion; chat and instruction-following models trained through RLHF are oriented towards text replies; there is an ambiguous boundary between RLVR and RLHF. And we hope that the outputs of this type of model can be directly consumed by code, that's why our company is called TypeSafe.

What we really want is to make AI as powerful as possible. And the way we think to achieve this is to combine it with software. Therefore, when designing, we not only consider the external usage of the model, but also optimize the deep internal mechanism of the model for software. Jev is our first large programmable model, you can also call it System 1 model, whatever you want to call it. Its optimization goal is "intelligence per dollar", which is where the name Jev comes from.

Swyx: Jevons Paradox.

Diogo Almeida: Yes, it is Jevons Paradox. Its goal is to achieve the most cost-effective intelligent model. I particularly like to discuss with people what is the most important among reliability, cost, calibration and speed. The name Jev will later represent a series of cutting-edge models that are leading in terms of "intelligence per dollar". Of course there are other optimization directions. In machine learning, at least for people who are good at machine learning, everything is about trade-offs. And we decided to go all out in this direction.

2 From mode collapse to calibration distortion: the other side of RLHF

Swyx: I think "calibration" is a problem that has only started to get attention recently. We previously invited Clementine Foreia from Hugging Face to do an episode, and we talked about this topic at that time. This is also one of your core views on RLHF: the model's responses tend to converge to what users want to hear, or the most likely content, rather than reflecting its real internal confidence in a matter.

Diogo Almeida: I heard that your audience has a strong technical background, so I just want to talk about this issue in depth. I put a lot of effort into checking every statement in the released video to ensure that it is accurate and true. Obviously, this practice is quite rare. One point in the video that almost no one notices is the shortcomings of RLHF, especially "mode dropping".

Swyx: Mode dropping or mode collapse?

Diogo Almeida: Here I mean the same thing. I want to write a special blog post about it later, but now I want to tell as many people as possible about this. I actually agree with many of Yann LeCun's judgments. But he has a very famous and controversial slide, which roughly means "large language models are doomed to fail".

Swyx: Are you talking about the "cake" metaphor?

Diogo Almeida: No, it's the slide about sequence length. His reasoning is: if there is a probability of error at each step of generation, the longer the text, the higher the probability of making at least one mistake. This reasoning seems mathematically intuitive, but the actual performance of the model does not simply follow this trend. I like to use it to ask: where is the difference between the mathematical derivation and the observation results?

Swyx: Where is the problem?

Diogo Almeida: The problem is that if the model tries to cover the entire distribution, or its probability distribution is well calibrated, then it will not be overly punished for producing a few outlier results. You would expect it to generate content within the common distribution sometimes, and content outside the distribution sometimes; covering the entire distribution will lead to this situation. You can think about image generation models before generative adversarial networks appeared: the images they generated are often blurry.

Generative adversarial networks will have mode collapse: they will directly discard categories with a small proportion, and only generate those most common categories. Because of this, the effect of "the longer the sequence, the error will inevitably accumulate to an unusable level" does not appear in the simplest way. In order to generate very long texts that are not prone to obvious errors, the model has to be extremely conservative, because once it makes a mistake, it is easy for humans to spot it; on the contrary, a seemingly correct response that actually misses subtle points is very difficult to detect. The requirements for keeping the model's probability distribution calibrated will have a great impact on the way text sequences are generated. This relationship is very subtle. I think it not only explains why that intuitive inference of error accumulation has not come true, but also explains why text models are not good at decision-making: letting models originally used for text generation undertake too many decision-making tasks often does not work well.

Swyx: Since we are talking about Yann, do you agree with his solution? That is, using world models, such as the method of embedding models like JEPA. Part of the problem seems to be that we let the model continue to reason based on the tokens it has already output, then send the output back, and loop repeatedly until a complete sentence is generated. The solution proposed by Yann is Joint Embedding Predictive Architecture (JEPA). Do you think this is the solution? What's your opinion on this?

Diogo Almeida: Maybe I shouldn't go into too much detail about the internal affairs of machine learning. However, apart from sometimes being more outspoken, I am actually very pragmatic in doing things. The judgment of the model just now is also based on the actual effect.

Am I a supporter of Scaling Law? It depends on what it can bring. Scaling Law tells you: how much a certain capability can be improved when a certain amount of resources are invested. Usually, resource investment needs to be greatly increased, but the return will not increase proportionally. Unless that capability improvement is very valuable, it doesn't seem to be a good investment. For me, the more important question is: with the resources we have on hand, how can we bring the greatest practical change?

My starting point is always practicality. For example, Yann LeCun's JEPA direction, I think the early research is wonderful, and I also like to see excellent research work. As for whether it is practical enough now, I don't want to make a judgment for the time being.

At present, there are many good results in the research community that have not been fully tapped, like unpolished diamonds. They have not been further turned into useful technologies, partly because everyone has not found tasks suitable for exerting their value.

The release of Jev is certainly beneficial to TypeSafe, but I hope its impact goes beyond that. On the one hand, developers may build a large number of new software based on Jev; on the other hand, some people will explore along different directions: what other ways can model capabilities be provided to programs to enable software to do more things that were impossible before. If these attempts appear at the same time, it feels a bit like the