73% of PRs are AI-generated. The father of Claude Code talks with Spotify: 2,900 engineers deploy 4,500 times a day, and can submit code even while taking the subway

In September last year, Boris Cherny, the founder of Anthropic Claude Code, had a conversation with Niklas Gustavsson, the vice president of engineering at Spotify, about the future of AI programming tools. At that time, Niklas made a radical judgment: by the end of the year, developers might no longer need traditional IDEs.

Boris' first reaction was that this was almost impossible to happen. In his view, it might take two years for AI programming tools to completely replace traditional IDEs, but it was impossible to achieve this in just two months. However, just two months later, Boris found that he really no longer used traditional IDEs, and his entire way of working had completely changed.

"This kind of change is something I've never seen in nearly 30 years of doing this kind of work," Boris said in a recent conversation between the two.

Today, Spotify has become one of the representative companies globally that have massively integrated AI programming into their software R & D systems. Inside this streaming giant with about 2,900 engineers, about 73% of PRs can now be directly generated by AI, and AI tools have increased the PR submission frequency by more than 75%.

Spotify has undergone an almost "visibly" reconstructed R & D approach: from traditional IDEs gradually withdrawing from the daily development scene to intelligent agents running simultaneously in the background; from engineers writing and modifying code manually to models completing most of the implementation and entering the production environment through automated verification; and from feature verifications that originally took weeks or even months to being compressed to just a few hours.

But Spotify's practice is not simply buying a set of Claude Code subscriptions for engineers. Behind it is the construction of code automation infrastructure, code library standardization, automated testing, continuous integration, permission systems, and verification closed - loops that have been ongoing for many years.

Spotify's experience shows that model capabilities are just the starting point. What really determines whether AI Coding can be implemented in large - scale enterprises is whether the enterprise has transformed its code, testing, tools, and R & D processes into a system that can be understood, invoked, and verified by intelligent agents.

TL;DR

Q: You always see the future earlier than others. Is it related to the culture or the participating teams?

A: About five or six years ago, we found that the growth rate of the company's code library far exceeded the growth rate of the number of engineers, about seven times faster. This means that as time goes by, more and more code needs to be maintained. And Spotify is a company with no shortage of product ideas. We always have a large number of features we want to deliver to users. If the team is dragged down by maintenance work for a long time, it's obviously not an ideal state.

Q: A common mistake many companies make is insufficient investment in the verification closed - loop.

A: This is indeed very common. When we started to automatically generate and submit PRs for source code, we had to change the requirements for each team. The team may no longer participate in every change. We will automatically merge most of these changes, and the relevant teams may not even see these PRs at all. This means that we must build a more complete automated testing system to ensure that all software can withstand this kind of automated change. This investment has helped us a great deal.

Q: How do you consider ROI?

A: Initially, the discussion about ROI was relatively simple because the improvements we saw were so significant. But as the technology matures and the cost structure improves, the requirement for the accuracy of ROI calculation is getting higher and higher. Part of it is to measure how much productivity has been improved; the other part is to calculate how much cost has been incurred to achieve these improvements, including how many tokens have been consumed, how many hours have been invested, and how much effective output has been produced in the end.

Q: What suggestions do you have for CTOs, VPs, and engineering managers of other companies?

A: In addition to automated testing and verification, standardization is also equally important. We have been promoting the code library to be more consistent and also promoting the unified use of tools and frameworks by the teams. These investments were initially made to reduce the work complexity of human engineers and improve human productivity. But we found that these practices can also be smoothly transferred to intelligent agents.

Q: A large amount of time previously spent on implementation has been freed up.

A: We are now investing heavily in prototype development. Moreover, this investment is not only for engineers but also for non - engineering personnel. Currently, one of our co - CEOs has also submitted a prototype in this internal application store.

Below is the detailed conversation between the two. We have translated it and made deletions without changing the original meaning to satisfy our readers.

Desired Code Automation Before Claude Existed

Boris: I still remember having a conversation with you around September last year. You said something like, "I think by the end of the year, maybe no one will be using IDEs anymore." When I heard that, I thought, "This is crazy. It's absolutely impossible to happen." I could imagine this kind of change might happen in about two years, but two months was just too exaggerated. But two months later, I found that I really no longer used it, and my entire way of working had completely changed. This kind of change is something I've never seen in nearly 30 years of doing this kind of work.

Niklas: Interestingly, the feeling inside our company is almost exactly the same as that of the outside world. It's just that we might be a few weeks ahead. Only a few weeks, but the feeling is really the same.

Boris: Now, intelligent agents and large - scale models are bringing about huge changes. In my opinion, both your personal usage and Spotify's usage are at the forefront of the industry. When did you first really feel that "AGI is here"?

Niklas: I think I've experienced several such moments, depending on what problems we were trying to solve at that time.

When large - scale models first appeared, we were among the early ones to try using them to automatically complete code changes. At the beginning, this was very difficult. But after a while, as we gradually figured out how to use large - scale models, evaluate models, and other methods, we started to get some very exciting results.

Boris: Was that several years ago?

Niklas: Yes. At that time, Claude didn't exist yet. It was in the very early stage of GPT. The results we got at that time couldn't solve all problems, but it showed us where this technology might be headed in the future. That definitely counted as an important moment.

But for me personally when writing code, the real breakthrough moment might be Opus 4.5 released in November or December last year. It changed from a "smart auto - completion tool" to something that I could really hand real - world problems to, and I no longer needed to do much prompt engineering.

For me, the biggest change was that I finally didn't need to edit code myself anymore. Before that, my workflow usually involved having the model write about 80% of the code, or 70% depending on the model's ability, and then I still had to go into the IDE to make the final modifications. Later, I suddenly didn't need to do this step anymore. That feeling was really crazy.

Boris: I think this is why that change felt like a huge leap. So what's your current workflow like?

Niklas: My usage is relatively basic. I usually open many Tmux sessions in the terminal. Whenever I need to do some work, usually several intelligent agents are running simultaneously in the background.

Boris: How many terminal tabs do you usually open?

Niklas: About 5 to 10. I also use some panes because I like to keep a separate terminal to view code differences and other information. So, I usually configure a matrix - style workspace now: on one side are multiple Claude sessions, and on the other side are the corresponding terminals, used in conjunction with multiple Git working trees.

We now have several very large monolithic code repositories, and we are gradually migrating to this architecture. But at the same time, we still maintain thousands of smaller multi - code repositories.

Most of my work takes place in these monolithic code repositories. So at any time, I usually run several Claude sessions and terminals simultaneously. If I need to enter a multi - code repository temporarily, I'll open a temporary Claude session there.

Boris: Do you think for Claude Code, which is more suitable, monolithic code repositories or multi - code repositories?

Niklas: To be honest, at first, I was a bit worried about the combination of monolithic code repositories and intelligent agents. Because when using early tools, we encountered problems with code indexing. And our code repositories are really very large. Just the backend monolithic code repository has more than 20 million lines of code.

But it turns out that Claude performs very well in these code repositories. We found that Claude is very good at looking at other code in the code repository and finding references and inspiration to solve the current problem.

The Code Growth Rate is 7 Times That of the Number of Engineers, Forced to Find an Automated Solution

Boris: I also want to talk about some of the infrastructure you've built. Spotify built Honk (an in - house developed AI automated programming system). From the earliest attempts with models to the later construction of Honk, and then to the development of background intelligent agents based on the Agent SDK, you always see the future earlier than others. Is this related to Spotify's culture or the team involved in this work?

Niklas: About five or six years ago, we found that the growth rate of the company's code library far exceeded the growth rate of the number of engineers, about seven times faster.

This means that as time goes by, more and more code needs to be maintained. And Spotify is a company with no shortage of product ideas. We always have a large number of features we want to deliver to users. If the team is dragged down by maintenance work for a long time, it's obviously not an ideal state.

So, we started to try to automate these maintenance tasks as much as possible. Many of these tasks are actually very boring, such as migrating to the latest Java version, upgrading dependency libraries, or completing similar updates. There are also many tasks involving migrating an API to another API across the entire code library. So, we built an infrastructure called "fleet management".

Before that, whenever a migration was needed, we usually sent migration instructions or tutorials to all teams and then asked each team to manually complete the migration of the components they were responsible for. But later, we started to think about whether we could do it differently, directly execute changes across the entire code library distributed in thousands of code repositories, instead of having each team repeat the same operation.

Boris: That means hundreds of teams have to manually perform the same operation on thousands of components.

Niklas: Exactly. Each migration could last for several months, and we could only complete about 10 such migrations a year, or even barely keep up with using the supported versions of various frameworks. So, we started to automate this work and built a complete infrastructure for it. Later, we merged millions of such PRs.

However, these changes initially all relied on deterministic scripts. The scripts would be applied to the code library to complete the corresponding code modifications or configuration changes. But we soon found that the interfaces, calling methods, and dependencies in the code were extremely complex. Once we tried to modify the code, the difficulty would increase rapidly. So, we quickly reached a limit: how complex code changes could be completed only by deterministic methods? Even when just replacing a method or an API, when there were 5 different calling methods, the problem would become very complicated.

Boris: That means you mainly relied on traditional static analysis and abstract syntax tree conversion to complete this work at that time? For example, if the return result of an API was first assigned to a variable, then you needed to track the variable and its state.

Niklas: Exactly.

Boris: This would be very troublesome.

Niklas: Yes. To migrate the code, each script we wrote would eventually expand to thousands of lines to handle various boundary cases in the code. This is why I mentioned earlier that as soon as early large - scale models appeared, we started thinking about whether we could apply them to this problem.

The initial results were not good. On the one hand, the capabilities of the models at that time were not strong enough. On the other hand, our approach to solving the problem was also very naive. We basically just presented the code directly to the model and asked the model to complete the modification in one go. The result was, of course, not ideal.

Later, as the model capabilities continued to improve and our understanding of such problems deepened, we started to use large - scale model evaluators to check whether the output results met our expectations, and we also started to break down and decompose tasks in different ways. We carried out many rounds of iterations and conducted a large number of experiments within the company, trying different methods to solve this problem. Later, we gradually integrated these methods, and finally formed what we call Honk today.

Honk was very different at the beginning from what it is now. It was not initially built on Claude, and there were many things developed by ourselves inside. But it gave us the first glimmer of hope that this problem could indeed be solved.

After that, we carried out many rounds of iterations on Honk. The version we released now is V2, but in fact, it might be around V8. We just didn't carefully record the version numbers for each round.

Honk was initially mainly used to automatically complete code changes and schedule and orchestrate these changes across all code repositories. But engineers soon realized that it could also be used for other things. For example, someone hoped to mention Honk directly in Slack and let it help complete a certain task, or use it to handle other similar work. Today, Honk has become a very widely used tool within Spotify.

The Most Important Thing in AI Programming Is Not Generation, but Verification

Boris: Can you introduce the overall architecture of Honk? What are its main components? You mentioned earlier that there is an intelligent agent responsible for writing code. Is this part built based on the Claude Agent SDK?

Niklas: Yes.

Boris: In addition, you also used to set up a verification step, such as an intelligent agent - style validator. Can you elaborate?

Niklas: Honk used to have an evaluator, but we removed it later. We found that starting from Claude 4.5, the intelligent agents and the models themselves were strong enough, and there was no longer a need for an additional evaluator. However, in the early versions of Honk, the evaluator was very important. If I remember correctly, it increased the success rate of PRs from about 20% - 30% to about 80%.

This was a very significant change. But as we discussed before, later the model capabilities caught up, and the Agent harness also caught up. So, we have now removed this evaluator from Honk.

In terms of architecture, Honk is actually quite simple. The Claude Agent SDK runs in a Kubernetes container group and has access to a set of tools. Before V2, these tools were a pre - defined permission list, and only the tools we trusted were open to the intelligent agents. In V2, users can add tools by themselves. Therefore, now the intelligent agents can call any tool within Spotify.

One of the most important tools is that it can execute the verification process, such as running continuous integration builds. These builds can run on both Linux and macOS. macOS is especially important to us because all iOS development requires builds to be completed in a macOS environment.

Boris: Does the verification mentioned here just mean completing the build? Or do you run the entire process, such as starting the iOS simulator, having the model open the application and perform operations? How in - depth can the entire verification be?

Niklas: It can execute such tests. We do have some scenarios where we integrate the simulator with Claude to automatically complete the entire process from Figma design drafts to user interface implementation. For example, we have been using this method to port the TV - end application from the existing iOS application.

Boris: I think verification is a topic we often discuss. But in this closed - loop development, after the intelligent agent gets a task, it may need to expand and break down the task and complete a large amount of work without human participation. At this time, verification is the most important thing. I've seen that a common mistake many companies make is insufficient investment in the verification closed - loop.

Niklas: This is indeed very common, and it's the same for us. In the process of promoting this work, one of the major adjustments we made to engineering practices was to strengthen automated testing.

We split the code library into thousands of components. Each component has a clearly defined ownership and is responsible for a specific team. This team needs to take full responsibility for this component. They may have designed it initially, then implemented it, and continue to be responsible for its operation.

Before we invested in code fleet management, every