Anthropic cancels programming interviews: newcomers get the first opportunity to compete directly with senior engineers.

"I have seen enough to confirm that it can no longer provide valid judgment signals. This month, I have removed the programming interview from the hiring process."

Felix Rieseberg, Engineering Manager at Anthropic and co-maintainer of Electron, stated this on Twitter.

Rieseberg currently takes charge of engineering management at Anthropic, and has previously led engineering teams for products including Claude.ai, Cowork and Claude Code Desktop. He has also worked at Notion, Stripe, Slack and Microsoft in sequence. The reason behind his judgment is that such tests can hardly continue to provide effective hiring signals.

He believes that in the past, a small React exercise or a tiny programming challenge extracted from real code, while unable to fully measure an engineer's full competence, could more or less reflect a candidate's ability to define problems and find solutions quickly. But with the rapid improvement of AI programming capabilities, this correlation is declining.

In Rieseberg's view, the significance of testing candidates with small React exercises is increasingly similar to assessing whether an engineer can skillfully write assembly code by hand today: Claude performs well enough on these tasks, and the most influential engineers in the team are not necessarily the ones who get the highest scores in traditional code tests.

Veteran engineers are constrained by unspoken rules

Behind this hiring adjustment, Rieseberg wants to discuss another question: when AI can complete more and more work that used to require years of programming experience to finish, what changes will happen to the gap between junior and senior engineers?

His judgment is quite optimistic. "Hunger beats tenure", which means strong willingness to act may be more important than seniority.

Rieseberg said he has already seen some engineers who have not been in the industry for very long quickly outperform peers with better programming fundamentals and longer working years with the help of AI. In the past, engineers needed years to accumulate knowledge of frameworks, libraries and tools, and this experience formed part of the career barrier; now AI is reducing the restriction of such knowledge accumulation on execution speed.

He believes that less experienced engineers have long had an easily underestimated advantage: they have not fully accepted the default judgments within the organization about "what cannot be done".

Rieseberg recalled that when he joined Stripe, founder Patrick Collison once reminded new employees to pay special attention to things that seem strange or unreasonable at first glance. Newcomers do not carry historical burdens, so they are more likely to find frictions and opportunities that veteran employees have long taken for granted.

He also went through a similar experience at Notion. During the interview, Rieseberg was asked to design an offline mode architecture. The interviewer told him it was essentially a "trap" question, because the team generally believed that Notion could not implement offline mode: the data model of the workspace was too complex, and downloading pages separately was considered unable to provide a good enough user experience.

The judgment about the complexity of the data model later proved to be valid, but the product restriction that everyone thought "the founders would not accept" was not as unshakable as imagined. Many people took it as a consensus for so long that few people re-examined it. As a new employee who just joined the company, Rieseberg happened to be unaware of this unspoken rule, so he chose to ask about it again.

Interestingly, a year later, he himself became the veteran employee who told newcomers "don't bother to make changes".

Rieseberg once tried to equip the team with a large TV to display a real-time Dashboard in the office, but he did not push it forward after encountering organizational resistance. A year later, new employee Ji Pei put forward the same idea. Rieseberg told him that he had tried it before and it was not worth further effort, but Ji Pei still went to ask about it, and the next day the TV appeared in the office.

Rieseberg uses this story to illustrate that so-called "experience" easily carries two things at the same time: the understanding of real technical restrictions, and the organizational inertia formed after past failures, and it is hard to distinguish between the two after a long time.

Two new employees built the company's hit product over a weekend

In the past, even if junior engineers found problems, it was difficult for them to push the organization to take action. A new idea often required writing a PRD, winning manager support, coordinating product managers and designers, and then getting engineering resources. The higher the execution cost, the more the organization tends to give decision-making power to people with higher seniority and more internal influence.

AI is changing this premise.

The first piece of advice Rieseberg gives to young engineers is to focus on business impact. He quotes a 2011 line from Patrick McKenzie: engineers are hired to create business value, not just to write code. Engineers need to first understand what the company, business department and supervisor really want to achieve, and then look for opportunities that can significantly improve results around these goals.

Next, Rieseberg suggests young engineers pick a bold enough idea, then build a working prototype over a weekend.

In some cases he observed, small teams or even individual engineers can bypass the long product approval process in the past, and directly build a working prototype with AI. When leaders can see a clickable, runnable and experiential product, a newcomer's idea no longer needs to rely entirely on position, seniority and internal connections to get attention.

However, he does not recommend stuffing too many ideas into the prototype. Young engineers tend to add a dozen features at once, which eventually makes several really valuable ideas submerged by a large number of secondary functions. A good prototype first serves as a communication tool, which allows decision-makers to quickly understand where the opportunity lies and why the person who proposed the solution is suitable to continue leading it.

For consumer products, new employees have another advantage: they are closer to ordinary users. Rieseberg recalled that when he worked at Slack and Notion, employees who had been involved in product development for a long time often could no longer experience what it feels like for an external user to use these products for the first time. If they are responsible for infrastructure and development tools, newcomers also do not carry so many existing perceptions of "this was tried before" and "this part can never be modified".

Rieseberg also shared a very representative example from inside Anthropic.

As the usage of Claude Code increased, internal employees at Anthropic gradually got used to using Claude Code as a command-line tool. Then two relatively new engineers Anthony Morris and Alice Zhao built a working prototype with a graphical interface over a weekend. About 8 days later, the first public version was officially released.

Rieseberg mentioned that the graphical interface of Claude Code later became a very popular product. This kind of project in the past might require a series of processes including requirement discussion, PRD, product design, and engineering resource coordination, while AI allows a very small team to finish the product first, then prove the existence of demand with the finished product.

This change is also the reason why he believes opportunities for young engineers are increasing. People without position and organizational influence could hardly get the company to invest resources in an unproven idea before; when a person can push an idea to a runnable stage with the help of AI, the seniority barrier within the organization will be lowered accordingly.

No more traditional coding tests, how should new engineers be interviewed?

Then, if traditional code interviews are increasingly unable to measure engineers' competence, what should replace them?

Rieseberg admits that he does not have a complete answer yet. His current approach is to look for "small prototypes" in resumes and portfolios, and focus on observing the product taste, user empathy reflected in these works, and whether the candidate has a clear and systematic way of thinking.

This is also the part that was questioned after Rieseberg's X post was released. "We all know that in the AI era, traditional programming interviews can no longer provide sufficiently effective judgment signals. The problem is that in a 60-minute interview, no better alternative indicator has been found so far. I did not see such an answer in your post either."

A candidate who had just participated in Anthropic's recruitment also mentioned that he took a take-home test, and compared with traditional code interviews, the new evaluation method also has uncertainty. Some other developers suggested that candidates can be given a code repository where "AI writes very confidently but there are actually errors", to observe whether candidates can identify the model's mistakes and stick to their own judgment.

Not everyone in the community agrees with the judgment that "programming interviews will be canceled". Some developers believe that AI companies like Anthropic have particularities, and a large number of traditional enterprises may continue to use algorithm and code tests for many years in the future. Others pointed out that the biggest current difficulty is not whether young engineers can show strong execution ability with the help of AI, but how they can get the opportunity to demonstrate this ability at the resume screening stage. After all, some enterprises are increasingly reluctant to recruit new junior engineers now.

However, Rieseberg still believes that traditional programming interviews can hardly return to their previous status. Software engineering has experienced multiple shifts up the abstraction hierarchy, from assembly to high-level languages, from writing a large amount of basic code by hand to calling frameworks and cloud services. Now AI has abstracted part of programming implementation one level higher.

For engineers who have just entered the industry, this change also brings a different way of competition from the past. Years of working experience still means experience, but the binding between execution ability and working years is weakening. For a person who can understand business goals, find problems ignored by organizational inertia, and quickly turn ideas into products with the help of AI, the time he needs to wait to gain influence may become shorter and shorter.

A brand new hiring paradigm has not yet taken shape

At present, not many people directly cancel programming interviews like Rieseberg, and more organizations are examining candidates' performance after they are allowed to use AI programming tools.

In January this year, Tristan Hume, one of the heads of the performance optimization team at Anthropic, made public a quite dramatic hiring experience: in order to recruit performance engineers, he designed a take-home test simulating accelerator optimization starting from 2024, and clearly allowed candidates to use AI tools. They wanted to evaluate whether candidates could still demonstrate their judgment and optimization ability with AI assistance.

About 1000 candidates have completed this set of tests, and Anthropic has recruited a large number of members of the performance engineering team through it. It is particularly effective for young candidates: some engineers with unremarkable resumes who just graduated from undergraduate programs got offers with excellent performance in the test, and proved their competence after joining the company.

During this period, Claude gradually "mastered" the interview questions. Claude 3.7 Sonnet has reached a critical point: more than half of candidates can get better scores if they directly give the questions to Claude Code to finish; Opus 4 further exceeds the scores achieved by the vast majority of people within 4 hours; the team then upgraded the questions to the second version, and shortened the test time from 4 hours to 2 hours; a few months later, Opus 4.5 caught up again, and it could reach the level of the strongest human candidates under the 2-hour limit. The team finally had to design new questions again, and deliberately added constraints, mini instruction sets and new types of puzzles that the model had not seen before.

Hume himself admitted that this new test sacrificed the similarity with real work, because the closer the questions are to real development work, the easier it is for Claude to complete them.

It is worth noting that the official website of Anthropic still shows that some technical positions will use tools such as Colab and CodeSignal for live coding. At the same time, the AI usage guideline for candidates still stipulates that candidates cannot use Claude by default for take-home tests, and AI assistance is also not allowed by default in live interviews unless otherwise specified.

OpenAI is also using a "hybrid" mode now. Different teams may arrange pair coding, take-home or technical tests; some interviews allow AI, and some interviews specifically observe candidates' independent problem-solving ability without AI help. OpenAI has made it clear that they care not only about the final answer, but also about how candidates analyze problems, make decisions and explain their thinking.

Similar practices have also emerged in China. Tang Feihu, who is engaged in R&D and responsible for developer relations at Moonshot AI, previously mentioned in an interview with InfoQ that he will not stop candidates from using AI when he works as an interviewer. He will design tasks that can only be completed through human-AI collaboration, and deliberately leave problems to observe whether candidates can understand what is happening, identify model errors and adjust directions.

Canva has gone even further. Canva redesigned the technical interviews for backend, frontend and machine learning positions last year, and explicitly required candidates to use AI tools such as Copilot, Cursor and Claude. The team previously gave traditional basic computer science questions directly to AI for testing, and the result was that AI could give correct answers with complete explanations in just a few seconds. Therefore, continuing to ban AI will only make interviews further and further away from the way engineers actually work.

As a result, Canva modified the questions to make them more complex, more ambiguous, and more similar to real product development. Candidates need to clarify requirements, break down tasks, guide AI, check generated code, debug, and take responsibility for the final technical solution.

After testing for a period of time, Canva found that the best performers usually do not hand over the entire task to AI. They will actively control the problem boundary, only hand over appropriate subtasks to the model, continuously review the code generated by AI, and quickly take over when the model makes mistakes. Many of the difficulties encountered by candidates with little AI experience are not about writing code itself, but the lack of ability to judge the quality of AI outputs.

From the data, this change has just begun.

Karat surveyed 400 engineering leaders in the United States, India and China this year. Only 38% of enterprises in the United States allow candidates to use AI in on-site technical interviews, while the proportion of Chinese enterprises reaches 68%; at the same time, 62% of the surveyed organizations still ban AI from participating in technical interviews.

The survey results of about 650 developers and recruiters by CoderPad this year are similar: 34% of organizations completely ban AI, 46% allow it or allow it under certain constraints, and the remaining 20% decide on a case-by-case basis.

Interestingly, after AI is allowed, enterprises begin to look for completely different hiring requirements: 66% of recruiters value candidates' ability to discover and fix AI errors the most; 56% care about the ability to explain technical tradeoffs and correctness; only 7% of recruiters believe that asynchronous technical tests can best reflect actual work ability, and technical discussions and live coding get significantly higher recognition.

This article is from the WeChat official account "InfoQ" (ID: infoqchina), written by CHU Xingjuan, and republished by 36Kr with authorization.