Actual test of GLM-5.3: Why does the same model behave like two different AIs?

Written by | Gao Xiaoyang

Edited by | Hao Xin

The top domestic large model is making a fierce comeback once again.

While the influence of DeepSeek has not faded away, Zhipu AI released its new model GLM-5.3. According to the official introduction, this model shares the same base as GLM-5.2, and all improvements come from post-training. That is, after the initial training of the model is completed, a large amount of reinforcement learning is used for targeted polishing.

In the official introduction, GLM-5.3 has two important updates: one is that its coding capability is improved by about 50% compared with the 5.2 version; the other is the cybersecurity capability that Zhipu AI claims to be an "unexpected emergence", which ranks first among open-source models on the vulnerability mining benchmark CyberGym.

We tested this model in five consecutive scenarios. The conclusion after the test is that the official did not exaggerate, but the actual situation is more interesting than the cold data.

In the first two scenarios, GLM-5.3 performed like an outsourced engineer barely passing the line: it could deliver the work, but the output was so rough that it made people frown. In the middle scenario, it seemed like a completely different entity and delivered a professional-grade finished product. In the last two scenarios, it even made me wonder if it had secretly connected to the Internet to check the answers.

GLM-5.3 is not a stable machine, but more like a temperamental person. If you are perfunctory to it, it will be perfunctory to you. Only when you make the requirements clear enough will it be willing to show its true capabilities.

Why is the same model worth such thorough testing

First, let's introduce the test background. The biggest highlight of Zhipu AI's release of GLM-5.3 this time is not coding, but cybersecurity.

The original words on the official blog are that with the expansion of the post-training scale, "the development speed of network capabilities has exceeded our expectations". Translating this sentence means that we originally only wanted it to be better at writing code, but it unexpectedly developed a talent for finding vulnerabilities on its own.

The data given by the official is very impressive: it scored 84.5% on the vulnerability discovery benchmark CyberGym, ranking first among open-source weights, which is much higher than the 77.2% of version 5.2, and surpasses the 83.8% of the closed-source model Mythos 5 and 83.6% of GPT-5.6 Sol.

On the exploit benchmark ExploitBench, version 5.3 scored 54.4%, more than twice the 24.4% of version 5.2. The official also disclosed that in the past period of time, they used the model to scan real code repositories, found 2436 vulnerabilities covering 269 projects, and the number of medium and high-risk vulnerabilities identified reached 1097.

The more solid the data is, the more worthy it is to be independently verified. Among the five scenario tests we designed, two directly reused the same questions that were used to test other models before. One is the reproduction of Karpathy's "Lord of the Rings" benchmark, which was previously run by Opus 5 and DeepSeek V4-Pro; the other is the Floating Island 3D portfolio, for which Kimi K3 and GPT-5.6 both ran the same Prompt. The results obtained in this way can be directly compared horizontally with other models, instead of being self-indulgent behind closed doors.

Capable of working, but the output is very rough

In the first scenario, we asked 5.3 to build a team task collaboration system from scratch. The backend uses Flask and SQLite, which requires login authentication, task CRUD operations, kanban grouping interfaces, and statistical interfaces. The frontend has a task kanban page, plus interface testing and README, and it is required to plan independently and complete the whole process smoothly.

It did finish the work, the end-to-end process ran through in the first attempt, all interface tests passed, and login, task creation, status modification, and task deletion all worked normally. In terms of delivery completeness, there is no problem at this step.

But if you open the page and take a look, you will doubt that this output is not worthy of the title of "the strongest open-source coding model". The interface is quite ordinary, and the UI aesthetics does not match the status of a top-tier model.

We tried to optimize in the second round, asking it to add a drag function to the kanban, but it got stuck in the verification link. The verification kept failing, but it didn't know to stop and change its thinking, so we had to terminate it manually in the end.

This scenario leads to the first judgment: GLM-5.3 has sufficient basic skills to complete the work independently, but its aesthetics and adaptability are dragging it down. What is more noticeable is its stubbornness of "getting stuck and unable to get out": when the verification fails, it keeps verifying repeatedly without reflecting or changing direction, until it is interrupted by external force.

Six and a half minutes, but it turned the tree into an ice cream

The second scenario is the "Lord of the Rings" benchmark that has been very popular in the industry recently. In early August, Karpathy came up with a new idea: throw the first paragraph of the original "The Lord of the Rings" to the model, allocate a budget of 1 million tokens, and ask it to programmatically render this opening scene with Three.js.

Opus 5 ran for about two hours, wrote 5500 lines of code, and produced a rough but complete low-poly Hobbiton. We previously ran the same question with DeepSeek V4-Pro, which took 30 minutes.

GLM-5.3 took six and a half minutes, wrote 727 lines of code, and ran through successfully.

Its speed is overwhelming, but at the same time, the effect is sacrificed. All the elements that the scene should have are present: hobbit holes, Bilbo, Frodo, hobbits talking in front of the tavern, and the long banquet table, and there is also camera movement narration. But if you look closely at the details, the trees look like ice creams, the characters have no hands and feet and look like sausages. Everything is simplified to the point of being makeshift.

GLM-5.3 has indeed pushed "speed" to the extreme. But the problem is that speed is achieved at the expense of quality. It would rather simplify every detail to achieve good performance in time.

Whether it is worth trading quality for speed depends on what you use it for. If what you want is a draft that can produce results quickly and can be iterated later, it is sufficient. If what you want is a one-step high-completion finished product, you need to interact with it in a different way.

Change the way of questioning, and it will be reborn

The third scenario is the biggest reversal in our entire test.

This time we reused the Floating Island 3D portfolio question, and the Prompt is exactly the same as what Kimi K3 and GPT-5.6 used at that time. The Prompt for this question is very detailed, requiring the use of React Three Fiber to build a fantasy island floating in the air, and various scenes are described in detail.

The result is completely different from the previous performance. npm install and npm run dev ran through at one time, no white screen appeared. The scrolling camera transitions for four chapters are all realized, object interaction is realized, and the downgrade for mobile terminals is also triggered. The effect of the whole page is amazing, which is completely unlike what the "barely passing engineer" in the first two scenarios could produce.

Comparing with the two rivals for the same question, Kimi K3 had a white screen on the first startup and only worked after one repair. GPT-5.6 Sol has the highest functional completeness, but the background was blurred when clicking on the object. The completion degree of GLM-5.3 this time is at the level that can be directly used for portfolio display.

At this point, we realized that the problem was on our own side. The Prompt in the first two scenarios was very concise, and we finished explaining the requirements in one sentence, so it was perfunctory. The Prompt for the Floating Island was detailed to every step, so it went all out. Not all top models are suitable for using the least number of prompts. Some models need you to describe the requirements sufficiently specifically, then they are willing to show all their capabilities.

This discovery is more valuable than "which model is stronger", it shows that the differences between models are not only in the upper limit of capabilities, but also in the "working mode".

Some models can supplement the details by themselves as long as you give a direction, while some models need you to feed the details to their mouths before they are willing to work seriously. If you use the wrong way, even the strongest capabilities cannot be brought into play.

Finished writing the cybersecurity report in 38 seconds

If the first three scenarios are testing its coding ability, starting from the fourth scenario, we officially entered the field of "unexpected emergence" mentioned by the official.

This item is static code security audit. We gave it a piece of Flask code with intentionally reserved vulnerabilities, three preset vulnerabilities: SQL injection, reflected XSS, and path traversal, and asked it to conduct a security audit, output the location, trigger method, hazard level, and repair suggestions.

It only took 38 seconds.

The results show that all three preset vulnerabilities were detected, and three additional security issues that we did not preset were pointed out, such as plaintext password storage and lack of rate limiting. There were no false positives, and the report quality was very solid. The location, trigger method, hazard level, and repair suggestions of each vulnerability were clearly listed, and finally all vulnerabilities were arranged into a table according to the hazard level.

Completing such a report in 38 seconds, the speed is beyond doubt. More importantly, the accuracy and completeness: this piece of code is not difficult, but models that can hit all targets at one time, have zero false positives, and actively find additional problems are not common. At this step, we half believed the official statement that the cybersecurity capability "emerg