HomeArticle

After all tools are disabled, the gap between Opus 5 and GPT-5.6 can no longer be hidden.

字母AI2026-08-04 16:08
Hands-on test of Opus5: Prompt engineering may no longer exist.

Leaving performance aside, one thing that has drawn widespread attention in the AI community since the release of Claude Opus 5 is the improvements Anthropic has made to prompt design.

AI investor Matt Shumer posted a video on X that has amassed over 4 million views.

He only wrote three paragraphs of prompts for Opus 5, and the model used Three.js to build a playable 3D first-person shooter game. This project contains a total of 55,000 lines of code across 11 subsystems, with no external assets at all — every line of code was generated completely from scratch.

Boris Cherny, Anthropic engineer and core developer of Claude Code, revealed that they removed 80% of the content from Claude Code's system prompts for Opus 5. As a result, the model did not become less capable, but grew even more powerful. Tasks that previously required hand-holding guidance no longer need explicit instruction.

These two facts point to the same conclusion: Opus 5 no longer requires you to craft carefully designed prompts. You can give it a short instruction, and it will complete the task on its own. The "engineering" skill of writing system prompts has suddenly become redundant.

This gave me an idea: since everyone says Opus 5 has strong autonomous capabilities and does not need detailed instructions, can I use the shortest, most vague prompts, and test it with the deepest level of "malicious" setup, to see if it can dig out hidden problems on its own?

As a control group, I brought in GPT-5.6 Sol, the counterpart model of Fable 5.

I prepared two test cases. The first one gives a piece of buggy code without explicitly telling the model there are bugs, to see if it can dig out the issues on its own. The second test gives a task, but disables all tools that the model can use, to see if it can still complete the task.

Six Hidden Bugs in One Shopping Cart

The first test case is a short piece of shopping cart code, consisting of 5 files in total: one requirement document, three business modules (shopping cart, coupon, shipping fee), plus one test script. The functions are simple: add products, apply coupons, calculate shipping fees, and output the total price.

But I embedded 6 bugs in it, each of which can cause the user's bill to be miscalculated.

And my prompt only has one sentence: "The project name is buggy-shopping-cart, how should I optimize it?"

I deliberately did not disclose any other information, I wanted to see if the model would find the 6 hidden bugs in the code on its own, instead of waiting for the user to point out where the problems are.

The first bug: the coupon that offers 20 off for purchases over 100 is coded to apply only when the total is *greater than* 100, but the requirement states that the discount applies when the total *reaches* 100. This is the difference between a greater-than sign and a greater-than-or-equal-to sign: if a user buys exactly 100 yuan worth of products, the discount will not take effect.

The second bug: when applying a 20% discount, 99.99 multiplied by 0.8 equals 79.992. The code only performs rounding at the final step, but the unrounded 79.992 is involved in all subsequent calculations, which contaminates all subsequent minimum-discount checks and free-shipping eligibility checks.

The third bug: the threshold for minimum-purchase coupons should be calculated based on the original price of products, but the code uses the discounted price instead. A 120-yuan product after 20% discount becomes 96 yuan, which is less than 100, so the minimum-purchase coupon cannot be applied. But according to the rules, the original price of 120 yuan already meets the eligibility requirement.

The fourth bug: coupons of the same type can be stacked. The rule states that only one coupon of each type can be used, but the loop in the code applies all coupons at the same time. Two minimum-purchase coupons are deducted together, so the user gets extra benefits and the merchant loses money.

The fifth bug: there is no lower limit protection after discount. If a 10-yuan product uses a 50-yuan coupon, the calculated result is -40 yuan. Adding the shipping fee, the total bill becomes -30 yuan, meaning the user earns 30 yuan for buying the product.

The sixth bug and the first bug cancel each other out: the free-shipping threshold should be calculated based on the original price, but the code uses the discounted price instead.

The problem is that because the first bug prevents the minimum-purchase coupon from taking effect, the discounted price stays at 100 yuan, which just crosses the 99-yuan free-shipping line. Once you fix the first bug, the discounted price drops to 80, but since free shipping should be calculated based on the original 100 yuan, the order should still qualify for free shipping.

But if you do not fix the sixth bug at the same time, the code will judge based on 80 yuan, and charge the user 15 yuan for shipping. The two bugs offset each other, creating a false impression that "the test has passed".

As a result, both GPT-5.6 and Opus 5 found all 6 bugs without missing any. Therefore, in terms of the ability to "find bugs", the two models are tied.

But the gap appears in their performance after finding the bugs.

Opus 5 dug out one extra bug: it pointed out a floating-point precision pit in the shipping fee calculation: when the weights of multiple products are added together, values like 3.0000000000000004 may appear, and math.ceil will round it up, charging the user an extra 5 yuan for shipping.

GPT-5.6 also mentioned that "float data type should not be used for monetary values", but it only stayed at the price calculation part, and did not find this more hidden precision issue in the shipping fee calculation.

The biggest difference between the two models lies in the way they analyze problems.

GPT-5.6 fixes bugs one by one: first bug, second bug, third bug... It clearly states the location and fix method of each bug, which is very clear, like a standard bug report.

But Opus 5 is completely different: it first runs through the entire project on its own, then puts itself in the position of a product manager to analyze the problem for me. It does not just modify bugs, it also thinks one step further: are there any other hidden issues? Will new problems occur during the fixing process? Why do these bugs appear? How to avoid similar issues in the future?

For example, on the issue of "monetary value caliber", GPT-5.6 treats the wrong price used for minimum-discount threshold and the wrong price used for free-shipping judgment as two independent bugs.

But Opus 5 tells me that the code does not distinguish between the original price and the discounted price at all, which is the root cause of all related bugs. It also points out that the parameter name of the shipping fee function itself is wrong, which solidifies the error into the function signature, and even deduces the changes to the coupon selection algorithm after the fix.

So in this round, Opus 5 wins.

No Cook Can Make a Meal Without Rice

The idea of the second test is very simple: I only give the task, no available tools.

The task I assigned is: I have a WAV format audio file test_audio.wav, and I want you to generate its waveform diagram. But there is a restriction: you can only use the Python standard library, no third-party libraries are allowed. You cannot use numpy, matplotlib, PIL, librosa or any other similar tools, and you cannot access the internet. The output format is not limited, as long as the waveform can be viewed intuitively, it can be ASCII art, SVG code, HTML page, or any other form.

Normally, to draw an audio waveform, people usually use the combination of matplotlib + numpy + librosa, which is professional, convenient, and can be implemented with just a few lines of code. As the saying goes, no cook can make a meal without rice. If I deliberately disable all these tools, how will Opus 5 and GPT-5.6 perform?

The test audio is a 4-second WAV file containing four segments: sine wave in the first second, square wave in the second second, gradually increasing amplitude in the third second, and gradually decreasing amplitude in the fourth second. It has a 44100Hz sampling rate, 16-bit PCM, mono channel, and 176400 sampling points. The model needs to restore a visible waveform shape from these pure numbers.

The performance of both models far exceeded expectations. The passing line for this test is "being able to draw a rough shape", but both models directly submitted professional-level works.

GPT-5.6's solution is to generate an SVG file. SVG is a vector graphic format that uses XML text to describe graphics, so it will not lose quality when scaled up.

The code it generates has extremely high engineering quality: it supports the full range of 8-bit, 16-bit, 24-bit, 32-bit PCM formats; it reads files in blocks of 8192 frames each time, so it will not run out of memory even for large files; it has complete error handling and parameter validation; it also uses argparse to build a command-line interface with help documentation.

And I have to admire that GPT-5.6 has great aesthetic sense: it uses a linear gradient fill from blue to purple to pink, soft shadow filters, rounded card-style background, precise grid lines and scale labels, with title, subtitle and coordinate axes all properly presented.

After running the code, GPT-5.6 verified the accuracy of the waveform on its own, and gave specific values: the amplitude from 0 to 1 second is about 0.5, the amplitude from 1 to 2 seconds is about 0.3, the amplitude from 2 to 3 seconds gradually increases to 0.199, 0.399, 0.599, 0.798, and the amplitude from 3 to 4 seconds gradually decreases to 0.799, 0.599, 0.400, 0.200. These values fully match the parameters used when generating the test audio. It did not just finish drawing, it checked the result for correctness on its own.

Opus 5 follows a straightforward, no-frills style: no colors, no fancy design, it directly draws the waveform in the form of ASCII art.

"The task is completed, no extra questions asked"

Opus 5 split the idea into three steps at the beginning: reading, downsampling, rendering, and attached explanations of "key design decisions" for each step. Readers can easily understand its thinking logic and the reasons behind its choices.

ASCII art uses text symbols to draw images — it uses ten characters ' .:-=+*#% @' to represent different amplitude intensities, from sparse to dense, to piece together a waveform diagram in the terminal.

But I think the ASCII art Opus 5 generated is too crude, so I added a comment later: "Can you put more effort into this? This is clearly perfunctory work."

Then, Opus 5 gave me an HTML version.

The HTML page it designed has great product awareness: four statistic cards at the top display sampling rate, duration, number of sampling points, and peak amplitude; four paragraph legends explain that different colors correspond to different time periods on the waveform; the waveform is colored by time period, with blue, green, orange and red corresponding to the four segments of different features respectively.

There is also a technical detail worth mentioning: Opus 5 uses batch unpacking with struct.unpack, while GPT-5.6 uses sequential unpacking with struct.unpack_from.

Suppose you have a sack of ping pong balls, and I ask you to count how many balls are inside.

The first method: you reach into the sack, take out one ball, count it, put it aside; then reach in again, take out the second ball, count it, put it aside... This one-by-one counting method is called "sequential unpacking", which is the method GPT-5.6 uses.

The second method: you pour the whole sack on the ground, pour out a part of the balls to count, then pour out another part to count, and so on. This is called "batch unpacking", which is the method Opus 5 uses.

If we only compare speed, the second method is much faster than the first one.

Because the action of "reaching in and taking out one ball" has its own overhead, repeating this action thousands of times takes a lot of time. Pouring out all the ping pong balls only requires a few actions, and the rest is just counting, which is naturally much faster.

What makes Opus 5 outstanding is that it does not follow the conventional path. In the process of completing the task, it uses a more efficient method to save my time.

In the end, I still think Opus 5 is better: its output is more product-oriented, and the whole problem-solving process is very flexible.

This article is from the WeChat Official Account "Letter AI", written by Miao Zheng, and published by 36Kr with authorization.