PostgreSQL 19 can't be released? AI is spotting defects so fast that humans cannot keep up with the speed of bug fixing.

Recently, the release of PostgreSQL 19 has encountered rare setbacks. The PostgreSQL team announced that PostgreSQL 19, originally scheduled for official release on October 29, will be postponed to November 19. Before that, the project will additionally add an RC2 candidate version, which will be released on November 12. The official reason given is that there have been a large number of fixes recently added to all maintenance branches, and the team hopes to first submit these fixes to the community for verification through RC2 before proceeding to the official release.

At the same time, after the start of the Beta phase, approximately 8 to 10 major features were withdrawn from PostgreSQL 19, some of which were removed at a fairly late stage. Normally, it is not uncommon for a major PostgreSQL version cycle to see one major feature rolled back, but withdrawing nearly 10 features at once has clearly gone beyond common practice.

Peter Eisentraut, a core PostgreSQL committer and Vice President of EDB, subsequently wrote a dedicated article to explain this anomaly, and one of the key variables is precisely the rapidly maturing large model-assisted code review and fuzz testing.

AI has uncovered issues that would have taken 5 years to discover in the past ahead of schedule

Peter Eisentraut dismissed a highly plausible speculation: these issues are largely not caused by vibe coding.

He explained that most of the features that were affected and eventually withdrawn were completed long before LLM-assisted programming and code review matured. That is to say, it is not that developers let models such as Claude and Codex generate a large number of low-quality PostgreSQL codes, which then exploded intensively during the Beta phase; on the contrary, increasingly powerful AI review tools later began to re-examine the code developed in accordance with traditional processes before.

Peter roughly divided the discovered issues into two categories. The first category is indeed relatively serious: some withdrawn features have architectural-level defects that are difficult to completely fix in a short period of time during the Beta phase. But in addition, there is a series of long-tail issues: a large number of problems that are not serious when viewed individually and will only be triggered under specific edge conditions.

In the past, such issues might not be discovered before release. PostgreSQL would be released normally first, and then patched little by little over the next few years as real users encountered various rare combinations. Peter estimates that many of these issues would have gradually emerged over the next 5 years.

LLM has changed this process. With the help of AI-assisted code review, edge defects that were scattered over several years in the past can now be intensively scanned out within days or weeks.

Peter gave an example: developers suddenly faced about 40 defect reports. Even if each issue only needs an average of 3 lines of code to modify, processing each issue still requires fixed costs. As a result, the code itself may only take a few minutes to modify, but when dozens of issues are superimposed, human time is quickly exhausted.

This also leads to a very counter-intuitive result: AI improves software quality, but may reduce software release speed in the short term.

Peter summed up this phenomenon as "the effective quality threshold has been raised". When these features were developed and submitted, the industry did not have the LLM review capability it has today; by the time PostgreSQL 19 was ready for release, new tools could already discover issues that were difficult to find in the past. Code that would have met the release criteria suddenly became "not good enough" under the new review capabilities.

AI-assisted fuzz testing may be directly integrated into feature development

Databases in particular are prone to encountering such problems.

An important feature of PostgreSQL is that developers often expect different features to be freely combined. A SQL feature not only needs to run in the most common scenarios, but may also be placed in various environments such as domains, composite types, partitioned tables, views, security-definer functions, triggers, and these capabilities can continue to be nested with each other.

Peter cited an example: a new feature at the SQL level may need to face such a combination: a domain is nested on a composite type, and the field in the composite type is itself a domain; the domain has a NOT NULL constraint; the column was once deleted and then re-added; the table partition was detached and then attached again in a different column order; in the end, all of this appears in a view, which is called from a security-definer function in a trigger.

Theoretically, developers expect all these things to work properly. The problem is that humans can hardly even enumerate all combinations, let alone test them manually one by one.

LLM-assisted fuzz testing is exactly very good at this kind of work. The model can continuously generate edge combinations, construct test inputs, observe anomalies, and then continue to expand along the anomaly path. Peter therefore judged that this type of AI-assisted fuzz testing will most likely need to directly enter the PostgreSQL feature development process in the future, instead of being used intensively only in the late Beta stage.

Similar signs have already appeared in the actual development process. In August this year, Robert Haas, a member of the PostgreSQL community, once asked Claude to conduct a risk assessment on patches for PostgreSQL 19 based on the number and type of bugs fixed after the feature freeze. Among them, the SQL property graph query was listed as one of the high-risk features, and it had already undergone 17 fixes at that time.

Core developer Tom Lane later stated that if SQL/PGQ entered PostgreSQL 19 in its state at that time, he would even be willing to "bet a dinner" that issues that could only be completely resolved until PostgreSQL 20 would still be discovered after release. In the end, SQL/PGQ was indeed withdrawn from PostgreSQL 19.

Another member of the PostgreSQL community later also mentioned that after launching AI review for this feature, some serious issues were discovered and manually confirmed. This means that during this release cycle, AI did not just act as a "chat-style code commentator", but has already begun to participate in risk screening for real open source projects.

However, Peter also emphasized that AI is not the only reason for this large-scale withdrawal. Some codes do have problems with their own quality and architectural design. This year, the PostgreSQL community also went through a round of intensive security issue handling, and a large number of senior developers were busy fixing security issues from April to August, which affected the additional reviews and tests that should have been carried out in the early Beta phase. When these developers re-engaged in PostgreSQL 19, many issues were exposed only in the later stage of the release cycle. In addition, the continuously growing complexity of PostgreSQL itself is also one of the fundamental problems.

A large number of issues eventually forced the PostgreSQL community to make a rather conservative decision.

Peter said that depending on the statistical method, about 8 to 10 major features have been withdrawn from the PostgreSQL 19 branch, many of which are expected to re-enter PostgreSQL 20 after being corrected. One representative feature is SQL/PGQ, which is used to query property graph data composed of nodes and edges according to SQL standards, but due to unresolved issues, it finally failed to enter PostgreSQL 19.

It is worth noting that the PostgreSQL community once even thought that PostgreSQL 20 might be mainly used for bug fixes and would not have many new features; now the situation is reversed: a batch of features withdrawn from PostgreSQL 19 have left a batch of nearly completed candidate features for PostgreSQL 20.

PostgreSQL 19 itself has not become a "minor version" because of this. Peter mentioned that it will still include important capabilities such as REPACK CONCURRENTLY, logical replication of sequences, and query planner hints implemented through pg_plan_advice. After this round of additional reviews, he instead expects PostgreSQL 19 to become a more robust version.

RC1 was advanced as planned on October 15, but the official version was postponed from the original October 29 to November 19. The release team also added RC2 on November 12, hoping that a large number of fixes accumulated recently will go through another round of verification. The official called this year an "unusual major version release cycle" and emphasized that the ultimate goal is still to ensure the stability and reliability of PostgreSQL.

AI can discover problems infinitely, so to what extent should humans fix them?

This incident quickly fermented in the developer community later, and many developers believe that the new bottleneck of software development may be shifting from "discovering bugs" to "classifying bugs".

"This situation will not disappear soon. The next step that really needs to be solved is how to classify the problems found by AI, and only prioritize the most serious problems for manual review." Other developers also pointed out that when the cost of finding vulnerabilities and constructing abnormal inputs suddenly drops, the speed of "discovering problems" and "fixing problems" no longer match. In the past, security researchers might take days or even weeks to find a complex defect, while AI can explore a large number of paths at the same time, but human maintainers can only confirm them one by one.

But another group of developers are worried that AI will cause "flood of real and fake bugs".

Some developers said that their experience is that AI will report a large number of problems that seem reasonable from the local code but are not valid in the context of the entire system. If the development team mechanically fixes every AI suggestion, they may end up with an over-designed system with worse efficiency. What really takes time is that humans must judge item by item whether it is a real bug, an extreme but acceptable edge case, or a pure false positive.

A developer engaged in data engineering said that in their internal AI code review system, even if the model can access the context such as code repository, Confluence and JIRA, only about one-eighth of the reports are ultimately considered valid; for more general review bots, this proportion may even drop further. Another developer familiar with static analysis tools said that a large number of false positives have always been an old problem of automated analysis. Once the number of defect reports accumulates to 100,000, it is almost unrealistic to require humans to achieve 100% confirmation.

However, some users also took their own experience as an example to prove that even if they are dissatisfied with false positives, they are still unwilling to give up AI review, because it can indeed dig out technical debts hidden for many years. They once asked AI to conduct centralized code review on scripts, tasks and configurations in a customer system that has been running stably for about 10 years, and finally found 39 problems, 30 of which were confirmed to be real bugs that had never been triggered before.

Similar things do not only happen in open source projects like PostgreSQL. In some closed-source software, features that have been developed for many years also cannot be released as planned due to new problems discovered by LLM. The past practice might be to go online first and then patch gradually in future versions; but once the development team already clearly knows that a certain bug or theoretical vulnerability exists, it will become more and more difficult to explain at the security and responsibility level if they continue to bring it into the official version.

Some developers pointed out that many problems in complex basic software such as databases cannot be solved by just throwing the bug report back to LLM and clicking "auto-fix". Some defects actually correspond to trade-offs in architecture and product semantics: a certain behavior may be a bug in usage scenario A, a reasonable design in scenario B, and in scenario C, it may mean that the basic design a year ago itself needs to be redone. AI can find out this contradiction very cheaply, but deciding "what the correct behavior is" still requires experienced maintainers.

Some people are even worried that another extreme will appear: "release paralysis". If AI can continuously dig out new edge problems from any complex code base, then theoretically there will always be the next bug. To what extent a project should be fixed before it is allowed to be released may become a more difficult problem than "how many bugs AI can find" in the future. Some developers suggest that instead of pursuing to eliminate all defects found by AI in a single version, it is better to register and grade them systematically and incorporate them into the subsequent development roadmap, otherwise no large software can achieve "zero defects" in the true sense.

Closing Remarks

After the coding capabilities of tools such as Claude Code and Codex become stronger and stronger, AI writing code is no longer a problem, and the PostgreSQL 19 incident shows that when AI can also review, test and attack code on a large scale, "discovering problems" is no longer a problem either.

The next truly scarce resource may become human judgment: which problems must be fixed, which problems are acceptable; whether an edge defect that can be fixed with 3 lines of code is worth delaying an important feature for; among dozens of problems given by AI, which ones mean that the architecture itself needs to be redesigned; and when the machine can look for anomalies endlessly, what level a piece of software needs to reach to be considered "good enough to release".

This article is from the WeChat Official Account "InfoQ" (ID: infoqchina), author: Chu Xingjuan, published with authorization from 36Kr.