Can AI predict "black swan" events in the stock market?

Philip K. Dick, the author of *Do Androids Dream of Electric Sheep?*, once asked what distinguishes humans from androids. AI is rapidly gaining traction in the asset management industry, yet financial markets have repeatedly experienced "black swan" events that cannot be explained by historical data...

Philip K. Dick, the author of the science fiction novel *Do Androids Dream of Electric Sheep?* which was adapted into the film *Blade Runner*, once posed the question of what sets humans apart from androids.

In the field of asset management, artificial intelligence (AI) is also spreading at a rapid pace, stepping into domains that once relied solely on humans for analysis and investment decision-making. AI demonstrates overwhelming capabilities in data processing. However, financial markets have repeatedly witnessed "black swan" events that defy explanation using past datasets. The pressing question now is: "Do AIs dream of black swans?"

The Landscape of "Asset Management Relies on Experience and Intuition" Is Undergoing Transformation

UK-based hedge fund Man Group is developing its own proprietary AI system, named "AlphaGPT". The AI can operate nonstop around the clock to process massive volumes of financial data at high speed, continuously generating investment propositions such as "Do companies with efficient recruitment strategies deliver better performance?" as potential investment ideas.

AI agents combine seemingly unrelated factors to formulate hypotheses in large quantities. It is reported that the system can generate dozens of hypotheses within minutes, expanding the scope of exploration exponentially far beyond the limits of human processing capacity.

Other dedicated AI agents are tasked with writing code to validate these hypotheses, enabling rapid iterative testing using automatically generated programs. The results are then passed to a third set of AI agents, which evaluate whether the hypotheses hold economic validity and logical coherence.

While this framework currently operates under human oversight, the firm states it will explore the feasibility of expanding the scope of automated monitoring in the future.

With the advancement of generative AI, asset management is shifting from a world that depends entirely on professional investors' experience and intuition to a new paradigm where AI takes on a leading role.

For instance, hedge fund Numerai has garnered attention after J.P. Morgan Asset Management pledged to invest up to 500 million USD in it. The platform operates on a mechanism that invites data scientists from across the globe to submit predictive models, pitting each model against one another to compete for performance rankings. The fund aggregates these collective insights to execute investment strategies, and distributes performance-based rewards using its native token.

The emergence of generative AI is even eroding the once-rigid boundary between professional investors and retail investors. By simply instructing AI to process massive datasets, users can instantly generate executable code for trading operations. We are entering an era where individuals without extensive formal financial expertise or advanced programming skills can easily build autonomous trading systems.

Other emerging use cases are also emerging, such as learning the "tacit knowledge" embedded in veteran fund managers' intuition and accumulated experience, and the Multi-Agent framework that facilitates multi-party discussions among multiple AI agents to refine investment ideas. The frontier of AI applications in finance is continuously expanding.

Of course, financial markets cannot be fully explained solely by historical data. "Black swan" events such as wars, global pandemics, and financial crises often unfold in ways that completely break away from past trends. How AI will respond to events entirely absent from its training datasets remains a major unknown.

In the United States, vigilance toward new risks introduced by autonomous AI trading is on the rise. In late June, Democratic members of Congress raised specific concerns in a letter requesting feedback from the U.S. Securities and Exchange Commission (SEC).

One key concern is herd behavior: if multiple AIs trained on similar datasets and optimized for aligned objectives respond synchronously to the same market signals, it could amplify one-sided trading flows and exacerbate market volatility.

Furthermore, when large-scale buying and selling operations are executed automatically by AI within extremely short timeframes without any human intervention, the market impact can spread exponentially in an instant, creating inherent systemic risks.

Market reactions to misinformation are equally worrisome. A previous incident saw a fabricated image purporting to show an explosion near the U.S. Department of Defense (the Pentagon) circulate online, triggering temporary volatility across financial markets. Market participants fear that malicious actors could exploit AI's automated response mechanisms to manipulate markets, a risk that can never be fully eliminated.

AI technology could also potentially be weaponized to conceal misconduct such as conflicts of interest and market manipulation. Research indicates that in simulated market environments, AIs acting to maximize profits may naturally gravitate toward behavior that borders on market rigging. There are also arguments that even when individual AIs hold no malicious intent, they may inadvertently act in ways that resemble coordinated collusion under default conditions.

On the other hand, AI is also widely regarded as a promising powerful tool for risk management, with the potential to take over tasks such as identifying anomalies that are nearly imperceptible to human observers.

Professor Kiyoshi Izumi from the University of Tokyo has conducted two research projects focused on crisis response. The first project constructs crisis scenarios in a simulated virtual market for options trading. Valid hedging strategies verified in this controlled virtual environment are expected to enhance the overall stability of real-world financial markets.

The second project centers on the concept of an Artificial Market. The research team builds a "Digital Twin" that replicates real financial markets by deploying a large number of AI agents representing diverse types of market participants. Within this virtual simulation, researchers can manually trigger extreme scenarios such as market crashes, then track the behavioral patterns of all AI agents to simulate and analyze questions like what factors could trigger a market collapse, and which combinations of risk factors create the most dangerous outcomes.

When asked whether "Flash Crashes" characterized by sudden evaporations of market liquidity will occur again in the future, Professor Izumi stated: "The answer is both yes and no". He argues that on one hand, the risk remains that AIs could act in complete lockstep within extremely short time windows, amplifying extreme price fluctuations. On the other hand, some AIs will automatically identify mispricing and correct deviations from fair value. Whether the market will spiral into extreme volatility or stabilize will ultimately depend on which of these two opposing forces gains the upper hand.

Regarding whether financial crises can be predicted in advance, Professor Izumi notes that this remains far from straightforward. "This is somewhat analogous to earthquake prediction: while early warnings might be issued when a crisis is imminent, long-term forecasting remains extremely challenging. This is precisely why conducting pre-emptive simulations using artificial market frameworks is so critical".

Takanobu Mizuta, a board member of the Japanese Society for Artificial Intelligence and a senior executive at SPARX Asset Management, believes there are equal possibilities that AI could either amplify financial crises or mitigate them. For example, in high-frequency trading, AIs can still be vulnerable to misleading signals. However, if AIs deployed across the market adopt more diverse decision-making frameworks, they could actively contribute to market stability.

Mizuta argues that a far more concerning threat is "Prompt Injection". This refers to attacks launched by malicious third parties who tamper with or hijack the instructions received by AI systems, manipulating them to make erroneous decisions without the user's awareness.

The Trap of Innovation

In the coming future, discussions focused on establishing effective "Guardrails" to preserve fair and orderly market operations will undoubtedly intensify.

AI systems will never consciously recognize that they are facing a so-called "black swan event". They will simply keep executing what they calculate to be the most rational decisions according to their predefined objective functions. However, the cumulative effect of these seemingly individually rational behaviors could potentially lead to unforeseen systemic instability across the entire market, a risk that cannot be dismissed.

Keiichi Omura, Professor Emeritus at Waseda University, points out that the historical evolution of financial innovation is simultaneously a story of continuously improving market efficiency and convenience, and a chronicle that constantly introduces new sources of market fragility. "The same holds true for AI, which is not immune to the 'Trap of Innovation'. Therefore, we must implement effective governance for new technologies equipped with robust safety mechanisms", he states. He emphasizes that while advancing AI adoption, governance frameworks, ethical norms, and third-party oversight systems must be developed at an equally rapid pace.

Dick's novel poses the fundamental question of "What makes us human?" This same question is now emerging in financial markets where AI is integrated into investment decision-making processes. Trained on massive volumes of historical data, AI can act as a catalyst that exacerbates crises, a tool that detects early warning signs of risks, or a mechanism that contains the spread of turmoil. Ultimately, however, the responsibility still falls on humans to imagine unprecedented "black swan" events that have never occurred in history, and prepare proactively for their possibility. AI may never dream of black swans. For that very reason, it must be humans who keep dreaming of them.

This article originates from the WeChat Official Account "Nikkei Chinese" (ID: rijingzhongwenwang), authored by Kazuaki Fujita, and published by 36Kr with authorized permission.