How far is AI from "understanding everything"? Let's first try it out on cancer cells and planetary orbits.

It is quite a revelation to see two completely unrelated research subjects appearing in a single technical report, which really shows how broad the horizons are when working with AI.

One is patient-derived organoids, while the other extends all the way to the macroscopic universe...

Let's start with the former. Researchers used the internal factors learned by the model to propose an intervention scheme of cytokine combined with antibody blockade, which was verified step by step through cell line, organoid and mouse experiments.

As for the latter, researchers fed the simulated planetary position and velocity trajectories to the model without informing it of Kepler's laws.

After it finished learning, researchers analyzed its internal prediction pattern, and the fitted slope of the extracted "orbital frequency - semi-major axis" relationship reached -1.4991 (the theoretical value of Kepler's third law is -1.5), with R²=0.99999999.

Two fields as unrelated as tumor research and planetary orbit calculation can surprisingly be handled by the same predictive learning method.

The boundless world is full of huge diversities, where weather, molecules, cells, patients, robots and even planets all follow completely different laws.

But when AI understands "how they change", is it possible to share a more underlying set of methods?

Recently, JEPA-Anything, newly released by PhAI Labs in collaboration with teams from universities including Oxford, Stanford, Princeton and the Chinese University of Hong Kong, is trying to answer this question.

According to the paper, JEPA-Anything is a cross-domain framework for learning predictive models across different "worlds".

Each domain retains its own observation form, context-target construction and encoder, but shares the subsequent prediction core and a unified latent world-state interface.

It involves a key mechanism called OPF (Orthogonal Predictive Factorization).

It can be simply understood as:

Do not pile up all kinds of changes in the world together for modeling and learning.

Modeling the world, one single prediction path may not be enough

First, let's clarify a concept: what exactly does "world model" mean in the research of JEPA-Anything?

The definition given in the JEPA-Anything paper balances broad applicability and operability.

The paper states that as long as a model can establish an internal state based on existing information (Context) and use it to predict another state in the same system, it can be classified into the category of Latent World Model.

The "another state" here can be the future in time, the hidden area or different perspectives in space, or even the result after applying a certain intervention.

Following this line of thinking, the industry has developed increasingly mature technical paths in the past few years.

For example, DreamerV3 focuses on building an environment model first, and then "conceiving" the consequences of actions inside the model; V-JEPA 2 combines video representation learning with robot control, allowing the robotic arm to predict the physical feedback brought by different behaviors before planning actions.

And JEPA — Joint-Embedding Predictive Architecture — is a quite representative route among them.

Compared with directly reconstructing the target state in the original space, JEPA chooses to make predictions in the representation space, without the need to restore every pixel or detail.

It first encodes the observed information into the internal representation in the latent space, and then completes the prediction in the representation space.

This design allows the model to allocate more representation capabilities to structures with predictive value, and reduces the interference of original space details such as texture and noise.

However, the standard JEPA usually lets all information flow into the same target embedding and go through the same prediction path.

But the changes in the real world are often not single-threaded.

For example, the overall translation of molecules is often accompanied by the microscopic vibration of internal atoms; the slow evolution of the macroscopic meteorological system is mixed with severe local disturbances; biological systems are even more complex, with multi-scale and multi-path changes occurring synchronously at all times.

When signals of different scales, different entities and different difficulty levels are forced to squeeze together, those signals that are easier to predict and have more significant changes may preferentially occupy more prediction capacity.

In contrast, those relatively weak but equally critical information is either canceled out in gradient conflicts, or completely covered by strong signals.

Researchers attribute this to the problem of "how to allocate prediction capacity", and put forward a new solution at the same time.

Since congestion will lead to extrusion, spatial decoupling should be carried out.

Thus came JEPA-Anything.

Its core mechanism OPF (Orthogonal Predictive Factorization) divides the originally complete target state into multiple mutually complementary subspaces, which are processed independently by different prediction branches, and finally recombined into the complete world state.

In order to prevent multiple branches from evolving into a redundant state where "multiple teams learn the same thing", OPF introduces orthogonal constraints to make different factors occupy non-overlapping prediction directions as much as possible, reducing repeated learning of the same type of information by multiple branches; at the same time, through factor activity constraints and encoder variance constraints, it avoids partial factor inactivation and reduces the risk of representation collapse.

It is worth noting that these factors are not pre-specified with rules like "you are responsible for speed", "you are responsible for temperature" or "you are responsible for a certain biological pathway".

What they learn is determined by the predictable structure in the data.

After the training is completed, these factors thus retain an internal interface available for subsequent analysis.

Same prediction core, verified across seven types of systems

No matter how elegant the methodology is, it ultimately needs to withstand the test of cross-domain generalization.

This is where the unique feature of JEPA-Anything's experimental design comes in.

It is not limited to traditional fields such as video representation or robot control, but directly applies the same set of Predictive Core to seven completely different systems for verification: vision, biology, clinical practice, control, molecule, physical field and weather.

The so-called "Anything" does not mean forcibly breaking and reconstructing data from all domains into the same input format. Images, single-cell sequences, molecular structures and even meteorological data still retain their most suitable encoders and data forms.

What it unifies is the following set of underlying rules:

After the data is converted into latent space representation, how the prediction part should build the model and complete the learning.

The paper first conducts experiments in the controlled dynamic environment CITRIS Interventional Pong constructed based on the simplified Pong, to verify whether the model can "decompose and learn the composite changes, and then recombine them".

During the training phase, the model only receives change data of a single factor; in the test phase, it needs to face multi-factor combined interventions that have never appeared before.

If the model only rigidly records the complete patterns in the training set, it will inevitably fail when facing new combinations; if the model learns the reusable state change rules between different changes, it will have a better chance to recombine them in unseen combinations.

Experimental results show that in the single intervention within the distribution, JEPA-Anything reduces the prediction error by about 11.7% compared with the standard JEPA; while in the unseen multi-factor combined intervention, the mean square error (MSE) also drops by about 3.5%, with improvements observed across all five paired training seeds.

Subsequently, the verification moves to more complex real systems.

The paper constructs a strictly aligned Matched Dynamics Benchmark, which ensures that the standard JEPA and JEPA-Anything use exactly the same training data, encoder, state transition backbone network, computing budget and evaluation split across ten tasks.

Among the nine prediction tasks, the performance metrics of JEPA-Anything have all been improved.

For example, for the Burgers equation in PDEBench, MSE drops by 39.7%; the shallow water equation drops by 39.3%; WeatherBench 2 drops by 10.5%.

The paper also specifically conducts multi-step rollout, making each prediction continue to be the input of the next prediction, to observe what happens after errors are continuously propagated.

The molecular system extends the timeline even longer.

In four systems: liquid water, α-quartz, paracetamol and benzene, the model needs to continuously predict the atomic positions and velocities in the future, and keep feeding its own output back into the model.

Finally, whether it is the MAE of one-step prediction or the RMSD of the final position after 100-step free rollout, JEPA-Anything achieves the lowest error in all four molecular systems.

However, it is worth noting that the experimental data does not show the idealized curve of "one single mechanism completely outperforms all others".

In the continuous control test, JEPA-Anything does not perform better in every task: on Walker2d and HalfCheetah, its average score is higher than that of the standard JEPA; but on Hopper, the standard JEPA performs better.

The longer the distance of continuous prediction, the more errors will continue to accumulate.

In the additional Burgers test, when the prediction length increases from 20 steps to 50 steps, although JEPA-Anything still leads, its advantage has been significantly reduced.

Therefore, this report verifies that "the general prediction principle has cross-domain applicability", rather than "a world model that unifies all domains has been realized".

But these results at least show that this set of prediction principles has the potential for cross-domain reuse.

Can the structure learned by JEPA-Anything withstand external verification?

The previous experiments mainly answered whether the prediction and state representation are effective. In the third set of experiments, the paper continues to ask a further question:

Do the factors formed in the model's latent space really have scientific exploration value?

Thus came the two research cases with completely different styles but echoing each other mentioned at the beginning of this article.

Case 1: From internal factors to verifiable candidate interventions

In liver cancer research, researchers did not simply regard JEPA-Anything as a "black box predictor", but carried out domain analysis on the Orthogonal Factor Coordinates formed inside it, and screened out a candidate combination: IL-18 combined with NT5E/CD73 blockade.

The research team advanced this scheme to the Huh7-PBMC co-culture system, then to patient-derived hepatocellular carcinoma organoids and tumor tissue fragments, and later added immunocompetent mice for verification.

Experimental data show that in 3 patient organoids and 3 tumor tissue fragments, the combination of IL-18 and CD73 blocker showed the strongest tumor cell killing effect, accompanied by significantly enhanced immune cell activation.

This means that the Latent Factor inside the model is no longer only responsible for making the prediction figures "look good".

It has become an interface for researchers to analyze system states and put forward experimentally verifiable candidate hypotheses, thus forming the chain of "internal model structure → domain analysis → candidate intervention → experimental verification".

Case 2: Checking against known physical laws from latent patterns

The planetary orbit case uses Kepler's third law, which humans have long known, to check whether the content learned inside the model has physical meaning.

Researchers only input the simulated planetary position and velocity trajectories into the model, without attaching any prior formulas.

After the training is completed, researchers perform spectrum analysis on the model's Latent Modes, and pair the extracted frequencies with the orbital semi-major axes.

The results show that even without artificially setting formulas in advance, researchers can still extract a scaling law highly consistent with the relationship of these variables from the latent modes learned by the model — the fitted slope is -1.4991, which is very close to the theoretical value of -1.5 with extremely small error.