The "Spatial Intelligence" benchmark test designed by Princeton AI Lab shows that humans scored 83.3%, while the highest score achieved by large models is only 39.7%.
Can AI truly understand the three-dimensional world?
Planning the next action, adjusting promptly after a cup is moved, and completing operations before the environment changes — dynamic tasks continuously pose more complex spatial requirements for AI. One fundamental basis supporting these behaviors is the reliable understanding of the space in front of us: where are the objects, and can observations from different perspectives be consistent with each other?
Researchers from Princeton AI Lab have released a new assessment for spatial intelligence: Same Space or Not (SSoN). It removes real-time changes and motion control, only provides four static indoor images, and asks the model to find the one that does not belong to the same scene, and explain why.
- Project Homepage: https://sson.aisexam.com/
- Paper Link: https://arxiv.org/abs/2610.23959
- Dataset: https://huggingface.co/datasets/HuiziCao/SSoN
- Code Link: https://github.com/sson-benchmark/SSoN
- Fine-tuned Weights: https://huggingface.co/HuiziCao/SSoN-Qwen2.5-VL-7B-LoRA
The results show a clear performance gap: on the full set of 1,000 questions, human accuracy reaches 83.3%. Among the six tested Vision-Language Models (VLMs), GPT-6 Astra Pro leads with 39.7%, and the remaining five all score below the 25% random baseline for four-option questions. Without the complexity of dynamic interaction, the human-AI gap has already emerged in the more fundamental static cross-perspective understanding task.
Why do some scores fall below 25%? This is the expected value of blind random guessing, not the lower limit of model capability. SSoN is specially constructed with strong interference views that "look altered but are spatially correct". If a model can detect appearance differences but cannot judge whether they come from perspective changes, it may be systematically misled, resulting in performance even worse than random guessing. This does not mean the model has no visual capability at all, but exposes the gap between recognizing differences and understanding spatial logic.
And the 39.7% score is only the accuracy of selecting the correct answer, while the reasoning provided in some of these correct responses is still not valid.
Why are all 17 visual encoders scoring below the random baseline? When the models are asked to perform actual 3D reconstruction, why do they still fail to get half of the questions right? This assessment breaks down layer by layer where spatial intelligence is stuck for current AI models.
Figure 1 | SSoN task and performance on the full 1000-question set. Human answer accuracy reaches 83.3%, while the highest score among tested VLMs is 39.7%; the bar chart further distinguishes correct choices supported by sufficient evidence from correct choices with invalid reasoning.
Collective Partial Weakness in Spatial Intelligence:
Swaps and Detail Changes Are Relatively Easy, While Rotation and Movement Become Common Blind Spots
The rules of SSoN are very simple: among the four images, three come from an unmodified 3D scene, and one comes from an edited version. There is no designated "correct reference image", nor are abnormal objects marked in advance. The model needs to determine for itself which observations can mutually confirm each other, and which one cannot coexist with the others.
The basketball in the figure is a typical example. Image B and D jointly confirm its position; Image C does not capture that position, and shows objects not presented in other views, all of which can be explained by perspective changes. But Image A captures the position where the basketball should be, while the basketball is missing. The key lies in whether the spatial relationships are self-consistent, and it is very easy to make wrong choices only by judging from "they look different".
When the results are broken down by question type, the six types of edits roughly present three difficulty levels, which also outline different gaps in the spatial capabilities of current models.
Swaps and detail changes are the relative advantage areas of models. Swapping the positions of two objects, or changing the appearance of a single object such as its shape, color, and texture, can provide clues for judgment regarding object correspondence and appearance differences. The accuracy of the six tested models on these two categories is higher than their respective overall scores; the top-performing Astra Pro achieves over 80% accuracy on both categories.
Mirroring and Deletion are in the middle layer, but still very difficult for most models. Mirroring questions require judging whether the room structure is actually flipped left and right, excluding visual changes caused by the camera moving to the other side; deletion questions need to distinguish between "the object has truly disappeared" and "it is only occluded or moved out of the field of view". Not seeing an object does not mean it does not exist. For mirroring questions, except for Astra Pro, the remaining five models all score below the 25% random baseline; for deletion questions, the highest score among these five models is only 42.57% achieved by Sol Pro.
Movement and Rotation expose more common shortcomings in geometric reasoning. The highest accuracy of the six models on movement questions is only 27.20%; the score for rotation questions is even lower, with the highest value reaching only 9.54%, while humans achieve over 70% accuracy on both categories. In subsequent experiments, even when the models were explicitly informed what the six types of edits were, the scores for these two categories still could not be effectively improved.
When a chair changes its position and orientation in the frame, is it the chair that moved, or the camera that moved? When there are several chairs with identical appearances in the room, the model must first identify the same instance, then combine references such as tables and walls to judge its position and orientation. After recognizing objects and detecting appearance differences, the model still needs to cross the barrier of "understanding 3D relationships".
SSoN extends this kind of test to 7 indoor scene sources, covering 6 types of 3D edits: swap, detail change, mirroring, deletion, movement and rotation. With 1000 questions and 4000 images, the differences in capabilities under the total score are revealed one by one: which changes can be easily captured by the models, and which spatial relationships will still cause them to lose points collectively.
Figure 2 | 7 scene sources, 6 types of 3D edits, as well as different image formats and object scales. Perspective changes can create visual differences, and real edits may only leave local clues.
Even With 17 Visual Encoders, All Still Score Below the Random Baseline
What if we try another solution? Since there is only one abnormal image, is it enough to directly find the one that is the most dissimilar from the other three?
Researchers tested this idea using 17 visual encoders from the CLIP, ViT and DINOv2 series. Selecting the outlier view based on image feature similarity only yields an accuracy ranging from 16.9% to 21.7%, all below the 25% random baseline.
This result concretely demonstrates the effect of strong interference: a valid new perspective may change a large part of the visible content, while a real edit may only leave local clues. Selection rules that rely on global appearance differences cannot reliably distinguish between the two.
VLMs can also be misled by such superficial differences. In diagnostic experiments during the development phase, legitimate but misleading views attracted more wrong choices; after replacing them with clear frontal references, the performance of multiple types of questions improved significantly. The replacement reduces interference and adds direct evidence at the same time, which also makes the core difficulty clearer: being able to recognize changes when they are placed directly in front of you does not guarantee that you can still reason about them when the perspective changes.
Humans can temporarily accept an unfamiliar perspective, then check the objects in it that are associated with other images. What SSoN tests is exactly whether the model can complete this step of cross-perspective spatial integration.
Nearly 30% of Even Correct Answers Have Unqualified Reasoning
What is more noteworthy than the low scores is that some correct choices do not necessarily correspond to correct spatial judgments.
A model may select the correct answer A, but claim that an object that has not changed at all has been moved; or it may provide clues that are true but not sufficient to exclude other views. If only the final choice of A, B, C, D is counted, these responses will all be marked as correct.
SSoN therefore checks both the answer and the supporting evidence at the same time. "True solve" requires that the choice is correct, and the reasoning conforms to the image facts and is sufficient to support the cross-perspective contradiction judgment. Among a total of 1368 correct choices made by the six models, 29.5% failed this check. The highest model performance dropped from an answer accuracy of 39.7% to a True solve rate of 34.9%.
Nearly 30% of correct choices lack valid supporting evidence, which indicates that "selecting the right answer" still cannot replace "clearly explaining why". This check targets the reasoning written by the model, and does not infer how it thinks internally.
Then, could it be that the questions themselves are too tricky and do not have sufficient evidence at all?
SSoN has been verified on both the question generation and grading sides: during construction, it is required that the abnormal image and at least two normal views provide usable evidence, excluding multi-solution questions and obvious rendering shortcuts; during grading, all model responses with correct choices are manually reviewed for evidence. Through iteration of more than 10,000 development candidate questions, the problems manually identified are continuously turned into new generation constraints, and finally these 1000 questions are retained.
The assessment needs to have a valid solution, and the answer needs to be well-founded, so that the model can be required to truly point out the contradictions in the scene.
Figure 3 | SSoN ensures sufficient evidence and unique answers for each question through automatic checks and manual verification of every single question, and eliminates visual shortcuts. The reasoning evaluation separately covers all correct choices, with the final label confirmed by human reviewers.
Prompting, Reconstruction, Camera Parameters: Which Kind of Help Can Turn the Tide?
Since the difficulty lies in cross-perspective reasoning, if we teach the model more problem-solving methods, or let it actually reconstruct the 3D scene, will the performance improve?
Researchers selected another 100 questions to form an intervention set, whose original answer accuracy is 24% and True solve rate is 12%. All subsequent comparisons are conducted within this fixed set of questions.
First, let's look at prompting. Researchers guide the model to find environmental references, match objects, judge occlusion, or first describe the scene layout; they also directly inform the model of the six possible edit types without revealing the type of the current question. When comparative guidance and edit type knowledge are combined, the accuracy reaches 48%, and the True solve rate reaches 34%. Prompting can provide help, but it still does not enable the model to get half of the questions right.
What if we let the model actually perform 3D reconstruction, then check the camera perspective? This complete workflow with execution verification achieves 47% accuracy and 34% True solve rate. The deletion and detail change categories perform well, but movement and rotation are still difficult. Simply adding 3D reconstruction to the problem-solving process is not enough to eliminate the shortcomings in geometric judgment.
The most obvious improvement comes from directly providing real camera information.
On the basis of comparative guidance, after adding the ground truth camera intrinsic parameters, extrinsic parameters and image size, the accuracy reaches 61%, and the True solve rate reaches 48%. The model thus obtains information about imaging geometry, the relative position and orientation of the camera, and does not need to infer these relationships entirely from the images.
The improvement is particularly prominent for movement questions: under the comparative guidance condition, both accuracy and True solve rate are 20%; after adding camera information, both rise to 70%. This suggests that camera geometry can help models distinguish between "the object moved in the frame" and "the object moved in the room".
But there is still a gap between 61% and 48%. Even with real camera parameters, less than half of the correct responses to these 100 questions are sufficiently supported by evidence. Prompting, tools and geometric information can alleviate the difficulty, but they still cannot make cross-perspective understanding reliable.
Can Spatial Intelligence Be Trained Out? Performance on Unseen Rooms Nearly Doubles
If the help provided during reasoning is not enough, what if we specifically train this capability?
Researchers fine-tuned Qwen2.5-VL-7B, and separated the train, val and test sets by associated rooms and scene families. The model is finally tested on 201 test questions from completely unseen rooms.
Without fine-tuning, the accuracy is 21.89%; with only answer supervision, it rises to 33.33%; with both reasoning basis and answer supervision, it reaches 41.79%. Compared with the original model, the accuracy nearly doubles.
The difference here lies in whether the model is told during training "why the evidence supports this answer". Reasoning basis supervision further provides the change type, related objects and supporting views, so that the model learns to connect cross-perspective evidence with the final choice; during testing, the model still needs to look at the images and answer the questions by itself.
The most prominent change also appears in rotation questions. Answer supervision has already made the accuracy of rotation questions reach 32.84%, and after adding reasoning basis supervision, the accuracy of rotation questions rises to 58.21%. After changing the option order of the four images, the average accuracy and the proportion of questions answered correctly consistently across different orders also improve.
This points out a direction worth further exploration: Spatial consistency judgment can be improved through targeted supervision, and the capability can be transferred to rooms that have never been seen during training. The improvement is still uneven, and the additional overall gain brought by reasoning basis supervision also requires more training repetitions for verification, but this assessment has begun to point out which capabilities can be supplemented.
Before AI Enters the Physical World, It Needs to Learn to Understand the Same World First
The requirement put forward by SSoN is very simple: the camera can move while the scene remains unchanged; the images can look different while the spatial relationships are still consistent.
Dynamic spatial intelligence requires transforming observations into planning, decision-making and action, and reliably understanding the current space is the foundation supporting these behaviors. SSoN removes motion control and real-time changes, allowing this foundation to be tested separately: can the model integrate different perspectives, accept reasonable visual differences, and then find the real scene contradictions?
This assessment released by AI Vision Exam uses controlled indoor scenes to test this capability separately. The performance gap between 83.3% and 39.7% reveals current shortcomings, and experiments on prompting, geometry and training point out directions for further