One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models

The Depth Illusion: Why Monocular AI Models See a Flat World

A new paper from arXiv (2606.29600v1) tackles a fundamental blind spot in modern computer vision: the assumption that each pixel corresponds to exactly one depth value. The researchers demonstrate that current monocular depth estimation foundation models—trained on massive datasets to predict a single depth per pixel—systematically fail when confronted with scenes containing layered geometry, such as a person standing behind a glass window or a bird partially occluded by leaves.

What the Research Reveals

The core insight is deceptively simple. In the real world, a single camera ray often passes through multiple surfaces: the glass, the person behind it, and the wall beyond. Standard monocular depth models collapse this rich, layered structure into a single scalar, typically choosing the nearest visible surface. The paper shows that this compression is not just a minor loss of information—it actively creates geometric ambiguity that leads to incorrect scene interpretations. When the same scene is viewed from slightly different angles, the model's single-depth output can contradict itself, producing physically impossible geometries.

The researchers propose a framework to probe this ambiguity, measuring how often and under what conditions these "depth collisions" occur. Their findings suggest the problem is pervasive, especially in scenes with transparency, reflections, or fine occluding boundaries.

Why This Matters

This work strikes at a critical assumption underpinning autonomous driving, robotics, and AR/VR systems. If a self-driving car's depth perception treats a rain-streaked windshield as a solid wall, or a robot's grasp planner mistakes a reflection for a reachable object, the consequences range from navigation failures to safety hazards. The paper exposes that scaling up data and model size alone cannot fix a structural flaw in the task definition itself.

For AI practitioners, this is a wake-up call. The current paradigm of "one depth per pixel" is a convenient simplification, but it creates a ceiling on performance in real-world deployment. The research implicitly argues for a shift toward volumetric or multi-surface representations, where each ray can carry multiple depth hypotheses. This aligns with emerging work in neural radiance fields (NeRFs) and 3D Gaussian splatting, which naturally handle transparency and occlusion.

Implications for Practitioners

Engineers building on top of monocular depth models should treat current outputs as "most likely single surface" estimates, not ground truth geometry. Validation pipelines must include adversarial testing with layered scenes—glass, mesh, foliage, reflections. For safety-critical applications, fusing monocular depth with stereo or LiDAR remains essential, not optional.

The paper also suggests that training datasets need to include explicit multi-surface annotations, moving beyond the standard single-depth ground truth from LiDAR or structured light sensors. Without this, models will continue to learn a flawed mapping from 2D to 3D.

Key Takeaways

• Monocular depth models fundamentally collapse layered 3D geometry into a single depth per pixel, creating systematic errors in scenes with transparency, reflections, or fine occlusion.
• This structural limitation cannot be fixed by scaling data or model size—it requires rethinking the output representation to support multiple depth hypotheses per ray.
• Practitioners must validate depth models on adversarial layered scenes and avoid treating single-depth outputs as reliable geometry for safety-critical applications.
• The research points toward volumetric representations (NeRFs, Gaussian splatting) as a more faithful approach for scenes with geometric ambiguity.