KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs
Accepted at ECCV 2026
Aravindh Mahendran1*,
Michael King1*,
Matthew Koichi Grimes1*,
Antoine Yang1,
Tyler Zhu2,
Joseph Heyward1,
Tengda Han1,
Shiry Ginosar3,
Chen Sun1,
Dima Damen1,
Simon Osindero1,
Noah Snavely1,
Simon Lynen4,
João Carreira1,
Viorica Pătrăucean1✉
1Google DeepMind
2Princeton University
3Toyota Technological Institute at Chicago
4Google
*core contributors
✉corresponding author
Abstract
We push the frontier of large-scale spatial intelligence in Vision-Language Models (VLMs) and introduce the first benchmark that probes geographical layout understanding from real-world videos, spanning up to 1km distances. Inspired by the cognitive science literature, we evaluate models against the hierarchical stages of human spatial awareness: anchoring via landmarks, connecting them through routes, and integrating these into global mental maps. Extensive experiments reveal a fundamental divergence in how current AI models process spatial information. Instead of utilising true path integration or forming geometric survey knowledge, we find that VLMs rely almost entirely on 2D visual recognition and text-matching to bypass complex spatial reasoning. The benchmark will be made publicly available.
The Landmark-Route-Map Paradigm
Our benchmark evaluates VLMs across three cognitive stages of spatial awareness based on 288 hours of real-world YouTube Walking Tour videos grounded via Google's Visual Positioning System (VPS):
- Landmarks: The ability to identify and localise salient objects.
- Routes: Egocentric turn-by-turn representations linking locations.
- Maps (Survey Knowledge): Allocentric maps integrating multiple routes.
Key Findings
We evaluated state-of-the-art models including Gemini 2.5 Flash, PLM-8B, Claude Opus, and GPT-5 against a human baseline. Our experiments reveal:
- Visual recognition vs. spatial grounding: VLMs excel at finding "needle-in-a-haystack" landmarks visually but fail to infer Euclidean distance to those landmarks.
- Semantic knowledge over path integration: Models bypass path integration by relying extensively on reading semantic cues like street signs or text labels on maps.
- Illusion of survey knowledge: VLMs possess only an illusion of map-level understanding. While some achieve super-human performance on loop closure, they do so via brute-force visual frame matching rather than forming an underlying geometric mental map.
Citation
@inproceedings{mahendran2026kilometer,
title={Kilometer Vision: A New Frontier for Large-Scale Spatial Intelligence in VLMS},
author={Mahendran, Aravindh and King, Michael and Grimes, Matthew Koichi and Yang, Antoine and Zhu, Tyler and Heyward, Joseph and Han, Tengda and Ginosar, Shiry and Sun, Chen and Damen, Dima and Osindero, Simon and Snavely, Noah and Lynen, Simon and Carreira, Jo{\~a}o and P{\v{a}}tr{\v{a}}ucean, Viorica},
booktitle={European Conference on Computer Vision (ECCV)},
year={2026}
}
