Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
arXiv:2508.04928v5 Announce Type: replace-cross Abstract: We propose a method to extend foundational monocular depth estimators (FMDEs), trained on perspective images, to fisheye images. Despite being trained on tens of millions of images, FMDEs are susceptible to the covariate shift introduced by changes in camera calibration (intrinsic, distortion) parameters, leading to erroneous depth estimates. Our method aligns the distribution of latent embeddings encoding fisheye images to those of perspective images, enabling the reuse of FMDEs for fisheye cameras without retraining or finetuning. To
This development arises as highly capable foundational models encounter real-world challenges in diverse sensor environments, necessitating adaptable solutions beyond their original training scope.
It allows for the immediate and efficient deployment of advanced AI depth perception capabilities in dynamic environments using fisheye cameras, critical for robotics, autonomous vehicles, and surveillance.
Foundational monocular depth estimators can now be effectively used with fisheye cameras without requiring costly retraining or fine-tuning, accelerating their adoption in varied applications.
- · Robotics companies
- · Autonomous vehicle developers
- · Surveillance technology providers
- · AI model developers
- · Companies specializing in fisheye-specific depth models
Wider deployment of AI-powered 3D perception in environments historically challenging for standard cameras accelerates.
Reduced development costs for systems integrating AI depth perception and unconventional camera types, fostering innovation.
Enhanced situational awareness for autonomous systems operating in complex, wide-angle viewing conditions becomes standard, impacting safety and efficiency across industries.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI