Systems and methods for training a depth estimator to be scale-aware using weak sonar supervision
AssigneesToyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
InventorsVitor Campagnolo Guizilini, Edwardo Martinez
Life of this patent
Estimating the term…
Abstract
Systems and methods described herein relate to training a depth estimator to be scale-aware using weak sonar supervision. In one embodiment, a depth and scale training system trains a self-supervised monocular depth estimation (MDE) network by minimizing a photometric reprojection error between an original target image of a scene and a viewpoint-synthesized image. The depth and scale training system also supervises weakly the training of the MDE network by minimizing a loss computed between a depth image generated by the MDE network and a set of sparse sonar data points collected from the scene to teach the MDE network to estimate metric scale in connection with controlling a robot.
Claims17
A system for training a depth estimator to be scale-aware using weak sonar supervision, the system comprising:
- a processor;
- and
- a memory storing machine-readable instructions that, when executed by the processor, cause the processor to:
- train a self-supervised monocular depth estimation (MDE) network by minimizing a photometric reprojection error between an original target image of a scene and a viewpoint-synthesized image;
- and
- supervise weakly the training of the MDE network by minimizing a loss computed between a depth image generated by the MDE network and a set of sparse sonar data points collected from the scene to teach the MDE network to estimate metric scale in connection with controlling a robot, wherein the set of sparse sonar data points is transmitted to the robot from one of an electric-vehicle (EV) charging station external to the robot and an infrastructure device external to the robot.
A non-transitory computer-readable medium for training a depth estimator to be scale-aware using weak sonar supervision and storing instructions that, when executed by a processor, cause the processor to:
- train a self-supervised monocular depth estimation (MDE) network by minimizing a photometric reprojection error between an original target image of a scene and a viewpoint-synthesized image;
- and
- supervise weakly the training of the MDE network by minimizing a loss computed between a depth image generated by the MDE network and a set of sparse sonar data points collected from the scene to teach the MDE network to estimate metric scale in connection with controlling a robot, wherein the set of sparse sonar data points is transmitted to the robot from one of an electric-vehicle (EV) charging station external to the robot and an infrastructure device external to the robot.
A method, comprising:
- training a self-supervised monocular depth estimation (MDE) network by minimizing a photometric reprojection error between an original target image of a scene and a viewpoint-synthesized image;
- and
- supervising weakly the training of the MDE network by minimizing a loss computed between a depth image generated by the MDE network and a set of sparse sonar data points collected from the scene to teach the MDE network to estimate metric scale in connection with controlling a robot, wherein the set of sparse sonar data points is transmitted to the robot from one of an electric-vehicle (EV) charging station external to the robot and an infrastructure device external to the robot.
Dependencies are inferred from the claim text.