Retrofit vision assist with monocular depth estimation
AssigneesToyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
InventorsYuki Wakayama, Erin A McColl
Life of this patent
Estimating the term…
Abstract
Systems, methods, and other embodiments described herein relate to retrofit vision assist systems for vehicles based on monocular depth estimation. In one embodiment, a method for operating a depth system of a vehicle includes acquiring a camera image from a monocular camera that is retrofitted to a vehicle. The method also includes estimating depth information of features in the camera image to output a depth map including three-dimensional data associated with the camera image. The method also includes generating an annotated image based on the three-dimensional data. The method further includes displaying the annotated image to aid a driver in understanding a surrounding environment of the vehicle.
Claims14
A system comprising:
- a processor;
- a memory communicably coupled to the processor and storing:
- a depth estimation module including instructions that, when executed by the processor, cause the processor to:
- calibrate a monocular camera retrofitted to a vehicle by receiving information regarding one or more dimensions of the vehicle, constructing a bounding box to relate the one or more dimensions with the monocular camera, and defining a location of the monocular camera relative to the vehicle using the bounding box;
- acquire a camera image from the monocular camera and additional images from additional cameras on the vehicle;
- estimate depth information of features including objects in the camera image and the additional images to by performing depth estimation using the bounding box to identify locations of the objects relative to the vehicle and output depth maps including three-dimensional data associated with the camera image and the additional images;
- generate an annotated image including a three-dimensional representation of a scene surrounding the vehicle based on the three-dimensional data;
- and
- display the annotated image to aid a driver in understanding the scene by superimposing colors onto the objects according to the depth information.
A non-transitory computer-readable medium including instructions that, when executed by a processor, cause the processor to:
- calibrate a monocular camera retrofitted to a vehicle by receiving information regarding one or more dimensions of the vehicle, constructing a bounding box to relate the one or more dimensions with the monocular camera, and defining a location of the monocular camera relative to the vehicle using the bounding box;
- acquire a camera image from the monocular camera and additional images from additional cameras on the vehicle;
- estimate depth information of features including objects in the camera image and the additional images by performing depth estimation using the bounding box to identify locations of the objects relative to the vehicle and output depth maps including three-dimensional data associated with the camera image and the additional images;
- generate an annotated image including a three-dimensional representation of a scene surrounding the vehicle based on the three-dimensional data;
- and
- display the annotated image to aid a driver in understanding the scene by superimposing colors onto the objects according to the depth information.
A method comprising:
- calibrating a monocular camera retrofitted to a vehicle by receiving information regarding one or more dimensions of the vehicle, constructing a bounding box to relate the one or more dimensions with the monocular camera, and defining a location of the monocular camera relative to the vehicle using the bounding box;
- acquiring a camera image from the monocular camera and additional images from additional cameras on the vehicle;
- estimating depth information of features including objects in the camera image and the additional images by performing depth estimation using the bounding box to identify locations of the objects relative to the vehicle and output depth maps including three-dimensional data associated with the camera image and the additional images;
- generating an annotated image including a three-dimensional representation of a scene surrounding the vehicle based on the three-dimensional data;
- and
- displaying the annotated image to aid a driver in understanding the scene by superimposing colors onto the objects according to the depth information.
Dependencies are inferred from the claim text.