System and method for monocular depth estimation from semantic information
AssigneeToyota Research Institute, Inc.
InventorsVitor GUIZILINI, Jie LI, Rares A. AMBRUS, Sudeep PILLAI, Adrien GAIDON
Life of this patent
Estimating the term…
Abstract
A method for controlling an ego agent includes capturing a two-dimensional (2D) image of an environment adjacent to the ego agent. The method also includes generating a semantically segmented image of the environment based on the 2D image. The method further includes generating a depth map of the environment based on the semantically segmented image. The method additionally includes generating a three-dimensional (3D) estimate of the environment based on the depth map. The method also includes controlling an action of the ego agent based on the identified location.
Claims20
A method for controlling an ego agent comprising:
- capturing a two-dimensional (2D) image of an environment adjacent to the ego agent;
- generating a semantically segmented image of the environment based on the 2D image;
- generating a depth map of the environment based on the semantically segmented image;
- generating a three-dimensional (3D) estimate of the environment based on the depth map;
- and
- controlling an action of the ego agent based on the identified location.
An apparatus for controlling an ego agent, comprising:
- a processor;
- a memory coupled with the processor;
- and
- instructions stored in the memory and operable, when executed by the processor, to cause the apparatus:
- to capture a two-dimensional (2D) image of an environment adjacent to the ego agent;
- to generate a semantically segmented image of the environment based on the 2D image;
- to generate a depth map of the environment based on the semantically segmented image;
- to generate a three-dimensional (3D) estimate of the environment based on the depth map;
- and
- to control an action of the ego agent based on the identified location.
A non-transitory computer-readable medium having program code recorded thereon for controlling an ego agent, the program code executed by a processor and comprising:
- program code to capture a two-dimensional (2D) image of an environment adjacent to the ego agent;
- program code to generate a semantically segmented image of the environment based on the 2D image;
- program code to generate a depth map of the environment based on the semantically segmented image;
- program code to generate a three-dimensional (3D) estimate of the environment based on the depth map;
- and
- program code to control an action of the ego agent based on the identified location.
Dependencies are inferred from the claim text.