Scene recognition using volumetric substitution of real world objects

A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:

• obtaining, by one or more sensors, a two-dimensional (2D) image of a scene through which a robot is to navigate, wherein the 2D image that lacks depth information about objects within the scene;
• identifying, based on matching (i) a portion of the 2D image of the scene that lacks depth information about objects within the scene with (ii) a stored, 2D image, an object within the image of the scene that is pre-associated with the stored, 2D image that matches the portion of the 2D image of the scene;
• after the object within the image of the scene has been identified based on matching (i) the portion of the 2D image of the scene that lacks depth information with (ii) the stored, 2D image, obtaining a three-dimensional model that corresponds to the object, the three-dimensional model being obtained from the database of three-dimensional models;
• determining, based on data from the three-dimensional model, an estimated distance between the one or more sensors and the object;
• generating or updating a three-dimensional representation of the scene through which the robot is to navigate based at least on (i) the estimated distance between the one or more sensors and the object, and (ii) dimensions of the object that are indicated by the three-dimensional model;
• providing the three-dimensional representation of the scene to a scene analyzer;
• and
• generating, for the robot to navigate and based at least on the three-dimensional representation of the scene, a navigation path through the scene that avoids contact between the robot and the object.

A system, comprising:

• a computing device;
• and
• a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations, the operations comprising:
• obtaining, by one or more sensors, a two-dimensional (2D) image of a scene through which a robot is to navigate, wherein the 2D image that lacks depth information about objects within the scene;
• identifying, based on matching (i) a portion of the 2D image of the scene that lacks depth information about objects within the scene with (ii) a stored, 2D image, an object within the image of the scene that is pre-associated with the stored, 2D image that matches the portion of the 2D image of the scene;
• after the object within the image of the scene has been identified based on matching (i) the portion of the 2D image of the scene that lacks depth information with (ii) the stored, 2D image, obtaining a three-dimensional model that corresponds to the object, the three-dimensional model being obtained from the database of three-dimensional models;
• determining, based on data from the three-dimensional model, an estimated distance between the one or more sensors and the object;
• generating or updating a three-dimensional representation of the scene through which the robot is to navigate based at least on (i) the estimated distance between the one or more sensors and the object, and (ii) dimensions of the object that are indicated by the three-dimensional model;
• providing the three-dimensional representation of the scene to a scene analyzer;
• and
• generating, for the robot to navigate and based at least on the three-dimensional representation of the scene, a navigation path through the scene that avoids contact between the robot and the object.