2-D image reconstruction in a 3-D simulation

A system for presenting a 3-D representation of an object captured in a 2-D image of the object in an environment in a 3-D simulation of the environment, the system comprising:

• a storage configured to store instructions;
• and
• a processor configured to execute the instructions and cause the processor to:
• receive 3-D data representing the environment, the 3-D data representing the environment including a bounding box showing an approximate location of the object in the environment, image data of a visual representation of the object in the environment, and a label identifying the object,
• locate a position of the object in the 3-D simulation of the environment created from the received 3-D data representing the environment, the locate the position of the object in the 3-D simulation of the environment including,
• determine an average measurement of the object, a depth of the object in the environment, and an orientation of the object in the environment based on a semantic database of attributes associated with the object,
• determine the average measurement of the object based on a value stored in the semantic database of attributes associated with the object,
• determine the depth of the object in the environment based on a comparison of the average measurement of the object and an apparent measurement of the object in the 2-D image,
• determine the orientation of the object in the environment based on a type of the object and the approximate location of the object in the environment,
• obtain a 3-D representation of the object from the semantic database of attributes associated with the object, and
• project the 3-D representation of the object into the 3-D simulation of the environment at the determined position for the object.

A non-transitory computer readable medium for presenting a 3-D representation of an object captured in a 2-D image of the object in an environment in a 3-D simulation of the environment, the non-transitory computer readable medium comprising instructions, the instructions, when executed by a computing system, cause the computing system to:

• receive 3-D data representing the environment, the 3-D data representing the environment including a bounding box showing an approximate location of the object in the environment, image data of a visual representation of the object in the environment, and a label identifying the object;
• locate a position of the object in the 3-D simulation of the environment created from the received 3-D data representing the environment, the locate the position of the object in the 3-D simulation of the environment including;
• determine an average measurement of the object, a depth of the object in the environment, and an orientation of the object in the environment based on a semantic database of attributes associated with the object;
• determine the average measurement of the object based on a value stored in the semantic database of attributes associated with the object;
• determine the depth of the object in the environment based on a comparison of the average measurement of the object and an apparent measurement of the object in the 2-D image;
• determine the orientation of the object in the environment based on a type of the object and the approximate location of the object in the environment;
• obtain a 3-D representation of the object from the semantic database of attributes associated with the object;
• and
• project the 3-D representation of the object into the 3-D simulation of the environment at the determined position for the object.