System and method for reconstructing unoccupied 3D space

Machine readable and non-transitory medium having information recorded thereon for understanding a three dimensional (3D) scene, wherein the information, when read by the machine, causes the machine to perform:

• receiving image data acquired, by a camera at different time instances, with respect to the 3D scene which includes at least one of a user and one or more objects present therein;
• detecting a face of the user at each of the different time instances;
• and
• with respect to each of at least some of the different time instances,
• generating a 2D user profile of the user based on the face detected at the time instance, wherein the 2D user profile represents a region in the image data occupied by the user at the time instance,
• obtaining a 3D position of the face of the user in the 3D scene corresponding to the region in the image data,
• determining, based on a face based human model, a 3D prism in the 3D scene at the 3D position,
• estimating a 3D volumetric space occupied by the user in the 3D scene based on the 3D prism,
• determining spatial relationships between the one or more objects and the 3D volumetric space,
• estimating, based on the 3D volumetric space and the spatial relationships, a 3D space occupancy of the 3D scene, and
• dynamically updating a 3D space occupancy record of the 3D scene to reflect the estimated 3D space occupancy of the 3D scene.

A system for understanding a three dimensional (3D) scene, comprising:

• a face detection unit configured for
• receiving image data acquired, by a camera at different time instances, with respect to the 3D scene which includes at least one of a user and one or more objects present therein, and
• detecting a face of the user at each of the different time instances;
• and
• a faced based human tracking unit configured for, with respect to each of at least some of the different time instances, generating a 2D user profile of the user based on the face detected at the time instance, wherein the 2D user profile represents a region in the image data occupied by the user at the time instance;
• a human 3D occupancy estimator configured for, with respect to the at least some of the time instances,
• obtaining a 3D position of the face of the user in the 3D scene corresponding to the region in the image data,
• determining, based on a face based human model, a 3D prism in the 3D scene at the 3D position,
• estimating a 3D volumetric space occupied by the user in the 3D scene based on the 3D prism, and
• a spatial relationship identifier configured for determining spatial relationships between the one or more objects and the 3D volumetric space;
• and
• a dynamic occupancy updater configured for dynamically updating a 3D space occupancy record of the 3D scene to reflect a 3D space occupancy of the 3D scene that is estimated based on the 3D volumetric space and the spatial relationships.