Methods and devices for detecting and identifying features in an AR/VR scene

An electronic device, comprising:

• a display;
• one or more processors;
• a non-transitory memory;
• and
• one or more programs stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions, which, when executed by the electronic device, cause the electronic device to perform operations comprising:
• generating, from pass-through image data characterized by a plurality of poses of a space, a three-dimensional (3D) point cloud for the space, wherein each of the plurality of poses of the space is associated with a respective field of view of an image sensor;
• obtaining characterization vectors for points of the 3D point cloud, wherein each of the characterization vectors includes one or more labels;
• disambiguating a first group of points from the 3D point cloud corresponding to a first object in the space, wherein characterization vectors for the first group of points satisfy an object confidence threshold based on the labels included in the characterization vectors for the first group of points;
• generating a first volumetric region for the first group of points, wherein the first volumetric region corresponds to a 3D representation of the first object;
• and
• synthesizing a two-dimensional (2D) floorplan of the space including a 2D boundary of the space and a 2D representation of the first object within the 2D boundary of the space corresponding to a top-down view of the first volumetric region.

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with a display, cause the electronic device to:

• generate, from pass-through image data characterized by a plurality of poses of a space, a three-dimensional (3D) point cloud for the space, wherein each of the plurality of poses of the space is associated with a respective field of view of an image sensor;
• obtain characterization vectors for points of the 3D point cloud, wherein each of the characterization vectors includes one or more labels;
• disambiguate a first group of points from the 3D point cloud corresponding to a first object in the space, wherein characterization vectors for the first group of points satisfy an object confidence threshold based on the labels included in the characterization vectors for the first group of points;
• generate a first volumetric region for the first group of points, wherein the first volumetric region corresponds to a 3D representation of the first object;
• and
• synthesize a two-dimensional (2D) floorplan of the space including a 2D boundary of the space and a 2D representation of the first object within the 2D boundary of the space corresponding to a top-down view of the first volumetric region.