Generating a volumetric representation of a capture region

A electronic device comprising:

• a display;
• an input device;
• an image sensor;
• one or more processors;
• non-transitory memory;
• and
• one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
• capturing, via the image sensor, at least one image of a physical environment;
• generating an extended reality (XR) environment, including a representation of the physical environment and a computer-generated object, based on the at least one image of the physical environment and the computer-generated object;
• while displaying, on the display, environment including the representation of the physical environment and the computer-generated object:
• generating a three-dimensional (3D) point cloud characterizing the physical environment;
• and
• obtaining depth information characterizing the physical environment;
• detecting, via the input device, a user input defining a capture region that includes a portion of the XR environment, wherein the capture region includes a portion of the physical environment and a portion of the computer-generated object;
• and
• in response to detecting the user input:
• selecting a group of points from the 3D point cloud characterizing the physical environment based on the capture region, wherein the group of points satisfies a spatial threshold relative to the capture region;
• generating, based on a function of the depth information and the group of points, a volumetric representation of the capture region, wherein the volumetric representation of the capture region includes a volumetric representation of the portion of the physical environment and a volumetric representation of the portion of the computer-generated object, wherein the volumetric representation of the portion of the physical environment is different from the representation of the physical environment, and wherein the volumetric representation of the portion of the computer-generated object is different from the computer-generated object;
• and
• storing the volumetric representation of the capture region in the non-transitory memory.

A non-transitory computer-readable medium including instructions, which, when executed by an electronic device including one or more processors, an input device, an image sensor, and a display, cause the electronic device to:

• capture, via the image sensor, at least one image of a physical environment;
• generate an extended reality (XR) environment, including a representation of the physical environment and a computer-generated object, based on the at least one image of the physical environment and the computer-generated object;
• while displaying, on the display, the XR environment including the representation of the physical environment and the computer-generated object:
• generate a three-dimensional (3D) point cloud characterizing the physical environment;
• and
• obtain depth information characterizing the physical environment;
• detect, via the input device, a user input defining a capture region includes a portion of the XR environment, wherein the capture region includes a portion of the physical environment and a portion of the computer-generated object;
• and
• in response to detecting the user input
• select a group of points from the 3D point cloud characterizing the physical environment based on the capture region, wherein the group of points satisfies a spatial threshold relative to the capture region;
• generate, based on a function of the depth information and the group of points, a volumetric representation of the capture region, wherein the volumetric representation of the capture region includes a volumetric representation of the portion of the physical environment and a volumetric representation of the portion of the computer-generated object, wherein the volumetric representation of the portion of the physical environment is different from the representation of the physical environment, and wherein the volumetric representation of the portion of the computer-generated object is different from the computer-generated object;
• and
• store the volumetric representation of the capture region.