# US 2026/0065577 A1 · Systems, Methods, and Graphical User Interfaces for Scanning and Modeling Environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065577 A1
- Document ID: US20260065577A1
- Status: Published application
- Kind: A1
- Application number: 19/379,442
- Filed: 2025-11-04
- Published: 2026-03-05
- Earliest claimed: 2022-05-10
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Allison W. Dryer; Giancarlo Yerkes; Praveen Sharma; Joseph A. Malia
- Web page: https://patentgrep.com/patent/US20260065577A1

## CPC

- G06T 15/10: Geometric effects
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06T 15/50: Lighting effects
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer system displays graphical objects overlaying a representation of a field of view of one or more cameras, including displaying a first graphical object that represents one or more estimated spatial properties of a first physical feature that has been detected in a respective portion of the physical environment, and a second graphical object at that represents one or more estimated spatial properties of a second physical feature that has been detected in the respective portion of the physical environment. The computer system changes one or more visual properties of the first graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the first physical feature, and changes the one more visual properties of the second graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the second physical feature.

## Claims

### Claim 1

A method, comprising:
at a computer system that is in communication with a display generation component, one or more input devices, and one or more cameras:
during a scan of a physical environment to obtain depth information of at least a portion of the physical environment:
displaying, via the display generation component, a first user interface, wherein the first user interface includes a representation of a field of view of the one or more cameras; and
displaying a plurality of graphical objects overlaying the representation of the field of view of the one or more cameras, including displaying at least a first graphical object at a first location that represents one or more estimated spatial properties of a first physical feature that has been detected in a respective portion of the physical environment in the field of view of the one or more cameras, and a second graphical object at a second location that represents one or more estimated spatial properties of a second physical feature that has been detected in the respective portion of the physical environment in the field of view of the one or more cameras; and
while displaying the plurality of graphical objects overlaying the representation of the field of view of the one or more cameras, changing one or more visual properties of the first graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the first physical feature, and changing the one more visual properties of the second graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the second physical feature.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first graphical object includes a first set of one or more lines that represents the one or more estimated spatial properties of the first physical feature, and the second graphical object includes a second set of one or more lines that represents the one or more estimated spatial properties of the second physical feature.

### Claim 3 (depends on 2)

The method of claim 2, wherein displaying the first graphical object includes extending respective lengths of the first set of one or more lines in accordance with the respective predicted accuracy of the one or more estimated spatial properties of the first physical feature.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first graphical object includes a first filled area that represents the one or more estimated spatial properties of the first physical feature, and the second graphical object includes a second filled area that represents the one or more estimated spatial properties of the second physical feature.

### Claim 5 (depends on 1)

The method of claim 1, wherein changing the one or more visual properties of the first graphical object in accordance with the variations in the respective predicted accuracy of the estimated spatial properties of the first physical feature includes changing a respective opacity of the first graphical object in accordance with changes in the respective predicted accuracy of the estimated spatial properties of the first physical feature.

### Claim 6 (depends on 1)

The method of claim 1, wherein changing the one or more visual properties of the first graphical object in accordance with the variations in the respective predicted accuracy of the estimated spatial properties of the first physical feature includes changing a respective amount of feathering applied to edges of the first graphical object in accordance with changes in the respective predicted accuracy of the estimated spatial properties of the first physical feature.

### Claim 7 (depends on 6)

The method of claim 6, wherein changing the respective amount of feathering applied to edges of the first graphical object in accordance with changes in the respective predicted accuracy of the estimated spatial properties of the first physical feature includes:
in accordance with a determination that scanning of a corner corresponding to the first graphical object meets first criteria, decreasing the respective amount of feathering applied to the edges of the first graphical object; and
in accordance with a determination that scanning of the corner corresponding to the first graphical object has not met the first criteria, increasing the respective amount of feathering applied to the edges of the first graphical object.

### Claim 8 (depends on 7)

The method of claim 7, wherein increasing the respective amount of feathering and decreasing the respective amount of feathering the first criteria are executed in accordance with a determination that the first graphical object includes a structural object and not a non-structural object.

### Claim 9 (depends on 1)

The method of claim 1, wherein changing the one or more visual properties of the first graphical object in accordance with the variations in the respective predicted accuracy of the estimated spatial properties of the first physical feature includes changing a respective sharpness of the first graphical object in accordance with changes in the respective predicted accuracy of the estimated spatial properties of the first physical feature.

### Claim 10 (depends on 1)

The method of claim 1, wherein changing the one or more visual properties of the first graphical object in accordance with variations in the respective predicted accuracy of the estimated spatial properties of the first physical feature includes:
at a first time:
in accordance with a determination that the respective predicted accuracy of the estimated spatial properties of the first physical feature is a first accuracy value for a first portion of the first physical feature, displaying a first portion of the first graphical object with a first property value for a first visual property of the one or more visual properties, and
at a second time later than the first time:
in accordance with a determination that the respective predicted accuracy of the estimated spatial properties of the first physical feature is a second accuracy value for the first portion of the first physical feature, displaying the first portion of the first graphical object with a second property value for the first visual property of the one or more visual properties, wherein the second accuracy value is different from the first accuracy value, and the second property value is different from the first property value.

### Claim 11 (depends on 1)

The method of claim 1, wherein, at a third time,
in accordance with a determination that the respective predicted accuracy of the estimated spatial properties of the first physical feature is a third accuracy value for a third portion of the first physical feature, and a fourth accuracy value for a fourth portion of the first physical feature, displaying a third portion of the first graphical object with a third property value for a second visual property of the one or more visual properties, and displaying a fourth portion of the first graphical object with a fourth property value for the second visual property of the one or more visual properties, wherein the fourth portion of the first physical feature is different from the third portion of the first physical feature, the fourth accuracy value is different from the third accuracy value, and the fourth property value is different from the third property value.

### Claim 12 (depends on 1)

The method of claim 1, wherein:
the first physical feature includes a fifth portion of the first physical feature and a sixth portion of the first physical feature,
the fifth portion of the first physical feature is not visually occluded by another object in the field of view of the one or more cameras, and the sixth portion of the first physical feature is visually occluded by another object in the field of view of the one or more cameras, and
displaying the first graphical object includes:
displaying a fifth portion of the first graphical object corresponding to the fifth portion of the first physical feature with a fifth property value that corresponds to a fifth accuracy value of the respective predicted accuracy of the one or more estimated spatial properties of the first physical feature, and
displaying a sixth portion of the first graphical object corresponding to the sixth portion of the first physical feature with a sixth property value corresponding to a sixth accuracy value of the respective predicted accuracy of the one or more estimated spatial properties of the first physical feature,
wherein the sixth property value corresponds to a lower visibility than the fifth property value does in the first user interface.

### Claim 13 (depends on 1)

The method of claim 1, including:
in accordance with a determination that scanning of the first physical feature is completed:
displaying a respective change in the one or more visual properties of the first graphical object to indicate completion of the scan for the first physical feature; and
ceasing to change the one or more visual properties of the first graphical object in accordance with the variations in the respective predicted accuracy of the estimated spatial properties of the first physical feature.

### Claim 14 (depends on 13)

The method of claim 13, wherein displaying the respective change in the one or more visual properties of the first graphical object includes:
in accordance with a determination that the first physical feature is of a first feature type, displaying a first type of change in the one or more visual properties of the first graphical object to indicate completion of scanning of the first physical feature; and
in accordance with a determination that the first physical feature is of a second feature type different from the first feature type, displaying a second type of change, different from the first type of change, in the one or more visual properties of the first graphical object to indicate completion of scanning of the first physical feature.

### Claim 15 (depends on 14)

The method of claim 14, wherein the first graphical object includes a set of one or more lines, and displaying the first type of change in the one or more visual properties of the first graphical object to indicate completion of scanning of the first physical feature includes reducing an amount of feathering.

### Claim 16 (depends on 14)

The method of claim 14, wherein the first graphical object includes a surface, and displaying the second type of change in the one or more visual properties of the first graphical object to indicate completion of scanning of the first physical feature includes displaying a preset change sequence in one or more visual properties in the surface.

### Claim 17 (depends on 1)

The method of claim 1, including:
detecting that scanning of the first physical feature is completed; and,
in response to detecting that scanning of the first physical feature is completed, reducing visual prominence of the first graphical object from a first visibility level to a second visibility level lower than the first visibility level.

### Claim 18

A computer system in communication with a display generation component, one or more input devices, and one or more cameras, comprising:
one or more processors; and
memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:
during a scan of a physical environment to obtain depth information of at least a portion of the physical environment:
displaying, via the display generation component, a first user interface, wherein the first user interface includes a representation of a field of view of the one or more cameras; and
displaying a plurality of graphical objects overlaying the representation of the field of view of the one or more cameras, including displaying at least a first graphical object at a first location that represents one or more estimated spatial properties of a first physical feature that has been detected in a respective portion of the physical environment in the field of view of the one or more cameras, and a second graphical object at a second location that represents one or more estimated spatial properties of a second physical feature that has been detected in the respective portion of the physical environment in the field of view of the one or more cameras; and
while displaying the plurality of graphical objects overlaying the representation of the field of view of the one or more cameras, changing one or more visual properties of the first graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the first physical feature, and changing one more visual properties of the second graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the second physical feature.

### Claim 19

A computer readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by a computer system in communication with a display generation component, one or more input devices, and one or more cameras, cause the computer system to:
during a scan of a physical environment to obtain depth information of at least a portion of the physical environment:
display, via the display generation component, a first user interface, wherein the first user interface includes a representation of a field of view of the one or more cameras; and
display a plurality of graphical objects overlaying the representation of the field of view of the one or more cameras, including displaying at least a first graphical object at a first location that represents one or more estimated spatial properties of a first physical feature that has been detected in a respective portion of the physical environment in the field of view of the one or more cameras, and a second graphical object at a second location that represents one or more estimated spatial properties of a second physical feature that has been detected in the respective portion of the physical environment in the field of view of the one or more cameras; and
while displaying the plurality of graphical objects overlaying the representation of the field of view of the one or more cameras, change one or more visual properties of the first graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the first physical feature, and change one more visual properties of the second graphical object in accordance with variations in a respective predicted accuracy of the estimated spatial properties of the second physical feature.
