# US 2025/0157136 A1 · Methods for manipulating objects in an environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0157136 A1
- Document ID: US20250157136A1
- Status: Published application
- Kind: A1
- Application number: 19/026,161
- Filed: 2025-01-16
- Published: 2025-05-15
- Earliest claimed: 2020-09-11
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: William D. LINDMEIER; Matan STAUBER; Stephen O. LEMAY; Matthew J. SUNDSTROM; Madeleine S. CORDIER; Peter D. ANTON
- Web page: https://patentgrep.com/patent/US20250157136A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 15/20: Perspective computation
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/2016: Rotation, translation, scaling
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

In some embodiments, an electronic device enhances a two-dimensional drawing. In some embodiments, an electronic device manipulates an object in a three-dimensional environment. In some embodiments, an electronic device displays a contextual menu or moving an object in a three-dimensional environment. In some embodiments, an electronic device moves an object in a three-dimensional environment by an amount based on the distance of the object from the user. In some embodiments, an electronic device scans a real world object. In some embodiments, an electronic device manipulates a distant object.

## Claims

### Claim 1

A method comprising:
at an electronic device in communication with a display generation component and one or more input devices:
receiving, via the one or more input devices, a user input corresponding to a two-dimensional drawing;
in response to receiving the user input corresponding to the two-dimensional drawing, displaying, via the display generation component, a representation of the two-dimensional drawing; and
after receiving the user input and while displaying the representation of the two-dimensional drawing, displaying, via the display generation component, one or more selectable options;
while displaying the one or more selectable options, detecting selection of a respective selectable option of the one or more selectable options; and
in response to detecting selection of the respective selectable option, replacing display of the two-dimensional drawing with a three-dimensional object with an appearance that is determined based on an appearance of the two-dimensional drawing.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more selectable options include a first selectable option and a second selection option, and:
in accordance with a determination that the respective selectable option is the first selectable option of the one or more selectable options, the three-dimensional object is a first three-dimensional object; and
in accordance with a determination that the respective selectable option is the second selectable option of the one or more selectable options, different from the first selectable option, the three-dimensional object is a second three-dimensional object, different from the first three-dimensional object.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
in accordance with a determination that the two-dimensional drawing corresponds to a first shape, the one or more selectable options includes a first set of selectable options based on the first shape; and
in accordance with a determination that the two-dimensional drawing corresponds to a second shape, different from the first shape, the one or more selectable options includes a second set of selectable options, different from the first set of selectable options, based on the second shape and does not include the first set of options.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the respective selectable option of the one or more selectable options includes a representation of the three-dimensional object with the appearance that is determined based on the appearance of the two-dimensional drawing.

### Claim 5 (depends on 1)

The method of claim 1, wherein the one or more selectable options include one or more of:
a first option that is selectable to replace display of the two-dimensional drawing with a representation of a respective shape corresponding to the two-dimensional drawing, wherein a path of a border of the representation of the respective shape is different from a path of a border of the two-dimensional drawing;
a second option that is selectable to fill an interior of the two-dimensional drawing;
a third option that is selectable to replace display of the two-dimensional drawing with a first three-dimensional object that includes a first side and a second side, opposite of the first side, that have shapes corresponding to the two-dimensional drawing;
a fourth option that is selectable to replace display of the two-dimensional drawing with a second three-dimensional object that corresponds to a representation of a respective three-dimensional object, wherein the two-dimensional drawing corresponds to a two-dimensional drawing of the respective three-dimensional object; or
a fifth option that is selectable to replace display of the two-dimensional drawing with font-based text corresponding to the two-dimensional drawing.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
in accordance with a determination that the two-dimensional drawing corresponds to a two-dimensional drawing of a first respective three-dimensional object, the three-dimensional object corresponds to a representation of the first respective three-dimensional object; and
in accordance with a determination that the two-dimensional drawing corresponds to a two-dimensional drawing of a second respective three-dimensional object, different from the first respective three-dimensional object, the three-dimensional object corresponds to a representation of the second respective three-dimensional object.

### Claim 7 (depends on 6)

The method of claim 6, wherein in accordance with a determination that the two-dimensional drawing corresponds to a two-dimensional drawing of the first respective three-dimensional object and also corresponds to a two-dimensional drawing of a third respective three-dimensional object, different from the first respective three-dimensional object, the one or more selectable options include:
the respective selectable option that is selectable to replace display of the two-dimensional drawing with the representation of the first respective three-dimensional object, and
a second respective selectable option that is selectable to replace display of the two-dimensional drawing with a representation of the third respective three-dimensional object.

### Claim 8 (depends on 1)

The method of claim 1, wherein:
the user input corresponding to the two-dimensional drawing is received using a pointing device in communication with the electronic device, and
detecting selection of the respective selectable option of the one or more selectable options includes determining, via an eye tracking device in communication with the electronic device, that a gaze of a user of the electronic device is directed to the respective selectable option when a selection input is received at the pointing device.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the user input corresponding to the two-dimensional drawing is received using a pointing device in communication with the electronic device, and
detecting selection of the respective selectable option of the one or more selectable options includes determining that the pointing device is directed to the respective selectable option when a selection input is received at the pointing device.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
in response to receiving the selection input at the pointing device, causing a tactile output to be generated at the pointing device.

### Claim 11 (depends on 10)

The method of claim 10, wherein a color of the three-dimensional object is based on a color of the two-dimensional drawing.

### Claim 12 (depends on 1)

The method of claim 1, wherein:
the three-dimensional object is parametrically generated based on the two-dimensional drawing.

### Claim 13 (depends on 1)

The method of claim 1, wherein:
the three-dimensional object is selected, based on the two-dimensional drawing, from a predetermined library of three-dimensional objects.

### Claim 14 (depends on 1)

The method of claim 1, wherein:
the three-dimensional object includes a first side and a second side, opposite of the first side, that have shapes corresponding to the two-dimensional drawing,
a path of a border of the first side and the second side corresponds to one or more properties of the two-dimensional drawing, and
the path of the border of the first side and the second side is different from a path of a border of the two-dimensional drawing.

### Claim 15 (depends on 1)

The method of claim 1, wherein displaying the one or more selectable options is performed in response to determining, via an eye tracking device in communication with the electronic device, that a gaze of a user of the electronic device is directed to the representation of the two-dimensional drawing.

### Claim 16

An electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving, via one or more input devices, a user input corresponding to a two-dimensional drawing;
in response to receiving the user input corresponding to the two-dimensional drawing, displaying, via a display generation component, a representation of the two-dimensional drawing; and
after receiving the user input and while displaying the representation of the two-dimensional drawing, displaying, via the display generation component, one or more selectable options;
while displaying the one or more selectable options, detecting selection of a respective selectable option of the one or more selectable options; and
in response to detecting selection of the respective selectable option, replacing display of the two-dimensional drawing with a three-dimensional object with an appearance that is determined based on an appearance of the two-dimensional drawing.

### Claim 17

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:
receiving, via one or more input devices, a user input corresponding to a two-dimensional drawing;
in response to receiving the user input corresponding to the two-dimensional drawing, displaying, via a display generation component, a representation of the two-dimensional drawing; and
after receiving the user input and while displaying the representation of the two-dimensional drawing, displaying, via the display generation component, one or more selectable options;
while displaying the one or more selectable options, detecting selection of a respective selectable option of the one or more selectable options; and
in response to detecting selection of the respective selectable option, replacing display of the two-dimensional drawing with a three-dimensional object with an appearance that is determined based on an appearance of the two-dimensional drawing.
