# US 2022/0035508 A1 · Private control interfaces for extended reality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0035508 A1
- Document ID: US20220035508A1
- Status: Published application
- Kind: A1
- Application number: 17/385,055
- Filed: 2021-07-26
- Published: 2022-02-03
- Earliest claimed: 2020-07-28
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Wesley James HOLLAND; Ramesh CHANDRASEKHAR; Daniel James GUEST; Sebastien MOUNIER; Bijan FORUTANPOUR
- Web page: https://patentgrep.com/patent/US20220035508A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0421: {by interrupting or reflecting a light beam, e.g. optical touch-screen}
- G06F 3/0426: {tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886)}
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06F 21/604: {Tools and structures for managing or administering access control systems}
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06F 21/84: output devices, e.g. displays or monitors
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/20: in augmented reality scenes
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/024: Multi-user, collaborative environment
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods, and non-transitory media are provided for generating private control interfaces for extended reality (XR) experiences. An example method can include determining a pose of an XR device within a mapped scene of a physical environment associated with the XR device; detecting a private region in the physical environment and a location of the private region relative to the pose of the XR device, the private region including an area estimated to be within a field of view (FOV) of a user of the XR device and out of a FOV of a person in the physical environment, a recording device in the physical environment, and/or an object in the physical environment; based on the pose of the XR device and the location of the private region, mapping a virtual private control interface to the private region; and rendering the virtual private control interface within the private region.

## Claims

### Claim 1

An apparatus comprising:
at least one memory; and
one or more processors configured to:
receive a first user input identifying a first area in a mapped scene of a physical environment; and
generate a first indication that the first area in the mapped scene is within a field of view (FOV) of at least one of a person in the physical environment, a recording device in the physical environment, and an object that enables viewing access from outside of the physical environment.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
receive a second user input identifying a second area in the mapped scene; and
generate a second indication that indicates whether the second area in the mapped scene is within the FOV of at least one of the person in the physical environment, the recording device in the physical environment, and the object that enables viewing access from outside of the physical environment.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein, to generate the second indication, the one or more processors are configured to determine that the second area in the mapped scene is within the FOV of at least one of the person in the physical environment, the recording device in the physical environment, and the object, and wherein the one or more processors are configured to:
in response to a third user input selecting an option to render virtual content within the second area, render virtual content within the second area.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
in response to a third user input selecting an option to render virtual content within the first area in the mapped scene, render the virtual content within the first area in the mapped scene.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
generate a prompt providing an option to render virtual content within a second area in the mapped scene that is estimated to be outside of the FOV of the at least one of the person, the recording device, and the object; and
in response to a third user input selecting the option, render the virtual content within the second area in the mapped scene.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the one or more processors are configured to:
detect that, after rendering the virtual content within the second area, the second area is within one or more FOVs of at least one of the recording device in the physical environment, the object that enables viewing access from outside of the physical environment, and the person in the physical environment.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the one or more processors are configured to:
perform an action in response to detecting that the second area is within the one or more FOVs, the action comprising at least one of removing the virtual content, moving the virtual content to a different area in the mapped scene, and generating an additional prompt providing at least one of a second option to render the virtual content in a third area and a third option to render the virtual content in the different area.

### Claim 8 (depends on 6)

The apparatus of claim 6, wherein, to detect that the second area is within the one or more FOVs, the one or more processors are configured to:
detect one or more objects in the physical environment; and
determine that the FOV of the at least one of the person, the recording device, and the object is occluded by the one or more objects.

### Claim 9 (depends on 6)

The apparatus of claim 6, wherein, to detect that the second area is within the one or more FOVs, the one or more processors are configured to:
detect a face in an image corresponding to the mapped scene;
based on the detecting of the face in the image, determine a respective FOV of the person; and
determine, based on the FOV of the person and a location of the second area, that the second area is within the respective FOV of the person.

### Claim 10 (depends on 6)

The apparatus of claim 6, wherein the one or more processors are configured to:
detect a face in an image corresponding to the mapped scene;
based on the detecting of the face, determine a respective FOV of a user of the apparatus; and
determine, based on a location of the second area, that the second area is within the respective FOV of the user of the apparatus.

### Claim 11 (depends on 6)

The apparatus of claim 6, wherein the one or more processors are configured to:
determine that a user of the apparatus has moved at least one of the virtual content and a surface of the second area; and
determine that a third area associated with a different location of at least one of the virtual content and the surface is within one or more FOVs of at least one of the recording device in the physical environment, the object that enables viewing access from outside of the physical environment, and one or more persons in the physical environment.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the one or more processors are configured to:
perform an action in response to determining that the third area is within the one or more FOVs of at least one of the recording device, the object, and the one or more persons, the action comprising at least one of removing the virtual content, moving the virtual content to a different area in the mapped scene, and generating an additional prompt providing at least one of a second option to render the virtual content in the third area and a third option to render the virtual content in the different area.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
determine at least one of a first size of the first area and a first orientation of the first area;
determine, based on the at least one of the first size of the first area and the first orientation of the first area, at least one of a second size of a virtual content and a second orientation of the virtual content, wherein the second size of the virtual content matches or fits within the first size of the virtual content, and wherein the second orientation of the virtual content at least partially aligns with the first orientation of the first area;
generate the virtual content according to the at least one of the second size and the second orientation; and
align the virtual content with the first area.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the first area comprises a surface of a body part associated with a user of the apparatus, and wherein the one or more processors are configured to determine whether the surface of the body part is facing the user and is within the FOV of the at least one of the person in the physical environment, the recording device in the physical environment, and the object that enables the viewing access.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
based on a location of the first area in the mapped scene relative to a pose of at least one of the apparatus, the person in the physical environment, the recording device in the physical environment, and the object that enables viewing access from outside of the physical environment, determine that the first area is within the FOV of at least one of the person, the recording device, and the object.

### Claim 16 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
in response to a second user input, render virtual content within one of a second area selected by a user of the apparatus or a third area in the mapped scene that is estimated to be outside of the FOV of the at least one of the person, the recording device, and the object, wherein the virtual content comprises at least one of a virtual keyboard, a virtual poll, a virtual prompt, a virtual form, a virtual joystick, a virtual controller, and a virtual interface associated with an augmented reality application.

### Claim 17 (depends on 1)

The apparatus of claim 1, wherein the apparatus is a mobile device.

### Claim 18 (depends on 1)

The apparatus of claim 1, wherein the apparatus comprises an augmented reality device.

### Claim 19

A method of rendering virtual content, the method comprising:
receiving a first user input identifying a first area in a mapped scene of a physical environment; and
generating, via an electronic device, a first indication that the first area in the mapped scene is within a field of view (FOV) of at least one of a person in the physical environment, a recording device in the physical environment, and an object that enables viewing access from outside of the physical environment.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
receiving a second user input identifying a second area in the mapped scene; and
generating a second indication that indicates whether the second area in the mapped scene is within the FOV of at least one of the person in the physical environment, the recording device in the physical environment, and the object that enables viewing access from outside of the physical environment.

### Claim 21 (depends on 20)

The method of claim 20, wherein generating the second indication further comprises determining that the second area in the mapped scene is within the FOV of at least one of the person in the physical environment, the recording device in the physical environment, and the object, and wherein the method further comprises:
in response to a third user input selecting an option to render virtual content within the second area, rendering virtual content within the second area.

### Claim 22 (depends on 19)

The method of claim 19, further comprising:
in response to a third user input selecting an option to render virtual content within the first area in the mapped scene, rendering the virtual content within the first area in the mapped scene.

### Claim 23 (depends on 19)

The method of claim 19, further comprising:
generating a prompt providing an option to render virtual content within a second area in the mapped scene that is estimated to be outside of the FOV of the at least one of the person, the recording device, and the object; and
in response to a third user input selecting the option, rendering the virtual content within the second area in the mapped scene.

### Claim 24 (depends on 23)

The method of claim 23, further comprising:
detecting that, after rendering the virtual content within the second area, the second area is within one or more FOVs of at least one of the recording device in the physical environment, the object that enables viewing access from outside of the physical environment, and the person in the physical environment.

### Claim 25 (depends on 24)

The method of claim 24, further comprising:
performing an action in response to detecting that the second area is within the one or more FOVs, the action comprising at least one of removing the virtual content, moving the virtual content to a different area in the mapped scene, and generating an additional prompt providing at least one of a second option to render the virtual content in a third area and a third option to render the virtual content in the different area.

### Claim 26 (depends on 24)

The method of claim 24, wherein detecting that the second area is within the one or more FOVs further comprises:
detecting one or more objects in the physical environment; and
determining that the FOV of the at least one of the person, the recording device, and the object is occluded by the one or more objects.

### Claim 27 (depends on 24)

The method of claim 24, further comprising:
detecting a face in an image corresponding to the mapped scene;
based on the detecting of the face, determining a respective FOV of a user of the electronic device; and
determining, based on a location of the second area, that the second area is within the respective FOV of the user of the electronic device.

### Claim 28 (depends on 24)

The method of claim 24, further comprising:
determining that a user of the electronic device has moved at least one of the virtual content and a surface of the second area; and
determining that a third area associated with a different location of at least one of the virtual content and the surface is within one or more FOVs of at least one of the recording device in the physical environment, the object that enables viewing access from outside of the physical environment, and one or more persons in the physical environment.

### Claim 29 (depends on 28)

The method of claim 28, further comprising:
performing an action in response to determining that the third area is within the one or more FOVs of at least one of the recording device, the object, and the one or more persons, the action comprising at least one of removing the virtual content, moving the virtual content to a different area in the mapped scene, and generating an additional prompt providing at least one of a second option to render the virtual content in the third area and a third option to render the virtual content in the different area.

### Claim 30 (depends on 19)

The method of claim 19, further comprising:
in response to a second user input, rendering virtual content within one of a second area selected by a user of the electronic device or a third area in the mapped scene that is estimated to be outside of the FOV of the at least one of the person, the recording device, and the object, wherein the virtual content comprises at least one of a virtual keyboard, a virtual poll, a virtual prompt, a virtual form, a virtual joystick, a virtual controller, and a virtual interface associated with an augmented reality application.
