# US 2023/0273431 A1 · Methods and apparatuses for providing input for head-worn image display devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0273431 A1
- Document ID: US20230273431A1
- Status: Published application
- Kind: A1
- Application number: 18/312,886
- Filed: 2023-05-05
- Published: 2023-08-31
- Earliest claimed: 2018-06-21
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Lorena PAZMINO; Andrea Isabel MONTOYA; Savannah NILES; Alexander ROCHA; Mario Antonio BRAGG; Parag GOEL; Jeffrey Scott SOMMERS; David Charles LUNDMARK
- Web page: https://patentgrep.com/patent/US20230273431A1

## CPC

- G02B 27/017: {Head mounted}
- G06F 1/1626: {with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 1/1643: {the display being associated to a digitizer, e.g. laptops that can be used as penpads (details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
- G06F 3/0485: Scrolling or panning
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- G06T 2219/2016: Rotation, translation, scaling
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An apparatus for use with an image display device configured for head-worn by a user, includes: a screen; and a processing unit configured to assign a first area of the screen to sense finger-action of the user; wherein the processing unit is configured to generate an electronic signal to cause a change in a content displayed by the display device based on the finger-action of the user sensed by the assigned first area of the screen of the apparatus.

## Claims

### Claim 1

An apparatus comprising:
a touch-sensitive display;
a feedback component; and
one or more processors that are operatively coupled to the touch-sensitive display and the feedback component and are communicatively coupled to a wearable device, the one or more processors configured to:
receive, from the wearable device, data indicating whether a user of the wearable device is viewing the touch-sensitive display or not; and
based on the data, switch between: (i) a first mode in which the one or more processors are configured to present a user interface on the touch-sensitive display, and (ii) a second mode in the one or more processors are configured to convey one or more portions of the user interface to the user via the feedback component.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the feedback component is a haptic actuator.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the feedback component is a speaker.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein user interface is a graphical user interface for controlling one or more functions of the wearable device.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the graphical user interface comprises a plurality of graphical control elements.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein when in the second mode, the one or more processors are configured to refrain from presenting one or more of the plurality of graphical control elements on the touch-sensitive display.

### Claim 7 (depends on 5)

The apparatus of claim 5, wherein when in the second mode, the one or more processors are configured to present a limited version of the graphical user interface on the touch-sensitive display.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the limited version of the graphical user interface on the touch-sensitive display is one in which one or more of the plurality of graphical control elements are not shown, one in which brightness levels of one or more of the plurality of graphical control elements are reduced, or a combination thereof.

### Claim 9 (depends on 5)

The apparatus of claim 5, wherein when in the second mode, the one or more processors are configured to convey one or more locations on the touch-sensitive display at which one or more of the plurality of graphical control elements are otherwise presented in the first mode.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the one or more locations on the touch-sensitive display that are conveyed in the second mode correspond to one or more outer boundaries of the plurality of graphical control elements that are otherwise presented in the first mode.

### Claim 11 (depends on 5)

The apparatus of claim 5, wherein the graphical control elements correspond to a plurality of physical user input components of a dedicated input controller that is associated with the wearable device.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to assign an area of the touch-sensitive display to sense different finger-gliding actions of the user.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the apparatus is configured to monitor a peripheral part of the assigned area for provision of a control signal to operate the feedback component, wherein the peripheral part of the assigned area being monitored for provision of the control signal extends continuously across at least a majority of a side of the touch-sensitive display.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the data comprises camera data.

### Claim 15

A processor-implemented method includes:
receiving, from a wearable device, data indicating whether a user of the wearable device is viewing a touch-sensitive display or not; and
based on the data, switching between (i) a first mode in which a user interface is presented on the touch-sensitive display, and (ii) a second mode in which one or more portions of the user interface are conveyed to the user of the wearable device via a feedback component.

### Claim 16 (depends on 15)

The method of claim 15, wherein the data indicates that user is viewing the touch-sensitive display, and wherein the switching comprises switching to the first mode.

### Claim 17 (depends on 15)

The method of claim 15, wherein the data indicates that the user is not viewing the touch-sensitive display, and wherein the switching comprises switching to the second mode.

### Claim 18 (depends on 15)

The method of claim 15, wherein the data comprises camera data.

### Claim 19 (depends on 15)

The method of claim 15, further comprising assigning an area of the touch-sensitive display to sense different finger-gliding actions of the user.

### Claim 20 (depends on 19)

The method of claim 19, further comprising monitoring a peripheral part of the assigned area for provision of a control signal to operate the feedback component, wherein the peripheral part of the assigned area being monitored for provision of the control signal extends continuously across at least a majority of a side of the touch-sensitive display.
