# US 12,118,141 B2 · Guided object targeting based on physiological feedback

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,141 B2
- Document ID: US12118141B2
- Status: Granted
- Kind: B2
- Application number: 18/123,212
- Filed: 2023-03-17
- Published: 2024-09-19
- Granted: 2024-10-15
- Earliest claimed: 2023-03-17
- Assignees: Micrsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Yu-Te Wang; Jen-Tse Dong
- Web page: https://patentgrep.com/patent/US12118141B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/02: Input arrangements using manually operated switches, e.g. using keyboards or dials
- G06F 3/0236: {using selection techniques to select from displayed items}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04892: Arrangements for controlling cursor position based on codes indicative of cursor displacements from one discrete location to another, e.g. using cursor control keys associated to different directions or using the tab key (arrangements for controlling cursor position based on coordinate signals G06F3/038)
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

This document relates to employing physiological feedback to aid a user in targeting objects using a computing device. One example relates to a method or technique that can include receiving a user attention signal conveying where a user directs attention. The method or technique can also include, based on the user attention signal, identifying a predicted object that the user intends to target with a targeting mechanism and outputting a visual identification of the predicted object. The method or technique can also include receiving a user reaction signal conveying a physiological reaction of the user to the visual identification of the predicted object. The method or technique can also include, in an instance when the physiological reaction of the user indicates an error, identifying another predicted object that the user intends to target with the targeting mechanism.

## Claims

### Claim 1

A method comprising:
receiving a user attention signal conveying where a user directs attention;
based at least on the user attention signal, identifying a predicted object that the user intends to target with a targeting mechanism;
outputting a visual identification of the predicted object;
receiving a user reaction signal conveying an implicit physiological reaction of the user to the visual identification of the predicted object; and
in an instance when the implicit physiological reaction of the user indicates an error, identifying another predicted object that the user intends to target with the targeting mechanism.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
responsive to receiving a selection input, performing a selection action on the another predicted object.

### Claim 3 (depends on 2)

The method of claim 2, wherein the predicted object is a first character, the another predicted object is a second character, and the selection action involves entering the second character into a text entry region on a display.

### Claim 4 (depends on 3)

The method of claim 3, wherein the targeting mechanism comprises a cursor.

### Claim 5 (depends on 4)

The method of claim 4, wherein the visual identification of the predicted object involves changing at least one of color, size, or font of the first character.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
outputting haptic feedback concurrently with the visual identification of the predicted object.

### Claim 7 (depends on 6)

The method of claim 6, wherein the haptic feedback is provided via a controller.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
receiving navigation inputs from the controller, the navigation inputs moving the targeting mechanism on a trajectory toward the predicted object; and
when the targeting mechanism reaches a threshold distance from the predicted object, performing a jump action that moves the targeting mechanism to the predicted object.

### Claim 9 (depends on 1)

The method of claim 1, further comprising determining respective targeting probabilities for a plurality of objects based at least on the user attention signal.

### Claim 10 (depends on 9)

The method of claim 9, wherein the predicted object has a highest targeting probability and the another predicted object has a next-highest targeting probability.

### Claim 11 (depends on 10)

The method of claim 10, wherein the respective targeting probabilities are based at least on attention-based coordinates derived from the user attention signal.

### Claim 12 (depends on 11)

The method of claim 11, wherein the respective targeting probabilities are based at least on respective distances from the attention-based coordinates to respective centers of each of the objects.

### Claim 13 (depends on 1)

The method of claim 1, wherein the predicted object and the another predicted object are displayed in a virtual or augmented reality experience.

### Claim 14

A system comprising:
a processor; and
a computer-readable storage medium storing instructions which, when executed by the processor, cause the system to:
based at least on a user attention signal, identify a predicted object that a user intends to target;
output an identification of the predicted object;
receiving a user reaction signal conveying an implicit physiological reaction of the user to the identification of the predicted object;
in an instance when the user reaction signal indicates that the predicted object is erroneous;
identify another predicted object that the user intends to target; and
perform a selection action on the another predicted object responsive to a selection input.

### Claim 15 (depends on 14)

The system of claim 14, further comprising an electroencephalogram sensor configured to provide an electroencephalogram signal as the user reaction signal.

### Claim 16 (depends on 15)

The system of claim 15, wherein the instructions, when executed by the processor, cause the system to:
determine that the electroencephalogram signal indicates that the user perceives an error when the electroencephalogram signal includes an error-related potential within a specified time window after the outputting the identification of the predicted object.

### Claim 17 (depends on 14)

The system of claim 14, further comprising an eye tracking sensor configured to provide a gaze tracking signal as the user reaction signal.

### Claim 18 (depends on 17)

The system of claim 17, wherein the instructions, when executed by the processor, cause the system to:
calibrate the eye tracking sensor based at least on the user reaction signal.

### Claim 19

A computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform acts comprising:
receiving a user attention signal conveying where a user directs attention;
based at least on the user attention signal, identifying a predicted object that the user intends to target with a targeting mechanism;
outputting a visual identification of the predicted object;
receiving a user reaction signal conveying an implicit physiological reaction of the user to the visual identification of the predicted object; and
in an instance when the implicit physiological reaction of the user indicates an error, identifying another predicted object that the user intends to target with the targeting mechanism.

### Claim 20 (depends on 19)

The computer-readable storage medium of claim 19, wherein the user reaction signal comprises a pupil diameter measurement.