# US 2024/0256035 A1 · Controlling a function via gaze detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0256035 A1
- Document ID: US20240256035A1
- Status: Published application
- Kind: A1
- Application number: 18/588,168
- Filed: 2024-02-27
- Published: 2024-08-01
- Earliest claimed: 2021-01-12
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Steven N. BATHICHE; Eric Chris Wolfgang Sommerlade; Vivek PRADEEP; Alexandros NEOFYTOU
- Web page: https://patentgrep.com/patent/US20240256035A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 21/31: User authentication
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06T 7/73: using feature-based methods
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/82: using neural networks
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/18: Eye characteristics, e.g. of the iris
- H04N 23/611: where the recognised objects include parts of the human body
- G06T 2207/20084: Artificial neural networks [ANN]
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Aspects of the present disclosure relate to systems and methods for controlling a function of a computing system using gaze detection. In examples, one or more images of a user are received and gaze information may be determined from the received one or more images. Non-gaze information may be received when the gaze information is determined to satisfy a condition. Accordingly, a function may be enabled based on the received non-gaze information. In examples, the gaze information may be determined by extracting a plurality of features from the received one or more images, providing the plurality of features to a neural network, and determining, utilizing the neural network, a location at a display device at which a gaze of the user is directed.

## Claims

### Claim 1

A method for controlling a function via gaze detection, the method comprising:
receiving one or more images of a user;
determining gaze information from the received one or more images;
receiving non-gaze information when the gaze information is determined to satisfy a condition; and
enabling a function based on the non-gaze information.

### Claim 2 (depends on 1)

The method of claim 1, wherein the non-gaze information includes a stored image of the user; and
enabling the function includes authenticating the user for access to an access controlled system.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving one or more second images including the user and a second user;
determining second gaze information for the second user from the received one or more second images;
identifying the second user associated with the second gaze information; and
enabling the function includes causing a display of content displayed at a display device to be modified based on the identity of the second user.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
determining content displayed at the display device; and
obscuring less than all of the content displayed at the display device based on the identity of the second user.

### Claim 5 (depends on 1)

The method of claim 1, wherein the function includes one or more of controlling a brightness of a display device, resetting an inactivity timer, and obtaining a second image.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining gaze information from the received one or more images includes:
extracting a plurality of features from the received one or more images;
providing the plurality of features to a neural network; and
determining, utilizing the neural network, a location at a display device at which a gaze of the user is directed.

### Claim 7 (depends on 6)

The method of claim 6, wherein the gaze information is determined to satisfy the condition when the location at the display device at which the gaze of the user is directed coincides with a region including a graphical element.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining gaze information from the received one or more images includes:
extracting a plurality of features from the received one or more images;
providing the plurality of features to a neural network; and
determining, utilizing the neural network, a location at a location other than a display device at which a gaze of the user is directed.

### Claim 9 (depends on 8)

The method of claim 8, further comprising determining, utilizing the neural network, a future predicted location at the display device associated with a future predicted gaze of the user.

### Claim 10

A system for controlling a function via gaze detection, the system comprising:
a processor;
an image sensor; and
memory including instructions, which when executed by the processor, cause the processor to:
receive one or more images of a user from the image sensor;
determine gaze information from the received one or more images;
determine if the gaze information satisfies a condition;
receive non-gaze information associated with at least one of a function or application of a computing system when the gaze information satisfies the condition; and
enable the at least one function or application based on non-gaze information.

### Claim 11 (depends on 10)

The system of claim 10, wherein at least one processor is a neural processing unit.

### Claim 12 (depends on 10)

The system of claim 10, wherein the instructions, when executed by the processor, cause the processor to:
receive second one or more images including the user and a second user;
determine second gaze information for the second user from the received second one or more images;
identify the second user associated with the second gaze information; and
cause a display of content displayed at a display device to be modified based on the identity of the second user.

### Claim 13 (depends on 12)

The system of claim 12, wherein the instructions, when executed by the processor, cause the processor to:
obscure less than all of the content displayed at the display device based on the identity of the second user.

### Claim 14 (depends on 10)

The system of claim 10, wherein the at least one function or application at least one of controls a brightness of a display device, resets an inactivity timer, obtains a second image, changes a font size, changes resolution, and changes a display refresh rate.

### Claim 15 (depends on 10)

The system of claim 10, wherein the instructions, when executed by the processor, cause the processor to:
extract a plurality of features from the received one or more images;
provide the plurality of features to a neural network; and
determine, utilizing the neural network, a location at a display device at which a gaze of the user is directed.

### Claim 16 (depends on 15)

The system of claim 15, wherein the gaze information satisfies the condition when the location at the display device at which the gaze of the user is directed coincides with a region including a graphical element.

### Claim 17 (depends on 15)

The system of claim 15, wherein the instructions, when executed by the processor, cause the processor to:
extract a plurality of features from the received one or more images;
provide the plurality of features to a neural network; and
determine, utilizing the neural network, a location at a location other than a display device at which a gaze of the user is directed.

### Claim 18 (depends on 17)

The system of claim 17, wherein the instructions, when executed by the processor, cause the processor to determine a predicted location at the display device associated with a predicted gaze of the user.

### Claim 19

A computer storage medium including instructions, which when executed by a processor, cause the processor to:
receive one or more images of a user from an image sensor;
determine gaze information from the received one or more images;
determine if the gaze information satisfies a condition;
receive device dependent information associated with at least one of a function or application of a computing system when the gaze information is determined to satisfy the condition; and
enable at least one function or application based on the device dependent information.

### Claim 20 (depends on 19)

The computer storage medium of claim 19, wherein the instructions, which when executed by a processor, cause the processor to:
extract a plurality of features from the received one or more images;
provide the plurality of features to a neural network; and
determine, utilizing the neural network, a location at a display device at which a gaze of the user is directed,
wherein, the gaze information satisfies the condition when the location at a display device at which the gaze of the user is directed coincides with a region including a graphical element associated with the at least one function or application.
