# US 12,182,322 B2 · Visibility improvement method based on eye tracking, machine-readable storage medium and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,322 B2
- Document ID: US12182322B2
- Status: Granted
- Kind: B2
- Application number: 17/478,529
- Filed: 2021-09-17
- Published: 2022-01-06
- Granted: 2024-12-31
- Earliest claimed: 2013-06-11
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Myung-Sik Kim; Chang-Yeong Kim; Ick-Sang Han; Joon-Oo Kim; Jinho Kim; Yun-Je Oh; Sun-Tae Jung
- Web page: https://patentgrep.com/patent/US12182322B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G02B 27/017: {Head mounted}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G02B 2027/0127: {comprising devices increasing the depth of field}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

A visibility improvement method using gaze tracking includes detecting a gaze of a user using a camera; determining a focus object at which the user gazes from among at least one object viewed through a display unit of an electronic device; and displaying, on the display unit, an image with high visibility that corresponds to the focus object and has higher visibility than the focus object.

## Claims

### Claim 1

A head mounted device (HMD) comprising:
a display;
at least one first camera provided on an inner surface of the HMD to face an eye of a user when the HMD is worn on the user;
at least one memory storing instructions; and
at least one processor configured to execute the instructions to:
obtain gaze information of the user of the HMD,
based on the gaze information, select a first object as a selected graphical object,
obtain object information for identifying the selected graphical object,
obtain HMD direction information indicating a direction of the HMD,
transmit the object information of the selected graphical object and the HMD direction information to a server,
receive, from the server, a second object that displays the selected graphical object as a three dimensional (3D) object having multiple views, based on the object information for identifying the selected graphical object and the HMD direction information indicating the direction of the HMD, being transmitted to the server, and
control the display to replace the selected graphical object with the 3D object received from the server in response to the object information of the selected graphical object and the HMD direction information being provided to the server.

### Claim 2 (depends on 1)

The HMD of claim 1, wherein the second object has a resolution, a definition, and a brightness that are different from a resolution, a definition, and a brightness of the first object.

### Claim 3 (depends on 1)

The HMD of claim 1, further comprising at least one second camera provided on an outer surface of the HMD,
wherein the direction of the HMD includes a camera facing direction of the at least one second camera, and
wherein the at least one processor is further configured to receive the second object from the server, in response to the object information of the selected graphical object and the camera facing direction of the at least one second camera being transmitted to the server.

### Claim 4 (depends on 1)

The HMD of claim 1, wherein the at least one processor is further configured to execute the instructions to:
control the display to display a plurality of objects including the first object,
identify the first object at which the user of the HMD gazes among the plurality of objects,
receive, from the server, the second object having a higher visibility than the first object,
control the plurality of objects to disappear from the display, and
control the display to display the second object while the plurality of objects is not displayed on the display of the HMD.

### Claim 5 (depends on 4)

The HMD of claim 4, wherein the at least one processor is further configured to execute the instructions to:
obtain a user command related to the first object,
transmit, to the server, third information related to the user command together with the object information and the HMD direction information, and
receive, from the server, the second object having the higher visibility that is set according to the user command.

### Claim 6 (depends on 5)

The HMD of claim 5, wherein the at least one processor is further configured to execute the instructions to:
based on the user command being a command for changing a size of the first object, receive, from the server, the first object of which the size is changed as the second object.

### Claim 7 (depends on 6)

The HMD of claim 6, wherein the user command includes at least one of a voice command or a gesture command.

### Claim 8 (depends on 1)

The HMD of claim 1, further comprising a sensor,
wherein the at least one processor is further configured to execute the instructions to:
obtain an amount of light surrounding the HMD using the sensor,
transmit the amount of the light to the server,
based on the amount of the light being a first amount, receive a third object having a first visibility as the second object, and
based on the amount of the light being a second amount, receive a fourth object having a second visibility as the second object.

### Claim 9

A method performed by a head mounted device (HMD), the method comprising:
obtaining gaze information of a user of the HMD;
based on the gaze information, selecting a first object as a selected graphical object;
obtaining object information for identifying the selected graphical object;
obtaining HMD direction information indicating a direction of the HMD;
transmitting the object information of the selected graphical object and the HMD direction information to a server;
receiving, from the server, a second object that displays the selected graphical object as a three dimensional (3D) object having multiple views, based on the object information for identifying the selected graphical object and the HMD direction information indicating the direction of the HMD, being transmitted to the server; and
displaying the second object by replacing the selected graphical object with the 3D object received from the server in response to the object information of the selected graphical object and the HMD direction information being provided to the server.

### Claim 10 (depends on 9)

The method of claim 9, wherein the second object has a resolution, a definition, and a brightness that are different from a resolution, a definition, and a brightness of the first object.

### Claim 11 (depends on 9)

The method of claim 9, wherein the direction of the HMD includes a camera facing direction of at least one second camera provided on an outer surface of the HMD, and
wherein the second object is received from the server, in response to the object information and the camera facing direction of the at least one second camera being transmitted to the server.

### Claim 12 (depends on 9)

The method of claim 9, further comprising:
displaying a plurality of objects including the first object,
identifying the first object at which the user of the HMD gazes among the plurality of objects;
controlling the plurality of objects to disappear from a display of the HMD; and
displaying the second object while the plurality of objects is not displayed on the display of the HMD.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
obtaining a user command related to the first object;
transmitting third information related to the user command together with the object information and the HMD direction information, to the server; and
receiving, from the server, the second object having the different visibility related to the user command.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
based on the user command being a command for changing a size of the first object, receiving, from the server, the first object of which the size is changed as the second object.

### Claim 15 (depends on 14)

The method of claim 14, wherein the user command includes at least one of a voice command or a gesture command.

### Claim 16 (depends on 9)

The method of claim 9, further comprising:
obtaining an amount of light surrounding the HMD using a sensor;
transmitting the amount of the light to the server;
based on the amount of the light being a first amount, receiving a third object having a first visibility as the second object; and
based on the amount of the light being a second amount, receiving a fourth object having a second visibility as the second object.

### Claim 17 (depends on 1)

The HMD of claim 1, wherein the first object has a single view in two dimensions.

### Claim 18 (depends on 9)

The method of claim 9, wherein the first object has a single view in two dimensions.