# US 2020/0394408 A1 · Electronic device and method for providing function by using corneal image in electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394408 A1
- Document ID: US20200394408A1
- Status: Published application
- Kind: A1
- Application number: 16/903,624
- Filed: 2020-06-17
- Published: 2020-12-17
- Earliest claimed: 2019-06-17
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Dmitry SYDORENKO; Roman KUSHNIRENKO; Svitlana ALKHIMOVA; Andrii BOGACHENKO
- Web page: https://patentgrep.com/patent/US20200394408A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 1/0007: {Image acquisition}
- G06T 7/73: using feature-based methods
- G06T 7/75: {involving models}
- G06V 10/443: {by matching or filtering}
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 20/20: in augmented reality scenes
- G06V 30/142: using hand-held instruments; Constructional details of the instruments
- G06V 40/19: Sensors therefor
- G06V 40/193: {Preprocessing; Feature extraction}
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/63: by using electronic viewfinders
- H04N 23/634: {Warning indications}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30041: Eye; Retina; Ophthalmic
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An electronic device includes a camera; a display; a memory; and a processor configured to be operatively connected to the camera, the display, and the memory, wherein the memory is configured to store instructions that, when executed, cause the processor to: receive image data including a first image, acquired by the camera, of at least a part of a user's body; extract a corneal region of an eye from the image data; acquire surrounding information formed in the extracted corneal region; recognize at least one object in the surrounding information; and provide, on the display, a visual feedback corresponding to the recognized at least one object.

## Claims

### Claim 1

An electronic device comprising:
a camera;
a display;
a memory; and
a processor configured to be operatively connected to the camera, the display, and the memory,
wherein the memory is configured to store instructions that, when executed, cause the processor to:
receive image data comprising a first image, acquired by the camera, of at least a part of a body of a user;
extract a corneal region of an eye from the image data;
acquire surrounding information formed in the extracted corneal region;
recognize at least one object in the surrounding information; and
provide, on the display, a visual feedback corresponding to the recognized at least one object.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to display, on the display, a second image of the recognized at least one object or a predetermined image.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to provide, as the visual feedback, an alarm for danger recognition, when the recognized at least one object corresponds to a predetermined type of object.

### Claim 4 (depends on 3)

The electronic device of claim 3, further comprising a sensor configured to measure a movement speed of the electronic device,
wherein the instructions, when executed, further cause the processor to provide the alarm for the danger recognition, when the movement speed measured by the sensor is greater than or equal to a predetermined speed.

### Claim 5 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to:
acquire location information of the electronic device based on the recognized at least one object; and
provide the acquired location information to an executed location-based application.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to:
display an augmented reality content on the display; and
display the visual feedback in the augmented reality content.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to acquire an ambient brightness from the surrounding information.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to:
recognize at least one text in the surrounding information; and
provide information related to the at least one text.

### Claim 9 (depends on 1)

The electronic device of claim 1, wherein the camera is disposed on a front surface of a housing of the electronic device.

### Claim 10 (depends on 1)

The electronic device of claim 1, wherein the extracted corneal region is of at least one eye of the user.

### Claim 11 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed, further cause the processor to:
detect an eye-gaze position of the user from the image data; and
provide the visual feedback when the eye-gaze position is directed to at least a part of the electronic device.

### Claim 12

A method for providing a function by an electronic device, the method comprising:
acquiring image data comprising a first image, acquired using a camera, of at least a part of a body of a user;
extracting a corneal region of an eye from the image data;
acquiring surrounding information formed in the extracted corneal region;
recognizing at least one object in the surrounding information; and
providing, on a display, a visual feedback corresponding to the recognized at least one object.

### Claim 13 (depends on 12)

The method of claim 12, wherein the providing of the visual feedback comprises providing, as the visual feedback, an alarm for danger recognition, when the recognized at least one object corresponds to a predetermined type of object.

### Claim 14 (depends on 12)

The method of claim 12, wherein the providing of the visual feedback comprises:
acquiring location information of the electronic device based on the recognized at least one object; and
providing the acquired location information to an executed location-based application.

### Claim 15 (depends on 12)

The method of claim 12, wherein the providing of the visual feedback comprises:
recognizing at least one text in the surrounding information; and
providing information related to the at least one text.

### Claim 16 (depends on 12)

The method of claim 12, further comprising detecting an eye-gaze position of the user from the image data,
wherein the providing of the visual feedback comprises providing the visual feedback when the eye-gaze position is directed to at least a part of the electronic device.

### Claim 17

An electronic device comprising:
a camera;
a display;
a processor; and
a memory configured to store instructions that, when executed, cause the processor to:
receive, from the camera, a first image reflected by a first cornea of a user viewing the display, the first image comprising a first sub-image of a real-world object located near the user; and
provide a visual feedback on the display based on the real-world object.

### Claim 18 (depends on 17)

The electronic device of claim 17, wherein the first image further comprises a second sub-image of a light source located near the user,
wherein the instructions, when executed, further cause the processor to acquire an ambient brightness near the user based on the light source, and
wherein the visual feedback comprises an augmented reality element added, based on the ambient brightness, to a displayed image of the real-world object.

### Claim 19 (depends on 17)

The electronic device of claim 17, wherein the instructions, when executed, further cause the processor to:
receive, from the camera, a second image reflected by a second cornea of the user, the second image comprising the first sub-image of the real-world object; and
generate a panorama image based on the first image and the second image and having a wider angle of view than angles of view of the first image and the second image.

### Claim 20 (depends on 17)

The electronic device of claim 17, wherein the first image is captured by the camera at a first time,
wherein the instructions, when executed, further cause the processor to receive, from the camera, a second image reflected at a second time by the first cornea of the user, and
wherein the second time is different from the first time.
