# US 2019/0244005 A1 · System and method for tracking a focal point for a head mounted device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244005 A1
- Document ID: US20190244005A1
- Status: Published application
- Kind: A1
- Application number: 16/128,322
- Filed: 2018-09-11
- Published: 2019-08-08
- Earliest claimed: 2018-02-05
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Masaki Suzuki; Sergio Perdices-Gonzalez; Pranav Mistry
- Web page: https://patentgrep.com/patent/US20190244005A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 40/193: {Preprocessing; Feature extraction}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An electronic device, method, and computer readable medium for augmented reality system is provided. The electronic device includes a light source that emits a light towards an eye of a user of the device, an adjustable lens having an adjustable focal point, a light sensor, configured to detect a reflected light that is reflected from the eye of the user, a processor, and a display screen. The processor determines a first distance corresponding to a focal point of the eye based on one or more positions of the reflected light, and adjusts the focal point of the lens unit based on the focal point of the eye. The display screen is configured to present an image of the object at a second distance based on the focal point of the lens unit to create a perception for the user that the image is placed at the first distance.

## Claims

### Claim 1

An electronic device comprising:
a light source configured to emit a light towards an eye of a user of the device;
an adjustable lens having an adjustable focal point;
a light sensor, coupled to the adjustable lens, configured to detect a reflected light that is reflected from the eye of the user;
a processor, coupled to the adjustable lens, configured to:
determine a first distance corresponding to a focal point of the eye of the user based on one or more positions of the reflected light, and
adjust the focal point of the adjustable lens based on the focal point of the eye; and
a display screen, coupled to the processor, configured to present an image of an object at a second distance based on the focal point of the adjustable lens to create a perception for the user that the object is placed at the first distance.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the first distance of the object is determined based on:
eye rotation and anterior surface curvature of the eye of the user and an inner lens of the eye of the user while the user is looking at the object, and
a database comprising a plurality of eye rotation, anterior surface and inner lens curvature data of the user looking at a particular object placed at different positions during a calibration mode.

### Claim 3 (depends on 2)

The electronic device of claim 2, wherein the database is formulated based on a first vector representing anterior surface curvature and a change in an inner lens thickness of the eye and a second vector representing eye rotation changes caused by position changes of the particular object.

### Claim 4 (depends on 1)

The electronic device of claim 1, wherein the processor is configured to:
detect a plurality of reflection points in a pupil of the eye of the user;
calculate a distance between at least two points among the plurality of points; and
determine the focal point of the eye based on the distance between at least two points.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the processor is further configured to filter at least one reflection point from the plurality of reflection points.

### Claim 6 (depends on 5)

The electronic device of claim 5, wherein the processor is configured to filter the at least one reflection point by one or more of:
varying a location of the light source that emits the light towards the eye of the user;
emitting light from a different light source from a plurality of light sources;
varying an orientation of a reflective surface, wherein the reflective surface is configured to reflect the light towards the eye of the user;
varying a location of the light sensor;
varying a position of the device on a head of the user;
adjusting a size of a captured image of the eye;
adjusting a resolution of an image of the eye; or
using binarization and morphology to remove the at least one reflection point.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the processor is configured to determine a gaze of the user by calculating vector between a center of a pupil of the eye and a reflection point corresponding to light reflected from an outer surface of a cornea of the eye.

### Claim 8

A method comprising:
emitting, by a light source, a light towards an eye of a user of an electronic device;
detecting, by a light sensor, a reflected light that is reflected from the eye of the user;
determining a first distance corresponding to a focal point of the eye of the user based on one or more positions of the reflected light;
adjusting the focal point of an adjustable lens based on the focal point of the eye; and
displaying an image of an object at a second distance based on the focal point of the adjustable lens to create a perception for the user that the object is placed at the first distance.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first distance of the object is determined based on:
eye rotation and anterior surface curvature of the eye of the user and an inner lens of the eye of the user while the user is looking at the object, and
a database comprising a plurality of eye rotation, anterior surface and inner lens curvature data of the user looking at a particular object placed at different positions during a calibration mode.

### Claim 10 (depends on 9)

The method of claim 9, wherein the database is formulated based on a first vector representing anterior surface curvature and a change in an inner lens thickness of the eye and a second vector representing eye rotation changes caused by position changes of the particular object.

### Claim 11 (depends on 9)

The method of claim 9, further comprising:
detecting a plurality of reflection points in a pupil of the eye of the user;
calculating a distance between at least two points among the plurality of points; and
determining the focal point of the eye based on the distance between at least two points.

### Claim 12 (depends on 11)

The method of claim 11, further comprising filtering at least one reflection point from the plurality of reflection points.

### Claim 13 (depends on 12)

The method of claim 12, wherein filtering the at least one reflection point comprises one or more of:
varying a location of a light source that emits a light towards the eye of the user;
emitting light from a different light source from a plurality of light sources;
varying an orientation of a reflective surface, wherein the reflective surface is configured to reflect the light towards the eye of the user;
varying a location of the light sensor;
varying a position of the electronic device on a head of the user;
adjusting a size of a captured image of the eye;
adjusting a resolution of an image of the eye; or
using binarization and morphology to remove the at least one reflection point.

### Claim 14 (depends on 8)

The method of claim 8, further comprising determining a gaze of the user by calculating a vector between a center of a pupil of the eye and a reflection point corresponding to light reflected from an outer surface of a cornea of the eye.

### Claim 15

A non-transitory computer readable medium embodying a computer program for operating an electronic device including a memory and at least one processor, the computer program comprising computer readable program code that, when executed by the at least one processor, causes the electronic device to:
emit, by a light source, a light towards an eye of a user of the device;
detect, by a light sensor, a reflected light that is reflected from the eye of the user;
determine a first distance corresponding to a focal point of the eye of the user based on one or more positions of the reflected light;
adjust the focal point of an adjustable lens based on the focal point of the eye; and
display an image of an object at a second distance based on the focal point of the adjustable lens to create a perception for the user that the object is placed at the first distance.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the first distance of the object is determined based on:
eye rotation and anterior surface curvature of the eye of the user and an inner lens of the eye of the user while the user is looking at the object, and
a database comprising a plurality of eye rotation, anterior surface and inner lens curvature data of the user looking at a particular object placed at different positions during a calibration mode.

### Claim 17 (depends on 15)

The computer readable medium of claim 15, wherein the computer readable program code is configured to cause the at least one processor to:
detect a plurality of reflection points in a pupil of the eye of the user;
calculate a distance between at least two points among the plurality of points; and
determine the focal point of the eye based on the distance between at least two points.

### Claim 18 (depends on 17)

The computer readable medium of claim 17, wherein the computer readable program code is configured to cause the at least one processor to filter at least one reflection point from the plurality of reflection points.

### Claim 19 (depends on 18)

The computer readable medium of claim 18, wherein the computer readable program code is configured to cause the at least one processor to filter the at least one reflection point by one or more of:
varying a location of a light source that emits a light towards the eye of the user;
emitting light from a different light source from a plurality of light sources;
varying an orientation of a reflective surface, wherein the reflective surface is configured to reflect the light towards the eye of the user;
varying a location of the light sensor;
varying a position of the device on a head of the user;
adjusting a size of a captured image of the eye;
adjusting a resolution of an image of the eye; or
using binarization and morphology to remove the at least one reflection point.

### Claim 20 (depends on 15)

The computer readable medium of claim 15, wherein the computer readable program code is configured to cause the at least one processor to calculate a vector between a center of a pupil of the eye and a reflection point corresponding to light reflected from an outer surface of a cornea of the eye.
