# US 2022/0036582 A1 · Controller, position determination device, position determination system, display system, program, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036582 A1
- Document ID: US20220036582A1
- Status: Published application
- Kind: A1
- Application number: 17/290,980
- Filed: 2019-10-28
- Published: 2022-02-03
- Earliest claimed: 2018-11-05
- Assignees: KYOCERA Corporation
- Applicants: KYOCERA Corporation
- Inventors: Yusuke HAYASHI; Kaoru KUSAFUKA
- Web page: https://patentgrep.com/patent/US20220036582A1

## CPC

- G06T 7/593: from stereo images
- G06T 7/73: using feature-based methods
- G02B 27/0172: {characterised by optical features}
- G02B 27/0179: {Display position adjusting means not related to the information to be displayed}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 7/60: Analysis of geometric attributes
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06T 2207/30041: Eye; Retina; Ophthalmic
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A controller is configured to determine a position of an object point in a real space, based on a cornea image of a first eye and a cornea image of a second eye of a user who is viewing the object point, in a captured image generated by imaging the first and second eyes of the user by an imaging device.

## Claims

### Claim 1

A controller that is configured to determine a position of an object point in a real space, based on a cornea image of a first eye and a cornea image of a second eye of a user who is viewing the object point, in a captured image generated by imaging the first and second eyes of the user by an imaging device.

### Claim 2 (depends on 1)

The controller according to claim 1, wherein
the controller is configured
to determine, based on the cornea image of the first eye, a first cornea center position which is a center of a boundary between a cornea and a sclera of the first eye in the real space and determine, based on the cornea image of the second eye, a second cornea center position which is a center of a boundary between a cornea and a sclera of the second eye in the real space, and
to determine a position of the object point based on the first cornea center position and the second cornea center position.

### Claim 3 (depends on 2)

The controller according to claim 2, wherein
the controller is configured
to determine a first cornea direction which is a normal direction of a tangent plane at a center of the cornea of the first eye and determine a second cornea direction which is a normal direction of a tangent plane at a center of the cornea of the second eye, and
to determine an intersection of a first straight line from the first cornea center position toward the first cornea direction and a second straight line from the second cornea center position toward the second cornea direction, as the position of the object point.

### Claim 4 (depends on 2)

The controller according to claim 2, wherein
the controller is configured to determine the position of the object point, based on a position of an imaging plane of the imaging device, positions of images of the object point in the cornea images of the first eye and the second eye in the captured image, and a focal length of the imaging device in the real space.

### Claim 5 (depends on 4)

The controller according to claim 4, wherein
the controller is configured
to determine projection positions of the object point projected in the corneas of the first eye and the second eye, based on the positions of the images of the object point in the cornea images of the first eye and the second eye, and
to determine the position of the object point, based on the position of the imaging plane, the focal length, and the projection positions.

### Claim 6 (depends on 5)

The controller according to claim 5, wherein
the controller is configured
to determine the projection position of the object point projected in the first eye, based on the position of the image of the object point in the cornea image of the first eye in the captured image, and
to extract the image of the object point from the cornea image of the second eye in the captured image, based on an intersection line of an epipolar plane comprising the first cornea center position, the second cornea center position, and the projection position of the first eye, and the cornea of the right eye.

### Claim 7 (depends on 2)

The controller according to claim 2, wherein
the controller is configured to determine a first cornea center position, based on major and minor axes of an ellipse which is formed by a part of the cornea image of the first eye.

### Claim 8

A position determination device, comprising:
a communication module that is configured to receive a captured image generated by imaging first and second eyes of a user who is viewing an object point, by an imaging device; and
a controller that is configured to determine a position of the object point in a real space, based on a cornea image of the first eye and a cornea image of the second eye in the captured image.

### Claim 9

A position determination system, comprising:
an imaging device; and
a position determination device comprising
a communication module that is configured to receive a captured image generated by imaging first and second eyes of a user who is viewing an object point, by the imaging device, and
a controller that is configured to determine a position of the object point in a real space, based on a cornea image of the first eye and a cornea image of the second eye in the captured image.

### Claim 10

A display system, comprising:
an imaging device;
a position determination device comprising
a communication module that is configured to receive a captured image generated by imaging first and second eyes of a user who is viewing an object point, by an imaging device, and
a controller that is configured to determine a position of the object point in a real space, based on a cornea image of the first eye and a cornea image of the second eye in the captured image; and
a head-up display that is configured to display, based on the position of the object point determined by the controller, a virtual image that the eyes of the user view at a position related to the position of the object point.

### Claim 11

(canceled)

### Claim 12

A computer-readable recording medium recording a program for causing a controller to determine a position of an object point in a real space, based on a cornea image of a first eye and a cornea image of a second eye of a user who is viewing the object point, in a captured image generated by imaging the first and second eyes of the user by an imaging device.
