# US 2017/0371406 A1 · Program, device, and calibration method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0371406 A1
- Document ID: US20170371406A1
- Status: Published application
- Kind: A1
- Application number: 15/612,293
- Filed: 2017-06-02
- Published: 2017-12-28
- Earliest claimed: 2016-06-15
- Assignees: Seiko Epson Corporation
- Applicants: Seiko Epson Corporation
- Inventors: Yang YANG; Guoyi FU
- Web page: https://patentgrep.com/patent/US20170371406A1

## CPC

- G06F 3/012: {Head tracking input arrangements}
- G02B 27/017: {Head mounted}
- G02B 27/0172: {characterised by optical features}
- G06T 7/74: {involving reference images or patches}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 2207/30204: Marker
- G06T 2207/30244: Camera pose
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method of calibration for a head-mounted display device including an camera, an inertial sensor and a processor causes the processor to realize functions of : acquiring a first image and a second image that are captured by imaging a first real marker with the camera when the inertial sensor is respectively at a first pose and a second pose; acquiring a difference in pose between the first pose and the second pose based on output of the inertial sensor; and deriving a spatial relationship between the camera and the inertial sensor based at least on the first image, the second image and the difference in pose.

## Claims

### Claim 1

A non-transitory storage medium containing program instructions that, when executed by a computer processor, cause the computer processor to perform a method, the method comprising:
acquiring a first image and a second image that are captured by imaging a real marker with a camera when an inertial sensor is respectively at a first pose and a second pose, the camera and the inertial sensor being included in a head-mounted display device;
acquiring a difference in pose between the first pose and the second pose based on output of the inertial sensor; and
deriving a spatial relationship between the camera and the inertial sensor using at least the first image, the second image and the difference in pose.

### Claim 2 (depends on 1)

A non-transitory storage medium according to claim 1,
wherein the head-mounted display device includes a first base portion and a second base portion, the camera is fixed with respect to the first base portion and the inertial sensor is fixed to the second base portion, so that at least one of the camera and the inertial sensor rotates with respect to one another.

### Claim 3 (depends on 1)

A non-transitory storage medium according to claim 1,
wherein the head-mounted display device further includes a display section that displays an image, and
wherein at least one of the camera and the display section is movable with respect to one another.

### Claim 4

A head-mounted display device comprising:
a camera;
an inertial sensor, and
a processor,
wherein the processor is configured to act:
acquiring a first image and a second image that are captured by imaging a first real marker with the camera when the inertial sensor is respectively at a first pose and a second pose;
acquiring a difference in pose between the first pose and the second pose based on output of the inertial sensor, and
deriving a spatial relationship between the camera and the inertial sensor based at least on the first image, the second image and the difference in pose.

### Claim 5

A method of calibration for a head-mounted display device including a camera, an inertial sensor and a processor, the method comprising steps of:
acquiring a first image and a second image that are captured by imaging a first real marker with the camera when the inertial sensor is respectively at a first pose and a second pose;
acquiring a difference in pose between the first pose and the second pose based on output of the inertial sensor, and
deriving a spatial relationship between the camera and the inertial sensor based at least on the first image, the second image and the difference in pose.
