# US 2017/0285734 A1 · Head mounted display, detection device, control method for head mounted display, and computer program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0285734 A1
- Document ID: US20170285734A1
- Status: Published application
- Kind: A1
- Application number: 15/631,654
- Filed: 2017-06-23
- Published: 2017-10-05
- Earliest claimed: 2014-06-06
- Assignees: Seiko Epson Corporation
- Applicants: Seiko Epson Corporation
- Inventors: Hitoshi SAITO; Masahide TAKANO; Hiroyuki BABA
- Web page: https://patentgrep.com/patent/US20170285734A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06T 1/0007: {Image acquisition}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip
- G09B 19/0038: {Sports}
- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/344: with head-mounted left-right displays
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/61: based on recognised objects
- H04N 23/63: by using electronic viewfinders
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- H04N 5/2621: {Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A head mounted display for supporting improvement in an act of moving a body is provided. The head mounted display includes image display unit that transmits external scenery therethrough and forms an image, a body motion detection unit that detects motions of a body of a user and at least a part of a tool bundled with the body, a model moving image acquisition unit that accesses a model moving image storage unit which stores, as a model moving image, a moving image showing a motion of the body, used as a reference of the act and stores the model moving image for each type of act, and acquires one or a plurality of model moving images corresponding to a detection result from the body motion detection unit, and a display control unit that causes the image display unit to form the acquired model moving image.

## Claims

### Claim 1

A head mounted display for supporting improvement in an act of moving a body, comprising:
an image display unit that transmits external scenery therethrough and forms an image;
a body motion detection unit that detects motions of a user and at least a part of a tool bundled with the body;
a reference image acquisition unit that accesses a reference image storage unit which stores a reference image used as a reference of the act for each type of act, and acquires one or a plurality of reference images corresponding to a detection result from the body motion detection unit, wherein the acquisition of the one or plurality of reference images are based upon motion of the head of the user and at least one focused part of the body moving image; and
a display control unit that causes the image display unit to form the acquired reference image.

### Claim 2 (depends on 1)

The head mounted display according to claim 1,
wherein the reference image is an image showing a motion of the body.

### Claim 3 (depends on 2)

The head mounted display according to claim 2,
wherein the body motion detection unit includes a body imaging camera that is installed at a spectacle portion supporting the image display unit.

### Claim 4 (depends on 3)

The head mounted display according to claim 3,
wherein the body imaging camera is installed so that an imaging direction is a downward direction.

### Claim 5 (depends on 2)

The head mounted display according to claim 2, further comprising:
a user moving image creation unit that creates an illustration moving image of the user on the basis of a detection result from the body motion detection unit,
wherein the reference image is a moving image, and
wherein the display control unit causes an illustration moving image of the user to be formed in addition to the reference image.

### Claim 6 (depends on 2)

The head mounted display according to claim 2,
wherein the body motion detection unit includes an external camera that is disposed at a position spaced apart from a spectacle portion supporting the image display unit and images the user.

### Claim 7 (depends on 6)

The head mounted display according to claim 6,
wherein the display control unit causes an image of the user captured by the external camera to be formed in addition to the reference image.

### Claim 8

A head mounted display for supporting improvement in an act of moving a body, comprising:
an image display unit that transmits external scenery therethrough and forms an image;
a body motion detection unit that estimates a motion of a body of a user and at least a part of a tool bundled with the body;
a head motion detection unit that detects a motion of the head of the user, the motion of the head of the user including positional movement and shift angle of the head;
a reference image acquisition unit that accesses a reference image storage unit which stores a reference image used as a reference of the act for each type of act, and acquires one or a plurality of reference images corresponding to a detection result from the body motion detection unit; and
a display control unit that causes the image display unit to form the acquired reference image.

### Claim 9 (depends on 8)

The head mounted display according to claim 8,
wherein the motion of the head is changed in synchronization with timing of a change of frames stored in a storage unit that stores detected motions of the body of the user.

### Claim 10 (depends on 8)

The head mounted display according to claim 8,
wherein a focused part from each frame is specified, and a motion of each focused part is estimated on the basis of the detected motion of the head of the user.

### Claim 11 (depends on 8)

The head mounted display according to claim 8,
wherein motions of positions in an X axis direction, Y axis direction and Z axis direction are estimated on the basis of the detected motion of the head of the user.

### Claim 12 (depends on 8)

The head mounted display according to claim 8,
wherein the reference image is an image showing a motion of the body.

### Claim 13 (depends on 12)

The head mounted display according to claim 12,
wherein the body motion detection unit includes a body imaging camera that is installed at a spectacle portion supporting the image display unit.

### Claim 14 (depends on 13)

The head mounted display according to claim 13,
wherein the body imaging camera is installed so that an imaging direction is a downward direction.

### Claim 15 (depends on 12)

The head mounted display according to claim 12, further comprising:
a user moving image creation unit that creates an illustration moving image of the user on the basis of a detection result from the body motion detection unit,
wherein the reference image is a moving image, and
wherein the display control unit causes an illustration moving image of the user to be formed in addition to the reference image.

### Claim 16 (depends on 12)

The head mounted display according to claim 12,
wherein the body motion detection unit includes an external camera that is disposed at a position spaced apart from a spectacle portion supporting the image display unit and images the user.

### Claim 17 (depends on 16)

The head mounted display according to claim 16,
wherein the display control unit causes an image of the user captured by the external camera to be formed in addition to the reference image.

### Claim 18 (depends on 8)

The head mounted display according to claim 8,
wherein in a case where a predetermined part of the image has a difference of a predetermined level or more with the reference image at the same position, a marker is attached to the predetermined part.

### Claim 19 (depends on 9)

The head mounted display according to claim 9,
wherein the storage unit stores imaged motions of the body of the user from a shoulder to a lower portion of the body.

### Claim 20 (depends on 8)

The head mounted display according to claim 8,
wherein the body motion detection unit further detects motions of muscles and skeletons using an electromyograph.

### Claim 21

A control method for a head mounted display for supporting improvement in an act of moving a body, the head mounted display including an image display unit that transmits external scenery therethrough and forms an image, the method comprising:
detecting motions of a user and at least a part of a tool bundled with the body;
accessing a reference image storage unit which stores a reference image used as a reference of the act for each type of act, and acquires one or a plurality of reference images corresponding to a detection result from the detecting, wherein the acquisition of the one or plurality of reference images are based upon motion of the head of the user and at least one focused part of the body moving image; and
causing the image display unit to form the acquired reference image.

### Claim 22

A control method for a head mounted display for supporting improvement in an act of moving a body, the head mounted display including an image display unit that transmits external scenery therethrough and forms an image, the method comprising:
estimating a motion of a body of a user and at least a part of a tool bundled with the body;
detecting a motion of the head of the user, the motion of the head of the user including positional movement and shift angle of the head;
accessing a reference image storage unit which stores a reference image used as a reference of the act for each type of act, and acquires one or a plurality of reference images corresponding to a detection result from the detecting; and
causing the image display unit to form the acquired reference image.

### Claim 23

A non-transitory computer readable medium having embodied thereon a computer program for controlling a head mounted display for supporting improvement in an act of moving a body, the head mounted display including an image display unit that transmits external scenery therethrough and forms an image, the computer program causing the computer readable medium to implement:
a function of detecting motions of a user and at least a part of a tool bundled with the body;
a function of accessing a reference image storage unit which stores a reference image used as a reference of the act for each type of act, and acquires one or a plurality of reference images corresponding to a detection result from the detecting, wherein the acquisition of the one or plurality of reference images are based upon motion of the head of the user and at least one focused part of the body moving image; and
a function of causing the image display unit to form the acquired reference image.

### Claim 24

A non-transitory computer readable medium having embodied thereon a computer program for controlling a head mounted display for supporting improvement in an act of moving a body, the head mounted display including an image display unit that transmits external scenery therethrough and forms an image, the computer program causing the computer readable medium to implement:
a function of estimating a motion of a body of a user and at least a part of a tool bundled with the body;
a function of detecting a motion of the head of the user, the motion of the head of the user including positional movement and shift angle of the head;
a function of accessing a reference image storage unit which stores a reference image used as a reference of the act for each type of act, and acquires one or a plurality of reference images corresponding to a detection result from the detecting; and
a function of causing the image display unit to form the acquired reference image.
