# US 9,750,452 B1 · Technique for determining sleeping state of user

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,750,452 B1
- Document ID: US9750452B1
- Status: Granted
- Kind: B1
- Application number: 15/409,804
- Filed: 2017-01-19
- Published: 2017-08-17
- Granted: 2017-09-05
- Earliest claimed: 2016-02-17
- Assignees: Koei Tecmo Games Co., Ltd.
- Applicants: Koei Tecmo Games Co., Ltd.
- Inventors: Takashi Saito
- Web page: https://patentgrep.com/patent/US9750452B1

## CPC

- A61B 5/4812: {Detecting sleep stages or cycles}
- A61B 5/1114: {Tracking parts of the body}
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/4836: {Diagnosis combined with treatment in closed-loop systems or methods (A61B5/0036 takes precedence)}
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- A61B 5/7278: {Artificial waveform generation or derivation, e.g. synthesizing signals from measured signals}
- A61B 5/742: {using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339)}
- A61B 5/744: {Displaying an avatar, e.g. an animated cartoon character}
- A61B 5/746: {Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms}
- A61B 2503/12: Healthy persons not otherwise provided for, e.g. subjects of a marketing survey
- A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
- A63B 2071/0666: {worn on the head or face, e.g. combined with goggles or glasses}
- A63B 2220/803: Motion sensors
- A63B 2220/806: Video cameras
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)

## Abstract

A non-transitory computer-readable recording medium having recorded thereon a program that causes a computer to execute a method to display an image of a predetermined process on a body-mounted display device worn by a user, the method includes tracking a head or the like of the user, and calculating the position and the direction of the head or the like; displaying the image of the predetermined process in a virtual space of the display device; determining that the user is in the sleeping state when a candidate state, in which at least either of the calculated position and direction of the head or the like of the user is shifted by greater than or equal to a predetermined set amount, continues for greater than or equal to a predetermined set period; and performing a predetermined sleeping countermeasure process when the user is determined to be in the sleeping state.

## Claims

### Claim 1

A non-transitory computer-readable recording medium having recorded thereon a data processing program that causes a computer to execute a method to display an image of a predetermined process on a body-mounted display device worn by a user, the method comprising:
tracking a head, a face or a neck of the user, and calculating the position and the direction of the head, the face or the neck of the user;
displaying the image of the predetermined process in a virtual space of the display device such that a position and a direction of the predetermined image corresponds to the calculated position and direction of the head, the face or the neck of the user;
determining that the user is in the sleeping state when a candidate state, in which at least either of the calculated position and direction of the head, the face or the neck of the user is shifted by greater than or equal to a predetermined set amount, continues for greater than or equal to a predetermined set period; and
performing a predetermined sleeping countermeasure process when it is determined that the user is in the sleeping state.

### Claim 2 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein the candidate state is that the calculated position of the head, the face or the neck of the user is below a set reference position by greater than or equal to a predetermined set distance.

### Claim 3 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein the candidate state is that the calculated direction of the head, the face or the neck of the user directs below with respect to a set reference attitude by greater than or equal to a predetermined set angle.

### Claim 4 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein the predetermined sleeping countermeasure process is at least one of
(a) storing information indicating a content of a process performed by the user regarding the predetermined process in a storage unit,
(b) finishing the predetermined process,
(c) switching off a power source of the display device,
(d) alarming the user that the user is in the sleeping state,
(e) temporarily suspending the predetermined process,
(f) switching control by the user to an automatic control for the process performed by the user regarding the predetermined process, and
(g) notifying that the user is in the sleeping state to other users.

### Claim 5 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein in the determining that the user is in the sleeping state, the user is determined to in the sleeping state when the candidate state continues for greater than or equal to the predetermined set period and also a state in which the user closes its eyes for greater than or equal to another predetermined set period.

### Claim 6

A data processing apparatus that controls to display an image of a predetermined process on a body-mounted display device worn by a user, comprising:
a position and attitude calculation unit that tracks a head, a face or a neck of the user, and calculates the position and the direction of the head, the face or the neck of the user;
a display control unit that displays the image of the predetermined process in a virtual space of the display device such that a position and a direction of the predetermined image corresponds to the calculated position and direction of the head, the face or the neck of the user;
a determining unit that determines that the user is in the sleeping state when a candidate state, in which at least either of the calculated position and direction of the head, the face or the neck of the user is shifted by greater than or equal to a predetermined set amount, continues for greater than or equal to a predetermined set period; and
a process performing unit that performs a predetermined sleeping countermeasure process when it is determined that the user is in the sleeping state.