# US 2021/0275089 A1 · Deriving information about a person's sleep and wake states from a sequence of video frames

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0275089 A1
- Document ID: US20210275089A1
- Status: Published application
- Kind: A1
- Application number: 17/272,959
- Filed: 2019-09-23
- Published: 2021-09-09
- Earliest claimed: 2018-09-25
- Assignees: Koninklijke Philips N.V.
- Applicants: Koninklijke Philips N.V.
- Inventors: Xi LONG; Rick BEZEMER
- Web page: https://patentgrep.com/patent/US20210275089A1

## CPC

- A61B 5/4809: {Sleep detection, i.e. determining whether a subject is asleep or not}
- A61B 5/0033: {Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room}
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/4812: {Detecting sleep stages or cycles}
- A61B 5/7225: {Details of analogue processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation (input circuits for detecting, measuring, or recording bioelectric or biomagnetic signals A61B5/30; specific diagnostic methods using bioelectric or biomagnetic signals A61B5/316)}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- A61B 2503/045: Newborns, e.g. premature baby monitoring
- A61B 2576/00: Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence)
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

For the purpose of obtaining information about a person's sleep and wake states, an arrangement (100) comprising a video camera (10) and a processing unit (20) is used. The video camera (10) serves for capturing a sequence of video frames during a time period, and the processing unit (20) is configured to process video frames provided by the video camera (10) and to provide output representative of the person's sleep and wake states during the time period. In particular, the processing unit (20) is configured to execute an algorithm according to which (i) a motion value-time relation, (ii) sets of features relating to respective epochs in the motion value-time relation and (iii) classifiers of the respective epochs are determined, wherein the algorithm is further configured to apply an adaptive prior probability determined for the particular person in dependence of the motion values of the respective epochs to the classifiers.

## Claims

### Claim 1

An arrangement designed to derive information about a person's sleep and wake states from a sequence of video frames, comprising:
a video camera for capturing a sequence of video frames during a time period, and
a processing unit configured to process video frames provided by the video camera and to provide output representative of the person's sleep and wake states during the time period, wherein the processing unit is configured to execute an algorithm according to which
a motion value-time relation is determined from the video frames,
sets of features relating to respective epochs in the motion value-time relation are determined by extracting a number of different features from the motion values in each of the respective epochs,
classifiers of the respective epochs are determined by classifying the respective sets of features relating to the respective epochs as being representative of the person's sleep or wake state, and
an adaptive prior probability determined for the particular person in dependence of the motion values of the respective epochs is applied to the classifiers.

### Claim 2 (depends on 1)

The arrangement according to claim 1, wherein the algorithm is configured to determine the adaptive prior probability in dependence of a distribution of the motion values over the respective epochs.

### Claim 3 (depends on 2)

The arrangement according to claim 2, wherein the algorithm is configured to determine the adaptive prior probability in dependence of a number value of epochs with motion values which are larger than a reference motion value that is equal to or higher than zero in a total of epochs.

### Claim 4 (depends on 3)

The arrangement according to claim 3, wherein the algorithm is configured to assign a high prior probability for wake when the number value of epochs with motion values which are larger than the reference motion value is equal to or higher than a threshold, and to assign a low prior probability for wake when the number value of epochs with motion values which are larger than the reference motion value is lower than the threshold.

### Claim 5 (depends on 3)

The arrangement according to claim 3, wherein the algorithm is configured to determine an optimal relation between the number value of epochs with motion values which are larger than the reference motion value and a prior probability for wake.

### Claim 6 (depends on 1)

The arrangement according to claim 1, wherein the algorithm is further configured to apply a smoothing filter removing short period deviations from an overall pattern of classifiers in respect of successive epochs to the classifiers.

### Claim 7 (depends on 6)

The arrangement according to claim 6, wherein, in order to have the smoothing filter, the algorithm is configured to assess whether two sequences of epochs both representing a minimum length of time and involving sets of features to be classified as being representative of only one of the person's sleep or wake state are interrupted by one epoch or a limited number of epochs involving (a) set(s) of features to be classified as being representative of the other of the person's sleep or wake state, and if such is found to be the case, to set the classifier(s) of the one epoch or the limited number of epochs to be the same as the classifiers of the two sequences of epochs.

### Claim 8 (depends on 1)

The arrangement according to claim 1, wherein the features to be extracted from the motion values in each of the respective epochs include at least one of (i) the mean of the motion values in each of the respective epochs and (ii) the number of motion values which are larger than a reference motion value that is equal to or higher than zero in each of the respective epochs.

### Claim 9 (depends on 8)

The arrangement according to claim 8, wherein the features to be extracted from the motion values in each of the respective epochs include a relative possibility of sleep, and wherein the algorithm is configured to include a step of determining a time distance of each of the respective epochs to a nearest epoch with a high activity level in a process of determining the relative possibility of sleep.

### Claim 10 (depends on 8)

The arrangement according to claim 8, wherein the features to be extracted from the motion values in each of the respective epochs include at least one of (i) a relative possibility of sleep, wherein the algorithm is configured to include a step of determining a time distance of each of the respective epochs to a nearest epoch with a high activity level in a process of determining the relative possibility of sleep, and wherein the algorithm is configured to identify the epochs with a high activity level by taking the epochs with the highest mean of the motion values, up to a predetermined maximum percentage of a total number of epochs, and (ii) a relative possibility of sleep, wherein the algorithm is configured to include a step of determining a time distance of each of the respective epochs to a nearest epoch with a high activity level in a process of determining the relative possibility of sleep, and wherein the algorithm is configured to identify the epochs with a high activity level by taking the epochs with the highest number of motion values which are larger than the reference motion value, up to a predetermined maximum percentage of a total number of epochs.

### Claim 11 (depends on 1)

The arrangement according to claim 1, wherein the algorithm is configured to normalize the features.

### Claim 12 (depends on 1)

The arrangement according to claim 1, wherein the algorithm is configured to determine machine learning classifiers on the basis of differences between (i) an initial set of classifiers determined on the basis of the features and (ii) a final set of classifiers determined by applying at least the adaptive prior probability, and to use the machine learning classifiers for making adjustments in the algorithm as far as determining the classifiers of the respective epochs is concerned.

### Claim 13 (depends on 1)

The arrangement according to claim 1, wherein the algorithm is configured to apply 3D recursive search motion estimation for determining the motion value-time relation from the video frames and/or to apply Bayesian-based linear discriminant analysis for determining the classifiers of the respective epochs.

### Claim 14 (depends on 1)

The arrangement according to claim 1, designed for use in infant care, wherein the algorithm is configured to classify the respective sets of features relating to the respective epochs as being representative of a care state of the infant besides the infant's sleep or wake state, and wherein the algorithm is configured to set the classifier of an epoch to be a care state classifier when the epoch is an epoch with an activity level that is above a threshold chosen to distinguish between the wake state and the care state.

### Claim 15 (depends on 1)

A computer program product comprising a program code of a computer program to make a computer execute the algorithm of the arrangement according to claim 1 when the computer program is loaded on the computer.
