# US 11,998,366 B2 · Generating a breathing alert

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,998,366 B2
- Document ID: US11998366B2
- Status: Granted
- Kind: B2
- Application number: 17/862,266
- Filed: 2022-07-11
- Published: 2022-11-17
- Granted: 2024-06-04
- Earliest claimed: 2016-09-06
- Assignees: Nutrits Ltd
- Applicants: Nutrits, Ltd.
- Inventors: Jacob Gunther; Nathan E. Ruben
- Web page: https://patentgrep.com/patent/US11998366B2

## CPC

- A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots}
- G06T 7/0016: {involving temporal comparison}
- A61B 5/0816: {Measuring devices for examining respiratory frequency (measuring frequency of electric signals G01R23/00)}
- A61B 5/107: Measuring physical dimensions, e.g. size of the entire body or parts thereof
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/113: occurring during breathing
- A61B 5/4815: {Sleep quality}
- A61B 5/4818: {Sleep apnoea}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- A61B 5/7267: {involving training the classification device}
- A61B 5/741: {using synthesised speech}
- A61B 5/746: {Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms}
- A61B 5/747: {in case of emergency, i.e. alerting emergency services}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/13: Edge detection
- G06T 7/143: involving probabilistic approaches, e.g. Markov random field [MRF] modelling
- G06T 7/194: involving foreground-background segmentation
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/44: using image operators, e.g. filters, edge density metrics or local histograms
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/30061: Lung
- G06T 2207/30088: Skin; Dermal
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

For generating a breathing alert is disclosed, a method receives a video stream of a subject. The method further estimates a breathing signal from the video stream. The method determines one of a large-scale motion and/or a breathing event of the subject based on the breathing signal. The method generates an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval.

## Claims

### Claim 1

A method comprising:
training, by a processing apparatus, a breathing model on training video streams of a plurality of subjects;
receiving, by the processing apparatus, a first video stream of a subject;
estimating, by the processing apparatus, a breathing signal from the first video stream using the breathing model;
determining, by the processing apparatus, one of a large-scale motion and/or a breathing event of the subject based on the breathing signal using the breathing model;
generating, by the processing apparatus, an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval;
presenting, by the processing apparatus, the first video stream, a breathing report, and/or an audio stream, wherein the breathing report comprises a numerical representation, a heat map, a graph, and/or a streaming sinusoidal graph of a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, and inter-breath interval histogram, motion frequency, motion magnitude, motion duration, sleep length, sleep quality, sleep intervals, a sleep position, a sleep status, and/or apnea event data;
detecting, by a processing apparatus, a sleep position and/or a sleep status of the subject from the first video stream, wherein the sleep position comprises thrashing, a stomach orientation, a back orientation, and/or a side orientation and the sleep status comprises asleep and awake; and
communicating, by the processing apparatus, a notification in response to a specified sleep position, a specified sleep status, a change of the sleep position, and/or a change of sleep status.

### Claim 2 (depends on 1)

The method of claim 1, the method further comprising:
training, by a processing apparatus, a crying model on a plurality of subjects crying;
receiving, by the processing apparatus, an audio stream; and
identifying, by the processing apparatus, the subject crying from the audio stream using the crying model.

### Claim 3 (depends on 1)

The method of claim 1, the method further comprising:
training, by a processing apparatus, a smiling model on a plurality of subjects smiling; and
identifying, by the processing apparatus, the subject smiling from the first video stream using the smiling model.

### Claim 4 (depends on 1)

The method of claim 1, the method further comprising:
measuring, by a processing apparatus, ambient light; and
in response to the ambient light being less than a light threshold, activating, by the processing apparatus, an infrared light, wherein the first video stream captures the subject in infrared.

### Claim 5 (depends on 1)

The method of claim 1, the method further comprising:
detecting, by the processing apparatus, a face from the first video stream;
scaling, by the processing apparatus, a face region comprising the face to a specified pixel size;
calculating, by the processing apparatus, a skin histogram on a color space of face region;
selecting, by the processing apparatus, vectors of pixels based on the skin histogram;
iteratively generating, by the processing apparatus, a skin mask from the vectors; and
iteratively tracking, by the processing apparatus, the face.

### Claim 6

An apparatus comprising:
a processing apparatus;
a memory storing code executable by the processing apparatus to perform:
training a breathing model on training video streams of a plurality of subjects;
receiving a first video stream of a subject;
estimating a breathing signal from the first video stream using the breathing model;
determining one of a large-scale motion and/or a breathing event of the subject based on the breathing signal using the breathing model;
generating an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval;
presenting the first video stream, a breathing report, and/or an audio stream, wherein the breathing report comprises a numerical representation, a heat map, a graph, and/or a streaming sinusoidal graph of a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, and inter-breath interval histogram, motion frequency, motion magnitude, motion duration, sleep length, sleep quality, sleep intervals, a sleep position, a sleep status, and/or apnea event data;
detecting a sleep position and/or a sleep status of the subject from the first video stream, wherein the sleep position comprises thrashing, a stomach orientation, a back orientation, and/or a side orientation and the sleep status comprises asleep and awake; and
communicating a notification in response to a specified sleep position, a specified sleep status, a change of the sleep position, and/or a change of sleep status.

### Claim 7 (depends on 6)

The apparatus of claim 6, the processing apparatus further:
training a crying model on a plurality of subjects crying;
receiving an audio stream; and
identifying the subject crying from the audio stream using the crying model.

### Claim 8 (depends on 6)

The apparatus of claim 6, the processing apparatus further:
training a smiling model on a plurality of subjects smiling; and
identifying the subject smiling from the first video stream using the smiling model.

### Claim 9 (depends on 6)

The apparatus of claim 6, the processing apparatus further:
measuring ambient light; and
in response to the ambient light being less than a light threshold, activating an infrared light, wherein the first video stream captures the subject in infrared.

### Claim 10 (depends on 6)

The apparatus of claim 6, the processing apparatus further:
detecting a face from the first video stream;
scaling a face region comprising the face to a specified pixel size;
calculating a skin histogram on a color space of face region;
selecting vectors of pixels based on the skin histogram;
iteratively generating a skin mask from the vectors; and
iteratively tracking the face.

### Claim 11

A system comprising:
a camera;
a processing apparatus;
a memory storing code executable by the processing apparatus to perform:
training a breathing model on training video streams of a plurality of subjects;
receiving a first video stream from the camera of a subject;
estimating a breathing signal from the first video stream using the breathing model;
determining one of a large-scale motion and/or a breathing event of the subject based on the breathing signal using the breathing model;
generating an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval;
presenting the first video stream, a breathing report, and/or an audio stream, wherein the breathing report comprises a numerical representation, a heat map, a graph, and/or a streaming sinusoidal graph of a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, an inter-breath interval histogram, motion frequency, motion magnitude, motion duration, sleep length, sleep quality, sleep intervals, a sleep position, a sleep status, and/or apnea event data;
detecting a sleep position and/or a sleep status of the subject from the first video stream, wherein the sleep position comprises thrashing, a stomach orientation, a back orientation, and/or a side orientation and the sleep status comprises asleep and awake; and
communicating a notification in response to a specified sleep position, a specified sleep status, a change of the sleep position, and/or a change of sleep status.

### Claim 12 (depends on 11)

The system of claim 11, the processing apparatus further:
training a crying model on a plurality of subjects crying;
receiving an audio stream; and
identifying the subject crying from the audio stream using the crying model.

### Claim 13 (depends on 11)

The system of claim 11, the processing apparatus further:
training a smiling model on a plurality of subjects smiling; and
identifying the subject smiling from the first video stream using the smiling model.

### Claim 14 (depends on 11)

The system of claim 11, the processing apparatus further:
measuring ambient light; and
in response to the ambient light being less than a light threshold, activating an infrared light, wherein the first video stream captures the subject in infrared.

### Claim 15 (depends on 11)

The system of claim 11, the processing apparatus further:
detecting a face from the first video stream;
scaling a face region comprising the face to a specified pixel size;
calculating a skin histogram on a color space of face region;
selecting vectors of pixels based on the skin histogram;
iteratively generating a skin mask from the vectors; and
iteratively tracking the face.