# US 12,008,880 B2 · Utilizing artificial intelligence to detect objects or patient safety events in a patient room

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,880 B2
- Document ID: US12008880B2
- Status: Granted
- Kind: B2
- Application number: 18/314,502
- Filed: 2023-05-09
- Published: 2023-09-07
- Granted: 2024-06-11
- Earliest claimed: 2017-12-28
- Assignees: Cerner Innovation, Inc.
- Applicants: Cerner Innovation, Inc.
- Inventors: Bruce Howard Kusens; Michael Kusens
- Web page: https://patentgrep.com/patent/US12008880B2

## CPC

- G08B 21/0476: {Cameras to detect unsafe condition, e.g. video cameras}
- G08B 25/08: using communication transmission lines {(G08B13/19658, G08B21/0286, G08B25/016 take precedence)}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B 29/185: {Signal analysis techniques for reducing or preventing false alarms or for enhancing the reliability of the system}
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods and systems are provided for detecting objects or patient safety events in a patient room. Artificial intelligence is utilized to enhance safety issue recognition capabilities by the methods and systems. Sensors collect a series of images and depth data in a room of a patient. Data (corresponding to images and depth data of an object or patent safety event) is received from the sensors and compared to stored data to identify the object or patient safety event. The images are communicated to a central video monitoring system and a user may be prompted to confirm if the object requires learning or a patient safety event occurred (or identify the object or patient safety event) or to provide additional parameters or actions. A patient safety learning system analyzes the data and incorporates the user response to enhance safety issue recognition capabilities of the system and reduce false alerts.

## Claims

### Claim 1

A method comprising:
collecting images of a scene including at least one object;
detecting the at least one object within at least one image of the images, wherein the at least one object is detected by comparing one or more features of the at least one object to one or more reference features;
in response to detecting that the at least one object is within the at least one image, presenting the at least one object in a graphical user interface;
receiving an indication that the at least one object has been selected in the graphical user interface;
in response to receiving the indication that the at least one object has been selected, updating a machine learning model using the one or more features of the at least one object, the updated machine learning model configured to predict that an event has occurred in the scene;
detecting that an event has occurred within the scene;
in response to detecting that the event has occurred, presenting information associated with the event in the graphical user interface;
receiving an indication that the event has been confirmed in the graphical user interface; and
in response to receiving the indication that the event has been confirmed, further updating the updated machine learning model.

### Claim 2 (depends on 1)

The method of claim 1, wherein at least one image of the images is collected by an image capturing device that is configured to capture images of an environment.

### Claim 3 (depends on 2)

The method of claim 2, wherein the environment is a patient room.

### Claim 4 (depends on 1)

The method of claim 1, wherein comparing the one or more features of the at least one object to one or more reference features comprises assigning one or more reference points to the one or more features of the at least one object.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving, from one or more image capturing devices, a set of images of an environment; and
predicting, using the updated machine learning model, that at least one event has occurred in the environment from the set of images.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
in response to predicting that the at least one event has occurred in the environment, generating an alert indicating that the at least one event occurred in the environment.

### Claim 7

A system comprising:
a processing system; and
one or more computer readable storage media storing instructions which, when executed by the processing system, cause the processing system to perform operations comprising:
collecting images of a scene including at least one object;
detecting the at least one object within at least one image of the images, wherein the at least one object is detected by comparing one or more features of the at least one object to one or more reference features;
in response to detecting that the at least one object is within the at least one image, presenting the at least one object in a graphical user interface;
receiving an indication that the at least one object has been selected in the graphical user interface;
in response to receiving the indication that the at least one object has been selected, updating a machine learning model using the one or more features of the at least one object, the updated machine learning model configured to predict that an event has occurred in the scene;
detecting that an event has occurred within the scene;
in response to detecting that the event has occurred, presenting information associated with the event in the graphical user interface;
receiving an indication that the event has been confirmed in the graphical user interface; and
in response to receiving the indication that the event has been confirmed, further updating the updated machine learning model.

### Claim 8 (depends on 7)

The system of claim 7,
wherein at least one image of the images is collected by an image capturing device that is configured to capture images of an environment.

### Claim 9 (depends on 8)

The system of claim 8, wherein the environment is a patient room.

### Claim 10 (depends on 7)

The system of claim 7, wherein comparing the one or more features of the at least one object to one or more reference features comprises assigning one or more reference points to the one or more features of the at least one object.

### Claim 11 (depends on 7)

The system of claim 7, the operations further comprising:
receiving, from one or more image capturing devices, a set of images of an environment; and
predicting, using the updated machine learning model, that at least one event has occurred in the environment from the set of images.

### Claim 12 (depends on 11)

The system of claim 11, the operations further comprising:
in response to predicting that the at least one event has occurred in the environment, generating an alert indicating that the at least one event occurred in the environment.

### Claim 13

One or more non-transitory computer-readable media storing computer-readable instructions that, when executed by a processing system, cause a system to perform operations comprising:
collecting images of a scene including at least one object;
detecting that at least one object within at least one image of the images, wherein the at least one object is detected by comparing one or more features of the at least one object to one or more reference features;
in response to detecting that the at least one object is within the at least one image, presenting the at least one object in a graphical user interface;
receiving an indication that the at least one object has been selected in the graphical user interface;
in response to receiving the indication that the at least one object has been selected, updating a machine learning model using the one or more features of the at least one object, the updated machine learning model configured to predict that an event has occurred in the scene;
detecting that an event has occurred within the scene;
in response to detecting that the event has occurred, presenting information associated with the event in the graphical user interface;
receiving an indication that the event has been confirmed in the graphical user interface; and
in response to receiving the indication that the event has been confirmed, further updating the updated machine learning model.

### Claim 14 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, the operations further comprising:
wherein at least one image of the images is collected by an image capturing device that is configured to capture images of an environment.

### Claim 15 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein comparing the one or more features of the at least one object to one or more reference features comprises assigning one or more reference points to the one or more features of the at least one object.

### Claim 16 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, the operations further comprising:
receiving, from one or more image capturing devices, a set of images of an environment; and
predicting, using the updated machine learning model, that at least one event has occurred in the environment from the set of images.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, the operations further comprising:
in response to predicting that the at least one event has occurred in the environment, generating an alert indicating that the at least one event occurred in the environment.