# US 2022/0036094 A1 · Method and system for monitoring subjects for conditions or occurrences of interest

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036094 A1
- Document ID: US20220036094A1
- Status: Published application
- Kind: A1
- Application number: 17/390,819
- Filed: 2021-07-30
- Published: 2022-02-03
- Earliest claimed: 2020-08-03
- Assignees: Healthcare Integrated Technologies, Inc.
- Applicants: Healthcare Integrated Technologies Inc.
- Inventors: Kenneth M. GREENWOOD; Scott Michael BORUFF; Jurgen VOLLRATH
- Web page: https://patentgrep.com/patent/US20220036094A1

## CPC

- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/251: {of input or preprocessed data}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G10L 25/51: for comparison or discrimination
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method and system for monitoring subjects in facilities, such as hospitals, continuous care retirement communities, and prisons, to identify anomalies in behavior and compliance with rules.

## Claims

### Claim 1

A system for monitoring human or robotic subjects in a defined location, comprising
at least one image capture device,
a memory containing logic defining at least one of: the subject(s) that are required or permitted in the defined location, and under what circumstances such subject(s) may enter or leave the defined location;
a data store for capturing information about one or more of: anomalies, illicit behavior, unsafe conditions, suspicious behavior, abusive behavior, and changes in interactions between subjects (collectively referred to as trigger events), in the defined location based on information provided by the at least one image capture device,
a processor configured to process logic contained in the memory, and
an artificial intelligence (AI) network for identifying trigger events, determining whether a trigger event rises to the level of a flaggable event that requires third-party attention based on type and degree of the event or based on corroboration by data from a second source, and notifying at least one third-party if a flaggable event is identified.

### Claim 2 (depends on 1)

The system of claim 1, wherein the third-party is a predefined or dynamically determined person, entity, or secondary system based on the nature of the flaggable event.

### Claim 3 (depends on 1)

The system of claim 1, wherein the second source includes a second camera or a microphone.

### Claim 4 (depends on 1)

The system of claim 1, wherein the AI network is configured using training data provided by the at least one image capture device observing the subjects in the defined location.

### Claim 5 (depends on 4)

The system of claim 4, wherein the AI network compares raw data or derived incoming data from the at least one image capture device to pre-recorded raw or derived image files indicative of flaggable events.

### Claim 6 (depends on 1)

The system of claim 1, wherein the at least one image capture device includes one or more of: a radio frequency image capture device, a thermal frequency image capture device, and a video camera.

### Claim 7 (depends on 1)

The system of claim 1, wherein the trigger event includes one or more of, a subject falling, a subject being immobile in an unexpected area or during an unexpected time of day or for excessive periods of time, changes in a subject's routine for a particular time of day or over the course of a defined period, changes or odd behavior in the interactions between two or more subjects, attempts by a subject to do things that the subject is not authorized to do, and insufficient performance of required or expected duties or tasks by a subject.

### Claim 8 (depends on 1)

The system of claim 1, further comprising one or more additional sensors for capturing other forms of data of different modalities about the one or more subjects and their location.

### Claim 9 (depends on 8)

The system of claim 8, wherein the AI network is configured to use at least one of timer information, and data from one or more of the additional sensors, to corroborate image data or supplement image data where image data is insufficient or non-existent.

### Claim 10 (depends on 8)

The system of claim 8, wherein the one or more additional sensors include sensors to capture data about the environmental conditions of the defined location, for purposes of detecting unexpected changes or anomalies in said environment.

### Claim 11

A method of monitoring one or more subjects that are associated with a defined location, comprising
capturing information about the one or more subjects,
identifying when a monitored subject enters or leaves the defined location,
defining the leaving and entering of the defined location as trigger events,
comparing the information for each subject to one or more of: information previously captured for said subject, a predefined schedule for said subject, and data from other subjects in similar situations or with similar physical conditions, to detect deviations, which constitute a trigger event, time stamping trigger events,
identifying those trigger events that rise to the level of a flaggable event, and notifying authorized parties or entities about flaggable events.

### Claim 12 (depends on 11)

The method of claim 11, wherein the captured information includes image data from one or more image capture devices operating in one or more frequency ranges, including data in raw or processed form.

### Claim 13 (depends on 12)

The method of claim 12, wherein the processed data includes data that has been transformed by an AI system or subsystem.

### Claim 14 (depends on 11)

The method of claim 11, further comprising defining opaque zones where image data is not captured, or where image quality is limited or convoluted to protect privacy.

### Claim 15 (depends on 14)

The method of claim 14, wherein image data is supplemented with alternative sensor information or timing information, to monitor subjects in the opaque zones or monitor their time in the opaque zones.

### Claim 16 (depends on 11)

The method of claim 11, wherein comparing of information includes identifying anomalies or unexpected or notable changes in the information, using an artificial intelligence network.

### Claim 17 (depends on 11)

The method of claim 11, wherein a flaggable event includes one or more of: certain trigger events that have been pre-defined as flaggable events, the same trigger event being repeated more than once, and a trigger event based on a first sensor's data being corroborated by at least one other sensor.

### Claim 18 (depends on 17)

The method of claim 17, wherein pre-defined flaggable events include one or more of, a subject leaving or entering the location without being expected or authorized to do so, and changes in interactions with other subjects as defined by the nature of the interaction or the identity of the other subject.

### Claim 19

A method of monitoring one or more subjects that are associated with a defined location, comprising;=:
capturing image information about the one or more subjects, using one or more image capture devices operating in one or more frequency ranges, wherein the privacy of subjects is protected by defining opaque zones where image data is not captured, or is convoluted,
supplementing the image information with non-image sensor information to monitor subjects in the opaque zones, or capturing timing information to monitor their time in the opaque zones,
comparing the image information, and at least one of the non-image information, and timing information to previously recorded data defining the routine of the one or more subjects, and
defining a flaggable event if an anomaly is detected in the routine of the one or more subjects.

### Claim 20 (depends on 19)

The method of claim 19, wherein the defining of a flaggable event includes the use of an artificial intelligence network.
