# US 2021/0081645 A1 · Systems and methods for coherent monitoring

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0081645 A1
- Document ID: US20210081645A1
- Status: Published application
- Kind: A1
- Application number: 17/102,215
- Filed: 2020-11-23
- Published: 2021-03-18
- Earliest claimed: 2019-01-31
- Assignees: Palantir Technologies Inc.
- Applicants: Palantir Technologies Inc.
- Inventors: Daniel Cervelli; Anand Gupta; Andrew Elder; Robert Imig; Praveen Kumar Ramalingam; Reese Glidden; Matthew Fedderly
- Web page: https://patentgrep.com/patent/US20210081645A1

## CPC

- G06Q 10/00: Administration; Management
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 18/24: Classification techniques
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 20/20: in augmented reality scenes
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/64: Three-dimensional [3D] objects
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods are provided for intelligently monitoring environments, classifying objects within such environments, detecting events within such environments, receiving and propagating input concerning image information from multiple users in a collaborative environment, identifying and responding to situational abnormalities or situations of interest based on such detections and/or user inputs.

## Claims

### Claim 1

A system for intelligently monitoring an environment, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to:
obtain content representing an environment, the content comprising a plurality of frames;
identify, based on the content, one or more discrete objects observed within the environment;
track an object of the one or more discrete objects across the frames;
detect one or more events associated with the tracked object;
learn one or more patterns associated with the detected one or more events;
determine whether a second event deviates from the learned one or more patterns; and
in response to determining that the second event deviates from the learned one or more patterns, flag the second event.

### Claim 2 (depends on 1)

The system of claim 1, wherein the learning the one or more patterns comprises learning seasonal changes in the one or more patterns.

### Claim 3 (depends on 1)

The system of claim 1, wherein the content comprises video content including the object captured from two angles, and the instructions further cause the system to:
generate a three-dimensional (3D) model of the object according to a parallax based 3D reconstruction algorithm; and
augment a map display of the environment with the generated 3D model.

### Claim 4 (depends on 1)

The system of claim 1, wherein the learning the one or more patterns associated with the detected one or more events comprises learning a range associated with the detected one or more events; and the determining whether a second event deviates from the learned one or more patterns comprises determining whether the second event is outside the range.

### Claim 5 (depends on 3)

The system of claim 3, wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
present a map of the environment; and
simultaneously present a playback of the tracked object in a separate pane.

### Claim 6 (depends on 5)

The system of claim 5, wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
identify a geographic boundary associated with the playback of the tracked object; and
determine a corresponding boundary in the map;
detect an input to zoom in the geographic boundary; and
change the corresponding boundary in the map based on the input to zoom in.

### Claim 7 (depends on 1)

The system of claim 1, wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
detect a number of occupants within the object based on a heat signature from thermal imagery; and
detect one or more attributes about an occupant of the occupants.

### Claim 8 (depends on 1)

The system of claim 1, wherein the detecting one or more events associated with the tracked object comprises:
detecting changes in the environment;
identifying candidate events based on the detected changes; and
comparing the candidate events with templates while accounting for a scaling, angle, or orientation difference between the object and corresponding objects in the templates.

### Claim 9 (depends on 1)

The system of claim 1, wherein the instructions further cause the system to:
determine a view field of a sensor capturing the content.

### Claim 10 (depends on 1)

The system of claim 1, wherein the instructions further cause the system to:
in response to detecting the one or more events, present a snapshot of a particular frame corresponding to the one or more detected events.

### Claim 11

A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:
obtaining content representing an environment, the content comprising a plurality of frames;
identifying, based on the content, one or more discrete objects observed within the environment;
tracking an object of the one or more discrete objects across the frames;
detecting one or more events associated with the tracked object;
learning one or more patterns associated with the detected one or more events;
determining whether a second event deviates from the learned one or more patterns; and
in response to determining that the second event deviates from the learned one or more patterns, flagging the second event.

### Claim 12 (depends on 11)

The method of claim 11, wherein the learning the one or more patterns comprises learning seasonal changes in the one or more patterns.

### Claim 13 (depends on 11)

The method of claim 11, wherein the content comprises video content including the object captured from two angles, and the method further comprises:
generating a three-dimensional (3D) model of the object according to a parallax based 3D reconstruction algorithm; and
augmenting a map display of the environment with the generated 3D model.

### Claim 14 (depends on 11)

The method of claim 11, wherein the learning the one or more patterns associated with the detected one or more events comprises learning a range associated with the detected one or more events; and the determining whether a second event deviates from the learned one or more patterns comprises determining whether the second event is outside the range.

### Claim 15 (depends on 13)

The method of claim 13, further comprising:
presenting a map of the environment; and
simultaneously presenting a playback of the tracked object in a separate pane.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
identifying a geographic boundary associated with the playback of the tracked object; and
determining a corresponding boundary in the map;
detecting an input to zoom in the geographic boundary; and
changing the corresponding boundary in the map based on the input to zoom in.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
detecting a number of occupants within the object based on a heat signature from thermal imagery; and
detecting one or more attributes about an occupant of the occupants.

### Claim 18 (depends on 11)

The method of claim 11, wherein the detecting one or more events associated with the tracked object comprises:
detecting changes in the environment;
identifying candidate events based on the detected changes; and
comparing the candidate events with templates while accounting for a scaling, angle, or orientation difference between the object and corresponding objects in the templates.

### Claim 19

A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
obtaining content representing an environment, the content comprising a plurality of frames;
identifying, based on the content, one or more discrete objects observed within the environment;
tracking an object of the one or more discrete objects across the frames;
detecting one or more events associated with the tracked object;
learning one or more patterns associated with the detected one or more events;
determining whether a second event deviates from the learned one or more patterns; and
in response to determining that the second event deviates from the learned one or more patterns, flagging the second event.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, wherein the content comprises video content including the object captured from two angles, and the instructions further cause the one or more processors to perform:
generating a three-dimensional (3D) model of the object according to a parallax based 3D reconstruction algorithm; and
augmenting a map display of the environment with the generated 3D model.
