# US 8,542,872 B2 · Motion-validating remote monitoring system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,542,872 B2
- Document ID: US8542872B2
- Status: Granted
- Kind: B2
- Application number: 12/167,877
- Filed: 2008-07-03
- Published: 2009-01-08
- Granted: 2013-09-24
- Earliest claimed: 2007-07-03
- Assignees: Pivotal Vision, LLC
- Applicants: John R. Gornick; Craig B. Moksnes
- Inventors: John R. Gornick; Craig B. Moksnes
- Web page: https://patentgrep.com/patent/US8542872B2

## CPC

- G08B 13/194: using image scanning and comparing systems
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/292: Multi-camera tracking
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 17/05: Geographic models
- G06V 10/24: Aligning, centring, orientation detection or correction of the image
- G06V 20/44: {Event detection}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B 13/19602: {Image analysis to detect motion of the intruder, e.g. by frame subtraction}
- G08B 13/19604: {involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change}
- G08B 13/19608: {Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position}
- G08B 13/1961: {Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image}
- G08B 13/1963: {Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]}
- G08B 13/19641: {Multiple cameras having overlapping views on a single scene}
- G08B 13/19645: {Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over}
- H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)}
- G06T 2207/30232: Surveillance
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of autonomously monitoring a remote site, including the steps of locating a primary detector at a site to be monitored; creating one or more geospatial maps of the site using an overhead image of the site; calibrating the primary detector to the geospatial map using a detector-specific model; detecting an object in motion at the site; tracking the moving object on the geospatial map; and alerting a user to the presence of motion at the site. In addition thermal image data from a infrared cameras, rather than optical/visual image data, is used to create detector-specific models and geospatial maps in substantially the same way that optical cameras and optical image data would be used.

## Claims

### Claim 1

A motion-validating monitoring system for detecting objects in motion at a site to be monitored, comprising:
a first detector for monitoring a site and transmitting first detector image data, the first detector comprising a pan-tilt-zoom (PTZ) detector; and
a detector controller adapted to:
receive the first detector image data from the first detector, and to use the first detector image data to create a three-dimensional (3D) terrain model of the site, the 3D terrain model defining an x dimension, a y dimension, and a z dimension, the x dimension and y dimension defining a reference plane of the site, the z dimension defining a distance above or below the reference plane of the site,
detect an object in motion at the site by analyzing the first detector image data transmitted from the first detector; and
track the object in motion within the 3D terrain model, including determining a first location of the object in motion, the first location defined by an x, y, and z coordinate of the 3D terrain model and including directing the PTZ detector to change a PTZ detector parameter so as to detect the object in motion at a second location;
wherein the creation of the 3D terrain model includes the steps of:
identifying a first reference point and a second reference point at the site to be monitored, the first reference point defined by a first x, y, and z coordinate set, and the second reference point defined by a second x, y, and z coordinate set, each coordinate corresponding to the respective x, y, and z dimensions of the 3D terrain model;
capturing a first reference image of the site to be monitored using the first detector configured at a first known PTZ parameter set that includes a pan angle, tilt angle, and zoom level, the first reference image comprising a plurality of image pixels, each of the image pixels having image pixel data, and one of the plurality of image pixels of the first reference image being a first reference pixel corresponding to the-first reference point, such that the first reference pixel also corresponds to the first x, y, and z coordinate set;
capturing a second reference image of the site to be monitored using the first detector configured at a second known PTZ parameter set that includes a pan angle, tilt angle, and zoom level, the second reference image comprising a plurality of image pixels, each of the image pixels having image pixel data, and one of the plurality of image pixels of the second reference image being a second reference pixel corresponding to the second-reference point, such that the second reference pixel also corresponds to the second x, y, and z coordinate set; and
scaling the 3D terrain model for substantially the entire site, including calculating x, y, and z coordinate sets and PTZ parameter sets for image pixels of the first image and the second image based on the first and second x, y, and z coordinate sets of the first and second reference pixels and the first and second PTZ parameter sets, such that image pixels captured by the first detector correspond to actual geospatial points of the site, each of the actual geospatial points of the site defined by its respective x, y, and z coordinate set.

### Claim 2

A method of autonomously monitoring a remote site, including:
locating a pan-tilt-zoom (PTZ) detector at a detector location at a site to be monitored;
creating a three-dimensional (3D) terrain model of the site to be monitored, the 3D terrain model defining an x dimension, a y dimension, and a z dimension, the x dimension and y dimension defining a reference plane of the site, the z dimension defining a distance above or below the reference plane of the site, including:
identifying a first reference point and a second reference point at the site to be monitored, the first reference point defined by a first x, y, and z coordinate set, and the second reference point defined by a second x, y, and z coordinate set, each coordinate corresponding to the respective x, y, and z dimensions of the 3D terrain model;
capturing a first reference image of the site to be monitored using the PTZ detector configured at a first known PTZ parameter set that includes a pan angle, tilt angle, and zoom level, the first reference image comprising a plurality of image pixels, each of the image pixels having image pixel data, and one of the plurality of image pixels of the first reference image being a first reference pixel corresponding to the-first reference point, such that the first reference pixel also corresponds to the first x, y, and z coordinate set;
capturing a second reference image of the site to be monitored using the PTZ detector configured at a second known PTZ parameter set that includes a pan angle, tilt angle, and zoom level, the second reference image comprising a plurality of image pixels, each of the image pixels having image pixel data, and one of the plurality of image pixels of the second reference image being a second reference pixel corresponding to the second-reference point, such that the second reference pixel also corresponds to the second x, y, and z coordinate set; and
scaling the 3D terrain model for substantially the entire site, including calculating x, y, and z coordinate sets and PTZ parameter sets for image pixels of the first image and the second image based on the first and second x, y, and z coordinate sets of the first and second reference pixels and the first and second PTZ parameter sets, such that image pixels captured by the PTZ detector correspond to actual geospatial points of the site, each of the actual geospatial points of the site defined by its respective x, y, and z coordinate set;
detecting an object in motion at the site by analyzing digital pixel data of images of the site transmitted from the PTZ detector;
tracking the object in motion within the 3D terrain model, including determining a first location of the object in motion, the first location defined by an x, y, and z coordinate of the 3D terrain model and including directing the PTZ detector to change a PTZ detector parameter so as to detect the object in motion at a second location.

### Claim 3 (depends on 1)

The system of claim 1, wherein the PTZ detector captures visual or thermal image data.

### Claim 4 (depends on 1)

The system of claim 1, wherein the controller is further adapted to validate the object in motion at the site by analyzing digital pixel data of images transmitted from the first detector to determine if it meets one of a characteristic of size, speed and uniform motion, relative to the reference plane of the 3D terrain model.

### Claim 5 (depends on 2)

The method of claim 2, wherein the detector is a pan, tilt, zoom (PTZ) camera.

### Claim 6 (depends on 5)

The method of claim 5, wherein capturing at least one additional reference image of the site to be monitored using the detector includes using the PTZ camera at a zoom setting different from a zoom setting of the first reference image.

### Claim 7 (depends on 2)

The method of claim 2, wherein tracking the object in motion within the 3D terrain model includes identifying the location of the object in motion with respect to the reference plane.

### Claim 8 (depends on 2)

The method of claim 2, further comprising validating the object in motion at the site by analyzing digital pixel data of images transmitted from the detector to determine if it meets one of a characteristic of size, speed and uniform motion, relative to the reference plane of the 3D terrain model.

### Claim 9 (depends on 1)

The system of claim 1, wherein scaling the 3D terrain model further includes calculating x, y, and z coordinate sets and PTZ parameter sets for all image pixels of the first image and the second image based on the first and second x, y, and z coordinate sets of the first and second reference pixels and the first and second PTZ parameter sets.

### Claim 10 (depends on 1)

The system of claim 1, wherein the first reference point and the second reference point share a common x or y coordinate, but do not share a common z coordinate.