# US 2023/0410325 A1 · Ground plane filtering of video events

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410325 A1
- Document ID: US20230410325A1
- Status: Published application
- Kind: A1
- Application number: 18/242,147
- Filed: 2023-09-05
- Published: 2023-12-21
- Earliest claimed: 2020-03-19
- Assignees: ObjectVideo Labs, LLC
- Applicants: ObjectVideo Labs, LLC
- Inventors: Narayanan Ramanathan; Kyoung-Jin Park; Gang Qian; Allison Beach; Donald Gerard Madden
- Web page: https://patentgrep.com/patent/US20230410325A1

## CPC

- G06T 7/215: Motion-based segmentation
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30232: Surveillance
- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods, systems, and apparatus for ground plane filtering of video events are disclosed. A method includes obtaining a first set of images of a scene from a camera; determining a ground plane from the first set of images of the scene; obtaining a second set of images of the scene after the first set of images of the scene is obtained; determining that movement shown by a group of pixels in the second set of images of the scene satisfies motion criteria; determining that the ground plane corresponds with at least a portion of the group of pixels; and in response to determining that movement shown by the group of pixels in the second set of images of the scene satisfies motion criteria, and that the ground plane corresponds with at least a portion of the group of pixels, classifying the group of pixels as showing ground plane based motion.

## Claims

### Claim 1

(canceled)

### Claim 2

A computer-implemented method comprising:
identifying a plurality of moving objects depicted in a set of images captured by a camera;
identifying, from the plurality of moving objects:
a first subset of moving objects that overlap at least in part with a depiction of a surface in the set of images; and
a second subset of moving objects that do not overlap with the depiction of the surface in the set of images;
tracking, in subsequent images captured by the camera, motion of the first subset of moving objects; and
determining not to track, in the subsequent images captured by the camera, motion of second subset of moving objects.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein each moving object of the plurality of moving objects is represented by a pixel group.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein identifying the plurality of moving objects comprises:
identifying a plurality of pixel groups each having an average motion flow that satisfies motion criteria, wherein each pixel group represents a respective moving object of the plurality of moving objects.

### Claim 5 (depends on 2)

The computer-implemented method of claim 2, wherein identifying the plurality of moving objects comprises:
identifying a plurality of pixel groups each having a uniformity of motion direction that satisfies motion criteria, wherein each pixel group represents a respective moving object of the plurality of moving objects.

### Claim 6 (depends on 2)

The computer-implemented method of claim 2, wherein identifying the plurality of moving objects comprises:
determining, for each pixel in the set of images, a motion flow for the pixel;
classifying, using the motion flow, each pixel as a motion pixel or a non-motion pixel; and
identifying a cluster of motion pixels in the set of images based on (i) a similarity of flow orientation of the motion pixels to each other and (ii) a proximity of the motion pixels to each other, wherein each moving object of the plurality of moving objects is depicted by a cluster of motion pixels.

### Claim 7 (depends on 2)

The computer-implemented method of claim 2, wherein identifying the first subset of moving objects comprises:
determining, for each moving object of the plurality of moving objects, a percentage of the depiction of the moving object that is represented by pixels in a ground plane of the set of images; and
selecting, for inclusion in the first subset of moving objects, moving objects for which the percentage of the depiction of the moving object that is represented by pixels in the ground plane of the set of images satisfies a threshold percentage.

### Claim 8 (depends on 2)

The computer-implemented method of claim 2, wherein identifying the first subset of moving objects comprises:
determining, for each moving object of the plurality of moving objects, a number of pixels of the depiction of the moving object that are in a ground plane of the set of images; and
selecting, for inclusion in the first subset of moving objects, moving objects for which the number of pixels of the depiction of the moving object that are in the ground plane of the set of images satisfies a threshold number of pixels.

### Claim 9 (depends on 2)

The computer-implemented method of claim 2, comprising:
in response to tracking the motion of the first subset of moving objects, determining that at least one ground plane motion event has occurred; and
performing one or more actions based on the at least one ground plane motion event.

### Claim 10 (depends on 2)

The computer-implemented method of claim 2, comprising:
obtaining a ground plane image including:
a first set of pixels corresponding to a ground plane region of a scene captured by the camera, each pixel of the first set of pixels having a first pixel value; and
a second set of pixels corresponding to a non-ground plane region of the scene captured by the camera, each pixel of the second set of pixels having a second pixel value,
wherein each moving object of the first subset of moving objects includes at least one pixel that has a pixel location corresponding to a pixel in the ground plane image that has the first pixel value.

### Claim 11 (depends on 2)

The computer-implemented method of claim 2, wherein tracking the motion of the first subset of moving objects comprises:
generating a bounding box around each moving object of the first subset of moving objects; and
tracking movements of the bounding boxes in the subsequent images captured by the camera.

### Claim 12 (depends on 2)

The computer-implemented method of claim 2, wherein the surface depicted in the set of images comprises a walkable surface.

### Claim 13 (depends on 2)

The computer-implemented method of claim 2, wherein the surface depicted in the set of images is defined in part by a horizon.

### Claim 14 (depends on 2)

The computer-implemented method of claim 2, wherein the surface depicted in the set of images comprises a surface on which one or more objects appear to be positioned.

### Claim 15

One or more non-transitory computer-readable media storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
identifying a plurality of moving objects depicted in a set of images captured by a camera;
identifying, from the plurality of moving objects:
a first subset of moving objects that overlap at least in part with a depiction of a surface in the set of images; and
a second subset of moving objects that do not overlap with the depiction of the surface in the set of images;
tracking, in subsequent images captured by the camera, motion of the first subset of moving objects; and
determining not to track, in the subsequent images captured by the camera, motion of second subset of moving objects.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein each moving object of the plurality of moving objects is represented by a pixel group.

### Claim 17 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein identifying the plurality of moving objects comprises:
identifying a plurality of pixel groups each having an average motion flow that satisfies motion criteria, wherein each pixel group represents a respective moving object of the plurality of moving objects.

### Claim 18 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein identifying the plurality of moving objects comprises:
identifying a plurality of pixel groups each having a uniformity of motion direction that satisfies motion criteria, wherein each pixel group represents a respective moving object of the plurality of moving objects.

### Claim 19 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein identifying the plurality of moving objects comprises:
determining, for each pixel in the set of images, a motion flow for the pixel;
classifying, using the motion flow, each pixel as a motion pixel or a non-motion pixel; and
identifying a cluster of motion pixels in the set of images based on (i) a similarity of flow orientation of the motion pixels to each other and (ii) a proximity of the motion pixels to each other, wherein each moving object of the plurality of moving objects is depicted by a cluster of motion pixels.

### Claim 20 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein identifying the first subset of moving objects comprises:
determining, for each moving object of the plurality of moving objects, a percentage of the depiction of the moving object that is represented by pixels in a ground plane of the set of images; and
selecting, for inclusion in the first subset of moving objects, moving objects for which the percentage of the depiction of the moving object that is represented by pixels in the ground plane of the set of images satisfies a threshold percentage.

### Claim 21

A system comprising:
one or more computers and one or more computer storage media storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
identifying a plurality of moving objects depicted in a set of images captured by a camera;
identifying, from the plurality of moving objects:
a first subset of moving objects that overlap at least in part with a depiction of a surface in the set of images; and
a second subset of moving objects that do not overlap with the depiction of the surface in the set of images;
tracking, in subsequent images captured by the camera, motion of the first subset of moving objects; and
determining not to track, in the subsequent images captured by the camera, motion of second subset of moving objects.
