# US 10,319,098 B2 · System for real-time moving target detection using vision based image segmentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,319,098 B2
- Document ID: US10319098B2
- Status: Granted
- Kind: B2
- Application number: 16/119,088
- Filed: 2018-08-31
- Published: 2019-04-18
- Granted: 2019-06-11
- Earliest claimed: 2014-10-31
- Assignees: Raytheon Company
- Applicants: Raytheon Company
- Inventors: Stephen R. Nash; Christopher A. Leddy; Hector A. Quevedo
- Web page: https://patentgrep.com/patent/US10319098B2

## CPC

- G06T 7/215: Motion-based segmentation
- G06T 7/11: Region-based segmentation
- G06T 7/74: {involving reference images or patches}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 2207/20128: Atlas-based segmentation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system for processing video signal information to identify those pixels associated with a moving object in the presence of platform and/or sensor pointing induced motion. Frame differencing with self-adjusting noise thresholds is implemented to detect pixels associated with objects that are in motion with respect to the background and a field-by-field motion pixel map of pixels associated with the moving object is generated. A two (2) step pixel grouping process is used where the first pass runs in real-time as the video signal is received and writes the links between pixel groups into entries in a table. The second pass operates on a smaller set of link data and only needs to reorder entries in the table.

## Claims

### Claim 1

A method of detecting a moving object in an image signal, the method comprising:
providing, with a digital video camera, an image signal including a plurality of sequential image frames of a plurality of pixels (P);
providing, by a first processor coupled to the digital video camera, a motion map including, for each pixel (P) in the image frame, a corresponding motion map pixel (MP) set to either a first motion value (FMV) or a second motion value (SMV) indicating that the pixel (P) is associated or is not associated, respectively, with an object in the image frame that is in motion with respect to a background; and
by a second processor coupled to the first processor, for each image frame:
assigning a group count value (GCV) to each motion map pixel (MP) as a function of the motion values of adjacent motion map pixels;
maintaining an indexed grouping table comprising outer pixel coordinates for a boundary about each group of pixels having a same group count value (GCV) and information representing two groups of pixels having different group count values adjacent to one another;
determining if two different groups of pixels having different group count values are adjacent to one another by comparing the group count value (GCV) for each respective motion map pixel MP to the GCVs of two adjacent motion map pixels and, if so determined, updating the outer pixel coordinates of the group count value for the adjacent motion map pixel in the indexed grouping table;
maintaining a multiple link table including information identifying a motion map pixel having a group count value GCV adjacent to two other motion map pixels each having a different group count value GCV;
traversing the indexed grouping table and the multiple link table and determining boundary pixel coordinates for a boundary that encompasses two or more adjacent groups of pixels having different group count values, the boundary pixel coordinates identifying the moving object in the image signal; and
forwarding the boundary pixel coordinates to an object tracking system.

### Claim 2 (depends on 1)

The method of claim 1, wherein assigning a group count value GCV to each motion map pixel comprises:
comparing each motion map pixel MP(Y, X) to a first adjacent motion map pixel MP(Y, X −1) in the adjacent column (X−1) and a same row (Y) and to the motion map pixel MP(Y−1, X) in the adjacent row (Y−1) and same column (X),
wherein Y indicates the row starting from a top of the image frame, and
wherein X indicates the column starting from a left side of the image frame.

### Claim 3 (depends on 2)

The method of claim 2, wherein assigning each motion map pixel MP(Y, X) a group count value further comprises, for each motion map pixel MP(Y, X):
if motion map pixel MP(Y, X)=the first motion value FMV and neither of a first adjacent motion map pixel MP(Y−1, X) and a second adjacent motion map pixel MP(Y, X−1) equals the FMV then setting an indexed group count table entry GCV(Y, X)=a current group count value (CGCV) and then incrementing the current count value CGCV (CGCV=CGCV+1).

### Claim 4 (depends on 3)

The method of claim 3, wherein assigning each motion map pixel MP(Y, X) a group count value further comprises:
if motion map pixel MP(Y, X)=FMV and the first adjacent motion map pixel MP(Y−1, X)=FMV then setting the indexed group count table entry GCV(Y, X)=GCV(Y, X−1).

### Claim 5 (depends on 4)

The method of claim 4, wherein assigning each motion map pixel MP(Y, X) a group count value further comprises:
if motion map pixel MP(Y, X)=FMV and the second adjacent motion map pixel MP(Y, X−1)=FMV then setting the indexed group count table entry GCV(Y, X)=GCV(Y−1, X).

### Claim 6 (depends on 2)

The method of claim 2, further comprising:
receiving the respective motion values for the pixels in a row-by-row manner.

### Claim 7 (depends on 3)

The method of claim 3, further comprising:
storing a subset of all rows in the respective image frame in a memory.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
storing two adjacent rows of the respective image frame at a time.

### Claim 9 (depends on 1)

The method of claim 1, wherein providing the motion map pixels further comprises:
providing the corresponding motion map pixels in both a spatial and temporal order starting from a top left location to a bottom right location of the image frame.

### Claim 10 (depends on 1)

The method of claim 1, wherein determining if two different groups of pixels having different group count values are adjacent to one another further comprises, for a respective motion map pixel MP(Y, X):
comparing GCV(Y, X) to GCV(Y, X−1) and to GCV(Y−1, X),
wherein Y indicates the row starting from a top of the image frame, and
wherein X indicates the column starting from a left side of the image frame.

### Claim 11

A system for detecting a moving object in an image signal , the system comprising:
a digital video camera configured to provide an image signal including a plurality of sequential image frames of a plurality of pixels (P);
a first processor, coupled to the digital video camera, configured to provide a motion map including, for each pixel (P) in the image frame, a corresponding motion map pixel (MP) having either a first motion value (FMV) or a second motion value (SMV) indicating that the pixel (P) is associated or is not associated, respectively, with an object in the image frame that is in motion with respect to a background; and
a second processor, coupled to the first processor, configured to, for each image frame:
assign a group count value (GCV) to each motion map pixel (MP) as a function of the motion values of adjacent motion map pixels;
maintain an indexed grouping table comprising outer pixel coordinates for a boundary about each group of pixels having a same group count value (GCV) and information representing two groups of pixels having different group count values adjacent to one another;
determine if two different groups of pixels having different group count values are adjacent to one another by comparing the group count value (GCV) for each respective motion map pixel MP to the GCVs of two adjacent motion map pixels and, if so determined, update the outer pixel coordinates of the group count value for the adjacent motion map pixel in the indexed grouping table;
maintain a multiple link table including information identifying a motion map pixel having a group count value GCV adjacent to two other motion map pixels each having a different group count value GCV;
traverse the indexed grouping table and the multiple link table and determine boundary pixel coordinates for a boundary that encompasses two or more adjacent groups of pixels having different group count values, the boundary pixel coordinates identifying the moving object in the image signal; and
forward the boundary pixel coordinates to an object tracking system.

### Claim 12 (depends on 11)

The system of claim 11, wherein assigning a group count value GCV to each motion map pixel comprises:
comparing each motion map pixel MP(Y, X) to a first adjacent motion map pixel MP(Y, X−1) in the adjacent column (X−1) and a same row (Y) and to the motion map pixel MP(Y−1, X) in the adjacent row (Y−1) and same column (X),
wherein Y indicates the row starting from a top of the image frame, and
wherein X indicates the column starting from a left side of the image frame.

### Claim 13 (depends on 12)

The system of claim 12, wherein the second processor is further configured to assign each motion map pixel MP(Y, X) a group count value by, for each motion map pixel MP(Y, X):
both setting an indexed grouping table entry GCV(Y, X)=a current group count value (CGCV) and then incrementing the current count value CGCV (CGCV=CGCV+1) if motion map pixel MP(Y, X)=the first motion value FMV and neither of a first adjacent motion map pixel MP(Y−1, X) and a second adjacent motion map pixel MP(Y, X−1) equals the FMV then.

### Claim 14 (depends on 13)

The system of claim 13, wherein the second processor is further configured to assign each motion map pixel MP(Y, X) a group count value by:
setting the indexed grouping table entry GCV(Y, X)=GCV(Y, X−1) if motion map pixel MP(Y, X)=FMV and the first adjacent motion map pixel MP(Y−1, X)=FMV.

### Claim 15 (depends on 14)

The system of claim 14, wherein the second processor is further configured to assign each motion map pixel MP(Y, X) a group count value by:
setting the indexed grouping table entry GCV(Y, X)=GCV(Y−1, X) if motion map pixel MP(Y, X)=FMV and the second adjacent motion map pixel MP(Y, X−1)=FMV.

### Claim 16 (depends on 12)

The system of claim 12, wherein the first processor is further configured to:
provide the respective motion values for the pixels in a row-by-row manner.

### Claim 17 (depends on 13)

The system of claim 13, wherein the second processor is further configured to:
store a subset of all rows in the respective image frame in a memory.

### Claim 18 (depends on 17)

The system of claim 17, wherein the second processor is further configured to:
store two adjacent rows of the respective image frame at a time.

### Claim 19 (depends on 11)

The system of claim 11, wherein the first processor is further configured to:
provide the corresponding motion map pixels in both a spatial and temporal order starting from a top left location to a bottom right location of the image frame.

### Claim 20 (depends on 11)

The system of claim 11, wherein the second processor is further configured to determine if two different groups of pixels having different group count values are adjacent to one another by, for a respective motion map pixel MP(Y, X):
comparing GCV(Y, X) to GCV(Y, X−1) and to GCV(Y−1, X),
wherein Y indicates the row starting from a top of the image frame, and
wherein X indicates the column starting from a left side of the image frame.