# US 11,308,630 B2 · Auto-exclusion zone for contour-based object detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,630 B2
- Document ID: US11308630B2
- Status: Granted
- Kind: B2
- Application number: 16/664,426
- Filed: 2019-10-25
- Published: 2021-04-29
- Granted: 2022-04-19
- Earliest claimed: 2019-10-25
- Assignees: 7-Eleven, Inc.
- Applicants: 7-Eleven, Inc.
- Inventors: Sailesh Bharathwaaj Krishnamurthy; Shahmeer Ali Mirza
- Web page: https://patentgrep.com/patent/US11308630B2

## CPC

- G06T 7/248: {involving reference images or patches}
- G06Q 30/0601: Electronic shopping [e-shopping]
- G06T 7/13: Edge detection
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/254: involving subtraction of images
- G06T 7/292: Multi-camera tracking
- G06T 7/74: {involving reference images or patches}
- G06T 2207/30196: Human being; Person
- G06T 2207/30232: Surveillance
- G06T 2207/30241: Trajectory
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A system includes a first sensor and a sensor client. During an initial time interval, the sensor client receives top-view images generated by the first sensor and detects contours in the images. The sensor client determines, based on the contours, regions of the top-view images generated by the first sensor to exclude during object tracking. During a subsequent time interval, the sensor client receives a second top-view image generated by the first sensor and detects a contour in the image. The sensor client determines pixel coordinates of the second contour and determines whether at least a threshold percentage of the second pixel coordinates overlap with the region to exclude during object tracking. If at least the threshold percentage of the second pixel coordinates overlap with the region to exclude, a position for tracking the second contour is not determined.

## Claims

### Claim 1

A system, comprising:
a first sensor configured to generate top-view images of at least a first portion of a space; and
a sensor client communicatively coupled to the first sensor, the sensor client configured to:
during an initial time interval:
receive one or more first top-view images generated by the first sensor;
detect one or more first contours in the one or more first top-view images;
determine first pixel coordinates corresponding to the detected first contours, wherein the first pixel coordinates correspond to regions of the top-view images generated by the first sensor to exclude during object tracking; and
during a subsequent time interval after the initial time interval:
receive a second top-view image generated by the first sensor;
detect a second contour in the first top-view image;
determine second pixel coordinates corresponding to the detected second contour;
determine whether at least a threshold percentage of the second pixel coordinates overlap with the first pixel coordinates;
in response to determining that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates, do not determine a first pixel position for tracking the second contour; and
in response to determining that at least the threshold percentage of the second pixel coordinates do not overlap with the first pixel coordinates, determine the first pixel position for tracking the second contour.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
a second sensor communicatively coupled to the sensor client and configured to generate top-view images of at least a second portion of the space;
wherein the sensor client is further configured to:
during the initial time interval:
receive one or more third top-view images generated by the second sensor;
detect one or more third contours in the one or more third top-view images;
determine third pixel coordinates corresponding to the detected third contours, wherein the third pixel coordinates correspond to regions of the top-view images generated by the second sensor to exclude during object tracking; and
during the subsequent time interval after the initial time interval:
receive a fourth top-view image generated by the second sensor;
detect a fourth contour in the fourth top-view image;
determine fourth pixel coordinates corresponding to the detected fourth contour;
determine whether at least a threshold percentage of the fourth pixel coordinates overlap with the third pixel coordinates;
in response to determining that at least the threshold percentage of the fourth pixel coordinates overlap with the third pixel coordinates, determine a second pixel position for tracking the fourth contour; and
in response to determining that at least the threshold percentage of the second pixel coordinates do not overlap with the first pixel coordinates, do not determine the second pixel position for tracking the fourth contour.

### Claim 3 (depends on 2)

The system of claim 2, wherein the fourth contour and the second contour correspond to the same object in the space.

### Claim 4 (depends on 1)

The system of claim 1, wherein the sensor client is further configured to determine that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates by determining that at least the threshold percentage of the second pixel coordinates are the same as the first pixel coordinates.

### Claim 5 (depends on 1)

The system of claim 1, wherein the sensor client is further configured to determine that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates by:
determining a first continuous area associated with the first pixel positions;
determining a second continuous area associated with the second contour;
calculating a percentage of the second continuous area that is within the first continuous area; and
determining that the calculated percentage is greater than or equal to the threshold percentage.

### Claim 6 (depends on 1)

The system of claim 1, wherein the threshold percentage is 10%.

### Claim 7 (depends on 1)

The system of claim 1, further comprising:
a server communicatively coupled to the sensor client;
wherein the sensor client is further configured to:
in response to determining that the threshold percentage of the second pixel coordinates are the same as the first pixel coordinates, transmit the first pixel position for tracking the second contour to the server; and
wherein the server is configured to:
receive the first pixel position from the sensor client;
determine, based on the first pixel position, a corresponding physical position in the space using a homography, the homography associating pixel coordinates in the top-view images generated by the first sensor to physical coordinates in the space; and
track the physical position during the subsequent interval of time.

### Claim 8 (depends on 7)

The system of claim 7, the server further configured to:
receive the first pixel coordinates from the sensor client; and
generate a set of physical coordinates to exclude during tracking in the space, the set of physical coordinates corresponding to one or more physical regions in the space to exclude during object tracking.

### Claim 9 (depends on 1)

The system of claim 1, wherein during the initial time interval no person is in the space.

### Claim 10

A method, comprising:
during an initial time interval:
receiving one or more first top-view images generated by a first sensor, the first sensor configured to generate top-view images of at least a first portion of a space;
detecting one or more first contours in the one or more first top-view images;
determining first pixel coordinates corresponding to the detected first contours, wherein the first pixel coordinates correspond to regions of the top-view images generated by the first sensor to exclude during object tracking; and during a subsequent time interval after the initial time interval:
receiving a second top-view image generated by the first sensor;
detecting a second contour in the first top-view image;
determining second pixel coordinates corresponding to the detected second contour;
determining whether at least a threshold percentage of the second pixel coordinates overlap with the first pixel coordinates;
in response to determining that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates, not determining a first pixel position for tracking the second contour; and
in response to determining that at least the threshold percentage of the second pixel coordinates do not overlap with the first pixel coordinates, determining the first pixel position for tracking the second contour.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
during the initial time interval:
receiving one or more third top-view images generated by a second sensor, the second sensor configured to generate top-view images of at least a second portion of the space;
detecting one or more third contours in the one or more third top-view images;
determining third pixel coordinates corresponding to the detected third contours, wherein the third pixel coordinates correspond to regions of the top-view images generated by the second sensor to exclude during object tracking; and
during the subsequent time interval after the initial time interval:
receiving a fourth top-view image generated by the second sensor;
detecting a fourth contour in the fourth top-view image;
determining fourth pixel coordinates corresponding to the detected fourth contour;
determining whether at least a threshold percentage of the fourth pixel coordinates overlap with the third pixel coordinates;
in response to determining that at least the threshold percentage of the fourth pixel coordinates overlap with the third pixel coordinates, determining a second pixel position for tracking the fourth contour; and
in response to determining that at least the threshold percentage of the second pixel coordinates do not overlap with the first pixel coordinates, not determining the second pixel position for tracking the fourth contour.

### Claim 12 (depends on 11)

The method of claim 11, wherein the fourth contour and the second contour correspond to the same object in the space.

### Claim 13 (depends on 10)

The method of claim 10, further comprising determining that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates by determining that at least the threshold percentage of the second pixel coordinates are the same as the first pixel coordinates.

### Claim 14 (depends on 10)

The method of claim 10, further comprising determining that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates by:
determining a first continuous area associated with the first pixel positions;
determining a second continuous area associated with the second contour;
calculating a percentage of the second continuous area that is within the first continuous area; and
determining that the calculated percentage is greater than or equal to the threshold percentage.

### Claim 15 (depends on 10)

The method of claim 10, wherein the threshold percentage is 10%.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
determining, based on the first pixel position, a corresponding physical position in the space using a homography, the homography associating pixel coordinates in the top-view images generated by the first sensor to physical coordinates in the space; and
tracking the physical position during the subsequent interval of time.

### Claim 17 (depends on 16)

The method of claim 16, further comprising determining a set of physical coordinates to exclude during tracking in the space, the set of physical coordinates corresponding to one or more physical regions in the space to exclude during object tracking.

### Claim 18 (depends on 10)

The method of claim 10, wherein during the initial time interval no person is in the space.

### Claim 19

A device communicatively coupled to a first sensor, the first sensor configured to generate top-view images of at least a first portion of a space, the device configured to:
during an initial time interval:
receive one or more first top-view images generated by the first sensor;
detect one or more first contours in the one or more first top-view images;
determine first pixel coordinates corresponding to the detected first contours, wherein the first pixel coordinates correspond to regions of the top-view images generated by the first sensor to exclude during object tracking; and during a subsequent time interval after the initial time interval:
receive a second top-view image generated by the first sensor;
detect a second contour in the first top-view image;
determine second pixel coordinates corresponding to the detected second contour;
determine whether at least a threshold percentage of the second pixel coordinates overlap with the first pixel coordinates;
in response to determining that at least the threshold percentage of the second pixel coordinates overlap with the first pixel coordinates, do not determine a first pixel position for tracking the second contour; and
in response to determining that at least the threshold percentage of the second pixel coordinates do not overlap with the first pixel coordinates, determine the first pixel position for tracking the second contour.

### Claim 20 (depends on 19)

The device of claim 19, the device further coupled to a second sensor, the second sensor configured to generate top-view images of at least a second portion of the space;
wherein the device is further configured to:
during the initial time interval:
receive one or more third top-view images generated by the second sensor;
detect one or more third contours in the one or more third top-view images;
determine third pixel coordinates corresponding to the detected third contours, wherein the third pixel coordinates correspond to regions of the top-view images generated by the second sensor to exclude during object tracking; and
during the subsequent time interval after the initial time interval:
receive a fourth top-view image generated by the second sensor;
detect a fourth contour in the fourth top-view image, wherein the fourth contour and the second contour correspond to the same object in the space;
determine fourth pixel coordinates corresponding to the detected fourth contour;
determine whether at least a threshold percentage of the fourth pixel coordinates overlap with the third pixel coordinates;
in response to determining that at least the threshold percentage of the fourth pixel coordinates overlap with the third pixel coordinates, determine a second pixel position for tracking the fourth contour; and
in response to determining that at least the threshold percentage of the second pixel coordinates do not overlap with the first pixel coordinates, do not determine the second pixel position for tracking the fourth contour.