# US 12,183,014 B2 · System and method for object tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,014 B2
- Document ID: US12183014B2
- Status: Granted
- Kind: B2
- Application number: 18/515,838
- Filed: 2023-11-21
- Published: 2024-03-14
- Granted: 2024-12-31
- Earliest claimed: 2019-10-25
- Assignees: 7-Eleven, Inc.
- Applicants: 7-Eleven, Inc.
- Inventors: Shahmeer Ali Mirza; Sailesh Bharathwaaj Krishnamurthy; Sarath Vakacharla; Deepanjan Paul
- Web page: https://patentgrep.com/patent/US12183014B2

## CPC

- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G01G 19/414: using electronic computing means only
- G01G 19/4144: {for controlling weight of goods in commercial establishments, e.g. supermarket, P.O.S. systems}
- G01G 19/52: Weighing apparatus combined with other objects, e.g. furniture (with walking sticks A45B3/08)
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 20/00: Machine learning
- G06Q 30/0633: {Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087)}
- G06T 5/30: Erosion or dilatation, e.g. thinning
- G06T 7/13: Edge detection
- G06T 7/181: involving edge growing; involving edge linking
- G06T 7/50: Depth or shape recovery
- G06T 7/73: using feature-based methods
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- G06N 3/08: Learning methods
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20161: Level set
- G06T 2207/30196: Human being; Person
- G06T 2207/30232: Surveillance
- G06T 2207/30241: Trajectory
- G06T 2207/30242: Counting objects in image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01G: WEIGHING (sorting by weighing B07C5/16)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system includes a first sensor configured to generate images of at least a first portion of a space. A processor of the system is configured to determine a position of a possible object in the space based on generated images.

## Claims

### Claim 1

A system, comprising:
a first sensor configured to generate images of at least a first portion of a space; and
a processor configured to:
determine, based at least in part on one or more images generated during an initial time interval, first pixel coordinates corresponding to regions of the images generated by the first sensor to exclude during object tracking; and
determine, based at least in part on one or more images generated during a subsequent time interval after the initial time interval, second pixel coordinates corresponding to a first possible object in the space;
determine whether at least a threshold amount of the second pixel coordinates overlap with the first pixel coordinates; and
in response to determining that at least the threshold amount of the second pixel coordinates do not overlap with the first pixel coordinates, determine a first position of the first possible object.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
a second sensor configured to generate images of at least a second portion of the space;
wherein the processor is further configured to:
determine, based at least in part on one or more images generated by the second sensor during the initial time interval, third pixel coordinates corresponding to regions of the images generated by the second sensor to exclude during object tracking;
determine, based at least in part on one or more images generated during the subsequent time interval after the initial time interval, fourth pixel coordinates corresponding to a second possible object in the space;
determine whether at least a threshold amount of the fourth pixel coordinates overlap with the third pixel coordinates; and
in response to determining that at least the threshold amount of the second pixel coordinates do not overlap with the first pixel coordinates, determine the second position of the second possible object.

### Claim 3 (depends on 2)

The system of claim 2, wherein the first possible object and the second possible object correspond to the same object in the space.

### Claim 4 (depends on 1)

The system of claim 1, wherein the processor is further configured to determine that at least the threshold amount of the second pixel coordinates overlap with the first pixel coordinates by determining that at least the threshold amount of the second pixel coordinates are the same as the first pixel coordinates.

### Claim 5 (depends on 1)

The system of claim 1, wherein the processor is further configured to determine that at least the threshold amount of the second pixel coordinates overlap with the first pixel coordinates by:
determining a first continuous area associated with the first pixel coordinates;
determining a second continuous area associated with the second pixel coordinates;
calculating a percentage of the second continuous area that is within the first continuous area; and
determining that the calculated percentage is greater than or equal to the threshold amount.

### Claim 6 (depends on 1)

The system of claim 1, wherein the threshold amount is 10%.

### Claim 7 (depends on 1)

The system of claim 1, further comprising:
wherein the processor is further configured to:
determine, based on the first position for the first possible object, a corresponding physical position in the space using a homography, the homography associating pixel coordinates in the images generated by the first sensor to physical coordinates in the space; and
track the physical position during the subsequent interval of time.

### Claim 8 (depends on 7)

The system of claim 7, the processor further configured to
generate a set of physical coordinates to exclude during tracking in the space, the set of physical coordinates corresponding to one or more physical regions in the space to exclude during object tracking.

### Claim 9 (depends on 1)

The system of claim 1, wherein during the initial time interval no object is in the space.

### Claim 10

A method, comprising:
determining, based at least in part on one or more images of a first portion of a space generated by a first sensor during an initial time interval, first pixel coordinates corresponding to regions of the images generated by the first sensor to exclude during object tracking;
determining, based at least in part on one or more images generated by the first sensor during a subsequent time interval after the initial time interval, second pixel coordinates corresponding to a first possible object in the space;
determining whether at least a threshold amount of the second pixel coordinates overlap with the first pixel coordinates; and
in response to determining that at least the threshold amount of the second pixel coordinates do not overlap with the first pixel coordinates, determining a first position of the first possible object.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
determining, based at least in part on one or more images generated by a second sensor during the initial time interval, third pixel coordinates corresponding to regions of the images generated by the second sensor to exclude during object tracking;
determining, based at least in part on one or more images generated during the subsequent time interval after the initial time interval, fourth pixel coordinates corresponding to a second possible object in the space;
determining whether at least a threshold amount of the fourth pixel coordinates overlap with the third pixel coordinates; and
in response to determining that at least the threshold amount of the second pixel coordinates do not overlap with the first pixel coordinates, determining the second position of the second possible object.

### Claim 12 (depends on 11)

The method of claim 11, wherein the first possible object and the second possible object correspond to the same object in the space.

### Claim 13 (depends on 10)

The method of claim 10, further comprising determining that at least the threshold amount of the second pixel coordinates overlap with the first pixel coordinates by determining that at least the threshold amount of the second pixel coordinates are the same as the first pixel coordinates.

### Claim 14 (depends on 10)

The method of claim 10, further comprising determining that at least the threshold amount of the second pixel coordinates overlap with the first pixel coordinates by:
determining a first continuous area associated with the first pixel coordinates;
determining a second continuous area associated with the second pixel coordinates;
calculating a percentage of the second continuous area that is within the first continuous area; and
determining that the calculated percentage is greater than or equal to the threshold amount.

### Claim 15 (depends on 10)

The method of claim 10, wherein the threshold amount is 10%.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
determining, based on the first position for the first possible object, a corresponding physical position in the space using a homography, the homography associating pixel coordinates in the images generated by the first sensor to physical coordinates in the space; and
tracking the physical position during the subsequent interval of time.

### Claim 17 (depends on 16)

The method of claim 16, further comprising determining a set of physical coordinates to exclude during tracking in the space, the set of physical coordinates corresponding to one or more physical regions in the space to exclude during object tracking.

### Claim 18 (depends on 10)

The method of claim 10, wherein during the initial time interval no object is in the space.

### Claim 19

A device communicatively coupled to a first sensor, the first sensor configured to generate images of at least a first portion of a space, the device configured to:
determine, based at least in part on one or more images generated by the first sensor during the initial time interval, first pixel coordinates corresponding to regions of the images generated by the first sensor to exclude during object tracking;
determine, based at least in part on one or more images generated during a subsequent time interval after the initial time interval, second pixel coordinates corresponding to a first possible object in the space;
determine whether at least a threshold amount of the second pixel coordinates overlap with the first pixel coordinates; and
in response to determining that at least the threshold amount of the second pixel coordinates do not overlap with the first pixel coordinates, determine a first position of the first possible object.

### Claim 20 (depends on 19)

The device of claim 19, the device further coupled to a second sensor, the second sensor configured to generate images of at least a second portion of the space;
wherein the device is further configured to:
determine, based at least in part on one or more images generated by the second sensor during the initial time interval, third pixel coordinates corresponding to regions of the images generated by the second sensor to exclude during object tracking;
determine, based at least in part on one or more images generated during the subsequent time interval after the initial time interval, fourth pixel coordinates corresponding to a second possible object in the space;
determine whether at least a threshold amount of the fourth pixel coordinates overlap with the third pixel coordinates; and
in response to determining that at least the threshold amount of the second pixel coordinates do not overlap with the first pixel coordinates, determine the second position of the second possible object.