# US 10,372,998 B2 · Object recognition for bottom of basket detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,998 B2
- Document ID: US10372998B2
- Status: Granted
- Kind: B2
- Application number: 15/671,618
- Filed: 2017-08-08
- Published: 2018-02-08
- Granted: 2019-08-06
- Earliest claimed: 2016-08-08
- Applicants: Indaflow LLC
- Inventors: Jacob D. Richards; Joel Thiessen
- Web page: https://patentgrep.com/patent/US10372998B2

## CPC

- G06K 19/07: with integrated circuit chips
- G06K 7/10722: {Photodetector array or CCD scanning}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/90: Determination of colour characteristics
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06Q 30/0633: {Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/30232: Surveillance
- G06V 2201/07: Target detection
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A checkout lane management system is described that uses object recognition to detect whether one or more items is under a shopping cart during a customer checkout process. The checkout lane management system may comprise one or more of cameras for collecting a stream of images focused on a checkout lane. The checkout lane management system may also comprise one or more of a display device and a speaker device as part of a checkout lane device for displaying a BOB indicator or sounding the BOB indicator.

## Claims

### Claim 1

A method of object recognition for a cart in a checkout lane at a retail store location, the method comprising:
collecting a stream of images from a checkout lane;
evaluating the stream of images to identify a cart tracking module;
selecting a detection image when identifying the cart tracking module;
analyzing recognition data from the detection image to determine a lower tray status for the checkout lane, where the recognition data includes color data and coordinate data and analyzing recognition data includes calculating a set of delta color values between a plurality of pixels from sampling the recognition data and a plurality of pixels from baseline data,
transforming the set of delta color values to a delta checksum,
comparing the delta checksum against a delta threshold, and
setting the lower tray status when the delta checksum is equal to or greater than the delta threshold;
translating the lower tray status to a lower tray indicator; and
communicating the lower tray indicator to a checkout lane device.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
displaying the lower tray indicator on a display device of the checkout lane device.

### Claim 3 (depends on 2)

The method of claim 2 further comprising:
presenting a lower tray accounting input on the display device of the checkout lane device that requires a response prior to permitting any further checkout activity.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
sounding the lower tray indicator on a speaker device of the checkout lane device.

### Claim 5

An object recognition system for a cart in a checkout lane at a retail store location, the system comprising:
at least one camera that collects a stream of images from a checkout lane and captures a detection image when a cart tracking module is identified;
a computing device that receives the stream of images from the checkout lane, the computing device including
an evaluation component that evaluates the stream of images to identify a cart tracking module and selects the detection image,
a collection component that collects recognition data from the detection image, where the recognition data includes color data and coordinate data,
an analyzing component that analyzes the recognition data to determine a lower tray status for the checkout lane, where analysis of recognition data includes
calculation of set of delta color values between a plurality of pixels from sampling the recognition data and a plurality of pixels from baseline data,
transformation of the set of delta color values to a delta checksum,
comparison of the delta checksum against a delta threshold, and
setting the lower tray status when the delta checksum is equal to or greater than the delta threshold,
a translation component that translates the lower tray status to a lower tray indicator, and
a communication component that communicates the lower tray indicator to a checkout lane device.

### Claim 6 (depends on 5)

The system of claim 5, wherein the checkout lane device comprises a display device to display the lower tray indicator.

### Claim 7 (depends on 6)

The system of claim 6, wherein the checkout lane device displays a lower tray accounting input on the display device of the checkout lane device that requires a response prior to permitting any further checkout activity.

### Claim 8 (depends on 5)

The system of claim 5, wherein the checkout lane device comprises a speaker device to sound the lower tray indicator.

### Claim 9

A computer program product for object recognition for a cart in a checkout lane at a retail store location, the computer program product stored on a non-transitory computer-readable medium and including instructions adapted to cause a computer to execute steps comprising:
collecting a stream of images from a checkout lane;
evaluating the stream of images to identify a cart tracking module; selecting a detection image when identifying the cart tracking module;
analyzing recognition data to determine a lower tray status for the checkout lane, where the recognition data includes color data and coordinate data and analyzing recognition data includes
calculating a set of delta color values between a plurality of pixels from sampling the recognition data and a plurality of pixels from baseline data,
transforming the set of delta color values to a delta checksum,
comparing the delta checksum against a delta threshold, and
setting the lower tray status when the delta checksum is equal to or greater than the delta threshold;
translating the lower tray status to a lower tray indicator; and
communicating the lower tray indicator to a checkout lane device.

### Claim 10 (depends on 9)

The computer program product of claim 9 further comprising:
displaying the lower tray indicator on a display device of the checkout lane device.

### Claim 11 (depends on 10)

The computer program product of claim 10 further comprising:
presenting a lower tray accounting input on the display device of the checkout lane device that requires a response prior to permitting any further checkout activity.