# US 2024/0005750 A1 · Event-triggered capture of item image data and generation and storage of enhanced item identification data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005750 A1
- Document ID: US20240005750A1
- Status: Published application
- Kind: A1
- Application number: 18/217,459
- Filed: 2023-06-30
- Published: 2024-01-04
- Earliest claimed: 2022-07-01
- Assignees: NCR Corporation
- Applicants: NCR Corporation
- Inventors: Eric Xavier Schoch; Marc Haberkorn; Brent Vance Zucker; Jay Arcement; Frank Hinek; Yogesh Kamat; Amit Acharya; Matthew Farrow
- Web page: https://patentgrep.com/patent/US20240005750A1

## CPC

- G06Q 20/18: involving self-service terminals [SST], vending machines, kiosks or multimedia terminals
- G07G 1/0036: {Checkout procedures}
- G06Q 20/208: {Input by product or record sensing, e.g. weighing or scanner processing}
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/44: {Event detection}
- G06V 20/50: Context or environment of the image
- G07G 1/0063: {with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration}
- H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G07G: REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems, machines, methods, and computer-readable media are disclosed for detecting an event at a self-checkout (SCO) machine, determining that the event is an item-identifying event, triggering capture of image data of the item identified by the item-identifying event responsive to detecting the event and/or responsive to determining that the event is an item-identifying event, and generating enhanced item identification data that associates the captured image data with identifying information of the item. The enhanced item identification data may be stored in one or more datastores, which may include a local datastore in a retail environment in which a self-checkout machine is located or a remote datastore such as cloud-based data storage.

## Claims

### Claim 1

A method of event-triggered capture of an image of an item, the method comprising:
detecting an event at a self-checkout (SCO) machine;
determining that the event is an item-identifying event;
triggering capture of image data of the item identified by the item-identifying event; and
generating enhanced item identification data that associates the captured image data with identifying information of the item.

### Claim 2 (depends on 1)

The method of claim 1, wherein the image data is captured responsive to detecting the event.

### Claim 3 (depends on 1)

The method of claim 1, wherein the image data is captured responsive to determining that the event is an item-identifying event

### Claim 4 (depends on 1)

The method of claim 1, wherein the enhanced item identification data is stored in a local datastore in a retail environment in which the SCO machine is located.

### Claim 5 (depends on 1)

The method of claim 1, wherein the enhanced item identification data is stored in a remote datastore provided in a cloud-based environment.

### Claim 6 (depends on 1)

The method of claim 1, wherein the image data is captured by a camera embedded in a scanner of the SCO machine.

### Claim 7 (depends on 1)

The method of claim 1, further comprising annotating the enhanced item identification data.

### Claim 8

A method of item identification in connection with a self-checkout transaction, the method comprising:
determining whether an item can be identified using a first learning methodology; and
identifying the item using a second alternative learning methodology responsive to determining that the item cannot be identified using the first learning methodology.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first learning methodology is a computer vision machine learning model (CVMLM).

### Claim 10 (depends on 9)

The method of claim 9, wherein determining whether the item can be identified using the first learning methodology comprises:
providing image data as input to the CVMLM;
receiving item identification data as output from the CVMLM; and
determining whether the item can be identified based on the item identification data.

### Claim 11 (depends on 10)

The method of claim 10, wherein the item identification data comprises a candidate item identifier of the item and a confidence value associated with the candidate item identifier.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining whether the item can be identified based on the item identification data comprises:
determining that the confidence value fails to satisfy a first threshold value; and
determining, based on predetermined transaction logic, that the candidate item identifier cannot be accepted as a true identifier of the item for the self-checkout transaction due to the confidence value failing to satisfy the first threshold value.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
determining that additional learning input is needed to determine the true identifier for the item;
obtaining the additional learning input;
determining the true identifier for the item based on the additional learning input; and
generating enhanced item identification data based on the additional learning input.

### Claim 14 (depends on 13)

The method of claim 13, wherein the enhanced item identification data associates the true identifier with at least a portion of the image data that includes the item.

### Claim 15 (depends on 14)

The method of claim 14, wherein the enhanced item identification data associates the true identifier with coordinates of a bounding box around the item in the image data.

### Claim 16 (depends on 13)

The method of claim 13, further comprising:
re-training the CVMLM based on the enhanced item identification data to improve an item identification accuracy of the CVMLM.

### Claim 17 (depends on 12)

The method of claim 12, further comprising:
determining that the confidence value satisfies a second threshold confidence level less than the first threshold confidence level;
identifying a set of candidate items;
presenting a representation of the set of candidate items on a display of a self-checkout (SCO) machine; and
receiving a selection of a particular candidate item or other input indicative of an alternative item.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
accessing item identifier data to determine an item identifier corresponding to the selected particular candidate item or corresponding to the alternative item; and
generating enhanced item identification data that associates the item identifier with at least a portion of the image data that includes the item.

### Claim 19 (depends on 12)

The method of claim 12, further comprising:
determining that the confidence value fails to satisfy a second threshold confidence level less than the first threshold confidence level;
prompting a customer for scanner input;
receiving the scanner input at a self-checkout (SCO) machine, the scanner input obtained from a scan of machine-readable indicia associated with the item;
accessing item identifier data to determine an item identifier corresponding to the scanner input; and
generating enhanced item identification data that associates the item identifier with at least a portion of the image data that includes the item.

### Claim 20

A self-checkout (SCO) apparatus, comprising:
at least one memory storing computer-executable instructions; and
at least one processor configured to access the at least one memory and execute the computer-executable instructions to perform operations comprising:
determining whether an item can be identified using a first learning methodology; and
identifying the item using a second alternative learning methodology responsive to determining that the item cannot be identified using the first learning methodology.
