# US 12,183,170 B2 · Self-checkout workflow management system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,170 B2
- Document ID: US12183170B2
- Status: Granted
- Kind: B2
- Application number: 18/164,510
- Filed: 2023-02-03
- Published: 2024-08-08
- Granted: 2024-12-31
- Earliest claimed: 2023-02-03
- Assignees: Toshiba Global Commerce Solutions, Inc.
- Applicants: Toshiba Global Commerce Solutions, Inc.
- Inventors: Judith L. Atallah; Charles Ray Kirk; Evgeny Shevtsov; Michelle M. Crompton
- Web page: https://patentgrep.com/patent/US12183170B2

## CPC

- G07G 1/0036: {Checkout procedures}
- G06N 20/00: Machine learning
- G06Q 20/18: involving self-service terminals [SST], vending machines, kiosks or multimedia terminals
- G06V 10/764: using classification, e.g. of video objects
- G07G 1/0018: {Constructional details, e.g. of drawer, printing means, input means}
- G07G 1/0045: {with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader}
- G07G 1/0063: {with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration}
- G07G 1/01: Details for indicating
- G07G: REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques to reconfigure self-checkout kiosks based on computer vision or interaction metrics. One or more graphical images are received. The one or more graphical images at least partially depict a physical area associated with a self-checkout kiosk during a transaction involving one or more items and a form of payment. The one or more graphical images include a first item. The self-checkout kiosk is configured to output one or more communications via one or more graphical user interface (GUI) screens associated with a workflow. An item type or a payment type is predicted for the transaction and using one or more classifiers. The predicted item type or payment type is provided as input to the workflow without requiring an intervention. The transaction is executed.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving one or more graphical images at least partially depicting a physical area associated with a self-checkout kiosk during a transaction associated with (i) one or more items and (ii) a form of payment at the self-checkout kiosk, the one or more graphical images comprising a first of the one or more items, the self-checkout kiosk configured to output one or more communications via one or more graphical user interface (GUI) screens associated with a workflow;
predicting, using one or more classifiers and based on the one or more graphical images, at least one of (i) an item type of the first item or (ii) a payment type of the form of payment, the one or more classifiers being executed by one or more computer processors;
providing, without requiring an intervention, at least one of the item type or the payment type as input to the workflow, which causes at least a given one of the one or more communications to be at least partially omitted from being output, wherein the given communication is associated with explicitly specifying at least one of the item type, the payment type, or payment readiness; and
executing the transaction at the self-checkout kiosk.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more graphical images depict the form of payment, and wherein the one or more classifiers includes at least one of:
a first classifier configured to predict the item type based on the first item in the one or more graphical images; or
a second classifier configured to predict the payment type based on the form of payment as depicted in the one or more graphical images.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the given communication includes at least one of a notification, a prompt, or a menu, and wherein the notification is selected from an instructional notification, an informational notification, an error notification, and a warning notification.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining that the first item has been placed on a scale associated with the self-checkout kiosk, whereupon the item type is predicted; and
determining that a window of time has elapsed in which no item has been added to the transaction at the self-checkout kiosk, whereupon payment readiness for the transaction is determined without requiring an intervention.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the classifier is trained via supervised learning and based on a set of training data, and wherein the classifier comprises a machine learning classification model.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining a measure of interaction for at least a portion of the workflow during the transaction at the self-checkout kiosk, the measure of interaction including at least one of a rate of interaction or a rate of repetition of steps of the workflow;
setting a communication level for the workflow for the transaction, wherein the communication level applies at least in part to at least a remainder of the workflow, the remainder being relative to the portion, wherein the communication level is determined based on the measure of interaction and without requiring an intervention;
determining, based on the communication level, a next communication to output, wherein the next communication is output via the self-checkout kiosk.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein at least one of the communication level or the next communication is determined based on a machine learning model.

### Claim 8 (depends on 6)

The computer-implemented method of claim 6, wherein the communication level is selected from a plurality of communication levels, the plurality of communication levels including (i) a first level, (ii) a second level of increased communication relative to the first level, and (iii) a third level of reduced communication relative to the first level;
wherein the second level causes, at least in part, more types of communication or a greater amount of communication to be output relative to the first level, and wherein the third level causes, at least in part, fewer types of communication or a lesser amount of communication to be output relative to the first level.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the one or more graphical images depict the form of payment, and wherein the one or more classifiers includes:
a first classifier configured to predict the item type based on the first item in the one or more graphical images; or
a second classifier configured to predict the payment type based on the form of payment as depicted in the one or more graphical images.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, further comprising:
determining that the first item has been placed on a scale associated with the self-checkout kiosk, whereupon the item type is predicted; and
determining that a window of time has elapsed in which no item has been added to the transaction at the self-checkout kiosk, whereupon the payment readiness for the transaction is determined without requiring an intervention.

### Claim 11

A system comprising:
one or more computer processors; and
a memory containing a program executable by the one or more computer processors to perform an operation comprising:
receiving one or more graphical images at least partially depicting a physical area associated with a self-checkout kiosk during a transaction associated with (i) one or more items and (ii) a form of payment at the self-checkout kiosk, the one or more graphical images comprising a first of the one or more items, the self-checkout kiosk configured to output one or more communications via one or more graphical user interface (GUI) screens associated with a workflow;
predicting, using one or more classifiers and based on the one or more graphical images, at least one of (i) an item type of the first item or (ii) a payment type of the form of payment;
providing, without requiring an intervention, at least one of the item type or the payment type as input to the workflow, which causes at least a given one of the one or more communications to be at least partially omitted from being output, wherein the given communication is associated with explicitly specifying at least one of the item type, the payment type, or payment readiness; and
executing the transaction at the self-checkout kiosk.

### Claim 12 (depends on 11)

The system of claim 11, wherein the one or more graphical images depict the form of payment, and wherein the one or more classifiers includes at least one of:
a first classifier configured to predict the item type based on the first item as depicted in the one or more graphical images; or
a second classifier configured to predict the payment type based on the form of payment as depicted in the one or more graphical images.

### Claim 13 (depends on 11)

The system of claim 11, the operation further comprising:
determining that the first item has been placed on a scale associated with the self-checkout kiosk, whereupon the item type is predicted; and
determining that a window of time has elapsed in which no item has been added to the transaction at the self-checkout kiosk, whereupon the payment readiness for the transaction is determined without requiring an intervention.

### Claim 14

A non-transitory computer-readable medium containing a program executable to perform an operation comprising:
receiving one or more graphical images at least partially depicting a physical area associated with a self-checkout kiosk during a transaction associated with (i) one or more items and (ii) a form of payment at the self-checkout kiosk, the one or more graphical images comprising a first of the one or more items, the self-checkout kiosk configured to output one or more communications via one or more graphical user interface (GUI) screens associated with a workflow;
predicting, using one or more classifiers and based on the one or more graphical images, at least one of (i) an item type of the first item or (ii) a payment type of the form of payment;
providing, without requiring an intervention, at least one of the item type or the payment type as input to the workflow, which causes at least a given one of the one or more communications to be at least partially omitted from being output, wherein the given communication is associated with explicitly specifying at least one of the item type, the payment type, or payment readiness; and
executing the transaction at the self-checkout kiosk.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the one or more classifiers includes at least one of:
a first classifier configured to predict the item type based on the first item as depicted in the one or more graphical images; or
a second classifier configured to predict the payment type based on the form of payment as depicted in the one or more graphical images.

### Claim 16 (depends on 14)

The non-transitory computer-readable medium of claim 14, the operation further comprising:
determining that the first item has been placed on a scale associated with the self-checkout kiosk, whereupon the item type is predicted; and
determining that a window of time has elapsed in which no item has been added to the transaction at the self-checkout kiosk, whereupon the payment readiness for the transaction is determined without requiring an intervention.

### Claim 17 (depends on 11)

The system of claim 11, wherein the operation further comprises:
determining a measure of interaction for at least a portion of the workflow during the transaction at the self-checkout kiosk, the measure of interaction including at least one of a rate of interaction or a rate of repetition of steps of the workflow;
setting a communication level for the workflow for the transaction, wherein the communication level applies at least in part to at least a remainder of the workflow, the remainder being relative to the portion, wherein the communication level is determined based on the measure of interaction and without requiring an intervention;
determining, based on the communication level, a next communication to output, wherein the next communication is output via the self-checkout kiosk.

### Claim 18 (depends on 17)

The system of claim 17, wherein at least one of the communication level or the next communication is determined based on a machine learning model.

### Claim 19 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the operation further comprises:
determining a measure of interaction for at least a portion of the workflow during the transaction at the self-checkout kiosk, the measure of interaction including at least one of a rate of interaction or a rate of repetition of steps of the workflow;
setting a communication level for the workflow for the transaction, wherein the communication level applies at least in part to at least a remainder of the workflow, the remainder being relative to the portion, wherein the communication level is determined based on the measure of interaction and without requiring an intervention;
determining, based on the communication level, a next communication to output, wherein the next communication is output via the self-checkout kiosk.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein at least one of the communication level or the next communication is determined based on a machine learning model.