# US 2022/0277299 A1 · Cart/basket fraud detection processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277299 A1
- Document ID: US20220277299A1
- Status: Published application
- Kind: A1
- Application number: 17/186,524
- Filed: 2021-02-26
- Published: 2022-09-01
- Earliest claimed: 2021-02-26
- Assignees: NCR Corporation
- Applicants: NCR Corporation
- Inventors: Joshua Migdal; Shayan Hemmatiyan
- Web page: https://patentgrep.com/patent/US20220277299A1

## CPC

- G06Q 20/389: {Keeping log of transactions for guaranteeing non-repudiation of a transaction}
- G06F 18/23213: with fixed number of clusters, e.g. K-means clustering
- G06F 18/24: Classification techniques
- G06N 20/00: Machine learning
- G06Q 20/208: {Input by product or record sensing, e.g. weighing or scanner processing}
- G06Q 20/401: {Transaction verification}
- G06Q 30/0633: {Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087)}
- G06Q 30/0641: {utilising user interfaces specially adapted for shopping}
- G06T 7/00: Image analysis
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions}
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 30/18: Extraction of features or characteristics of the image
- G07G 1/0063: {with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07G: REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS

## Abstract

An image of a cart/basket is captured based on a triggering event associated with a transaction at a transaction terminal. A determination is made whether the cart/basket is empty or nonempty from the image. When the cart/basket is nonempty a determination is made whether the nonempty cart includes one or more legitimate items or saleable items associated with a store. When the cart/basket is nonempty with one or more saleable items remaining in the cart/basket, and alert is raised to suspend completion of the transaction at the transaction terminal for intervention and audit of the transaction.

## Claims

### Claim 1

A method, comprising:
obtaining an image of a transaction area during a transaction at a transaction terminal based on a transaction event;
determining whether a cart/basket is present within the image;
determining whether the cart/basket is nonempty or empty when the cart/basket is present within the image;
determining whether the cart/basket includes at least one saleable item when the cart/basket is nonempty; and
causing the transaction to suspend for intervention of the transaction when the cart/basket includes the at least one saleable item.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining further includes detecting the transaction event as a payment request for a payment of the transaction made at the transaction terminal.

### Claim 3 (depends on 2)

The method of claim 2, wherein detecting further includes obtaining the image as a last image captured of the transaction area just before the detecting of the transaction event.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining whether the cart/basket is present within the image further includes providing the image to a machine-learning model as input and receiving as an output pixel coordinates within the image that define edges and an area associated with the cart/basket.

### Claim 5 (depends on 4)

The method of claim 4, wherein providing further includes defining a bounding box within the image based on the pixel coordinates.

### Claim 6 (depends on 5)

The method of claim 5, wherein defining further includes cropping the bounding box out of the image creating a modified version of the image.

### Claim 7 (depends on 6)

The method of claim 6, wherein cropping further includes scaling or resizing the modified version of the image to a canonical normalized size.

### Claim 8 (depends on 7)

The method of claim 7, wherein determining whether the cart/basket is nonempty or empty further includes providing the modified version of the image in the canonical normalized size to a second machine-learning model as input and receiving as output a nonempty classification or an empty classification from the second machine-learning model.

### Claim 9 (depends on 8)

The method of claim 8, wherein determining whether the cart/basket includes the at least one saleable item further includes providing, when output from the second machine-learning model was the nonempty classification, the modified version of the image in the canonical normalized size to a plurality of third machine-learning models as input and receiving as outputs indications as to whether the cart/basket comprises legitimate non-saleable items or the at least one saleable item.

### Claim 10 (depends on 9)

The method of claim 9, wherein causing further includes processing an Application Programming Interface (API) call to send an alert to a transaction system associated with the transaction terminal when at least one of the outputs from the third-machine learning models identifies the at least one saleable item causing the transaction terminal to suspend completion of the transaction.

### Claim 11 (depends on 10)

The method of claim 10, wherein processing the API call further includes providing at least one saleable item identifiers with the alert for the at least one saleable item.

### Claim 12

A method, comprising:
receiving a first image captured of a transaction area associated with a transaction at a transaction terminal;
identifying a cart/basket within the first image;
cropping out pixels of the first image associated with the cart/basket creating a second image;
classifying the cart/basket as a nonempty classification or an empty classification based on the second image;
upon detecting the nonempty classification further processing the second image and identifying objects within the cart/basket; and
upon identifying the objects raising an alert to the transaction terminal to suspend the transaction when at least one of the objects is associated with a saleable item.

### Claim 13 (depends on 12)

The method of claim 12, wherein receiving further includes identifying and selecting the first image from a plurality of other images captured of the transaction area based on receipt of a transaction event raised by a transaction system associated with the transaction terminal during the transaction.

### Claim 14 (depends on 12)

The method of claim 12, wherein identifying further includes providing the first image to a first machine-learning model as input and receiving as output pixel coordinates within the first image for an area within the first image that comprises the cart/basket.

### Claim 15 (depends on 14)

The method of claim 14, wherein cropping out the pixels further includes using the pixel coordinates to extract the area from the first image and creating the second image.

### Claim 16 (depends on 15)

The method of claim 15, wherein classifying further includes providing the second image to a second machine-learning model as input and receiving as output the nonempty classification of the empty classification.

### Claim 17 (depends on 16)

The method of claim 16, wherein upon detecting the nonempty classification further includes defining bounding boxes around each of the objects present within the cart/basket in the second image.

### Claim 18 (depends on 17)

The method of claim 17, wherein upon detecting the nonempty classification further includes providing bounding box pixels associated with each bounding box to a pipeline of third machine-leaning models and receiving as output legitimate non-saleable item identifiers for any legitimate non-saleable items present within the cart/basket and receiving saleable item identifiers for any saleable items present within the cart/basket.

### Claim 19

A system, comprising:
a cloud processing environment comprising at least one processor and a non-transitory computer-readable storage medium;
the non-transitory computer-readable storage medium comprises executable instructions;
the executable instructions when executed by the at least one processor from the non-transitory computer-readable storage medium cause the at least one processor to perform operations comprising:
detecting a transaction event at a transaction terminal during a transaction indicating that payment processing for the transaction was requested by an operator of the transaction terminal;
obtaining an image of a transaction area associated with the transaction terminal;
providing the image to a first machine-learning model and receiving as output pixel coordinates for a cart/basket present within the image;
cropping the pixel coordinates out of the image creating a second image;
resizing the second image into a canonical size;
providing the second image in the canonical size to a second machine-learning model and receiving as output a nonempty classification for the cart/basket or an empty classification for the cart/basket;
upon detecting the nonempty classification, providing the second image in the canonical size to a plurality of filtering machine-learning models as input and receiving as output one or more non-saleable item identifiers for legitimate items present within the cart/basket and one or more saleable item identifiers for saleable items present within the cart/basket; and
upon detecting at least one saleable item identifier, sending an alert to a transaction system associated with the transaction terminal that causes the transaction to be suspended without performing the payment processing for the transaction and that causes an attendant to be dispatched to the transaction terminal for an audit of the transaction.

### Claim 20 (depends on 19)

The system of claim 19, wherein the transaction terminal is a Self-Service Terminal (SST) or a Point-Of-Sale (POS) terminal.
