# US 2025/0094749 A1 · Image-based barcode decoding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094749 A1
- Document ID: US20250094749A1
- Status: Published application
- Kind: A1
- Application number: 18/969,074
- Filed: 2024-12-04
- Published: 2025-03-20
- Earliest claimed: 2021-03-24
- Assignees: Maplebear Inc.
- Applicants: Maplebear Inc.
- Inventors: Shiyuan Yang; Yilin Huang; Wentao Pan; Xiao Zhou
- Web page: https://patentgrep.com/patent/US20250094749A1

## CPC

- G06K 7/10861: {sensing of data fields affixed to objects or articles, e.g. coded labels (postal sorting B07C3/14, conveying articles B65G47/48)}
- G06K 7/1413: {1D bar codes}
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- G06V 10/242: {by image rotation, e.g. by 90 degrees}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/82: using neural networks
- G06V 20/80: Recognising image objects characterised by unique random patterns
- G06F 2218/12: Classification; Matching
- G06T 2207/20081: Training; Learning
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A barcode decoding system decodes item identifiers from images of barcodes. The barcode decoding system receives an image of a barcode and rotates the image to a pre-determined orientation. The barcode decoding system also may segment the barcode image to emphasize the portions of the image that correspond to the barcode. The barcode decoding system generates a binary sequence representation of the item identifier encoded in the barcode by applying a barcode classifier model to the barcode image, and decodes the item identifier from the barcode based on the binary sequence representation.

## Claims

### Claim 1

A cart system comprising:
a shopping cart;
a camera coupled to the shopping cart;
a processor; and
a non-transitory computer-readable medium storing instruction that, when executed by the processor, cause the processor to perform operations comprising:
receiving, from the camera coupled to the shopping cart, an image depicting a machine-readable label, wherein the machine-readable label represents an item identifier for an item comprising a sequence of values encoded in portions of the machine-readable label, wherein each portion corresponds to a encoding of a value of the sequence of values of the item identifier;
generating a rotated image of the machine-readable label by applying a rotation model to the image, wherein the rotation model comprises a machine-learning model stored on the computer-readable medium and trained to rotate images depicting machine-readable labels to a pre-determined orientation;
generating a segmented image of the machine-readable label by applying a segmentation model to the rotated image, wherein the segmentation model comprises a machine-learning model stored on the computer-readable medium and trained to remove backgrounds from rotated images of machine-readable labels;
generating a sequence representation of the item identifier for the item by applying a machine-readable label prediction model to the segmented image of the machine-readable label, wherein the sequence representation represents a prediction of the sequence of values, wherein the sequence representation represents one or more probabilistic predictions of the sequence of values encoded in the portions of the machine-readable label, and wherein the machine-readable label prediction model is a machine-learning model stored on the computer-readable medium and trained to generate sequence representations of item identifiers for machine-readable labels in segmented images; and
decoding the item identifier represented in the machine-readable label from the sequence representation of the item identifier generated by the machine-readable label prediction model.

### Claim 2 (depends on 1)

The cart system of claim 1, wherein the segmentation model comprises an attention mechanism.

### Claim 3 (depends on 1)

The cart system of claim 1, wherein the machine-readable label is a barcode.

### Claim 4 (depends on 1)

The cart system of claim 1, wherein the machine-readable label prediction model generates the prediction of the sequence of values by applying a connectionist temporal classification inference algorithm to the segmented image.

### Claim 5 (depends on 4)

The cart system of claim 4, wherein the machine-readable label prediction model is trained using a loss function comprising a connectionist temporal classification loss function.

### Claim 6 (depends on 1)

The cart system of claim 1, wherein the pre-determined orientation is horizontal.

### Claim 7 (depends on 1)

The cart system of claim 1, wherein the predicted sequence of values is an alphanumeric representation of the predicted item identifier.

### Claim 8 (depends on 1)

The cart system of claim 1, wherein the predicted sequence of values comprises a set of binary predictions that indicate probabilistic predictions of corresponding binary values of a binary encoding of the item identifier.

### Claim 9 (depends on 1)

The cart system of claim 1, further comprising:
adding the item represented by the item identifier to a shopping list associated with a user.

### Claim 10 (depends on 1)

The cart system of claim 1, further comprising:
causing information associated with the item to be displayed on a display device coupled to the shopping cart.

### Claim 11

A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, from a camera coupled to a shopping cart, an image depicting a machine-readable label, wherein the machine-readable label represents an item identifier for an item comprising a sequence of values encoded in portions of the machine-readable label, wherein each portion corresponds to a encoding of a value of the sequence of values of the item identifier;
generating a rotated image of the machine-readable label by applying a rotation model to the image, wherein the rotation model comprises a machine-learning model stored on the computer-readable medium and trained to rotate images depicting machine-readable labels to a pre-determined orientation;
generating a segmented image of the machine-readable label by applying a segmentation model to the rotated image, wherein the segmentation model comprises a machine-learning model stored on the computer-readable medium and trained to remove backgrounds from rotated images of machine-readable labels;
generating a sequence representation of the item identifier for the item by applying a machine-readable label prediction model to the segmented image of the machine-readable label, wherein the sequence representation represents a prediction of the sequence of values, wherein the sequence representation represents one or more probabilistic predictions of the sequence of values encoded in the portions of the machine-readable label, and wherein the machine-readable label prediction model is a machine-learning model stored on the computer-readable medium and trained to generate sequence representations of item identifiers for machine-readable labels in segmented images; and
decoding the item identifier represented in the machine-readable label from the sequence representation of the item identifier generated by the machine-readable label prediction model.

### Claim 12 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the segmentation model comprises an attention mechanism.

### Claim 13 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the machine-readable label is a barcode.

### Claim 14 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the machine-readable label prediction model generates the prediction of the sequence of values by applying a connectionist temporal classification inference algorithm to the segmented image.

### Claim 15 (depends on 14)

The non-transitory computer-readable storage medium of claim 14, wherein the machine-readable label prediction model is trained using a loss function comprising a connectionist temporal classification loss function.

### Claim 16 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the pre-determined orientation is horizontal.

### Claim 17 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the predicted sequence of values is an alphanumeric representation of the predicted item identifier.

### Claim 18 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the predicted sequence of values comprises a set of binary predictions that indicate probabilistic predictions of corresponding binary values of a binary encoding of the item identifier.

### Claim 19 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, further comprising:
adding the item represented by the item identifier to a shopping list associated with a user.

### Claim 20

A method comprising:
receiving, from a camera coupled to a shopping cart, an image depicting a machine-readable label, wherein the machine-readable label represents an item identifier for an item comprising a sequence of values encoded in portions of the machine-readable label, wherein each portion corresponds to a encoding of a value of the sequence of values of the item identifier;
generating a rotated image of the machine-readable label by applying a rotation model to the image, wherein the rotation model comprises a machine-learning model stored on a computer-readable medium coupled to the shopping cart and trained to rotate images depicting machine-readable labels to a pre-determined orientation;
generating a segmented image of the machine-readable label by applying a segmentation model to the rotated image, wherein the segmentation model comprises a machine-learning model stored on the computer-readable medium and trained to remove backgrounds from rotated images of machine-readable labels;
generating a sequence representation of the item identifier for the item by applying a machine-readable label prediction model to the segmented image of the machine-readable label, wherein the sequence representation represents a prediction of the sequence of values, wherein the sequence representation represents one or more probabilistic predictions of the sequence of values encoded in the portions of the machine-readable label, and wherein the machine-readable label prediction model is a machine-learning model stored on the computer-readable medium and trained to generate sequence representations of item identifiers for machine-readable labels in segmented images; and
decoding the item identifier represented in the machine-readable label from the sequence representation of the item identifier generated by the machine-readable label prediction model.
