# US 2026/0065253 A1 · Computer-Enabled Cart System Leveraging Machine Learning Models for Content Selection Based on Sensor Data Describing User Interactions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065253 A1
- Document ID: US20260065253A1
- Status: Published application
- Kind: A1
- Application number: 18/821,677
- Filed: 2024-08-30
- Published: 2026-03-05
- Earliest claimed: 2024-08-30
- Assignees: Maplebear Inc.
- Applicants: Maplebear Inc.
- Inventors: Brent Scheibelhut; Naval Shah; Charles Wesley; Mark Oberemk
- Web page: https://patentgrep.com/patent/US20260065253A1

## CPC

- G06Q 20/208: {Input by product or record sensing, e.g. weighing or scanner processing}
- G01G 19/52: Weighing apparatus combined with other objects, e.g. furniture (with walking sticks A45B3/08)
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G07G 1/0072: {with means for detecting the weight of the article of which the code is read, for the verification of the registration (weighing combined with recording means, e.g. in weighing scale, G01G19/415)}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G01G: WEIGHING (sorting by weighing B07C5/16)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07G: REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS

## Abstract

A smart cart system accounts for edge cases in user interactions by leveraging sensor data and machine-learning models of a smart cart system. For example, a smart cart system uses sensor data to detect when a user removes an item from the smart cart system and presents content to the user on a display of the smart cart system based on the removed item. The smart cart system captures images of the storage area and applies an item identification model to the images to identify the item removed from the storage area. The smart cart system identifies a set of candidate items based on location sensor data describing a location of the smart cart system when the item was removed and computes presentation scores for each of the set of candidate items based on item data for each item the removed item.

## Claims

### Claim 1

A method comprising:
detecting, by an on-cart computing system of a smart cart system, a change in contents of a storage area of the smart cart system based on sensor data captured by a set of sensors coupled to the smart cart system;
determining, based on the sensor data, that an item has been removed from the storage area of the smart cart system by a user of the smart cart system;
responsive to detecting the change in the contents of the storage area, capturing an image using a camera coupled to the smart cart system;
applying an item identification model to the captured image to identify the item removed from the storage area of the smart cart system, wherein the item identification model is a machine-learning model that is trained to identify items based on images that depict the items;
identifying a plurality of candidate items based on location data captured by a location sensor of the smart cart system, wherein the location data describes a location of the smart cart system when the item was removed from the storage area of the smart cart system;
computing a presentation score for each of the plurality of candidate items based on item data describing the item removed from the smart cart system and item data describing a corresponding candidate item;
selecting a candidate item for presentation to the user based on the computed presentation scores for the plurality of candidate items; and
updating a display of the smart cart system to present content describing the selected candidate item.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sensor data comprises load data from a load sensor coupled to the storage area of the smart cart system, and wherein determining that an item has been removed from the storage area comprises:
measuring, at a first time, a first weight of items in the storage area of the smart cart system based on load data captured by the load sensor;
measuring, at a second time after the first time, a second weight of items in the storage area of the smart cart system based on load data captured by the load sensor;
comparing the first weight to the second weight; and
responsive to the first weight being greater than the second weight, determining that an item has been removed from the storage area.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining that an item has been removed from the storage area comprises:
identifying a first set of items in the storage area at a first time based on a first image captured by the camera;
identifying a second set of items in the storage area at a second time after the first time based on a second image captured by the camera;
comparing the first set of items and the second set of items; and
responsive to the first set of items being larger than the second set of items, determining that an item has been removed from the storage area.

### Claim 4 (depends on 3)

The method of claim 3, wherein applying the item identification model to identify the item removed from the storage area comprises:
identifying an item that is in the first set of items and not in the second set of items.

### Claim 5 (depends on 1)

The method of claim 1, wherein the item identification model is at least one of a barcode detection model, an optical character recognition model, or an image embedding model.

### Claim 6 (depends on 1)

The method of claim 1, wherein identifying the plurality of candidate items based on location data comprises:
comparing the location of the smart cart system to a model of an environment around the smart cart system.

### Claim 7 (depends on 6)

The method of claim 6, wherein identifying the plurality of candidate items based on location data comprises:
identifying, based on the model of the environment, a set of items located within a threshold distance of the location of the smart cart system.

### Claim 8 (depends on 1)

The method of claim 1, wherein computing a presentation score for a candidate item comprises:
applying a machine-learning model to the item data describing the item removed from the smart cart system and item data describing the candidate item, wherein the machine-learning model is trained to generate presentation scores based on a set of training examples, wherein each training example comprises item data for a candidate item, item data for an item removed from a smart cart system, and a label indicating whether a user performed a target interaction in response to being presented with content relating to the candidate item after removing the removed item.

### Claim 9 (depends on 1)

The method of claim 1, wherein computing the presentation score for each candidate item comprises:
computing the presentation score based on user data describing the user of the smart cart system or context data describing a context of the smart cart system.

### Claim 10 (depends on 9)

The method of claim 9, wherein computing the presentation score for each candidate item comprises:
computing the presentation score based on the context data, wherein the context data comprises a selection of a user of a user interface element indicating a reason for removing the item removed from the smart cart system.

### Claim 11

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
detecting, by an on-cart computing system of a smart cart system, a change in contents of a storage area of the smart cart system based on sensor data captured by a set of sensors coupled to the smart cart system;
determining, based on the sensor data, that an item has been removed from the storage area of the smart cart system by a user of the smart cart system;
responsive to detecting the change in the contents of the storage area, capturing an image using a camera coupled to the smart cart system;
applying an item identification model to the captured image to identify the item removed from the storage area of the smart cart system, wherein the item identification model is a machine-learning model that is trained to identify items based on images that depict the items;
identifying a plurality of candidate items based on location data captured by a location sensor of the smart cart system, wherein the location data describes a location of the smart cart system when the item was removed from the storage area of the smart cart system;
computing a presentation score for each of the plurality of candidate items based on item data describing the item removed from the smart cart system and item data describing a corresponding candidate item;
selecting a candidate item for presentation to the user based on the computed presentation scores for the plurality of candidate items; and
updating a display of the smart cart system to present content describing the selected candidate item.

### Claim 12 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the sensor data comprises load data from a load sensor coupled to the storage area of the smart cart system, and wherein determining that an item has been removed from the storage area comprises:
measuring, at a first time, a first weight of items in the storage area of the smart cart system based on load data captured by the load sensor;
measuring, at a second time after the first time, a second weight of items in the storage area of the smart cart system based on load data captured by the load sensor;
comparing the first weight to the second weight; and
responsive to the first weight being greater than the second weight, determining that an item has been removed from the storage area.

### Claim 13 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein determining that an item has been removed from the storage area comprises:
identifying a first set of items in the storage area at a first time based on a first image captured by the camera;
identifying a second set of items in the storage area at a second time after the first time based on a second image captured by the camera;
comparing the first set of items and the second set of items; and
responsive to the first set of items being larger than the second set of items, determining that an item has been removed from the storage area.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein applying the item identification model to identify the item removed from the storage area comprises:
identifying an item that is in the first set of items and not in the second set of items.

### Claim 15 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the item identification model is at least one of a barcode detection model, an optical character recognition model, or an image embedding model.

### Claim 16 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein identifying the plurality of candidate items based on location data comprises:
comparing the location of the smart cart system to a model of an environment around the smart cart system.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein identifying the plurality of candidate items based on location data comprises:
identifying, based on the model of the environment, a set of items located within a threshold distance of the location of the smart cart system.

### Claim 18 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein computing a presentation score for a candidate item comprises:
applying a machine-learning model to the item data describing the item removed from the smart cart system and item data describing the candidate item, wherein the machine-learning model is trained to generate presentation scores based on a set of training examples, wherein each training example comprises item data for a candidate item, item data for an item removed from a smart cart system, and a label indicating whether a user performed a target interaction in response to being presented with content relating to the candidate item after removing the removed item.

### Claim 19 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein computing the presentation score for each candidate item comprises:
computing the presentation score based on user data describing the user of the smart cart system or context data describing a context of the smart cart system.

### Claim 20

A system comprising:
a processor; and
a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
detecting, by an on-cart computing system of a smart cart system, a change in contents of a storage area of the smart cart system based on sensor data captured by a set of sensors coupled to the smart cart system;
determining, based on the sensor data, that an item has been removed from the storage area of the smart cart system by a user of the smart cart system;
responsive to detecting the change in the contents of the storage area, capturing an image using a camera coupled to the smart cart system;
applying an item identification model to the captured image to identify the item removed from the storage area of the smart cart system, wherein the item identification model is a machine-learning model that is trained to identify items based on images that depict the items;
identifying a plurality of candidate items based on location data captured by a location sensor of the smart cart system, wherein the location data describes a location of the smart cart system when the item was removed from the storage area of the smart cart system;
computing a presentation score for each of the plurality of candidate items based on item data describing the item removed from the smart cart system and item data describing a corresponding candidate item;
selecting a candidate item for presentation to the user based on the computed presentation scores for the plurality of candidate items; and
updating a display of the smart cart system to present content describing the selected candidate item.
