Computer-Enabled Cart System Leveraging Machine Learning Models for Content Selection Based on Sensor Data Describing User Interactions

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

• detecting, by an on-cart computing system of a smart cart system, a change in contents of a storage area of the smart cart system based on sensor data captured by a set of sensors coupled to the smart cart system;
• determining, based on the sensor data, that an item has been removed from the storage area of the smart cart system by a user of the smart cart system;
• responsive to detecting the change in the contents of the storage area, capturing an image using a camera coupled to the smart cart system;
• applying an item identification model to the captured image to identify the item removed from the storage area of the smart cart system, wherein the item identification model is a machine-learning model that is trained to identify items based on images that depict the items;
• identifying a plurality of candidate items based on location data captured by a location sensor of the smart cart system, wherein the location data describes a location of the smart cart system when the item was removed from the storage area of the smart cart system;
• computing a presentation score for each of the plurality of candidate items based on item data describing the item removed from the smart cart system and item data describing a corresponding candidate item;
• selecting a candidate item for presentation to the user based on the computed presentation scores for the plurality of candidate items;
• and
• updating a display of the smart cart system to present content describing the selected candidate item.

A system comprising:

• a processor;
• and
• a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
• detecting, by an on-cart computing system of a smart cart system, a change in contents of a storage area of the smart cart system based on sensor data captured by a set of sensors coupled to the smart cart system;
• determining, based on the sensor data, that an item has been removed from the storage area of the smart cart system by a user of the smart cart system;
• responsive to detecting the change in the contents of the storage area, capturing an image using a camera coupled to the smart cart system;
• applying an item identification model to the captured image to identify the item removed from the storage area of the smart cart system, wherein the item identification model is a machine-learning model that is trained to identify items based on images that depict the items;
• identifying a plurality of candidate items based on location data captured by a location sensor of the smart cart system, wherein the location data describes a location of the smart cart system when the item was removed from the storage area of the smart cart system;
• computing a presentation score for each of the plurality of candidate items based on item data describing the item removed from the smart cart system and item data describing a corresponding candidate item;
• selecting a candidate item for presentation to the user based on the computed presentation scores for the plurality of candidate items;
• and
• updating a display of the smart cart system to present content describing the selected candidate item.