Training a model to identify items based on image data and load curve data

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

• capturing load data by a plurality of load sensors coupled to a storage area of a shopping cart, wherein the load data comprises partial load data describing a load imparted by an item on the storage area at different locations of the storage area;
• capturing image data by a plurality of cameras coupled to the shopping cart, wherein the image data comprises partial image data captured by each of the plurality of cameras, wherein the partial image data captured by a camera depicts the item from a different field of view of a portions of the storage area of the shopping cart;
• applying an item recognition model to the captured load data and the captured image data to generate an item identifier prediction for the item, wherein the item recognition model is trained to predict an item identifier for an item based on load data describing a load imparted by an item and image data depicting the item, wherein a set of parameters for the item recognition model were generated by training the item recognition model based on a plurality of training examples comprising load data for an item, image data depicting the item, and a label indicating an item identifier for the item, wherein applying the item recognition model comprises applying a set of parameters for the item recognition model to the captured load data and the captured image data, wherein the set of parameters for the item recognition model are stored by the computer-readable medium;
• and
• updating a list of items to indicate that the item is located within the storage area based on the item identifier prediction generated by the item recognition model.

A method comprising:

• capturing load data by a plurality of load sensors coupled to a storage area of a shopping cart, wherein the load data comprises partial load data describing a load imparted by an item on the storage area at different locations of the storage area;
• capturing image data by a plurality of cameras coupled to the shopping cart, wherein the image data comprises partial image data captured by each of the plurality of cameras, wherein the partial image data captured by a camera depicts the item from a different field of view of a portions of the storage area of the shopping cart;
• applying an item recognition model to the captured load data and the captured image data to generate an item identifier prediction for the item, wherein the item recognition model is trained to predict an item identifier for an item based on load data describing a load imparted by an item and image data depicting the item, wherein a set of parameters for the item recognition model were generated by training the item recognition model based on a plurality of training examples comprising load data for an item, image data depicting the item, and a label indicating an item identifier for the item, wherein applying the item recognition model comprises applying a set of parameters for the item recognition model to the captured load data and the captured image data;
• and
• updating a list of items to indicate that the item is located within the storage area based on the item identifier prediction generated by the item recognition model.