Large-scale automated image annotation system

A system comprising:

• a mobile device having a mobile application installed thereon, the mobile application including executable instructions which, when executed, cause the mobile device to perform:
• in a first mode, capturing a plurality of image frames via a camera of the mobile device, the plurality of image frames depicting a plurality of product items;
• for each of the plurality of image frames, generating one or more cropped images, the one or more cropped images representing each product item appearing in the image frame;
• detecting a plurality of objects appearing across the plurality of image frames;
• in a second mode, capturing a unique identifier associated with each detected object, the unique identifier being an identity of an object within a fine-grained image taxonomy;
• transmitting the plurality of cropped images, information regarding the plurality of objects, and the unique identifier associated with each of the plurality of objects to a remote computing system to form an annotated image collection.

A method comprising:

• capturing a plurality of image frames via a mobile device, the image frames depicting a plurality of product items offered by a retailer, the plurality of product items organized into an item classification;
• for each of the plurality of image frames, generating one or more cropped images, the one or more cropped images representing each product item appearing in the image frame;
• detecting a plurality of objects appearing across the plurality of image frames;
• associating at least one of the cropped images generated from an image frame with each detected object of the plurality of objects tracked across the plurality of image frames;
• capturing, via the mobile device, a unique identifier of each detected object;
• annotating each cropped image that is associated with a detected object of the plurality of objects by applying a label to the cropped image, wherein applying the label includes determining an item label of the detected object based on the unique identifier and associating the item label with the cropped image, thereby automatically forming an image classification training dataset.