Mobile device platform for automated visual retail product recognition

Non-transitory computer-readable storage media having computer-executable instructions stored thereon for configuring a mobile point-of-sale application to perform automated product-recognition checkout processes in a mobile device network, wherein when executed by at least one processor the computer-executable instructions cause the at least one processor to:

• provide a convolutional neural network including n hidden layers and corresponding model parameters, the n hidden layers including hidden layer n positioned downstream of the remaining hidden layers and downstream of an input segment of the convolutional neural network;
• append a fine classification arm to the convolutional neural network, the fine classification arm operating on the output from hidden layer n and being configured to perform object classification;
• append a second classification arm to the convolutional neural network, the second classification arm operating on the output from one of the remaining n hidden layers upstream from hidden layer n and being configured to detect a barcode in an image frame;
• append a third classification arm to the convolutional neural network and configure the third classification arm to perform objectness detection on the image frame, the third classification arm
• operating on the output from one of the remaining n hidden layers upstream from the upstream hidden layer corresponding to the second classification arm,
• sharing convolutional features of the convolutional neural network with the first and second classification arms;
• provide a plurality of image subsets, each image subset relating to a product class;
• provide a plurality of true labels, each true label being applicable to at least one of the product classes;
• provide a first loss function configured to generate output of the fine classification arm;
• provide a second loss function configured to generate output of the second classification arm;
• provide a third loss function configured to generate output of the third classification arm;
• train the convolutional neural network to generate a trained convolutional neural network at least in part by
• passing the plurality of image subsets through the convolutional neural network,
• iteratively adjusting values of the model parameters based at least in part on one or more of:
• (A) output of the first, second and third loss functions and (B) derivatives of the first, second and third loss functions with respect to the model parameters in response to passing the plurality of image subset through the convolutional neural network;
• and
• configure a control logic to perform:
• (i) aborting propagation through one or more of the n hidden layers which are downstream of the third classification arm if the third classification arm identifies only background objects in the image frame, or (ii) terminating object classification by the fine classification arm if the second classification arm reaches an output determination for the image frame.