Methods and systems of a multistage object detection and tracking checkout system

A system for multistage object detection and tracking for a cashierless checkout comprising:

• a mobile device comprising:
• a set of on-board sensors installed in the shopping container, wherein the on-board sensors comprise digital cameras viewing an interior of the shopping container from a set of different angles and positions, wherein the on-board sensors detect and provide a user activity data associated with respect to placement or removal of a set of shopping items into or from a shopping container, and wherein a digital camera obtains a set of video frames associated with these items entering and leaving the interior region of the shopping container and communicates these to a mobile device comprising an on-device machine learning (ML) detection;
• an on-device ML detection and tracking engine, wherein the on-device ML detection and tracking engine uses computer vision and specified ML algorithms to implement object detection of unique signatures associated with a shopping item of the set of shopping items and tracking the shopping item by interfacing with the set of on-board image sensors in the interior of the shopping container and determines a state of the item in the interior region of the shopping container;
• wherein the on-device ML detection and tracking engine detects the item in the interior of the shopping cart and determines the state of the item in the interior region of the shopping container based in two phases of analysis comprising:
• in a first phase, the on-device ML detection and tracking engine scans the set of video frames from different digital cameras of the interior region of the shopping container for a semantic change from a previous frame,
• in a second phase, the on-device ML detection and tracking engine implements an object detection process on the interior region, such that a presence or an absence of the item is detected.