Occlusion-resilient multi-camera action detection and recognition

A method, comprising:

• detecting, by processing circuitry, a user in each of a plurality of video streams of a plurality of cameras, wherein a first video stream comprises a plurality of frames;
• generating, by the processing circuitry, using each of the plurality of video streams, a plurality of bounding boxes, with a first bounding box representing the user and a first set of bounding boxes representing a set of products present in the corresponding video stream;
• detecting, by the processing circuitry, using each of the plurality of video streams, an action associated with the user, wherein the step of the detection of the action using the first video stream of the plurality of video streams further comprises:
• extracting, by the processing circuitry, a plurality of key-points associated with the user;
• determining, by the processing circuitry, for the set of products present in the first video stream, a set of proximity distances between the first set of bounding boxes and a first key-point of the plurality of key-points, wherein the set of proximity distances is determined for each frame of the plurality of frames;
• selecting, by the processing circuitry, based on the set of proximity distances, at least a first product from the set of products;
• determining, by the processing circuitry, for the first product, a first state indicating whether the first product is linked with the user;
• tracking, by the processing circuitry, the first state for a trajectory sequence associated with the user, wherein the action is detected based on the tracked first state;
• selecting, by the processing circuitry, one of the plurality of cameras based on a visibility of the first bounding box and an overlap associated with the first bounding box, in each video stream of the plurality of video streams;
• and
• designating, by the processing circuitry, the action detected using the selected camera to the user.