Image processing device and method of detecting objects crossing a crossline and a direction the objects crosses the crossline

A non-transitory computer-readable storage medium having stored thereon instructions for implementing a method when executed in a device having an image sensor and a processor, the method for detecting objects crossing a crossline in a scene captured by a video camera and a direction the object crosses the crossline, the method comprising:

• capturing a sequence of video image frames of the scene;
• creating a combined image frame by:
• extracting two or more lines of pixels of each image frame of the sequence of video image frames, wherein the two or more lines of pixels of each video image frame are parallel to, and correspond to, the crossline in the scene such that if a representation of an object crosses the two or more lines of pixels in the sequence of video image frames, this corresponds to the object having crossed the crossline in the scene, and
• constructing the combined image frame by arranging the extracted two or more lines of pixels of each video image frame of the sequence of video image frames adjacent to each other;
• sending the combined image frame to a machine learning model trained to detect a combined image frame created from a sequence of video image frames that represents an object crossing a crossline in a scene and a direction the object crosses the crossline during capturing of the sequence of video image frames;
• and
• receiving from the machine learning model, a detection of any object crossing the crossline in the scene and a direction the object crosses the crossline during capturing of the sequence of video image frames.

An image processing device for detecting objects crossing a crossline in a scene captured by a video camera and a direction the object crosses the crossline, the image processing device comprising:

• an image sensor for capturing a sequence of video image frames of the scene;
• and
• circuitry arranged to execute:
• a creating function arranged to create a combined image frame by:
• extracting two or more lines of pixels of each video image frame of the sequence of video image frames, wherein the two or more lines of pixels of each video image frame are parallel to, and correspond to, the crossline in the scene such that if a representation of an object crosses the two or more lines of pixels in the sequence of video image frames, this corresponds to the object having crossed the crossline in the scene, and
• constructing the combined image frame by arranging the extracted two or more lines of pixels of the sequence of video image frames adjacent to each other;
• a sending function arranged to send the combined image frame to a machine learning model trained to detect a combined image frame created from a sequence of video image frames that represents an object crossing a crossline in a scene and a direction the object crosses the crossline during capturing of the sequence of video image frames;
• and
• a receiving function arranged to receive, from the machine learning model, a detection of any object crossing the crossline in the scene and a direction the object crosses the crossline during capturing of the sequence of video image frames.

A method of training a machine learning model to detect that an object crosses a crossline in a scene captured by a video camera and a direction the object crosses the crossline, the method comprising:

• creating a combined image frame for training by:
• extracting two or more lines of pixels of each image frame of a training sequence of video image frames, wherein the two or more lines of pixels of each video image frame are parallel to, and correspond to, the crossline in the training scene, such that if a representation of an object crosses the two or more lines of pixels in the training sequence of video image frames, this corresponds to the object having crossed the crossline in the training scene, and wherein a known object crosses the crossline in the training scene in a known direction during capturing of the training sequence of video image frames, and
• constructing the combined image frame for training by arranging the extracted two or more lines of pixels of each video image frame of the sequence of video image frames adjacent to each other;
• labelling the combined image frame for training with information specifying that the combined image frame for training represents the known object crossing the crossline in the training scene in the known direction during capturing of the training sequence of video image frames;
• and
• having the labelled combined image frame for training as input, training the machine learning model to detect a combined image frame created from a sequence of video image frames that represents an object crossing a crossline in a scene and a direction the object crosses the crossline during capturing of the sequence of video image frames.