Binocular pedestrian detection system having dual-stream deep learning neural network and the methods of using the same

A binocular pedestrian detection system, comprising:

• a binocular camera having a first camera, and a second camera to capture a plurality of binocular images of a plurality of pedestrians passing through a predetermined area over a predetermined period of time, wherein each binocular image comprises a left image and a right image;
• an image/video processing application specific integrated circuit (ASIC) to preprocess the plurality of binocular images of the pedestrians captured by the binocular camera;
• and
• a binocular pedestrian detection system controller, having a processor, a network interface, and a memory storing an operating system, and computer executable instructions, wherein when executed in the processor, the computer executable instructions cause the processor to perform one or more of following:
• capturing, by the binocular camera, the plurality of binocular images of the pedestrians passing through the predetermined area over the predetermined period of time;
• binocularly rectifying, by the image/video processing ASIC, the plurality of binocular images of the pedestrians captured to obtain a plurality of rectified binocular images, wherein each of the plurality of rectified binocular images comprises a rectified left image and a rectified right image;
• calculating, by the image/video processing ASIC, disparity maps of the plurality of rectified binocular images;
• training a dual-stream deep learning neural network, wherein the dual-stream deep learning neural network comprises a neural network for extracting disparity features from the disparity maps of the plurality of binocular images of the pedestrians, and a neural network for learning and fusing features from the plurality of rectified left images and disparity maps of the plurality of binocular images of the pedestrians;
• and
• detecting, via the dual-stream deep learning neural network trained, the plurality of pedestrians passing through the predetermined area over the predetermined period of time.