Deep learning inference systems and methods for imaging systems

A method comprising:

• loading onto a vision processing unit of an imaging device a first trained inference network;
• capturing an image pair of a scene using two or more imaging components of the imaging device;
• processing the image pair through the first trained inference network to detect an object of interest and determine a first inference result comprising bounding boxes corresponding to two-dimensional locations in at least one of the images in the image pair of the detected object in at least one of the images, wherein at least one image feature computed during operation of the first trained inference network are used as features for stereo matching when objects of interest are detected;
• and
• determining, based on the first inference result, an action for the image pair;
• and
• controlling a robotic system to interact with at least one of the detected objects based on three dimensional data;
• performing stereo matching on regions of interest in the image pair based on the bounding boxes to produce spatial location information for the objects of interest, wherein the stereo matching is performed when objects of interest are detected;
• and
• wherein the stereo matching includes a correlation-based method and/or a feature-based method to generate a three-dimensional point cloud.

A system comprising:

• two or more image capture components configured to capture a pair of images of a scene;
• a vision processing unit configured to process the image pair through a first trained inference network to determine a first inference result, wherein the first trained inference network detects objects of interest in the image pair and generates bounding boxes of corresponding two-dimensional locations in at least one of the images in the image pair, wherein at least one image feature computed during operation of the first trained inference network are used as features for stereo matching when objects of interest are detected;
• and
• a processing component configured to process the first inference result and determine, based on the first inference result, an action for the image pair;
• wherein three-dimensional data is used to control a robotic system to interact with at least one of the detected objects;
• wherein the vision processing unit is further configured to perform stereo matching on regions of interest in the image pair based on the bounding boxes to produce spatial location information for the objects of interest;
• wherein the stereo matching is performed when objects of interest are detected;
• and
• wherein the stereo matching includes a correlation-based method and/or a feature-based method to generate a three-dimensional point cloud.