Binocular vision-based environment sensing method and apparatus, and unmanned aerial vehicle

A binocular vision-based environment sensing apparatus, applied to an unmanned aerial vehicle provided with five binocular cameras, the five binocular cameras comprising a first binocular camera, a second binocular camera, a third binocular camera, a fourth binocular camera and a fifth binocular camera, the first binocular camera being disposed at a front portion of a fuselage of the unmanned aerial vehicle, the second binocular camera being inclined upward and being disposed between a left side of the fuselage and an upper portion of the fuselage of the unmanned aerial vehicle, the third binocular camera being inclined upward and being disposed between a right side of the fuselage and the upper portion of the fuselage of the unmanned aerial vehicle, the fourth binocular camera being disposed at a lower portion of the fuselage of the unmanned aerial vehicle, and the fifth binocular camera being disposed at a rear portion of the fuselage of the unmanned aerial vehicle, the apparatus comprising:

• a disparity map generation module for acquiring a binocular view by using each binocular camera among the five binocular cameras, and generating disparity maps according to the binocular views;
• an initial point cloud data generation module for separately generating initial three-dimensional (3D) point cloud data according to the disparity maps and internal parameters of the binocular cameras corresponding thereto;
• a point cloud data generation module for converting coordinate systems of the initial 3D point cloud data into world coordinate systems, respectively, so as to acquire 3D point cloud data in five world coordinate systems;
• a three-dimensional map construction module for constructing a 3D map of a target scene according to the 3D point cloud data in the five world coordinate systems
• a control binocular view acquisition module for, in a pre-set environment, acquiring a control binocular view by using each binocular camera among the five binocular cameras;
• and
• a mask view generation module for, if it is determined that there is an occlusion area formed by a body of the unmanned aerial vehicle on left and right views in the control binocular view, respectively constructing a left mask and a right mask according to the occlusion area, generating mask views corresponding to the binocular camera according to the left mask and the right mask, and storing the mask views;
• the initial point cloud data generation module further for fusing the disparity map and a mask view corresponding to the binocular camera to generate a fused disparity map;
• generating initial three-dimensional point cloud data according to the fused disparity map and the internal parameters of the binocular camera.

An unmanned aerial vehicle, comprising:

• a first binocular camera disposed at a front portion of a fuselage of the unmanned aerial vehicle;
• a second binocular camera inclined upward and disposed between a left side of the fuselage and an upper portion of the fuselage of the unmanned aerial vehicle;
• a third binocular camera inclined upward and disposed between a right side of the fuselage and the upper portion of the fuselage of the unmanned aerial vehicle;
• a fourth binocular camera disposed at a lower portion of the fuselage of the unmanned aerial vehicle;
• a fifth binocular camera disposed at a rear portion of the fuselage of the unmanned aerial vehicle;
• a controller respectively connected to the first binocular camera, the second binocular camera, the third binocular camera, the fourth binocular camera and the fifth binocular camera, the controller comprising:
• at least one processor, and
• a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the following operations:
• acquiring a binocular view by using each binocular camera among the five binocular cameras, and generating disparity maps according to the binocular views;
• separately generating initial three-dimensional (3D) point cloud data according to the disparity maps and internal parameters of the binocular cameras corresponding thereto;
• converting coordinate systems of the initial 3D point cloud data into world coordinate systems, respectively, so as to acquire 3D point cloud data in five world coordinate systems;
• constructing a 3D map of a target scene according to the 3D point cloud data in the five world coordinate systems
• wherein the at least one processor further performs the following operations:
• acquiring a control binocular view in a pre-set environment by using each binocular camera among the five binocular cameras;
• if there is an occlusion area formed by a body of the unmanned aerial vehicle on left and right views in the control binocular view, respectively constructing a left mask and a right mask according to the occlusion area, generating mask views corresponding to the binocular camera according to the left mask and the right mask, and storing the mask views;
• fusing the disparity map and a mask view corresponding to the binocular camera to generate a fused disparity map;
• and
• generating initial three-dimensional point cloud data according to the fused disparity map and the internal parameters of the binocular camera.