Three-dimensional measurement method based on end-to-end deep learning for speckle projection

A three-dimensional (3D) measurement method based on end-to-end deep learning for speckle projection, the method being configured to substantially promote matching accuracy for 3D shape measurement by overcoming low matching accuracy of traditional stereo matching algorithms, the method comprising the steps of:

• providing a projector and a stereo camera;
• projecting a speckle pattern by the projector onto an object;
• collecting the speckle pattern projected onto the object simultaneously by the stereo camera to generate speckle images;
• stereo rectifying the speckle images;
• processing the rectified speckle images by a feature extraction sub-network based on shared weights to obtain a series of low-resolution 3D feature tensors with a customized size, wherein the feature extraction sub-network is configured to extract rich multi-scale feature information for constructing a 4D cost volume;
• feeding the series of low-resolution 3D feature tensors into a saliency object detection sub-network to detect foreground information in the speckle images, thereby producing a full-resolution valid mask map;
• generating a 4D matching cost volume using the feature tensors derived from a left view and a right view of the stereo camera based on a candidate disparity range;
• filtering the 4D matching cost volume by a lightweight 3D U-net network using 3D convolutional layers to achieve efficient cost aggregation and obtain an initial disparity map by disparity regression;
• and
• obtaining a final disparity map by combining the full-resolution valid mask map and the initial disparity map.