Method for determining alignment state, controller, and material handling equipment

A controller, configured to execute a program instruction to implement the following steps:

• controlling a material handling equipment to move a first stacking object to a stacking preparation position to complete a pre-alignment action relative to a second stacking object;
• acquiring an image of the first stacking object, and an image of the second stacking object by using a camera, and a point cloud of the first stacking object and a point cloud of the second stacking object by using a LiDAR, wherein the camera and the LiDAR are both installed on the material handling equipment;
• extracting first target image data of a first target region from the image of the first stacking object, and extracting second target image data of a second target region from the image of the second stacking object;
• extracting, based on the first target image data, first target point cloud data corresponding to the first target region from the point cloud of the first stacking object;
• extracting, based on the second target image data, second target point cloud data corresponding to the second target region from the point cloud of the second stacking object;
• determining a pose of the first stacking object based on the first target point cloud data, and determining a pose of the second stacking object based on the second target point cloud data;
• and
• determining a difference between the pose of the first stacking object and the pose of the second stacking object, and comparing the difference with a threshold to determine an alignment state between the first stacking object and the second stacking object;
• wherein the extracting, based on the first target image data, the first target point cloud data corresponding to the first target region from the point cloud of the first stacking object comprises:
• extracting, based on a joint calibration parameter and the first target image data, the first target point cloud data corresponding to the first target region from the point cloud of the first stacking object;
• and
• the extracting, based on the second target image data, the second point cloud data corresponding to the second target region from the target point cloud of the second stacking object comprises:
• extracting, based on a joint calibration parameter and the second target image data, the second target point cloud data corresponding to the second target region from the point cloud of the second stacking object.

A method for determining an alignment state, comprising:

• controlling a material handling equipment to move a first stacking object to a stacking preparation position to complete a pre-alignment action relative to a second stacking object;
• acquiring an image of the first stacking object;
• and an image of the second stacking object by using a camera, and a point cloud of the first stacking object and a point cloud of the second stacking object by using a LiDAR, wherein the camera and the LiDAR are both installed on the material handling equipment;
• extracting first target image data of a first target region from the image of the first stacking object, and extracting second target image data of a second target region from the image of the second stacking object;
• extracting, based on the first target image data, first target point cloud data corresponding to the first target region from the point cloud of the first stacking object;
• extracting, based on the second target image data, second target point cloud data corresponding to the second target region from the point cloud of the second stacking object;
• determining a pose of the first stacking object based on the first target point cloud data, and determining a pose of the second stacking object based on the second target point cloud data;
• and
• determining a difference between the pose of the first stacking object and the pose of the second stacking object, and comparing the difference with a threshold to determine an alignment state between the first stacking object and the second stacking object;
• wherein the extracting, based on the first target image data, the first target point cloud data corresponding to the first target region from the point cloud of the first stacking object comprises:
• extracting, based on a joint calibration parameter and the first target image data, the first target point cloud data corresponding to the first target region from the point cloud of the first stacking object;
• and
• the extracting, based on the second target image data, the second point cloud data corresponding to the second target region from the target point cloud of the second stacking object comprises:
• extracting, based on a joint calibration parameter and the second target image data, the second target point cloud data corresponding to the second target region from the point cloud of the second stacking object.