Fruit picking method based on three-dimensional parameter prediction model for fruit

A fruit picking device based on a three-dimensional parameter prediction model for a fruit, comprising:

• a first image acquisition unit for performing first-time image acquisition processing to obtain a first image of a to-be-picked fruit using a camera;
• a first range determination unit for determining a first range that accommodates the to-be-picked fruit based on the first image;
• a first-time moving unit for controlling a manipulator to move the manipulator to an outside of the first range and cover the first range from below, and guaranteeing that the manipulator does not enter the first range while moving;
• a gas injection unit for controlling a gas injection device to perform intermittent gas injection treatment in the first range to lead to forced vibration of the to-be-picked fruit in the first range, wherein the gas injection unit is further configured to control the gas injection device to perform intermittent gas injection treatment in the first range along a first direction to lead to forced vibration of the to-be-picked fruit located in the first range in the first direction, wherein an angle between the first direction and an image acquisition direction of the camera is not zero;
• a second image acquisition unit for performing second-time image acquisition processing on the to-be-picked fruit many times by using the camera in a forced vibration process of the to-be-picked fruit to obtain a plurality of second images, wherein the second image acquisition unit is further configured to:
• project, by a spot generator, an appointed spot on the to-be-picked fruit in the forced vibration process of the to-be-picked fruit, wherein a projection direction of the appointed spot projected by the spot generator is parallel to the image acquisition direction of the camera;
• and
• perform second-time image acquisition processing on the to-be-picked fruit projected by the appointed spot many times by using the camera to obtain a plurality of second images with spots;
• a second image screening unit for performing, by taking the first image as a reference object, screening processing on the plurality of second images to screen out two appointed second images deviating from an equilibrium position to a maximum extent;
• a predicted three-dimensional parameter acquisition unit configured to jointly input the first image and the two second images into a preset three-dimensional parameter prediction model for the fruit so as to obtain predicted three-dimensional parameters outputted by the three-dimensional parameter prediction model for the fruit, wherein the three-dimensional parameter prediction model for the fruit is trained by a preset neural network model;
• a second-time moving unit for determining a second range according to the predicted three-dimensional parameters and controlling the manipulator to perform second-time moving processing to make the manipulator reduce a coverage to the second range, wherein the second range is smaller than the first range;
• and
• a fruit picking unit for determining a fruit stem position in the first image and performing cutting processing on the fruit stem position by using a cutting device to make the to-be-picked fruit to fall onto the manipulator, so as to complete a picking process.