Fruit and vegetable picking method and device based on machine vision and storage medium

A picking device, comprising:

• a processor and a storage medium communicating with the processor, the storage medium stores a plurality of instructions;
• the processor follows the instructions stored in the storage medium to implement a plurality of steps of a fruit and vegetable picking method based on machine vision, the steps of the fruit and vegetable picking method based on machine vision comprising:
• collecting a data set of different types of fruits and vegetables, and building a Mask r-cnn network model by using a deep learning framework on the dataset, wherein the data set of different types of fruits and vegetables comprises a plurality of pulp images or plant images of different varieties of fruits and vegetables, a plurality of pulp images in different maturity levels of the fruits and vegetables, a plurality of environment images of a garden or a vegetable greenhouse, and a plurality of images of various fruits and vegetables in a to-be-picked state in the gardens or the vegetable greenhouses;
• acquiring a fruit image of a fruit or a vegetable current to be picked, calling the Mask r-cnn network model to identify the fruit image from the dataset, and determining a type of the fruit or the vegetable to be picked;
• acquiring the type of the fruit or the vegetable to be picked, and determining a cuttable area on a fruit stalk of the fruit or the vegetable to be picked, according to the type of the fruit or the vegetable, before locating a cutting point;
• and
• controlling an end picking apparatus to cut off the stalk of the fruit or the vegetable to be picked, according to the cutting point determined.

A storage medium, having a plurality of instructions stored, wherein the instructions are able to be loaded and executed by a processor, to implement a plurality of steps of a fruit and vegetable picking method based on machine vision, the steps of the fruit and vegetable picking method based on machine vision comprising:

• collecting a data set of different types of fruits and vegetables, and building a Mask r-cnn network model by using a deep learning framework on the dataset;
• wherein the data set of different types of fruits and vegetables comprises a plurality of pulp images or plant images of different varieties of fruits and vegetables, a plurality of pulp images in different maturity levels of the fruits and vegetables, a plurality of environment images of a garden or a vegetable greenhouse, and a plurality of images of various fruits and vegetables in a to-be-picked state in the gardens or the vegetable greenhouses;
• acquiring a fruit image of a fruit or a vegetable current to be picked, calling the Mask r-cnn network model to identify the fruit image from the dataset, and determining a type of the fruit or the vegetable to be picked;
• acquiring the type of the fruit or the vegetable to be picked, and determining a cuttable area on a fruit stalk of the fruit or the vegetable to be picked, according to the type of the fruit or the vegetable, before locating a cutting point;
• and
• controlling an end picking apparatus to cut off the stalk of the fruit or the vegetable to be picked, according to the cutting point determined.