# US 2025/0095389 A1 · Modeling method for picking target of fruit bunch picking robot

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095389 A1
- Document ID: US20250095389A1
- Status: Published application
- Kind: A1
- Application number: 18/757,863
- Filed: 2024-06-28
- Published: 2025-03-20
- Earliest claimed: 2023-09-19
- Assignees: Research Center of Intelligent Equipment, BAAFS
- Applicants: Research Center of Intelligent Equipment, BAAFS
- Inventors: Qingchun FENG; Yajun LI; Chuanlang PENG; Yifan ZHANG
- Web page: https://patentgrep.com/patent/US20250095389A1

## CPC

- G06V 20/68: Food, e.g. fruit or vegetables
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
- G06V 10/457: {by analysing connectivity, e.g. edge linking, connected component analysis or slices}
- G06V 10/7625: {Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06V 20/188: {Vegetation}
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/766: using regression, e.g. by projecting features on hyperplanes
- G06V 2201/07: Target detection
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})

## Abstract

Disclosed is a modeling method for a picking target of a fruit bunch picking robot, which relates to the technical field of general image data processing or generation. The modeling method includes: obtaining an image of a to-be-picked region of a picking robot, and extracting image features of each branch and fruit cluster in the image of the to-be-picked region through a multi-task perception network; determining a to-be-picked fruit cluster based on the image feature of the fruit cluster; inputting the image features of the branch and the fruit cluster into a subordinate decision model to determine a branch connected to the to-be-picked fruit cluster; and extracting key points of image features of the to-be-picked fruit cluster and the branch connected to the to-be-picked fruit cluster, and modeling the to-be-picked fruit cluster and the branch connected to the to-be-picked fruit cluster.

## Claims

### Claim 1

A modeling method for a picking target of a fruit bunch picking robot, wherein a fruit bunch comprises a branch and a fruit cluster connected to the branch, and
the modeling method for a picking target of a fruit bunch picking robot comprises:
obtaining an image of a to-be-picked region of a picking robot, and extracting image features of each branch and fruit cluster in the image of the to-be-picked region through a multi-task perception network;
determining a to-be-picked fruit cluster based on the image feature of the fruit cluster; and inputting the image features of the branch and the fruit cluster into a subordinate decision model to determine a branch connected to the to-be-picked fruit cluster; and
extracting key points of image features of the to-be-picked fruit cluster and the branch connected to the to-be-picked fruit cluster, and modeling the to-be-picked fruit cluster and the branch connected to the to-be-picked fruit cluster.

### Claim 2 (depends on 1)

The modeling method for a picking target of a fruit bunch picking robot according to claim 1, wherein the extracting image features of each branch and fruit cluster in the image of the to-be-picked region through a multi-task perception network comprises:
extracting an overall image feature of the fruit bunch; and
extracting a boundary frame of the fruit cluster and a segmentation mask of the branch based on the overall image feature of the fruit bunch.

### Claim 3 (depends on 2)

The modeling method for a picking target of a fruit bunch picking robot according to claim 2, wherein the multi-task perception network comprises:
a shared encoder configured to extract the overall image feature of the fruit bunch;
a target detection decoder communicatively connected to the shared encoder and configured to process the overall image feature of the fruit bunch to extract the boundary frame of the fruit cluster; and
an instance segmentation decoder communicatively connected to the shared encoder and configured to process the overall image feature of the fruit bunch to extract the segmentation mask of the branch.

### Claim 4 (depends on 1)

The modeling method for a picking target of a fruit bunch picking robot according to claim 1, before the obtaining an image of a to-be-picked region of a picking robot, and extracting image features of each branch and fruit cluster in the image of the to-be-picked region through a multi-task perception network, comprising:
collecting an image sample of the fruit bunch;
determining a subordinate relationship parameter based on the image sample of the fruit bunch; and
training a classification and regression tree model based on the subordinate relationship parameter of the image sample of the fruit bunch, and constructing the subordinate decision model.

### Claim 5 (depends on 4)

The modeling method for a picking target of a fruit bunch picking robot according to claim 4, wherein the branch comprises a main stem and a fruit stem, the main stem and the fruit cluster are respectively connected to two ends of the fruit stem, and the subordinate relationship parameter comprises a first parameter, a second parameter, a third parameter, a fourth parameter, and a fifth parameter; and the determining a subordinate relationship parameter based on the image sample of the fruit bunch comprises:
determining the first parameter based on a connection relationship between the fruit cluster and the fruit stem in the image sample;
determining the second parameter based on a connection relationship between the fruit stem and the main stem in the image sample;
determining the third parameter based on a positional relationship between the fruit cluster and the fruit stem in the image sample;
determining the fourth parameter based on a positional relationship between a lower endpoint of the fruit stem and the fruit cluster in the image sample; and
determining the fifth parameter based on a distance between an upper endpoint of the fruit stem and a center line of the main stem in the image sample.

### Claim 6 (depends on 4)

The modeling method for a picking target of a fruit bunch picking robot according to claim 4, wherein the inputting the image features of the branch and the fruit cluster into a subordinate decision model to determine a branch connected to the to-be-picked fruit cluster comprises:
determining the subordinate relationship parameter based on the image features of the branch and the fruit cluster; and
inputting the subordinate relationship parameter into the subordinate decision model to determine a branch connected to each fruit cluster, so as to determine the branch connected to the to-be-picked fruit cluster.

### Claim 7 (depends on 2)

The modeling method for a picking target of a fruit bunch picking robot according to claim 2, wherein the branch comprises a main stem and a fruit stem, the main stem and the fruit cluster are respectively connected to two ends of the fruit stem, and the key points comprise a key point of the fruit cluster, a key point of the fruit stem, and a key point of the main stem; and the extracting key points of image features of the to-be-picked fruit cluster and the branch connected to the to-be-picked fruit cluster, and modeling the to-be-picked fruit cluster and the branch connected to the to-be-picked fruit cluster comprises:
extracting a key point that is of the fruit cluster and located in the boundary frame of the fruit cluster, and constructing a fruit cluster model;
extracting a key point that is of the fruit stem and located within a segmentation mask of the fruit stem, and constructing a fruit stem model; and
extracting a key point that is of the main stem and located within a segmentation mask of the main stem, and constructing a main stem model.

### Claim 8 (depends on 7)

The modeling method for a picking target of a fruit bunch picking robot according to claim 7, wherein the key point of the fruit cluster comprises all vertices of the boundary frame of the fruit cluster.

### Claim 9 (depends on 7)

The modeling method for a picking target of a fruit bunch picking robot according to claim 7, wherein the key point of the fruit stem comprises a connection point between the fruit stem and the main stem, a connection point between the fruit stem and the fruit cluster, and an inflection point of a middle segment of the fruit stem.

### Claim 10 (depends on 7)

The modeling method for a picking target of a fruit bunch picking robot according to claim 7, wherein the key point of the main stem comprises a first key point and a plurality of second key points, the first key point is a connection point between the fruit stem and the main stem, and the second key points are spaced on both sides of the first key point along an extension direction of the main stem.
