# US 2023/0410362 A1 · Target object detection method and apparatus, and electronic device, storage medium and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410362 A1
- Document ID: US20230410362A1
- Status: Published application
- Kind: A1
- Application number: 18/248,617
- Filed: 2022-01-13
- Published: 2023-12-21
- Earliest claimed: 2021-01-20
- Assignees: Beijing Jingdong Qianshi Technology Co., Ltd.
- Applicants: Beijing Jingdong Qianshi Technology Co., Ltd.
- Inventors: Yunjian CHENG; Weifeng LIU
- Web page: https://patentgrep.com/patent/US20230410362A1

## CPC

- B07C 5/36: Sorting apparatus characterised by the means used for distribution {(sorting according to destination B07C3/003, B07C3/02)}
- G06T 7/74: {involving reference images or patches}
- G06T 7/248: {involving reference images or patches}
- G06T 7/60: Analysis of geometric attributes
- G06V 20/50: Context or environment of the image
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- B07C 2501/0063: Using robots
- G06V 2201/06: Recognition of objects for industrial automation
- B07C: POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING (specially adapted for a specific purpose covered by another class, see the relevant place, e.g. A43D33/06, B23Q7/12)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided in the embodiments of the present application are a target object detection method and apparatus, and an electronic device and a computer storage medium. The method comprises: when it is determined that a robot arm picks up a target object from any turnover box and the height of the target object is greater than the height of the turnover box, performing target tracking on the target object to obtain real-time position information of the target object; and when it is determined, according to the real-time position information of the target object, that the target object falls from the robot arm, determining an area in which the target object is located according to position information of the target object at the current moment and area position information, wherein the area position information comprises areas in which various types of turnover boxes are located.

## Claims

### Claim 1

A method for target object detection, performed by an electronic device, the method comprising:
in response to determining that a mechanical arm picks up a target object from any one of a plurality of turnover boxes and a height of the target object is greater than a height of the turnover box, performing target tracking on the target object to obtain real-time position information of the target object; and
in response to determining, based on the real-time position information of the target object, that the target object falls down from the mechanical arm, determining an area where a landing point of the target object is located based on current position information of the target object and area position information, wherein the area position information comprises information of areas where the plurality of turnover boxes are located.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
for each turnover box of the plurality of turnover boxes, determining a type of the turnover box based on an attribute of the turnover box, and obtaining type labeling information of the turnover box based on the type of the turnover box; and
obtaining the area position information based on type labeling information of all of the plurality of turnover boxes.

### Claim 3 (depends on 2)

The method of claim 2, wherein the attribute of the turnover box comprises at least one of: color, texture, shape, or size.

### Claim 4 (depends on 1)

The method of claim 1, wherein performing the target tracking on the target object comprises:
capturing, by using an image capturing device, the target object for multiple times to obtain a plurality of captured images; and performing the target tracking on the target object based on each of the plurality of captured images.

### Claim 5 (depends on 4)

The method of claim 4, wherein performing the target tracking on the target object based on each of the plurality of captured images comprises:
eliminating a background in each of the plurality of captured images to obtain a respective result after background elimination of the captured image; and
performing the target tracking on the target object based on the respective result after background elimination of the captured image, wherein a background of the target object represents an image of a preset background object.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
after determining the area where the landing point of the target object is located, determining a position of the landing point of the target object based on a relative position relationship between the image capturing device and the mechanical arm, and position and attitude information of an end of the mechanical arm.

### Claim 7 (depends on 1)

The method of claim 1, further comprising: after determining that the mechanical arm picks up the target object from the turnover box,
determining an initial position of the target object based on position and attitude information of an end of the mechanical arm; and
determining the height of the target object based on the initial position of the target object.

### Claim 8 (depends on 7)

The method of claim 7, wherein performing the target tracking on the target object comprises:
performing, based on the initial position of the target object, the target tracking on the target object, to obtain the real-time position information of the target object.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
when the area where the landing point of the target object is located is in a destination turnover box, determining that a sorting of the target object succeeds; and
when the area where the landing point of the target object is located is in a source turnover box or another area, determining that the sorting of the target object fails, wherein the another area is an area other than the destination turnover box and the source turnover box.

### Claim 10

An electronic device for target object detection, comprising:
a processor; and
a memory storing instructions executable by the processor,
wherein the processor is configured to:
in response to determining that a mechanical arm picks up a target object from any one of a plurality of turnover boxes and a height of the target object is greater than a height of the turnover box, perform target tracking on the target object to obtain real-time position information of the target object; and
in response to determining, based on the real-time position information of the target object, that the target object falls down from the mechanical arm, determine an area where a landing point of the target object is located based on current position information of the target object and area position information, wherein the area position information comprises information of areas where the plurality of turnover boxes are located.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein the processor is further configured to:
for each turnover box of the plurality of turnover boxes, determine a type of the turnover box based on an attribute of the turnover box, and obtain type labeling information of the turnover box based on the type of the turnover box; and
obtain the area position information based on type labeling information of all of the plurality of turnover boxes.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein the attribute of the turnover box comprises at least one of: color, texture, shape, or size.

### Claim 13 (depends on 10)

The electronic device of claim 10, wherein during performing the target tracking on the target object, the processor is further configured to:
capture, by using an image capturing device, the target object for multiple times to obtain a plurality of captured images; and perform the target tracking on the target object based on each of the plurality of captured images.

### Claim 14 (depends on 13)

The electronic device of claim 13, wherein during performing the target tracking on the target object based on each of the plurality of captured images, the processor is further configured to:
eliminate a background in each of the plurality of captured images to obtain a respective result after background elimination of the captured image; and
perform the target tracking on the target object based on the respective result after background elimination of the captured image, wherein a background of the target object represents an image of a preset background object.

### Claim 15 (depends on 13)

The electronic device of claim 13, wherein the processor is further configured to:
after determining the area where the landing point of the target object is located, determine a position of the landing point of the target object based on a relative position relationship between the image capturing device and the mechanical arm, and position and attitude information of an end of the mechanical arm.

### Claim 16 (depends on 10)

The electronic device of claim 10, wherein the processor is further configured to: after determining that the mechanical arm picks up the target object from the turnover box,
determine an initial position of the target object based on position and attitude information of an end of the mechanical arm; and
determine the height of the target object based on the initial position of the target object.

### Claim 17 (depends on 16)

The electronic device of claim 16, wherein during performing the target tracking on the target object, the processor is further configured to:
perform, based on the initial position of the target object, the target tracking on the target object, to obtain the real-time position information of the target object.

### Claim 18 (depends on 10)

The electronic device of claim 10, wherein the processor is further configured to:
when the area where the landing point of the target object is located is in a destination turnover box, determine that a sorting of the target object succeeds; and
when the area where the landing point of the target object is located is in a source turnover box or another area, determine that the sorting of the target object fails, wherein the another area is an area other than the destination turnover box and the source turnover box.

### Claim 19

(canceled)

### Claim 20

A computer storage medium having stored thereon a computer program, which, when executed by a processor, causes the processor to perform operations comprising:
in response to determining that a mechanical arm picks up a target object from any one of a plurality of turnover boxes and a height of the target object is greater than a height of the turnover box, performing target tracking on the target object to obtain real-time position information of the target object; and
in response to determining, based on the real-time position information of the target object, that the target object falls down from the mechanical arm, determining an area where a landing point of the target object is located based on current position information of the target object and area position information, wherein the area position information comprises information of areas where the plurality of turnover boxes are located.

### Claim 21

(canceled)

### Claim 22 (depends on 20)

The computer storage medium of claim 20, wherein the processor is further configured to:
after determining the area where the landing point of the target object is located, determine a position of the landing point of the target object based on a relative position relationship between the image capturing device and the mechanical arm, and position and attitude information of an end of the mechanical arm.
