# US 11,111,785 B2 · Method and device for acquiring three-dimensional coordinates of ore based on mining process

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,111,785 B2
- Document ID: US11111785B2
- Status: Granted
- Kind: B2
- Application number: 16/806,714
- Filed: 2020-03-02
- Published: 2021-03-04
- Granted: 2021-09-07
- Earliest claimed: 2019-09-04
- Assignees: Wuyi University
- Applicants: Wuyi University
- Inventors: Junying Zeng; Xuhua Li; Chuanbo Qin; Kaitian Wei; Fan Wang; Xiaowei Jiang; Weizhao He; Junhua He
- Web page: https://patentgrep.com/patent/US11111785B2

## CPC

- E21C 39/00: Devices for testing in situ the hardness or other properties of minerals, e.g. for giving information as to the selection of suitable mining tools
- E21C 41/22: for ores, e.g. mining placers
- G01V 8/02: Prospecting
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/82: using neural networks
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06V 10/56: relating to colour
- G06V 2201/07: Target detection
- E21C: MINING OR QUARRYING
- G01V: GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS (means for indicating the location of accidentally buried, e.g. snow-buried, persons A63B29/02)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method and a device for acquiring three-dimensional coordinates of ore based on mining process are disclosed. The method includes: obtaining a two-dimensional coordinate of the ore by using a YOLACT algorithm and a NMS algorithm to obtain a prediction mask map, obtaining depth information of the ore based on the color map and the infrared depth map, and combining the two-dimensional coordinate with the depth information to obtain a three-dimensional coordinate of the ore.

## Claims

### Claim 1

A method for acquiring three-dimensional coordinates of ore based on mining process, comprising:
acquiring an ore image;
processing the ore image by using a YOLACT algorithm and a NMS algorithm to obtain a prediction mask map;
drawing a rectangular box according to the prediction mask map and obtaining a two-dimensional coordinate of ore through a center point of the rectangular box;
acquiring a color map and an infrared depth map;
transmitting the color map into a pre-training model for recognition to obtain object contours;
selecting a contour of a target object, calculating the sum of coordinates of X axis and Y axis of all points on the contour, and then dividing the sum of the coordinates by the number of the points on the contour to obtain a gravity center of the target object;
calculating a distance between the gravity center of the target object and an origin to obtain a polar coordinate of the target object, and aligning a center point of the color map and a center point of the infrared depth map through affine transformation to obtain a scale of the color map in the infrared depth map;
multiplying a length of the polar coordinate by the scale to obtain a point in the infrared depth map corresponding to a point in the color map, thereby obtaining depth information of the point in the infrared depth map; and
combining the two-dimensional coordinate with the depth information to obtain a three-dimensional coordinate of the ore.

### Claim 2

An apparatus for acquiring three-dimensional coordinates of ore based on mining process, comprising: at least one control processor and a memory for communicating with the at least one control processor, wherein the memory stores instructions executable by the at least one control processor, and the instructions, when executed by the at least one control processor, cause the at least one control processor to perform steps of:
acquiring an ore image;
processing the ore image by using a YOLACT algorithm and a NMS algorithm to obtain a prediction mask map;
drawing a rectangular box according to the prediction mask map and obtaining a two-dimensional coordinate of ore through a center point of the rectangular box;
acquiring a color map and an infrared depth map;
transmitting the color map into a pre-training model for recognition to obtain object contours;
selecting a contour of a target object, calculating the sum of coordinates of X axis and Y axis of all points on the contour, and then dividing the sum of the coordinates by the number of the points on the contour to obtain a gravity center of the target object;
calculating a distance between the gravity center of the target object and an origin to obtain a polar coordinate of the target object, and aligning a center point of the color map and a center point of the infrared depth map through affine transformation to obtain a scale of the color map in the infrared depth map;
multiplying a length of the polar coordinate by the scale to obtain a point in the infrared depth map corresponding to a point in the color map, thereby obtaining depth information of the point in the infrared depth map; and
combining the two-dimensional coordinate with the depth information to obtain a three-dimensional coordinate of the ore.

### Claim 3

A computer readable storage medium having computer executable instructions stored thereon which, when executed by a computer, cause the computer to perform steps of:
acquiring an ore image;
processing the ore image by using a YOLACT algorithm and a NMS algorithm to obtain a prediction mask map;
drawing a rectangular box according to the prediction mask map and obtaining a two-dimensional coordinate of ore through a center point of the rectangular box;
acquiring a color map and an infrared depth map;
transmitting the color map into a pre-training model for recognition to obtain object contours;
selecting a contour of a target object, calculating the sum of coordinates of X axis and Y axis of all points on the contour, and then dividing the sum of the coordinates by the number of the points on the contour to obtain a gravity center of the target object;
calculating a distance between the gravity center of the target object and an origin to obtain a polar coordinate of the target object, and aligning a center point of the color map and a center point of the infrared depth map through affine transformation to obtain a scale of the color map in the infrared depth map;
multiplying a length of the polar coordinate by the scale to obtain a point in the infrared depth map corresponding to a point in the color map, thereby obtaining depth information of the point in the infrared depth map; and
combining the two-dimensional coordinate with the depth information to obtain a three-dimensional coordinate of the ore.