# US 11,991,944 B2 · Computer vision system and method for agriculture

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,991,944 B2
- Document ID: US11991944B2
- Status: Granted
- Kind: B2
- Application number: 17/306,202
- Filed: 2021-05-03
- Published: 2022-11-03
- Granted: 2024-05-28
- Earliest claimed: 2021-05-03
- Assignees: Crop Sentry Ltd.
- Applicants: Crop Sentry Ltd.
- Inventors: Christopher Brock Porth; Lysa M. Porth; Ehsan Tahmasebian; Vahab Khoshdel; Amin Hajihosseinlu
- Web page: https://patentgrep.com/patent/US11991944B2

## CPC

- A01B 79/005: {Precision agriculture}
- G01C 15/002: {Active optical surveying means (optical plumbing G01C15/105)}
- G01C 15/06: Surveyors' staffs; Movable markers
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/88: Lidar systems specially adapted for specific applications
- G01S 17/89: for mapping or imaging
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- H04N 23/531: {being rotatable or detachable}
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30188: Vegetation; Agriculture
- Y02A 40/10: in agriculture
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

There is provided a computer vision monitoring system and method for use in agriculture. The system has a frame; a battery mounted to the frame and providing electrical power; a Light Detection and Ranging (LiDAR) sensor mounted to the frame; a camera mounted to the frame; an enclosure mounted to the frame, the enclosure housing a processing structure. The processing structure executes instructions such as capturing point data from the LiDAR sensor; capturing image data from the camera; generating a 3D color-assigned cloud point data set from the image data and the point data; and determining crop characteristic data. The method involves similar steps as executed by the processing structure.

## Claims

### Claim 1

A crop monitoring system for a field comprising:
a frame;
a battery mounted to the frame and providing electrical power;
a Light Detection and Ranging (LiDAR) sensor mounted to the frame;
a camera mounted to the frame;
an enclosure mounted to the frame, the enclosure housing a processing structure; the processing structure executing a plurality of instructions stored on a tangible computer-readable memory;
the instructions comprise:
capturing point data from the LiDAR sensor;
capturing image data from the camera;
generating a 3D color-assigned cloud point data set from the image data and the point data; and
determining crop characteristic data based on the 3D color-assigned cloud point data set.

### Claim 2 (depends on 1)

The crop monitoring system according to claim 1, further comprising: a rotating platform mounted on the frame; the rotating platform supporting the LiDAR sensor and the camera; and the instructions further comprise: rotating the rotating platform before the capturing of the point data and the image data.

### Claim 3 (depends on 2)

The crop monitoring system according to claim 2, wherein the instructions further comprise: determining a field of view (FOV) of the LiDAR sensor; determining a rotational measurement associated with the FOV of the LiDAR sensor; and rotating the rotating platform by the rotational measurement before the capturing of the point data.

### Claim 4 (depends on 1)

The crop monitoring system according to claim 1, wherein the instructions further comprise: filtering the 3D color-assigned cloud point data set prior to determining the crop characteristic data.

### Claim 5 (depends on 1)

The crop monitoring system according to claim 1, wherein the instructions further comprise: filtering the point data set prior to generating the 3D color-assigned cloud point data set.

### Claim 6 (depends on 1)

The crop monitoring system according to claim 1, wherein the instructions further comprise: segmenting crop data points from the 3D color-assigned cloud point data set as part of the determining of the crop characteristic data.

### Claim 7 (depends on 1)

The crop monitoring system according to claim 1, wherein the processing structure comprises a graphical processing unit (GPU).

### Claim 8 (depends on 1)

The crop monitoring system according to claim 1, further comprising: a solar panel providing electrical charge to the battery.

### Claim 9 (depends on 8)

The crop monitoring system according to claim 8, wherein the instructions further comprise: a battery management system placing the processing structure in a standby state when the battery management system determines that the battery falls close to 80% of full capacity.

### Claim 10 (depends on 1)

The crop monitoring system according to claim 1, further comprising: at least one sensor providing at least one sensor measurement to the processing structure; and the instructions further comprise: determining at least one parameter to adjust the crop characteristic data.

### Claim 11 (depends on 10)

The crop monitoring system according to claim 10, wherein the at least one sensor is selected from at least one of: a temperature sensor, a humidity sensor, a pressure sensor, a soil moisture sensor, a soil resistance sensor, a nitrogen/phosphorus/potassium (NPK) sensor, a rain gauge, and a solar radiation sensor.

### Claim 12

A method for monitoring a field comprises:
capturing point data from a Light Detection and Ranging (LiDAR) sensor;
capturing image data from a camera;
generating, using a processor, a 3D color-assigned cloud point data set from the image data and the point data; and
determining, using the processor, crop characteristic data based on the 3D color-assigned cloud point data set.

### Claim 13 (depends on 12)

The method of claim 12 further comprise: rotating a rotating platform supporting the LiDAR sensor and the camera before the capturing of the point data and the image data.

### Claim 14 (depends on 13)

The method of claim 13 further comprise: determining a field of view (FOV) of the LiDAR sensor; determining a rotational measurement associated with the FOV of the LiDAR sensor; and rotating the rotating platform by the rotational measurement before the capturing of the point data.

### Claim 15 (depends on 12)

The method of claim 12 further comprise: filtering, by the processor, the 3D color-assigned cloud point data set prior to determining the crop characteristic data.

### Claim 16 (depends on 12)

The method of claim 12 further comprise: filtering, by the processor, the point data set prior to generating the 3D color-assigned cloud point data set.

### Claim 17 (depends on 12)

The method of claim 12 further comprise: segmenting, by the processor, crop data points from the 3D color-assigned cloud point data set as part of the determining of the crop characteristic data.

### Claim 18 (depends on 12)

The method of claim 12, wherein the processor comprises a graphical processing unit (GPU).

### Claim 19 (depends on 12)

The method of claim 12 further comprise: placing the processor in a standby state when a battery management system determines that the battery falls close to 80% of full capacity.

### Claim 20 (depends on 12)

The method of claim 12, further comprise: reading at least one sensor measurement from at least one sensor; and determining at least one parameter from the at least one sensor measurement to adjust the crop characteristic data.

### Claim 21 (depends on 20)

The method of claim 20, wherein the at least one sensor is selected from at least one of: a temperature sensor, a humidity sensor, a pressure sensor, a soil moisture sensor, a soil resistance sensor, a nitrogen/phosphorus/potassium (NPK) sensor, a rain gauge, and a solar radiation sensor.