# US 2022/0030763 A1 · Moisture and vegetative health mapping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0030763 A1
- Document ID: US20220030763A1
- Status: Published application
- Kind: A1
- Application number: 17/498,303
- Filed: 2021-10-11
- Published: 2022-02-03
- Earliest claimed: 2018-01-24
- Assignees: Scythe Robotics, Inc.
- Applicants: Scythe Robotics, Inc.
- Inventors: John Gordon Morrison; Davis Thorp Foster; Isaac Heath Roberts
- Web page: https://patentgrep.com/patent/US20220030763A1

## CPC

- A01C 21/007: {Determining fertilization requirements}
- A01B 69/004: {automatic}
- A01B 79/005: {Precision agriculture}
- A01C 7/06: Seeders combined with fertilising apparatus
- A01C 7/102: {Regulating or controlling the seed rate}
- A01C 23/007: {Metering or regulating systems}
- A01M 7/0089: {Regulating or controlling systems (the delivery being related to the movement of a vehicle B05B9/06)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0257: {using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 17/05: Geographic models
- G06N 3/04: Architecture, e.g. interconnection topology
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10044: Radar image
- G06T 2207/20081: Training; Learning
- G06T 2207/30188: Vegetation; Agriculture
- Y02A 90/40: Monitoring or fighting invasive species
- A01C: PLANTING; SOWING; FERTILISING (parts, details or accessories of agricultural machines or implements, in general A01B51/00 - A01B75/00)
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- A01M: CATCHING, TRAPPING OR SCARING OF ANIMALS (appliances for catching swarms or drone-catching A01K57/00; fishing A01K69/00 - {A01K99/00}; biocides, pest repellants or attractants A01N); APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

A vegetative health mapping system which creates two- or three-dimensional maps and associates moisture content, soil density, ambient light, surface temperature, and/or additional indications of vegetative health with the map. Moisture content is inferred using radar return signals of near-field and/or far-field radar. By tuning various parameters of the one or more radar (e.g. frequency, focus, power), additional data may be associated with the map from subterranean features (such as rocks, soil density, sprinklers, etc.). Additional sensors (camera(s), lidar, IMU, GPS, etc.) may be fused with radar returns to generate maps having associated moisture content, surface temperature, ambient light levels, additional indications of vegetative health (as may be determined by machine learned algorithms), etc. Such vegetative health maps may be provided to a user who, in turn, may indicate additional areas for the vegetative health device to scan or otherwise used to recommend and/or perform treatments.

## Claims

### Claim 1

A device comprising:
one or more non-transitory computer readable media storing instructions, which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a plurality of coordinates that define a bounded region of interest;
determining, based at least in part on the plurality of coordinates, a trajectory to sweep the bounded region of interest using at least one of minimal energy or minimal motion;
receiving sensor data from a sensor associated with the device;
determining, based at least in part on a portion of the sensor data, information about vegetation proximate the device;
determining, based at least in part on the information about vegetation, a location of the device; and
controlling, based at least in part on the location, the device to move along the trajectory.

### Claim 2 (depends on 1)

The device of claim 1, wherein the device further comprises a coupling configured to one or more of:
couple the device to a ground vehicle, or
couple the device to a handle to be pushed by an operator.

### Claim 3 (depends on 1)

The device of claim 1, wherein the operations further comprise:
providing a second portion of sensor data to a machine learning model;
receiving, from the machine learning model, an indication of vegetative health associated with the location; and
associating the indication of vegetative health with a three-dimensional map of the bounded region of interest.

### Claim 4 (depends on 1)

The device of claim 1, wherein the operations further comprise:
determining an indication of vegetative health associated with the location;
determining a degradation of vegetative health;
determining an area associated with the degradation;
determining, as a treatment and based at least in part on the area, an amount of one or more of herbicide, pesticide, fertilizer, or seed; and
applying the treatment.

### Claim 5 (depends on 4)

The device of claim 4, wherein determining a degradation of vegetative health comprises:
receiving a previous indication of vegetative health associated with the location from one or more of a previously generated three-dimensional map or from third party data;
determining a difference between the previous indication and the indication; and
determining the difference meets or exceeds a threshold.

### Claim 6 (depends on 1)

The device of claim 1,
wherein the sensor comprises a near-field radar,
wherein the information about vegetation is determined based at least in part on the near- field radar, and
wherein the operations further comprise one or more of:
altering an intensity of the radar; or
altering a frequency of the radar.

### Claim 7 (depends on 1)

The device of claim 1, wherein the operations further comprise transmitting a signal to a user device to cause the user device to display a three-dimensional map of the bounded region of interest,
wherein one or more of the information about vegetation or additional vegetative health indicators are displayed to the user on the user device over the three-dimensional map, and
wherein the plurality of coordinates are received from the user device.

### Claim 8 (depends on 1)

The device of claim 1, wherein a three-dimensional map of the bounded region of interest is determined based at least in part on sensor data from the sensor.

### Claim 9 (depends on 1)

The device of claim 1, wherein the information about vegetation is determined through semantic segmentation of the sensor data and comprises one or more of:
moisture content;
soil density;
an indication of no vegetation where vegetation is expected;
areas of discoloration of vegetation with respect to an expected coloration; or
a deviation of sensor returns as compared to an expected return associated with the area.

### Claim 10

A method for determining information about vegetation and controlling a device through a bounded region of interest using information determined about vegetation, the method comprising:
receiving a plurality of coordinates that define a bounded region of interest;
determining, based at least in part on the plurality of coordinates, a trajectory to sweep the bounded region of interest using at least one of minimal energy or minimal motion;
receiving sensor data from a sensor associated with the device;
determining, based at least in part on a portion of the sensor data, information about vegetation proximate the device;
determining, based at least in part on the information about vegetation, a location of the device; and
controlling, based at least in part on the location, the device to move along the trajectory.

### Claim 11 (depends on 10)

The method of claim 10, wherein the method further comprises:
providing a second portion of sensor data to a machine learning model;
receiving, from the machine learning model, an indication of vegetative health associated with the location; and
associating the indication of vegetative health with a three-dimensional map of the bounded region of interest.

### Claim 12 (depends on 10)

The method of claim 10, wherein determining a degradation of vegetative health comprises:
determining an indication of vegetative health associated with the location;
determining a degradation of vegetative health;
determining an area associated with the degradation;
determining, as a treatment and based at least in part on the area, an amount of one or more of herbicide, pesticide, fertilizer, or seed; and
applying the treatment.

### Claim 13

One or more non-transitory computer readable media having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a plurality of coordinates that define a bounded region of interest;
determining, based at least in part on the plurality of coordinates, a trajectory to sweep the bounded region of interest using at least one of minimal energy or minimal motion;
receiving sensor data from a sensor associated with a device;
determining, based at least in part on a portion of the sensor data, information about vegetation proximate the device;
determining, based at least in part on the information about vegetation, a location of the device; and
controlling, based at least in part on the location, the device to move along the trajectory.

### Claim 14 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the operations further comprise:
providing a second portion of sensor data to a machine learning model;
receiving, from the machine learning model, an indication of vegetative health associated with the location; and
associating the indication of vegetative health with a three-dimensional map of the bounded region of interest.

### Claim 15 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein determining a degradation of vegetative health comprises:
determining an indication of vegetative health associated with the location;
determining a degradation of vegetative health;
determining an area associated with the degradation;
determining, as a treatment and based at least in part on the area, an amount of one or more of herbicide, pesticide, fertilizer, or seed; and
applying the treatment.

### Claim 16 (depends on 15)

The one or more non-transitory computer readable media of claim 15, wherein determining a degradation of vegetative health comprises:
receiving a previous indication of vegetative health associated with the location from one or more of a previously generated three-dimensional map or from third party data;
determining a difference between the previous indication and the indication; and
determining the difference meets or exceeds a threshold.

### Claim 17 (depends on 13)

The one or more non-transitory computer readable media of claim 13,
wherein the sensor comprises a near-field radar,
wherein the information about vegetation is determined based at least in part on the near-field radar, and
wherein the operations further comprise one or more of:
altering an intensity of the radar; or
altering a frequency of the radar.

### Claim 18 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the operations further comprise transmitting a signal to a user device to cause the user device to display a three-dimensional map of the bounded region of interest,
wherein one or more of the information about vegetation or additional vegetative health indicators are displayed to the user on the user device over the three-dimensional map, and
wherein the plurality of coordinates are received from the user device.

### Claim 19 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein a three-dimensional map of the bounded region of interest is determined based at least in part on sensor data from the sensor.

### Claim 20 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the information about vegetation is determined through semantic segmentation of the sensor data and comprises one or more of:
moisture content;
soil density;
an indication of no vegetation where vegetation is expected;
areas of discoloration of vegetation with respect to an expected coloration; or
a deviation of sensor returns as compared to an expected return associated with the area.
