# US 2021/0240965 A1 · System and method for retrieving bales from a field

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240965 A1
- Document ID: US20210240965A1
- Status: Published application
- Kind: A1
- Application number: 16/776,788
- Filed: 2020-01-30
- Published: 2021-08-05
- Earliest claimed: 2020-01-30
- Assignees: Deere & Company
- Applicants: Deere & Company
- Inventors: MILAN YADAV; SATYAM NALAWADE; DHANASHREE DESAI; JERRY SAMUEL
- Web page: https://patentgrep.com/patent/US20210240965A1

## CPC

- G08G 5/55: for a single aircraft
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G06V 20/13: Satellite images
- G08G 5/26: Transmission of traffic-related information between aircraft and ground stations
- G08G 5/57: for unmanned aircraft
- G08G 5/74: for monitoring terrain
- B64U 10/13: Flying platforms
- B64U 2101/30: for imaging, photography or videography
- H04W 4/02: Services making use of location information
- H04W 4/029: Location-based management or tracking services
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A bale retrieval system includes an aerial vehicle operable to fly over a field. An imaging device is mounted on the aerial vehicle. The imaging device is operable to capture an image of the field. A position locater is operable to determine a position of multiple bales located on the field. A controller is operable to identify the bales included in images captured by the imaging device, and then determine a location of each bale identified in the image with the position locator. The location of each bale may be used to generate a map showing the bales in the field. The location of the bales may then be communicated to a bale retriever for pickup.

## Claims

### Claim 1

A bale retrieval system comprising:
an aerial vehicle operable to fly over a field;
an imaging device mounted on the aerial vehicle and operable to capture an image of the field;
a position locater operable to determine a position on the field;
a controller in communication with the imaging device and the position locater, the controller including a processor and a memory having a bale mapping algorithm stored thereon, wherein the processor is operable to execute the bale mapping algorithm to:
identify a bale included in an image captured by the imaging device;
determine a location of the bale identified in the image with the position locator; and
communicate the location of the bale in the field to a bale retriever.

### Claim 2 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the aerial vehicle is one of a manned aerial vehicle, an un-manned aerial vehicle, or a satellite.

### Claim 3 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the imaging device is positioned to face in a substantially downward vertical direction.

### Claim 4 (depends on 3)

The bale retrieval system set forth in claim 3, wherein the aerial vehicle includes a gimbal supporting the imaging device.

### Claim 5 (depends on 3)

The bale retrieval system set forth in claim 3, wherein the position locator is mounted to the aerial vehicle and is operable to determine a location of the aerial vehicle over the field such that the location of the aerial vehicle at the time the image including the bale is captured substantially corresponds to the position of the bale in the field.

### Claim 6 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the processor is operable to execute the bale mapping algorithm to generate a map of the field, wherein the map of the field indicates the location of the bale in the field.

### Claim 7 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the processor is operable to execute the bale mapping algorithm to calculate a retrieval sequence for retrieving multiple bales in the field, and communicate the retrieval sequence to the retriever.

### Claim 8 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the aerial vehicle is configured to operate autonomously within a perimeter of the field to identify and locate all bales located within the perimeter of the field.

### Claim 9 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the imaging device includes a camera.

### Claim 10 (depends on 1)

The bale retrieval system set forth in claim 1, wherein the position locator includes a Global Positioning Satellite (GPS) system.

### Claim 11

A bale retrieval system comprising:
an un-manned aerial vehicle operable to fly over a field;
an camera mounted on the un-manned aerial vehicle and operable to capture an image of the field;
a GPS receiver mounted on the un-manned aerial vehicle and operable to determine a position of the un-manned aerial vehicle over the field at the time the imaging device captures an image;
a controller in communication with the camera and the GPS receiver, the controller including a processor and a memory having a bale mapping algorithm stored thereon, wherein the processor is operable to execute the bale mapping algorithm to:
identify a bale included in an image captured by the camera;
determine a location of the bale identified in the image with the GPS receiver; and
communicate the location of the bale in the field to a bale retriever.

### Claim 12 (depends on 11)

The bale retrieval system set forth in claim 11, wherein the camera includes a lens positioned to face in a substantially downward vertical direction.

### Claim 13 (depends on 11)

The bale retrieval system set forth in claim 11, wherein the un-manned aerial vehicle includes a gimbal supporting the camera.

### Claim 14 (depends on 11)

The bale retrieval system set forth in claim 11, wherein the processor is operable to execute the bale mapping algorithm to generate a map of the field, wherein the map of the field indicates the location of the bale in the field.

### Claim 15 (depends on 11)

The bale retrieval system set forth in claim 11, wherein the processor is operable to execute the bale mapping algorithm to calculate a retrieval sequence for retrieving multiple bales in the field, and communicate the retrieval sequence to the retriever.

### Claim 16 (depends on 11)

The bale retrieval system set forth in claim 11, wherein the un-manned aerial vehicle is configured to operate autonomously within a perimeter of the field to identify and locate all bales located within the perimeter of the field.

### Claim 17

A method of retrieving formed bales of crop material from a field, the method comprising:
providing an aerial vehicle having an imaging device, a bale identifier, and a position locator;
flying the aerial vehicle over the field;
capturing an image of the field, with the imaging device;
identifying a bale in the captured image of the field, with the bale identifier;
determining a location of the identified bale with the position locator; and
communicating the location of the bale to a retriever.

### Claim 18 (depends on 17)

The method set forth in claim 17, wherein the imaging device is positioned to face and maintain a substantially downward vertical direction, and wherein capturing the image of the field is further defined as capturing the image of the field directly beneath the aerial vehicle.

### Claim 19 (depends on 17)

The method set forth in claim 17, wherein the position locator is operable to determine a position of the aerial vehicle at the time the image is captured, and wherein determining a location of the identified bale with the position locator is further defined as determining a position of the aerial vehicle with the position locator at the time the image is captured.

### Claim 20 (depends on 17)

The method set forth in claim 17, further comprising generating a map of the field showing the location of the bale thereon, and communicating the map of the field to the retriever.