# US 12,178,155 B2 · System and method of assisted or automated grain unload synchronization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,178,155 B2 - Document ID: US12178155B2 - Status: Granted - Kind: B2 - Application number: 17/377,357 - Filed: 2021-07-15 - Published: 2022-01-20 - Granted: 2024-12-31 - Earliest claimed: 2020-07-17 - Assignees: AGCO International GmbH - Applicants: AGCO International GmbH - Inventors: Martin Peter Christiansen; Ramon Buchaca Tarragona; Morten Leth Bilde - Web page: https://patentgrep.com/patent/US12178155B2 ## CPC - A01B 69/008: {automatic} - A01D 41/127: Control or measuring arrangements specially adapted for combines - A01D 90/10: Unloading means - G06T 7/73: using feature-based methods - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/30204: Marker - A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21) - A01D: HARVESTING; MOWING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An agricultural harvester includes a crop processor for reducing crop material to processed crop, an unload conveyor for transferring a stream of processed crop out of the harvester, and a camera for capturing images of an area proximate the harvester and generating image data from the captured images. One or more computing devices are configured for receiving the image data from the camera, identifying, from the image data, a visual marker corresponding to a receiving vehicle, and determining, from the visual marker, a location of the receiving vehicle relative to the agricultural harvester. An electronic device presents, on a graphical user interface of the device, a graphical representation of the relative locations of the unload conveyor and at least a portion of the receiving vehicle, the graphical representation based on the location of the receiving vehicle relative to the agricultural harvester determined by the one or more computing devices. ## Claims ### Claim 1 A system comprising: an agricultural harvester comprising: a crop processor for reducing crop material to processed crop; an unload conveyor for transferring a stream of processed crop out of the agricultural harvester; a first controller; and a camera for capturing images of an area proximate the agricultural harvester and generating image data from the captured images; a receiving vehicle comprising: a grain bin for storing processed crop received from the agricultural harvester; a visual marker; and a second controller, wherein the first controller is configured to: receive the image data from the camera; identify, from the image data, a visual marker corresponding to the receiving vehicle; and determine, from the visual marker, a location of the receiving vehicle relative to the agricultural harvester; and wherein the second controller is configured to: receive the location data from the first controller; and use the location data to generate a graphical representation of the relative locations of the unload conveyor and at least a portion of the receiving vehicle, the graphical representation based on the location data; and an electronic device including a graphical user interface, the electronic device for presenting, on the graphical user interface, the graphical representation of the relative locations of the unload conveyor and at least a portion of the receiving vehicle. ### Claim 2 (depends on 1) The system as set forth in claim 1, wherein the first controller is further configured to: identify, from the image data, a plurality of visual markers corresponding to the receiving vehicle; determine, from the plurality of visual markers, the location of the receiving vehicle relative to the agricultural harvester; and determine the location of the receiving vehicle using the plurality of visual markers. ### Claim 3 (depends on 2) The system as set forth in claim 2, wherein the controller is further configured to: associate a first one of the plurality of visual markers with a front of the receiving vehicle to determine a location of the front of the receiving vehicle relative to the agricultural harvester; associate a second one of the plurality of visual markers with a rear of the receiving vehicle to determine a location of the rear of the receiving vehicle relative to the agricultural harvester; and determine the location of the receiving vehicle relative to the agricultural harvester using the locations of the front and the rear of the receiving vehicle. ### Claim 4 (depends on 2) The system as set forth in claim 2, wherein the controller is further configured to: associate a first one of the plurality of visual markers with a first corner of the receiving vehicle to determine a location of the first corner relative to the agricultural harvester; associate a second one of the plurality of visual markers with a second corner of the receiving vehicle to determine a location of the second corner relative to the agricultural harvester; and determine the location of the receiving vehicle relative to the agricultural harvester using the locations of the first and second corners of the receiving vehicle. ### Claim 5 (depends on 4) The agricultural harvester as set forth in claim 4, wherein the controller is further configured to: associate a third one of the plurality of visual markers with a third corner of the receiving vehicle to determine a location of the third corner relative to the agricultural harvester, and determine the location of the receiving vehicle relative to the agricultural harvester using the locations of the first, second and third corners of the receiving vehicle. ### Claim 6 (depends on 1) The agricultural harvester as set forth in claim 1, wherein the controller is further configured to determine the location of the receiving vehicle relative to the agricultural harvester by identify a pattern on the visual marker and using the pattern to determine a distance between the agricultural harvester and the receiving vehicle. ### Claim 7 A system comprising: an agricultural harvester comprising: a crop processor for reducing crop material to processed crop; an unload conveyor for transferring a stream of processed crop out of the agricultural harvester; a first controller; a first communications gateway for sending and receiving wireless signals; and a camera for capturing images of an area proximate the agricultural harvester and generating image data; a receiving vehicle comprising: a grain bin for storing processed crop received from the agricultural harvester; a visual marker; a second controller; and a second communications gateway for sending and receiving wireless signals, wherein the first controller is configured to: receive the image data from the camera; identify, from the image data, the visual marker; generate, from the visual marker, location data indicating a location of the receiving vehicle relative to the agricultural harvester; and communicate the location data to the receiving vehicle using the first communications gateway, and wherein the second controller is configured to: receive the location data using the second communications gateway, and use the location data to generate a graphical representation of the relative locations of the unload conveyor and at least a portion of the receiving vehicle, the graphical representation based on the location data. ### Claim 8 (depends on 7) The system as set forth in claim 7, wherein: wherein the visual marker comprises a plurality of visual markers, and wherein the first controller is further configured to: identify, from the image data, the plurality of visual markers, and generate, from the plurality of visual markers, the location data. ### Claim 9 (depends on 8) The system as set forth in claim 8, wherein the first controller is further configured to: associate a first one of the plurality of visual markers with a front of the receiving vehicle to determine a location of the front of the receiving vehicle relative to the agricultural harvester; associate a second one of the plurality of visual markers with a rear of the receiving vehicle to determine a location of the rear of the receiving vehicle relative to the agricultural harvester; and generate the location data using the locations of the front and the rear of the receiving vehicle. ### Claim 10 (depends on 8) The system as set forth in claim 8, wherein the first controller is further configured to: associate a first one of the plurality of visual markers with a first corner of the receiving vehicle to determine a location of the first corner relative to the agricultural harvester; associate a second one of the plurality of visual markers with a second corner of the receiving vehicle to determine a location of the second corner relative to the agricultural harvester; and generate the location data using the locations of the first and second corners of the receiving vehicle. ### Claim 11 (depends on 10) The system as set forth in claim 10, wherein the first controller is further configured to: associate a third one of the plurality of visual markers with a third corner of the receiving vehicle to determine a location of the third corner relative to the agricultural harvester; and generate the location data using the locations of the first, second and third corners of the receiving vehicle. ### Claim 12 (depends on 7) The system as set forth in claim 7, wherein the first controller is further configured to generate the location data by identify a pattern on the visual marker and using the pattern to determine a distance between the agricultural harvester and the receiving vehicle. ### Claim 13 (depends on 7) The system as set forth in claim 7, wherein the agricultural harvester comprises a combine harvester. ### Claim 14 A method comprising: providing: an agricultural harvester comprising: a crop processor for reducing crop material to processed crop; an unload conveyor for transferring a stream of processed crop out of the agricultural harvester; a first controller; a first communications gateway for sending and receiving wireless signals; and a camera for capturing images of an area proximate the agricultural harvester and generating image data; a receiving vehicle comprising: a grain bin for storing processed crop received from the agricultural harvester; a visual marker; a second controller; and a second communications gateway for sending and receiving wireless signals, capturing images of an area proximate the agricultural harvester using the camera of the agricultural harvester and generating image data from the capture images; identify from the image data, using the first controller of the agricultural harvester, the visual marker of the receiving vehicle; determining from the visual marker, using the first controller, a location of the receiving vehicle relative to the agricultural harvester; and generating, using the second controller of the receiving vehicle, a graphical representation of the relative locations of the unload conveyor and at least a portion of the receiving vehicle, the graphical representation based on the location of the receiving vehicle relative to the agricultural harvester. ### Claim 15 (depends on 14) The method as set forth in claim 14, further comprising: identify from the image data, using the first controller, a plurality of visual markers of the receiving vehicle; determining from the plurality of visual markers, using the first controller, the location of the receiving vehicle relative to the agricultural harvester; and generating, using the second controller, automated navigation data based on the plurality of visual markers. ### Claim 16 (depends on 15) The method as set forth in claim 15, further comprising: associating, using the first controller, a first one of the plurality of visual markers with a front of the receiving vehicle to determine a location of the front of the receiving vehicle relative to the agricultural harvester; associating, using the first controller, a second one of the plurality of visual markers with a rear of the receiving vehicle to determine a location of the rear of the receiving vehicle relative to the agricultural harvester; and determining, using the first controller, the location of the receiving vehicle relative to the agricultural harvester using the locations of the front and the rear of the receiving vehicle. ### Claim 17 (depends on 15) The method as set forth in claim 15, further comprising: associating, using the first controller, a first one of the plurality of visual markers with a first corner of the receiving vehicle to determine a location of the first corner relative to the agricultural harvester; associating, using the first controller, a second one of the plurality of visual markers with a second corner of the receiving vehicle to determine a location of the second corner relative to the agricultural harvester; and determining, using the first controller, the location of the receiving vehicle relative to the agricultural harvester using the locations of the first and second corners of the receiving vehicle. ### Claim 18 (depends on 17) The method as set forth in claim 17, further comprising: associating, using the first controller, a third one of the plurality of visual markers with a third corner of the receiving vehicle to determine a location of the third corner relative to the agricultural harvester; and determining, using the first controller, the location of the receiving vehicle relative to the agricultural harvester using the locations of the first, second and third corners of the receiving vehicle. ### Claim 19 (depends on 14) The method as set forth in claim 14, further comprising determining, using the first controller, the location of the receiving vehicle relative to the agricultural harvester by identifying a pattern on the visual marker and using the pattern to determine a distance between the agricultural harvester and the receiving vehicle.