Obstacle detection system, agricultural work vehicle, obstacle detection program, recording medium on which obstacle detection program is recorded, and obstacle detection method

An agricultural work vehicle including an obstacle detection system, comprising:

• an obstacle sensor configured to detect an obstacle in a surrounding region of the agricultural work vehicle;
• an obstacle estimation unit configured to estimate a region based on a detection signal from the obstacle sensor in a region which the obstacle is present and output obstacle present region information;
• an image capturing unit configured to capture an image of a surrounding region of the agricultural work vehicle and output the captured image;
• an image preprocessing unit configured to generate a trimmed image based on the obstacle present region information and shooting-angle-of-view information regarding the image capturing unit which trimmed image is obtained by trimming the captured image so as to include the region in which the obstacle is present;
• and
• an obstacle detection unit configured to receive the trimmed image as an input image and output obstacle detection information that includes a result of the detection of the obstacle,
• wherein the obstacle sensor is a scanning sensor that uses ultrasonic beams, optical beams, or electromagnetic wave beams, and outputs the detection signal based on reflection beams that return to the obstacle sensor as a result of transmitted beams reflecting off a reflection body,
• wherein the obstacle estimation unit calculates a three-dimensional position of the obstacle from the detection signal with use of the reflection body as the obstacle, and outputs the obstacle present region information including the three-dimensional position,
• wherein the obstacle detection unit is configured as a machine-learned neural network, and
• wherein the obstacle detection information output based on the input image includes a type of the obstacle.

A non-transitory computer readable storage medium comprising an obstacle detection program for an agricultural work vehicle that, when executed by a computer comprising at least one processor, causes the computer to perform:

• an obstacle estimation function of, based on a detection signal from an obstacle sensor that detects an obstacle in a surrounding region of the agricultural work vehicle, estimating a region in which the obstacle is present, and outputting obstacle present region information;
• an image preprocessing function of generating a trimmed image, which is obtained by trimming a captured image of a surrounding region of the agricultural work vehicle so as to include the region in which the obstacle is present, based on the captured image, shooting-angle-of-view information regarding an image capturing unit that captured the captured image, and the obstacle present region information;
• and
• an obstacle detection function of receiving the trimmed image as an input image and outputting obstacle detection information that includes a result of the detection of the obstacle,
• wherein the obstacle sensor is a scanning sensor that uses ultrasonic beams, optical beams, or electromagnetic wave beams, and outputs the detection signal based on reflection beams that return to the obstacle sensor as a result of transmitted beams reflecting off a reflection body,
• wherein an obstacle estimation unit calculates a three-dimensional position of the obstacle from the detection signal with use of the reflection body as the obstacle, and outputs the obstacle present region information including the three-dimensional position,
• wherein the obstacle detection function is configured as a machine-learned neural network, and
• wherein the obstacle detection information output based on the input image includes a type of the obstacle.

A non-transitory computer readable recording medium comprising an obstacle detection program for an agricultural work vehicle,

• the obstacle detection program, when executed by a computer comprising at least one processor, causes the computer to perform:
• an obstacle estimation function of estimating a region based on a detection signal from an obstacle sensor that detects an obstacle in a surrounding region of the agricultural work vehicle in which region the obstacle is present, and outputting obstacle present region information;
• an image preprocessing function of generating a trimmed image, which is obtained by trimming a captured image of a surrounding region of the agricultural work vehicle so as to include the region in which the obstacle is present, based on the captured image, shooting-angle-of-view information regarding an image capturing unit that captured the captured image, and the obstacle present region information;
• and
• an obstacle detection function of receiving the trimmed image as an input image and outputting obstacle detection information that includes a result of the detection of the obstacle,
• wherein the obstacle sensor is a scanning sensor that uses ultrasonic beams, optical beams, or electromagnetic wave beams, and outputs the detection signal based on reflection beams that return to the obstacle sensor as a result of transmitted beams reflecting off a reflection body,
• wherein an obstacle estimation unit calculates a three-dimensional position of the obstacle from the detection signal with use of the reflection body as the obstacle, and outputs the obstacle present region information including the three-dimensional position,
• wherein the obstacle detection function is configured as a machine-learned neural network, and
• wherein the obstacle detection information output based on the input image includes a type of the obstacle.

An obstacle detection method for an agricultural work vehicle, comprising:

• estimating a region based on a detection signal from an obstacle sensor that detects an obstacle in a surrounding region of the agricultural work vehicle in which region the obstacle is present, and outputting obstacle present region information;
• generating a trimmed image, which is obtained by trimming a captured image of a surrounding region of the agricultural work vehicle so as to include the region in which the obstacle is present, based on the captured image, shooting-angle-of-view information regarding an image capturing unit that captured the captured image, and the obstacle present region information;
• and
• receiving the trimmed image as an input image and outputting obstacle detection information that includes a result of the detection of the obstacle,
• wherein the obstacle sensor is a scanning sensor that uses ultrasonic beams, optical beams, or electromagnetic wave beams, and outputs the detection signal based on reflection beams that return to the obstacle sensor as a result of transmitted beams reflecting off a reflection body,
• wherein an obstacle estimation unit calculates a three-dimensional position of the obstacle from the detection signal with use of the reflection body as the obstacle, and outputs the obstacle present region information including the three-dimensional position,
• wherein a machine-learned neural network is used for the receiving of the trimmed image and the outputting of the obstacle detection information, and
• wherein the obstacle detection information output based on the input image includes a type of the obstacle.