# US 2019/0244338 A1 · Maturity determination device and maturity determination method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244338 A1
- Document ID: US20190244338A1
- Status: Published application
- Kind: A1
- Application number: 16/379,843
- Filed: 2019-04-10
- Published: 2019-08-08
- Earliest claimed: 2016-07-05
- Assignees: Sharp Kabushiki Kaisha
- Applicants: Sharp Kabushiki Kaisha
- Inventors: Rieko OGAWA; Takashi NAKANO; Mitsuru NAKURA; Daiichiro NAKASHIMA
- Web page: https://patentgrep.com/patent/US20190244338A1

## CPC

- G06T 7/001: {using an image reference approach}
- G06T 7/0012: {Biomedical image inspection}
- A01D 46/30: Robotic devices for individually picking crops
- A01D 75/00: Accessories for harvesters or mowers
- A01D 91/04: Products growing above the soil (fruit, hops A01D46/00)
- G06T 2207/10024: Color image
- G06T 2207/10048: Infrared image
- G06T 2207/30004: Biomedical image processing
- G06T 2207/30128: Food products
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A01D: HARVESTING; MOWING

## Abstract

A maturity determination device includes an image capturing device including a plurality of pixels arrayed one-dimensionally or two-dimensionally, the image capturing device performing image capturing of at least a part of a fruit or vegetable product to acquire an image, the plurality of pixels including a plurality of first pixels each including a first light transmission filter selectively transmitting light of a first wavelength band, the intensity of the light of the first wavelength band reflected by the fruit or vegetable product varying in accordance with a maturity level; and a signal processing circuit configured to find an area size ratio of an intensity distribution of the light of the first wavelength band on the basis of a predetermined reference value based on a pixel value obtained from the plurality of first pixels, and to generate maturity determination information in accordance with the area size ratio.

## Claims

### Claim 1

A method for determining a maturity level of a fruit or vegetable product, the method comprising the steps of:
receiving an image including at least a part of the fruit or vegetable product, the image including an intensity distribution of light of at least a first wavelength band reflected by the fruit or vegetable product and varying in accordance with the maturity level;
finding an area size ratio of the intensity distribution of the light of the first wavelength band based on a predetermined reference value; and
generating maturity determination information in accordance with the area size ratio.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the first wavelength band is a wavelength band of near infrared light.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the first wavelength band is a wavelength band of near infrared light from 800 nm to 900 nm.

### Claim 4 (depends on 1)

The method according to claim 1, wherein:
in the step of finding the area size ratio, a number of pixels having a pixel value larger than, or equal to, a first threshold value or a pixel value smaller than, or equal to, the first threshold value is calculated from a plurality of first pixels including information representing the intensity distribution of the light of the first wavelength band and included in the image; and a first ratio of the calculated number of the pixels with respect to a number of the plurality of first pixels is determined; and
in the step of generating the maturity determination information, the maturity determination information is generated in accordance with a result of comparing the first ratio against a second threshold value.

### Claim 5 (depends on 1)

The method according to claim 1, wherein:
in the step of receiving the image, the image received further includes an intensity distribution of light of a second wavelength band reflected by the fruit or vegetable product and varying in accordance with the maturity level, the second wavelength band being different from the first wavelength band; and
in the step of generating the maturity determination information, the maturity determination information is generated based on the intensity distributions of the light of the first wavelength band and the light of the second wavelength band.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the second wavelength band is a wavelength band of red light.

### Claim 7 (depends on 5)

The method according to claim 5, wherein:
in the step of finding the area size ratio, a pixel value of each of the plurality of first pixels including information representing the intensity distribution of the light of the first wavelength band and included in the image, and a pixel value of each of the plurality of second pixels including information representing the intensity distribution of the light of the second wavelength band and included in the image, are added together on a pixel-by-pixel basis, said each second pixel being associated with said each first pixel; a number of pixels having a sum value larger than, or equal to, a third threshold value or a sum value smaller than, or equal to, the third threshold value is calculated from pixels as targets of addition; and a second ratio of the calculated number of the pixels with respect to the number of the pixels as the targets of addition is determined; and
in the step of generating the maturity determination information, the maturity determination information is generated in accordance with a result of comparing the second ratio with a fourth threshold value.

### Claim 8 (depends on 5)

The method according to claim 5, wherein:
in the step of finding the area size ratio, a pixel value of each of the plurality of second pixels including information representing the intensity distribution of the light of the second wavelength band and included in the image is subtracted, on a pixel-by-pixel basis, from a pixel value of each first pixel of the plurality of first pixels including information, representing the intensity distribution of the light of the first wavelength band, and included in the image, said each second pixel being associated with said each first pixel; a number of pixels having a difference value larger than, or equal to, a third threshold value or a difference value smaller than, or equal to, the third threshold value is calculated from pixels as targets of subtraction; and a third ratio of the calculated number of the pixels with respect to the number of the pixels as the targets of subtraction is determined; and
in the step of generating the maturity determination information, the maturity determination information is generated in accordance with a result of comparing the third ratio against a fourth threshold value.

### Claim 9 (depends on 5)

The method according to claim 5, wherein:
in the step of finding the area size ratio, a number of pixels having a pixel value larger than, or equal to, a fifth threshold value or a pixel value smaller than, or equal to, the fifth threshold value is calculated from the plurality of second pixels including information representing the intensity distribution of the light of the second wavelength band and included in the image; a fourth ratio of the calculated number of the pixels with respect to the number of the plurality of second pixels is determined; and the maturity determination information is generated in accordance with a result of comparing the first ratio against the second threshold value and a result of comparing the fourth ratio against a sixth threshold value.

### Claim 10 (depends on 6)

The method according to claim 6, wherein in the step of receiving the image, the image received further includes intensity distributions of blue light and green light;
the method further comprising generating a color image based on pixel values obtained from the plurality of pixels including information representing the intensity distributions of the red light, the blue light and the green light, and generating a maturity level image including information on the reference value and representing the maturity level, the maturity level image being generated based on the maturity determination information.

### Claim 11 (depends on 1)

The method according to claim 1, further comprising the step of outputting the maturity determination information to outside.

### Claim 12 (depends on 1)

The method according to claim 1, further comprising generating a driving signal to drive a notification device notifying the maturity level, the driving signal being generated in accordance with the maturity determination information.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the notification device includes at least one of an optical device to emit light in accordance with the driving signal, a sound output device to output a sound in accordance with the driving signal, a vibration device to vibrate in accordance with the driving signal, and a display device to display maturity level information in accordance with the driving signal.

### Claim 14 (depends on 10)

The method according to claim 10, further comprising displaying the maturity level image as overlapping the color image on a display device.

### Claim 15 (depends on 1)

The method according to claim 1, further comprising performing image capturing of at least a portion of the fruit or vegetable product to acquire the image.

### Claim 16 (depends on 15)

The method according to claim 15, wherein in the step of acquiring the image, the image is acquired by performing image capturing of a central portion and a vicinity thereof of the fruit or vegetable product.

### Claim 17 (depends on 15)

The method according to claim 15, wherein in the step of acquiring the image, the image is acquired a plurality of times.

### Claim 18 (depends on 1)

The method according to claim 1, further comprising determining whether the fruit or vegetable product is harvestable or not based on the maturity determination information.

### Claim 19 (depends on 1)

The method according to claim 1, wherein the light reflected by the fruit or vegetable product at a time of harvest thereof has a reflectance characteristic with an intensity that increases as the light has a longer wavelength in a wavelength band from a blue light to a near infrared light.

### Claim 20 (depends on 1)

The method according to claim 1, wherein the fruit or vegetable product is a bunch including a plurality of fruits.
