# US 2015/0177512 A1 · Camera device, three-dimensional object detection device, and lens cleaning method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0177512 A1
- Document ID: US20150177512A1
- Status: Published application
- Kind: A1
- Application number: 14/416,650
- Filed: 2013-07-26
- Published: 2015-06-25
- Earliest claimed: 2012-07-27
- Assignees: Nissan Motor Co., Ltd.; Clarion Co., Ltd.
- Applicants: Nissan Motor Co., Ltd.; Clarion Co., Ltd.
- Inventors: Yasuhisa Hayakawa; Osamu Fukata; Masayuki Takemura; Masahiro Kiyohara; Shoji Muramatsu; Kota Irie
- Web page: https://patentgrep.com/patent/US20150177512A1

## CPC

- G02B 27/0006: {with means to keep optical surfaces clean, e.g. by preventing or removing dirt, stains, contamination, condensation (G02B1/18 takes precedence; cleaning in general B08B)}
- B60S 1/56: specially adapted for cleaning other parts or devices than front windows or windscreens {(rear-view mirror arrangements mounted on vehicle exterior including cleaning devices B60R1/0602)}
- G03B 17/02: Bodies
- G03B 17/08: Waterproof bodies or housings
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- H04N 23/811: {by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor}
- B60S 1/0848: {Cleaning devices for cameras on vehicle}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- B60S: SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A camera device includes image capturing device, a lens cleaning device, an adhesion state assessment unit and a controller. The image capturing device is installed on a vehicle and has a lens for forming an image of the vehicle surroundings. The lens cleaning device cleans the lens by spraying a cleaning fluid on the lens in accordance with a predetermined lens cleaning step, in which at least a supply time for supplying cleaning fluid to the lens surface is predetermined. The adhesion state assessment unit is programmed to assess an adhesion state of contamination from a distribution of pixels corresponding to foreign matter adhered to the lens based on a captured image. The controller is programmed to extend the time until a time to start supplying the cleaning fluid in the lens cleaning step as the number of pixels corresponding to foreign matter adhered to the lens increases.

## Claims

### Claim 1

A camera device comprising:
an image capturing device installed on a vehicle and comprising a lens for forming an image of the vehicle surroundings;
a lens cleaning device configured to clean the lens by spraying a cleaning fluid on the lens in accordance with a predetermined lens cleaning step, in which at least a supply time for supplying cleaning fluid to the lens surface is predetermined;
an adhesion state assessment unit programmed to assess an adhesion state of contamination from a distribution of pixels corresponding to foreign matter adhered to the lens based on a captured image captured by the image capturing device; and
a lens cleaning controller programmed to extend the time until a time to start supplying the cleaning fluid in the lens cleaning step as the number of pixels corresponding to foreign matter adhered to the lens increases.

### Claim 2 (depends on 1)

The camera device as recited in claim 1, wherein
the adhesion state assessment unit is programmed to calculate an adhesion amount of the foreign matter as the adhesion degree, based on pixel values of pixels included in the obtained captured image, and the number of pixels corresponding to foreign matter adhered to the surface of the lens, and to assess a state of adhered lens contamination based on the adhesion degree.

### Claim 3

(canceled)

### Claim 4 (depends on 1)

The camera device as recited in claim 1, wherein
the adhesion state assessment unit is programmed to calculate a variation in the adhesion amount of foreign matter as a variation in the adhesion degree, based on pixel values of pixels included in the obtained captured image, and a variation in the number of pixels corresponding to foreign matter adhered to the surface of the lens in a predetermined time, and to assess a state of adhered lens contamination based on the variation in the adhesion degree.

### Claim at position 5

5-13. (canceled)

### Claim 14

A camera device characterized by comprising:
an image capturing device installed on a vehicle and comprising a lens for forming an image of the vehicle surroundings;
a lens cleaning device configured to clean the lens by spraying a cleaning fluid on the lens in accordance with a predetermined lens cleaning step, in which at least a supply time for supplying the cleaning fluid to the lens surface is predetermined;
an adhesion state assessment unit programmed to assess an adhesion state of contamination from a distribution of pixels corresponding to foreign matter adhered to the lens, based on a captured image captured by the image capturing device; and
a lens cleaning controller programmed to extend the time until a time to start supplying the cleaning fluid as the distribution of pixels corresponding to foreign matter adhered to the lens increases.

### Claim 15 (depends on 14)

The camera device as recited in claim 14, wherein
the adhesion state assessment unit is programmed to calculate an adhesion ratio of the foreign matter as the adhesion degree, based on pixel values of the pixels included in the obtained captured image, and the distribution of the pixels corresponding to foreign matter adhered to the surface of the lens, and to assess a state of adhered lens contamination based on the adhesion degree.

### Claim 16 (depends on 14)

The camera device as recited in claim 14, wherein
the adhesion state assessment unit is programmed to calculate a variation in the adhesion ratio of foreign matter as a variation in an adhesion degree, based on pixel values of the pixels included in the obtained captured image, and a variation in distribution of the pixels corresponding to foreign matter adhered to the surface of the lens in a predetermined time, and to assess a state of adhered lens contamination based on the variation in the adhesion degree.

### Claim 17 (depends on 2)

The camera device as recited in claim 2, wherein
the adhesion state assessment unit is programmed to assess that the state of adhered lens contamination is a first state when the adhesion degree of foreign matter is in a predetermined threshold value of equal to or greater than a first adhesion threshold and less than a second adhesion threshold, which are preset; and
the lens cleaning controller is programmed to execute a control to extend the time until a time to start supplying the cleaning fluid when the state of adhered lens contamination is assessed as being the first state.

### Claim 18 (depends on 17)

The camera device as recited in claim 17, wherein
the first adhesion threshold is an adhesion degree of foreign matter when the vehicle is traveling on a paved road without patterns on the road surface or a paved road without irregularities on the road surface.

### Claim 19 (depends on 17)

The camera device as recited in claim 17, wherein
the second adhesion threshold is an adhesion degree of foreign matter when the vehicle is traveling on one of a paved road with patterns on the road surface and a paved road with irregularities formed on the road surface.

### Claim 20 (depends on 17)

The camera device as recited in claim 17, wherein
the lens cleaning controller is programmed to cause the lens cleaning device to clean the lens following the predetermined lens cleaning step when the adhesion degree of foreign matter is assessed as being less than the first adhesion threshold or is equal to or greater than the second adhesion threshold, which is greater than the first adhesion threshold, and when the state of adhered lens contamination is assessed as not being the first state.

### Claim 21 (depends on 17)

The camera device as recited in claim 17, wherein
the first state is a state in which particles with a small average particle size have adhered to the lens.

### Claim 22 (depends on 1)

The camera device as recited in claim 1, further comprising:
a traveling path state assessment unit programmed to assess whether a traveling path on which the vehicle is traveling is one of a paved road and an unpaved road based on traveling information acquired by the vehicle; wherein
the lens cleaning controller supplies the cleaning fluid to the lens at a time at which the traveling path on which the host vehicle is traveling is assessed as being the paved road by the traveling path state assessment unit.

### Claim 23 (depends on 22)

The camera device as recited in claim 22, wherein
the traveling path state assessment means
acquires each wheel speed of a plurality of wheels provided to the vehicle, and when the variation of the plurality of acquired wheel speeds is less than a predetermined value, the traveling path on which the vehicle is traveling is assessed as being a paved road.

### Claim 24 (depends on 2)

A three-dimensional object detection device including the camera device as recited in claim 2, the three-dimensional object detection device comprising:
an image conversion unit programmed to perform viewpoint conversion of captured images obtained by the camera device into bird's-eye view images;
a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object, based on distribution information of pixels that is generated in the bird's-eye view images obtained by the image conversion unit, when a luminance difference is at a predetermined threshold value or greater along a collapsing direction of a three-dimensional object when the viewpoint is converted to the bird's-eye view image;
a three-dimensional object assessment unit programmed to assess whether the three-dimensional object detected by the three-dimensional object detection unit is another vehicle; and
a controller programmed to prevent a detection of the three-dimensional object and the three-dimensional object from being assessed as being another vehicle when the adhesion state assessment unit has assessed that the adhesion degree of foreign matter adhered to the lens is equal to or greater than a predetermined value.

### Claim 25

A lens cleaning method that is executed by a camera device comprising:
a lens; and
a lens cleaning device for cleaning the lens by supplying cleaning fluid on the lens; wherein
the camera device executes a step assessing an adhesion state of contamination from a number of pixels corresponding to foreign matter adhered to the lens, based on a captured image; and
extends the time until a time to start supplying the cleaning fluid as the number of pixels corresponding to foreign matter adhered to the lens increases.
