# US 2023/0069346 A1 · Methods and Systems for Detecting Weather Conditions using Vehicle Onboard Sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0069346 A1
- Document ID: US20230069346A1
- Status: Published application
- Kind: A1
- Application number: 18/049,920
- Filed: 2022-10-26
- Published: 2023-03-02
- Earliest claimed: 2013-05-07
- Assignees: Waymo LLC
- Applicants: Waymo LLC
- Inventors: Jiajun Zhu; Dmitri Dolgov; Dave Ferguson
- Web page: https://patentgrep.com/patent/US20230069346A1

## CPC

- G01W 1/00: Meteorology
- B60W 60/0053: {from vehicle to occupant}
- B60W 60/0059: {Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity}
- G01N 21/4738: {Diffuse reflection (precedence is given to G01N21/55 - G01N21/57 if specular component is taken into consideration), e.g. also for testing fluids, fibrous materials}
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/87: Combinations of systems using electromagnetic waves other than radio waves
- G01W 1/14: Rainfall or precipitation gauges (measuring volume in general G01F)
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0257: {using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931)}
- G05D 1/0276: {using signals provided by a source external to the vehicle (involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058)}
- H04N 7/183: {for receiving images from a single remote source}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2555/20: Ambient conditions, e.g. wind or rain
- Y02A 90/10: Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
- G01W: METEOROLOGY (radar, sonar, lidar or analogous systems, designed for meteorological use G01S13/95, G01S15/88, G01S17/95)
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

Example methods and systems for detecting weather conditions using vehicle onboard sensors are provided. An example method includes receiving laser data collected for an environment of a vehicle, and the laser data includes a plurality of laser data points. The method also includes associating, by a computing device, laser data points of the plurality of laser data points with one or more objects in the environment, and determining given laser data points of the plurality of laser data points that are unassociated with the one or more objects in the environment as being representative of an untracked object. The method also includes based on one or more untracked objects being determined, identifying by the computing device an indication of a weather condition of the environment.

## Claims

### Claim 1

A method comprising:
monitoring, at a computing device and using sensor data from one or more sensors coupled to a vehicle, a plurality of objects located in an environment of the vehicle,
receiving, at the computing device and from a lidar coupled to the vehicle, laser data corresponding to the environment of the vehicle;
performing a comparison between objects represented by the laser data and the plurality of objects monitored using sensor data from the one or more sensors;
based on the comparison, estimating a weather condition in the environment; and
controlling the vehicle based on the estimated weather condition.

### Claim 2 (depends on 1)

The method of claim 1, wherein performing the comparison comprises:
determining one or more objects represented by the laser data lacks a match in the plurality of objects monitored using sensor data from the one or more sensors.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
based on determining one or more objects represented by the laser data lack the match in the plurality of objects, estimating the weather condition in the environment.

### Claim 4 (depends on 3)

The method of claim 3, wherein estimating the weather condition in the environment comprises:
determining laser data points corresponding to the one or more objects exceeds a threshold number of data points; and
estimating the weather condition based on the laser data points corresponding to the one or more objects exceeding the threshold number of data points.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
comparing the laser data points corresponding to the one or more objects with stored laser data points representative of a pattern indicative of fog; and
wherein estimating the weather condition comprises:
estimating the weather condition in the environment includes fog based on the comparison.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
determining the one or more objects remained at a substantially same relative position with respect to the vehicle as the vehicle moves; and
wherein estimating the weather condition comprises:
estimating the weather condition in the environment includes a sunny condition.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
based on the comparison, determining laser data represents an object that lacks a match in the plurality of objects monitored using sensor data from the one or more sensors; and
based on the laser data that represents the object, determining that the laser data corresponds to reflections caused by road spray.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
based on the comparison, determining that the objects represented by the laser data matches the plurality of objects monitored using sensor data from the one or more sensors; and
determining that the weather condition of the environment does not impact performance of the lidar coupled to the vehicle.

### Claim 9 (depends on 1)

The method of claim 1, wherein the weather condition includes one or more of wetting of a roadway, a fog or sunlight.

### Claim 10 (depends on 1)

The method of claim 1, wherein monitoring the plurality of objects located in the environment of the vehicle comprises:
receiving sensor data from one or more radars coupled to the vehicle.

### Claim 11 (depends on 1)

The method of claim 1, wherein monitoring the plurality of objects located in the environment of the vehicle comprises:
receiving sensor data from one or more cameras coupled to the vehicle.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
receiving image data captured from a camera coupled to the vehicle; and
validating the estimated weather condition based on the image data.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
obtaining a temperature corresponding to the environment based on a location of the vehicle; and
wherein estimating the weather condition of the environment comprises:
estimating the weather condition of the environment further based on the temperature corresponding to the environment.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
determining a location of the vehicle and a time of day;
based on the location of the vehicle and the time of day, determining an approximate position of a sun relative to the vehicle; and
wherein estimating the weather condition of the environment comprises:
estimating the weather condition of the environment further based on the approximate position of the sun relative to the vehicle.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
determining a speed of the vehicle; and
wherein estimating the weather condition of the environment further comprises:
determining that a road under the vehicle is wet based on the speed of the vehicle being above a threshold.

### Claim 16

A system comprising:
a vehicle having a plurality of sensors, wherein the plurality of sensors includes a lidar;
a computing device coupled to the vehicle and configured to:
monitor, using sensor data from one or more sensors coupled to the vehicle, a plurality of objects located in an environment of the vehicle,
receive, from the lidar, laser data corresponding to the environment of the vehicle;
perform a comparison between objects represented by the laser data and the plurality of objects monitored using sensor data from the one or more sensors;
based on the comparison, estimate a weather condition in the environment; and
control the vehicle based on the estimated weather condition.

### Claim 17 (depends on 16)

The system of claim 16, wherein the computing device is further configured to:
determine one or more objects represented by the laser data lacks a match in the plurality of objects monitored using sensor data from the one or more sensors;
based on determining one or more objects represented by the laser data lack the match in the plurality of objects, estimate the weather condition in the environment.

### Claim 18 (depends on 16)

The system of claim 16, wherein the computing device is further configured to:
based on the comparison, determine laser data represents an object that lacks a match in the plurality of objects monitored using sensor data from the one or more sensors; and
based on the laser data that represents the object, determine that the laser data corresponds to reflections caused by road spray.

### Claim 19 (depends on 16)

The system of claim 16, wherein the weather condition includes one or more of wetting of a roadway, a fog or sunlight.

### Claim 20

A non-transitory computer readable storage medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions comprising:
monitoring, using sensor data from one or more sensors coupled to a vehicle, a plurality of objects located in an environment of the vehicle,
receiving, from a lidar coupled to the vehicle, laser data corresponding to the environment of the vehicle;
performing a comparison between objects represented by the laser data and the plurality of objects monitored using sensor data from the one or more sensors;
based on the comparison, estimating a weather condition in the environment; and
controlling the vehicle based on the estimated weather condition.
