# US 9,436,997 B2 · Estimating rainfall precipitation amounts by applying computer vision in cameras > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,997 B2 - Document ID: US9436997B2 - Status: Granted - Kind: B2 - Application number: 14/748,125 - Filed: 2015-06-23 - Published: 2016-05-26 - Granted: 2016-09-06 - Earliest claimed: 2014-11-24 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Renato F. Cerqueira; Kiran Mantripragada - Web page: https://patentgrep.com/patent/US9436997B2 ## CPC - G01W 1/14: Rainfall or precipitation gauges (measuring volume in general G01F) - G06F 16/50: of still image data - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation - G06V 10/48: by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation - G06V 20/38: {Outdoor scenes} - G01W: METEOROLOGY (radar, sonar, lidar or analogous systems, designed for meteorological use G01S13/95, G01S15/88, G01S17/95) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method and system are provided. The method includes storing a set of references images without rain and spanning a plurality of different light conditions. The method further includes capturing, using a camera, an image of a scene with rain. The method also includes selecting a reference image from the set of reference images based on the light condition of the captured image. The method additionally includes performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image. The method further includes estimating an amount of rain in the subtraction image based on previously calibrated values. ## Claims ### Claim 1 A method, comprising: storing a set of references images without rain and spanning a plurality of different light conditions; capturing, using a camera, an image of a scene with rain; selecting a reference image from the set of reference images based on the light condition of the captured image; performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image; cutting off corner pixels in the subtraction image; and estimating an amount of rain in the subtraction image based on previously calibrated values. ### Claim 2 (depends on 1) The method of claim 1, further comprising re-purposing the camera from another use. ### Claim 3 (depends on 1) The method of claim 1, further comprising: performing image border detection processing on the subtraction image to detect the rain; generating a binary mask responsive to a result of the border detection image processing, the binary mask having a first pixel value indicative of an absence of rain and having a second pixel value indicative of a presence of rain; and performing a count of pixels having rain based on a total number of occurrences of at least one of the first pixel value and the second pixel value. ### Claim 4 (depends on 3) The method of claim 3, wherein said estimating step generates an estimate of the amount of rain responsive to the count of pixels. ### Claim 5 (depends on 3) The method of claim 3, wherein the image border detection processing comprises border detection, line detection, and at least one of Fourier Transformation and wavelet transformation. ### Claim 6 (depends on 3) The method of claim 3, wherein the image border detection processing comprises applying at least one of a Fourier transform and a wavelet transform to the subtraction image to at least one of identify and enhance frequencies which describe textures of raindrops. ### Claim 7 (depends on 3) The method of claim 3, wherein the image border detection processing comprises performing line detection to detect lines of one or more particular orientations. ### Claim 8 (depends on 7) The method of claim 7, wherein the line detection comprises applying at least one Hough transform to the subtraction image. ### Claim 9 (depends on 1) The method of claim 1, wherein the reference image is selected from the set based upon a time correspondence between the reference image and the captured image. ### Claim 10 (depends on 1) The method of claim 1, wherein the arithmetic subtraction image processing operation comprises performing: C(x,y)−T(x,y)=O(x,y), where C denotes the captured image, T denotes the reference image, O denotes the subtraction image without a background and with at least one moving object, x denotes a first spatial orientation, and y denotes a second spatial orientation orthogonal with respect to the first spatial orientation. ### Claim 11 (depends on 1) The method of claim 1, further comprising performing a threshold filtering operation to cut off pixels in the subtraction image having an intensity below a threshold intensity. ### Claim 12 (depends on 1) The method of claim 1, wherein said step of estimating the amount of rain comprises setting a predetermined granularity and precision for at least one rain estimate. ### Claim 13 (depends on 1) The method of claim 1, further comprising generating a user-perceptible indication of the amount of rain. ### Claim 14 (depends on 1) The method of claim 1, further comprising training a set of calibrated values that include the previously calibrated values to improve a precision of a rain amount estimate generated by said estimating step. ### Claim 15 (depends on 1) The method of claim 1, wherein the captured image includes a plate glass from which the amount of rain is estimated.