# US 9,519,957 B1 · Automated selective application of fill light filter > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,957 B1 - Document ID: US9519957B1 - Status: Granted - Kind: B1 - Application number: 14/323,911 - Filed: 2014-07-03 - Granted: 2016-12-13 - Earliest claimed: 2014-07-03 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: Krzysztof Kulewski; Aravind Krishnaswamy; Sevket Derin Babacan - Web page: https://patentgrep.com/patent/US9519957B1 ## CPC - G06T 5/94: based on local image properties, e.g. for local contrast enhancement - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2207/10024: Color image - G06T 2207/20208: High dynamic range [HDR] image processing - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Aspects of the subject technology relate to automatically and selectively applying a fill light filter to an image. A process includes determining an edge-preserved, smoothed version of the image, and determining a grayscale version of the image. The process also includes comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image. The process also includes applying the fill light filter to the image based on the comparison. The fill light filter is automatically adjusted based on identifying regions in the image. Selectively applying the fill light filter can reduce artifacts and noise from forming or being amplified as a result of the fill light filter. ## Claims ### Claim 1 A method for image processing, the method comprising: determining an edge-preserved, smoothed version of an image; determining a grayscale version of the image; comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image to determine a comparison value for each pixel; determining a scaling value for each pixel based on the respective comparison value; and selectively applying a fill light filter to the image by scaling the applied fill light filter for each pixel based on the respective scaling value. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the edge-preserved, smoothed version of the image comprises: blurring non-edge areas of the image; and determining a grayscale version of the at least partially blurred image. ### Claim 3 (depends on 1) The method of claim 1, wherein comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises determining a difference between color values of each pixel of the edge-preserved, smoothed version of the image and color values of each corresponding pixel of the grayscale version of the image. ### Claim 4 (depends on 3) The method of claim 3, wherein applying the fill light filter comprises reducing the applied fill light filter to a pixel when the determined difference for the pixel is negative. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the edge-preserved, smoothed version of the image comprises applying a Local Statistical Convolution (LSC) function to each pixel of the image. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the grayscale version of the image comprises applying a luminance function to each pixel of the image. ### Claim 7 (depends on 1) The method of claim 1, wherein comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises determining one or more characteristics of the image. ### Claim 8 (depends on 1) The method of claim 1, wherein comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises calculating a linear combination of the edge-preserved, smoothed version of the image and the grayscale version of the image. ### Claim 9 A non-transitory machine-readable medium including instructions stored therein, which when executed by a machine, cause the machine to perform operations comprising: determining an edge-preserved, smoothed version of an image; determining a grayscale version of the image; comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image to determine a comparison value for each pixel; determining a scaling value for each pixel based on the respective comparison value; and selectively applying a fill light filter to the image by scaling the applied fill light filter for each pixel based on the respective scaling value. ### Claim 10 (depends on 9) The non-transitory machine-readable medium of claim 9, wherein the operation of determining the edge-preserved, smoothed version of the image comprises operations for: blurring non-edge areas of the image; and determining a grayscale version of the at least partially blurred image. ### Claim 11 (depends on 9) The non-transitory machine-readable medium of claim 9, wherein the operation of comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises operations for determining a difference between color values of each pixel of the edge-preserved, smoothed version of the image and color values of each corresponding pixel of the grayscale version of the image. ### Claim 12 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the operation for applying the fill light filter comprises operations for reducing the applied fill light filter to a pixel when the determined difference for the pixel is negative. ### Claim 13 (depends on 9) The non-transitory machine-readable medium of claim 9, wherein the operation for determining the edge-preserved, smoothed version of the image comprises operations for applying a Local Statistical Convolution (LSC) function to each pixel of the image. ### Claim 14 (depends on 9) The non-transitory machine-readable medium of claim 9, wherein the operation for determining the grayscale version of the image comprises operations for applying a luminance function to each pixel of the image. ### Claim 15 (depends on 9) The non-transitory machine-readable medium of claim 9, wherein the operation for comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises operations for determining one or more characteristics of the image. ### Claim 16 (depends on 9) The non-transitory machine-readable medium of claim 9, wherein the operation for comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises operations for calculating a linear combination of the edge-preserved, smoothed version of the image and the grayscale version of the image. ### Claim 17 A system comprising: one or more processors; and a non-transitory computer-readable medium comprising instructions stored therein, which, when executed by the one or more processors, cause the one or more processors to perform operation comprising: determining a first modified version of each pixel of an image; determining a second modified version of each pixel of the image; determining a comparison value for each pixel of the image by comparing at least the first modified version of each pixel to the second modified version of each pixel, respectively; determining a magnitude adjustment value for each pixel based on the respective comparison value; and selectively applying an image filter to each pixel based on the respective magnitude adjustment value for each pixel. ### Claim 18 (depends on 17) The system of claim 17, wherein determining the first modified version of each pixel of the image cause the one or more processors to perform operations comprising applying a first image filter function to each pixel of the image, and wherein determining the second modified version of each pixel of the image comprises applying a second image filter function to each pixel of the image. ### Claim 19 (depends on 18) The system of claim 18, wherein determining a comparison value for each pixel of the image cause the one or more processors to perform operations comprising calculating a linear combination of the first image filter function and the second image filter function for each pixel of the image. ### Claim 20 (depends on 19) The system of claim 19, wherein determining the magnitude adjustment value for each pixel cause the one or more processors to perform operations comprising calculating a sigmoid function based on the linear combination.