# US 10,121,231 B2 · Adaptive bilateral (BL) filtering for computer vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,231 B2 - Document ID: US10121231B2 - Status: Granted - Kind: B2 - Application number: 15/183,459 - Filed: 2016-06-15 - Published: 2017-08-03 - Granted: 2018-11-06 - Earliest claimed: 2016-02-01 - Assignees: Texas Instruments Incorporated - Applicants: Texas Instruments Incorporated - Inventors: Mihir Narendra Mody; Shashank Dabral; Jesse Gregory Villarreal, Jr.; William Wallace; Niraj Nandan - Web page: https://patentgrep.com/patent/US10121231B2 ## CPC - G06T 5/70: Denoising; Smoothing - G06T 5/20: using local operators - G06T 2200/28: involving image processing hardware - G06T 2207/20004: Adaptive image processing - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/20028: Bilateral filtering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method for filtering noise for imaging includes receiving an image frame having position and range data. A filter size divides the frame into filter windows for processing each of the filter windows. For the first pixel, a space to the center pixel and a range difference between this pixel and the center pixel is determined and used for choosing a selected weight from weights in a 2D weight LUT including weighting for space and range difference, a filtered range value is calculated by applying the selected 2D weight to the pixel, and the range, filtered range value and selected 2D weight are summed. The determining, choosing, calculating and summing are repeated for at least the second pixel. A total sum of contributions from the first and second pixel are divided by the sum of selected 2D weights to generate a final filtered range value for the center pixel. ## Claims ### Claim 1 A method of filtering noise for imaging, comprising: using a processor implementing a bilateral BL filter algorithm stored in a memory accessible by said processor or using (BL) filter hardware configured to perform said method on an image frame comprising image data including a position and a range from a plurality of pixels, said frame including a plurality of filter windows each including a center pixel and a plurality of other pixels including a first pixel and a second pixel, processing each of said plurality of filter windows including: for said first pixel, determining a space being its distance relative to said center pixel and a range difference between its range and said range of said center pixel, using said space and said range difference, selecting a combined 2D weight from a plurality of combined 2D weights which each include weighting for both said space and said range difference from at least one combined pre-computed 2D weight lookup table (2D weight LUT), calculating a filtered range value by applying said selected combined 2D weight to said first pixel range, and summing said filtered range value to determine a contribution of said first pixel, repeating said determining, said selecting, said calculating, and said summing for at least said second pixel, and dividing a total sum of said contributions from said first pixel and at least said second pixel by a sum of said selected combined 2D weights from said first pixel and said second pixel to generate a final filtered range value for said center pixel as an output pixel. ### Claim 2 (depends on 1) The method of claim 1, wherein said 2D weight LUT comprises a plurality of 2D weight sub-LUTs which each have a plurality of said combined 2D weights providing a plurality of different sigma (a) values for at least said range, wherein before said choosing said method further comprises: based on a pixel image attribute adaptively picking a selected one of said plurality of 2D weight sub-LUTs (selected 2D weight sub-LUT) from said plurality of 2D weight sub-LUTs, and wherein said choosing said selected 2D weight is from said selected 2D weight sub-LUT. ### Claim 3 (depends on 1) The method of claim 1, wherein said 2D weight LUT is quantized in at least one of said space and said range. ### Claim 4 (depends on 3) The method of claim 3, wherein said 2D weight LUT is quantized in both said space and said range. ### Claim 5 (depends on 3) The method of claim 3, wherein non-linear interpolation is used to quantize said range by utilizing a non-linear (NL) LUT for said 2D weight LUT including a finer step size at lower values for said range as compared to a larger step size at higher values for said range. ### Claim 6 (depends on 1) The method of claim 1, wherein said memory comprises flip flops or registers. ### Claim 7 (depends on 1) The method of claim 1, wherein said 2D weight LUT comprises a plurality of 2D weight sub-LUTs which each have a plurality of said combined 2D weights providing a plurality of different sigma (σ) values for said range and for said space, further comprising calculating a sub-table index and using said sub-table index for selecting one of said plurality of 2D weight sub-LUT's (selected 2D weight sub-LUT), and using said selected 2D weight sub-LUT for said processing across all of said plurality of filter windows in said image frame. ### Claim 8 (depends on 1) The method of claim 1, wherein said 2D weight LUT comprises a plurality of 2D weight sub-LUTs which each have a plurality of said combined 2D weights providing a plurality of different sigma (σ) values for said range, further comprising local averaging in each of said plurality of filter windows to determine an average range and using said average range for selecting one of said plurality of 2D weight sub-LUTs individually for said processing of said plurality of filter windows. ### Claim 9 (depends on 8) The method of claim 8, wherein only a portion of said plurality of other pixels are used to determine said average range. ### Claim 10 (depends on 1) The method of claim 1, further comprising using said output pixels from said processing said plurality of filter windows to generate a noise filtered image that is utilized for an Advanced Driver Assistance System (ADAS). ### Claim 11 A system for processing images, comprising: a processor and an associated memory; a bilateral (BL) filter program stored in said memory and executable by said processor or BL filter hardware, wherein the BL filter program or said BL filter hardware implements a method on an image frame comprising image data including a position and a range from a plurality of pixels, said frame including a plurality of filter windows each including a center pixel and a plurality of other pixels including a first pixel and a second pixel, processing each of said plurality of filter windows including: for said first pixel, determining a space being its distance relative to said center pixel and a range difference between its range and said range of said center pixel, using said space and said range difference, selecting a combined 2D weight from a plurality of combined 2D weights which each include weighting for both said space and said range difference from at least one combined pre-computed 2D weight lookup table (2D weight LUT), calculating a faltered range value by applying said selected combined 2D weight to said first pixel range, and summing said filtered range value to determine a contribution of said first pixel, repeating said determining, said selecting, said calculating, and said summing for at least said second pixel, and dividing a total sum of said contributions from said first pixel and at least said second pixel by a sum of said selected combined 2D weights from said first pixel and said second pixel to generate a final filtered range value for said center pixel as an output pixel. ### Claim 12 (depends on 11) The system of claim 11, wherein said 2D weight LUT comprises a plurality of 2D weight sub-LUTs which each have a plurality of said combined 2D weights providing a plurality of different variance (σ) values for at least said range, wherein before said choosing said method further comprises: based on a pixel image attribute including adaptively picking a selected one of said plurality of 2D weight sub-LUTs (selected 2D weight sub-LUT) from said plurality of 2D weight sub-LUTs, and wherein said choosing said selected 2D weight is from said selected 2D weight sub-LUT. ### Claim 13 (depends on 11) The system of claim 11, wherein said 2D weight LUT is quantized in at least one of said space and said range. ### Claim 14 (depends on 13) The system of claim 13, wherein said 2D weight LUT is quantized in both said space and said range. ### Claim 15 (depends on 13) The system of claim 13, wherein non-linear interpolation is used to quantize said range by utilizing a non-linen (NL) LUT for said 2D weight LUT including a finer step size at lower values for said range as compared to a larger step size at higher values for said range. ### Claim 16 (depends on 11) The system of claim 11, wherein said memory comprises flip flops or registers. ### Claim 17 (depends on 11) The system of claim 11, wherein said 2D weight LUT comprises a plurality of 2D weight sub-LUTs which each have a plurality of said combined 2D weights providing a plurality of different sigma (σ) values for said range and for said space, further comprising calculating a sub-table index and using said sub-table index for selecting one of said plurality of 2D weight sub-LUTs (selected 2D weight sub-LUT), and using said selected 2D weight sub-LUT for said processing across all of said plurality of filter windows in said image frame. ### Claim 18 (depends on 11) The system of claim 11, wherein said 2D weight LUT comprises a plurality of 2D weight sub-LUTs which each have a plurality of said combined 2D weights providing a plurality of different sigma (σ) values for at least said range, further comprising local averaging in each of said plurality of filter window to determine an average range and using said average range for selecting one of said plurality of 2D weight sub-LUTs individually for said processing of said plurality of filter windows. ### Claim 19 (depends on 18) The system of claim 18, wherein only a portion of said plurality of other pixels are used to determine said average range. ### Claim 20 (depends on 11) The system of claim 11, further comprising using said output pixels from said processing said plurality of filter windows to generate a noise filtered image that is utilized for an Advanced Driver Assistance System (ADAS). ### Claim 21 (depends on 11) The system of claim 11, wherein said system includes said BL filter hardware.