# US 2017/0257551 A1 · Method for processing high dynamic range (hdr) data from a nonlinear camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0257551 A1 - Document ID: US20170257551A1 - Status: Published application - Kind: A1 - Application number: 15/441,893 - Filed: 2017-02-24 - Published: 2017-09-07 - Earliest claimed: 2016-03-04 - Assignees: Delphi Technologies, Inc. - Applicants: Delphi Technologies, Inc. - Inventors: Mirko Meuter; Mateusz Komorkiewicz; Rajat Sanyal; Christof Petig - Web page: https://patentgrep.com/patent/US20170257551A1 ## CPC - H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors - B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle - G06T 5/92: based on global image properties - G06T 7/11: Region-based segmentation - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 19/428: {Recompression, e.g. by spatial or temporal decimation} - H04N 23/10: for generating image signals from different wavelengths - H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control - B60R 2300/10: characterised by the type of camera system used - B60R 2300/30: characterised by the type of image processing - G06T 2207/20208: High dynamic range [HDR] image processing - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for processing high dynamic range data from a nonlinear camera includes; generating an input image comprising a plurality of pixels, each pixel having an initial pixel value, wherein the initial pixel values are generated using a camera transition curve; generating a first lookup table representing a combination of an inverse function and a re-compression function, the first lookup table having input values and output values, wherein each input value is linked to one output value, the inverse function is the inverse of the camera transition curve, the re-compression function is a smooth and continuous function having a slope at each input value which is greater than or equal to a corresponding slope of the camera transition curve, the first lookup table is generated such that the inverse function precedes the re-compression function; and generating a first image by converting the initial pixel values using the first lookup table. ## Claims ### Claim 1 A method for processing High Dynamic Range (HDR) data from a nonlinear camera, the method comprising: generating an input image with the camera, the input image comprising a plurality of pixels, each pixel having an initial pixel value, wherein the initial pixel values are generated using a camera transition curve; generating a first lookup table representing a combination of an inverse function and a re-compression function, the first lookup table having input values and output values, wherein each input value is linked to one output value, the inverse function is the inverse of the camera transition curve, the re-compression function is a smooth and continuous function having a slope at each input value which is greater than or equal to a corresponding slope of the camera transition curve, the first lookup table is generated such that the inverse function precedes the re-compression function; and generating a first image by converting the initial pixel values using the first lookup table. ### Claim 2 (depends on 1) The method in accordance with claim 1, wherein the re-compression function is a logarithmic function y=a*ld(x)+b, wherein a and b are selected such that d dx  a * ld  ( x ) + b >= d dx  camera —  transition —  curve  ( x ) , wherein a and b are coefficients, x is an initial pixel value, and y is an output value. ### Claim 3 (depends on 1) The method in accordance with claim 1, wherein the first lookup table also represents a clipping function which maps all input values below or equal to one or all input values below or equal to a predefined threshold value to a value of zero, wherein the clipping function precedes the inverse function. ### Claim 4 (depends on 1) The method in accordance with claim 1, wherein an initial histogram of the input image is calculated and the initial histogram is transformed into a modified histogram using the first lookup table. ### Claim 5 (depends on 4) The method in accordance with claim 4, wherein the initial histogram is calculated as a weighted sum of sub-histograms of several regions of the input image, wherein each sub-histogram is preferably given equal weight. ### Claim 6 (depends on 5) The method in accordance with claim 5, wherein the modified histogram is transformed to a reduced histogram comprising of fewer bins than the modified histogram and a second lookup table is generated, which represents a modified histogram equalization based on the reduced histogram. ### Claim 7 (depends on 6) The method in accordance with claim 6, wherein the modified histogram equalization comprises altering the bin counts of the modified histogram such that all bin counts are between a predefined maximum value and a predefined minimum value, and wherein the modified histogram equalization further comprises convolving the modified histogram with a low-pass filter, preferably after the alteration of the bin counts of the modified histogram. ### Claim 8 (depends on 6) The method in accordance with claim 6, wherein the modified histogram equalization comprises generating the reduced histogram as a cumulative histogram from the modified histogram, preferably after the convolution, wherein the reduced histogram comprises of fewer bins than the modified histogram and wherein a histogram equalization using the reduced histogram is mapped in the second lookup table. ### Claim 9 (depends on 1) The method in accordance with claim 1, wherein the camera is a red/clear imager (RCCC imager), wherein one of the input image and the first image comprise red and grey pixels, wherein a second image is extracted from the first image, wherein the second image comprises the red pixels of the first image, and a third image is generated from the first image, wherein the red pixels of the first image are replaced by interpolating the values of neighboring grey pixels of the first image. ### Claim 10 (depends on 9) The method in accordance with claim 9, wherein the interpolated values of the third image are subtracted from the corresponding values of the red pixels of the second image thereby generating a fourth image and the fourth image is mapped to a red to clear ratio image using a linear mapping, wherein the value range of each pixel in the red to clear ratio image is smaller than in the fourth image. ### Claim 11 (depends on 9) The method in accordance with claim 9, wherein the third image is mapped to a grey image using the second lookup table, wherein the value range of each pixel in the grey image is smaller than in the third image. ### Claim 12 (depends on 9) The method in accordance with claim 9, wherein a white balance factor is calculated based on the function w b = 2 log 2   f - 1 ap  Σ  ( a   log 2  r g ) , wherein wb is the white balance factor, f is 1/3, p is a pixelcount, r is the value of a red pixel of the first image, and g is the value of a grey pixel of the first image. ### Claim 13 (depends on 12) The method in accordance with claim 12, wherein the resolution of the red to clear ratio image is interpolated to be equal to the resolution of the grey image, wherein an interpolated red to clear ratio image is generated and wherein to each pixel of the interpolated red to clear ratio image a value y=a*ld(gnew)+b is added, with gnew being the value of a grey pixel of the grey image, thereby creating a first modified interpolated red image, wherein the value y is generated by transforming the grey image using the first lookup table, and wherein a logarithm of the white balance factor is added to each pixel value of the first modified interpolated red image, thereby creating a second modified interpolated red image, a third lookup table is created, which represents the function y = 2 x - b a wherein the third lookup table is used to create a red image from the second modified red image, one third of each pixel value of the red image is subtracted from each corresponding pixel value of the grey image and the resulting pixel values are equally distributed to a blue color channel and a green color channel and the red image, the blue color channel and the green color channel are combined to create a color image. ### Claim 14 A camera system comprising a camera and a processing unit, the camera system being adapted to generate an input image with the camera, the input image comprising a plurality of pixels, each pixel having an initial pixel value, wherein the initial pixel values are generated using a camera transition curve; generate a first lookup table representing a combination of an inverse function and a re-compression function, the first lookup table having input values and output values, wherein each input value is linked to one output value, the inverse function is the inverse of the camera transition curve, the re-compression function is a smooth and continuous function having a slope at each input value which is greater than or equal to a corresponding slope of the camera transition curve, the first lookup table is generated such that the inverse function precedes the re-compression function; and generate a first image by converting the initial pixel values using the first lookup table. ### Claim 15 (depends on 14) A vehicle with the camera system of claim 14, wherein output data of the camera system is used for a machine vision application.