# US 2024/0202874 A1 · Bad pixel correction in image processing applications or other applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202874 A1
- Document ID: US20240202874A1
- Status: Published application
- Kind: A1
- Application number: 18/068,916
- Filed: 2022-12-20
- Published: 2024-06-20
- Earliest claimed: 2022-12-20
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Madhuri Suthar; Tyler Luu; Nguyen Thang Long Le; John W. Glotzbach; Hamid R. Sheikh
- Web page: https://patentgrep.com/patent/US20240202874A1

## CPC

- G06T 5/30: Erosion or dilatation, e.g. thinning
- G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/70: Denoising; Smoothing
- G06T 2207/20024: Filtering details
- G06T 2207/20036: Morphological image processing
- G06T 2207/20212: Image combination
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method includes obtaining input image frames, including at least two captured using different capture conditions. The method also includes separating color channels of each image frame and generating at least one bad pixel map for each color channel of each image frame. Each bad pixel map is generated by identifying one or more outliers in pixel values in the color channel based on an intensity distribution of pixel values in an operation window within the color channel. The operation window has a window size based on the capture condition and/or local image content in the corresponding image frame. The method further includes combining the bad pixel maps and performing a morphological operation to refine the combined bad pixel map. In addition, the method includes using one or more coordinates of one or more bad pixels in the refined bad pixel map to update one or more pixel values of at least one image frame.

## Claims

### Claim 1

A method comprising:
obtaining multiple input image frames, at least two of the input image frames captured using different capture conditions;
separating color channels of each of the input image frames;
generating at least one bad pixel map for each color channel of each input image frame, wherein each bad pixel map is generated by identifying one or more outliers in pixel values in the corresponding color channel based on an intensity distribution of the pixel values in a specified operation window within the corresponding color channel, the specified operation window having a window size based on at least one of: the capture condition associated with the corresponding image frame and local image content in the corresponding image frame;
combining the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;
performing a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and
using one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

### Claim 2 (depends on 1)

The method of claim 1, wherein the morphological operation comprises removing clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

### Claim 3 (depends on 1)

The method of claim 1, wherein the morphological operation comprises spatial filtering of the at least one combined bad pixel map to produce the at least one refined bad pixel map.

### Claim 4 (depends on 1)

The method of claim 1, wherein the morphological operation comprises performing erode operations and dilate operations, the erode operations replacing pixel values in the specified operation window with a minimum value, the dilate operations replacing the pixel values in the specified operation window with a maximum value.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the at least one bad pixel map for each color channel of each input image frame comprises:
for each color channel of each input image frame:
identifying first and Nth maximum and minimum pixel values in the color channel of the image frame, the first and Nth maximum and minimum pixel values based on pixel values within the specified operation window;
identifying adaptive thresholds based on differences between the first and Nth maximum and minimum pixel values; and
applying the adaptive thresholds to identify the one or more outliers in the pixel values of the color channel.

### Claim 6 (depends on 5)

The method of claim 5, wherein identifying the adaptive thresholds comprises:
linearly updating maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel values in the color channel of the image frame reach constant values.

### Claim 7 (depends on 1)

The method of claim 1, wherein the different capture conditions comprise different exposure settings.

### Claim 8

An electronic device comprising:
at least one imaging sensor configured to capture multiple input image frames such that at least two of the input image frames are captured using different capture conditions; and
at least one processing device configured to:
separate color channels of each of the input image frames;
generate at least one bad pixel map for each color channel of each input image frame, wherein, to generate each bad pixel map, the at least one processing device is configured to identify one or more outliers in pixel values in the corresponding color channel based on an intensity distribution of the pixel values in a specified operation window within the corresponding color channel, the specified operation window having a window size based on at least one of: the capture condition associated with the corresponding image frame and local image content in the corresponding image frame;
combine the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;
perform a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and
use one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein, to perform the morphological operation, the at least one processing device is configured to remove clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

### Claim 10 (depends on 8)

The electronic device of claim 8, wherein, to perform the morphological operation, the at least one processing device is configured to spatially filter the at least one combined bad pixel map to produce the at least one refined bad pixel map.

### Claim 11 (depends on 8)

The electronic device of claim 8, wherein:
to perform the morphological operation, the at least one processing device is configured to perform erode operations and dilate operations;
to perform the erode operations, the at least one processing device is configured to replace pixel values in the specified operation window with a minimum value; and
to perform the dilate operations, the at least one processing device is configured to replace the pixel values in the specified operation window with a maximum value.

### Claim 12 (depends on 8)

The electronic device of claim 8, wherein, to generate the at least one bad pixel map for each color channel of each input image frame, the at least one processing device is configured to:
for each color channel of each input image frame:
identify first and Nth maximum and minimum pixel values in the color channel of the image frame, the first and Nth maximum and minimum pixel values based on pixel values within the specified operation window;
identify adaptive thresholds based on differences between the first and Nth maximum and minimum pixel values; and
apply the adaptive thresholds to identify the one or more outliers in the pixel values of the color channel.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein, to identify the adaptive thresholds, the at least one processing device is configured to linearly update maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel values in the color channel of the image frame reach constant values.

### Claim 14 (depends on 8)

The electronic device of claim 8, wherein the different capture conditions comprise different exposure settings.

### Claim 15

A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:
obtain multiple input image frames, at least two of the input image frames captured using different capture conditions;
separate color channels of each of the input image frames;
generate at least one bad pixel map for each color channel of each input image frame, wherein the instructions that when executed cause the at least one processor to generate each bad pixel map comprise instructions that when executed cause the at least one processor to identify one or more outliers in pixel values in the corresponding color channel based on an intensity distribution of the pixel values in a specified operation window within the corresponding color channel, the specified operation window having a window size based on at least one of: the capture condition associated with the corresponding image frame and local image content in the corresponding image frame;
combine the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;
perform a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and
use one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the instructions that when executed cause the at least one processor to perform the morphological operation comprise:
instructions that when executed cause the at least one processor to remove clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

### Claim 17 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the instructions that when executed cause the at least one processor to perform the morphological operation comprise:
instructions that when executed cause the at least one processor to spatially filter the at least one combined bad pixel map to produce the at least one refined bad pixel map.

### Claim 18 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein:
The instructions that when executed cause the at least one processor to perform the morphological operation comprise instructions that when executed cause the at least one processor to perform erode operations and dilate operations;
the instructions that when executed cause the at least one processor to perform the erode operations comprise instructions that when executed cause the at least one processor to replace pixel values in the specified operation window with a minimum value; and
the instructions that when executed cause the at least one processor to perform the dilate operations comprise instructions that when executed cause the at least one processor to replace the pixel values in the specified operation window with a maximum value.

### Claim 19 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the instructions that when executed cause the at least one processor to generate the at least one bad pixel map for each color channel of each input image frame comprise:
instructions that when executed cause the at least one processor, for each color channel of each input iage frame, to:
identify first and Nth maximum and minimum pixel values in the color channel of the image frame, the first and Nth maximum and minimum pixel values based on pixel values within the specified operation window;
identify adaptive thresholds based on differences between the first and Nth maximum and minimum pixel values; and
apply the adaptive thresholds to identify the one or more outliers in the pixel values of the color channel.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, wherein the instructions that when executed cause the at least one processor to identify the adaptive thresholds comprise:
instructions that when executed cause the at least one processor to linearly update maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel values in the color channel of the image frame reach constant values.
