# US 2012/0250976 A1 · Wavelet transform on incomplete image data and its applications in image processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0250976 A1
- Document ID: US20120250976A1
- Status: Published application
- Kind: A1
- Application number: 13/065,729
- Filed: 2011-03-29
- Published: 2012-10-04
- Earliest claimed: 2011-03-29
- Assignees: Sony Corporation
- Inventors: Xiaogang Dong; Farhan Baqai; Kenichi Nishio
- Web page: https://patentgrep.com/patent/US20120250976A1

## CPC

- G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
- G06T 5/70: Denoising; Smoothing
- H04N 23/84: for processing colour signals
- H04N 25/134: {based on three different wavelength filter elements}
- G06T 2207/20064: Wavelet transform [DWT]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system and method for effectively performing wavelet transforms on incomplete image data includes an image processor that performs a green-pixel transformation procedure on incomplete color pixel matrices. The image processor then rearranges red, blue and transformed green-pixel into four quadrants of contiguous pixels and applies some two dimensional (2D) wavelet thresholding schemes on each quadrant. After thresholding, an inverse procedure is applied to reconstruct the pixel values on the incomplete color pixel matrices. For further de-correlation of image data, the image processor may stack similar image patches in a three dimensional (3D) array and apply incomplete-data wavelet thresholding on the 3D array. The incomplete-data wavelet thresholding procedure may be put in an improved local similarity measurement framework to achieve better performance of image processing tasks. A CPU device typically controls the image processor to effectively perform the image processing procedure.

## Claims

### Claim 1

A system for performing an enhanced image processing procedure with an electronic device, comprising:
an image processor that performs a 2D incomplete-data wavelet transform and thresholding procedure, said 2D incomplete-data wavelet transform being performed on each patch of a 3D array consisting of similar patches, a thresholding scheme being applied to the wavelet coefficients of said 3D array, said image processor building said 3D array for a selected target patch, said image processor performing said incomplete-data wavelet transform and thresholding procedure upon said 3D array to produce an intermediate result, said image processor combining the intermediate results of multiple 3D-arrays to produce final results; and
a processor controller device that controls said image processor to perform said image processing procedure upon image data.

### Claim 2 (depends on 1)

The system of claim 1 wherein said image data is initially configured according to a Bayer Pattern format of interleaved red pixels, green pixels, and blue pixels.

### Claim 3 (depends on 2)

The system of claim 2 wherein said image processor initially separates said image data into a red matrix of said red pixels, a green matrix of said green pixels, and a blue matrix of said blue pixels.

### Claim 4 (depends on 3)

The system of claim 3 wherein said image processor subsamples said red matrix to produce a red quadrant of said red pixels, said image processor also subsampling said blue matrix to produce a blue quadrant of said blue pixels.

### Claim 5 (depends on 4)

The system of claim 4 wherein said image processor performs said green-pixel transformation procedure on adjacent diagonal pairs of said green pixels from said green matrix.

### Claim 6 (depends on 5)

The system of claim 5 wherein said green-pixel transformation procedure includes a 1-level wavelet transform that converts said adjacent diagonal pairs of green pixels into corresponding A terms and D terms in a transformed green matrix.

### Claim 7 (depends on 6)

The system of claim 6 wherein said image processor subsamples said transformed green matrix to produce an A quadrant of said A terms and a D quadrant of said D terms.

### Claim 8 (depends on 7)

The system of claim 7 wherein said image processor positions said R quadrant, said A quadrant, said D quadrant, and said B quadrant to produce a four-quadrant array.

### Claim 9 (depends on 8)

The system of claim 8 wherein said image processor performs a 2D wavelet transform on each quadrant in said four-quadrant array to produce wavelet coefficients.

### Claim 10 (depends on 9)

The system of claim 9 wherein said image processor performs a wavelet coefficients thresholding procedure upon said wavelet coefficients of said four-quadrant array to produce thresholded coefficients.

### Claim 11 (depends on 10)

The system of claim 10 wherein said image processor performs an inverse 2D wavelet transform on said thresholded coefficients to produce four reconstructed quadrants for said four-quadrant array.

### Claim 12 (depends on 11)

The system of claim 11 wherein said image processor performs an inverse 1-level wavelet transform upon a reconstructed A quadrant and a reconstructed D quadrant from said reconstructed four-quadrant array to produce reconstructed green pixels.

### Claim 13 (depends on 12)

The system of claim 12 wherein said image processor rearranges a reconstructed R quadrant, a reconstructed B quadrant, and said green pixels into said Bayer Pattern format to populate a final processed version of said four-quadrant array.

### Claim 14 (depends on 1)

The system of claim 1 wherein said image processor selects a target patch and builds a 3D array of said target patch and one or more similar patches of said target patch.

### Claim 15 (depends on 14)

The system of claim 14 wherein said image processor performs a 2D incomplete-data wavelet transform on each patch in said 3D array to produce first wavelet coefficients, said image processor then performing a 1D wavelet transform on each set of corresponding first coefficients in a third dimension to produce second wavelet coefficients.

### Claim 16 (depends on 15)

The system of claim 15 wherein said image processor performs a wavelet coefficients thresholding procedure upon said second coefficients to produce thresholded coefficients.

### Claim 17 (depends on 16)

The system of claim 16 wherein said image processor performs an inverse 1D wavelet transform on corresponding sets of said thresholded coefficients to produce first reconstructed coefficients, said image processor then performing an inverse transform of said 2D incomplete-data wavelet transform on said first reconstructed coefficients to produce a reconstructed 3D array.

### Claim 18 (depends on 17)

The system of claim 17 wherein said final reconstructed 3D array provides an intermediate result values for storing into an intermediate results buffer.

### Claim 19 (depends on 18)

The system of claim 18 wherein said image processor combines said intermediate results of multiple 3D arrays to produce better final results of image processing tasks including denoising.

### Claim 20

A method for performing an enhanced image processing procedure with an electronic device, comprising the steps of:
performing an incomplete-data wavelet transform and thresholding procedure upon a 3D array consisting of a target patch and one or more similar patches, the reconstructed values of said 3D array being saved as intermediate results in an intermediate result buffer;
performing an intermediate-results combination procedure with said image processor by combining said intermediate result values to produce processed pixel values; and
utilizing a processor controller device to control said image processor for performing said image processing procedure.
