# US 10,764,507 B1 · Image processing system capable of generating a snapshot image with high image quality by using a zero-shutter-lag snapshot operation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,764,507 B1
- Document ID: US10764507B1
- Status: Granted
- Kind: B1
- Application number: 16/387,565
- Filed: 2019-04-18
- Granted: 2020-09-01
- Earliest claimed: 2019-04-18
- Assignees: Kneron (Taiwan) Co., Ltd.
- Applicants: Kneron (Taiwan) Co., Ltd.
- Inventors: Hsiang-Tsun Li; Bike Xie; Junjie Su; Yi-Chou Chen
- Web page: https://patentgrep.com/patent/US10764507B1

## CPC

- H04N 23/617: Upgrading or updating of programs or applications for camera control
- H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
- H04N 23/651: {for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera}
- H04N 23/76: by influencing the image signals
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- H04N 25/13: characterised by the spectral characteristics of the filter elements
- H04N 25/46: by combining or binning pixels
- H04N 25/60: Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing system includes an image capturing device, a pixel binning device, a temporal filter, a first memory, a re-mosaic device, a second memory, and a blending device. The image capturing device is used for capturing a raw image. The pixel binning device is coupled to the image capturing device for outputting an enhanced image according to the raw image. The temporal filter is coupled to the pixel binning device for outputting a preview image according to the enhanced image. The first memory is used for buffering the raw image. The re-mosaic device is coupled to the first memory for outputting a processed image. The second memory is used for buffering the enhanced image. The blending device is coupled to the re-mosaic device and the second memory for outputting a snapshot image according to the processed image and the enhanced image.

## Claims

### Claim 1

An image processing system comprising:
an image capturing device configured to capture a raw image with a first resolution;
a pixel binning device coupled to the image capturing device and configured to output an enhanced image with a second resolution according to the raw image;
a temporal filter coupled to the pixel binning device and configured to output a preview image with the second resolution according to the enhanced image;
a first memory coupled to the image capturing device and configured to buffer the raw image;
a re-mosaic device coupled to the first memory and configured to output a processed image with the first resolution;
a second memory coupled to the temporal filter and configured to buffer the enhanced image; and
a blending device coupled to the re-mosaic device and the second memory and configured to output a snapshot image with the first image resolution according to the processed image and the enhanced image;
wherein the image capturing device, the pixel binning device, and the temporal filter are always enabled, the re-mosaic device and the blending device are optionally enabled according to a snapshot signal, and the first resolution is greater than the second resolution.

### Claim 2 (depends on 1)

The system of claim 1, wherein the image capturing device is configured to generate Bayer patterns in form of a color filter array overlaid on a photo sensor of the image capturing device, the color filter array comprises a plurality of sub-arrays, each sub-array comprises N×N square grids of identical primary color filters, and N is a positive integer greater than one.

### Claim 3 (depends on 2)

The system of claim 2, wherein the Bayer patterns are four-cell based Bayer patterns, and the each sub-array comprises 2×2 square grids.

### Claim 4 (depends on 2)

The system of claim 2, wherein the pixel binning device is configured to combine information of a part of square grids and generate standard Bayer patterns in order to generate the enhanced image with the second resolution, and the enhanced image has a standard Bayer format.

### Claim 5 (depends on 2)

The system of claim 2, wherein the re-mosaic device is configured to combine information of a part of square grids, generate standard Bayer patterns, and interpolate square grids of the standard Bayer patterns in order to generate the processed image with the first resolution, and the processed image has a standard Bayer format.

### Claim 6 (depends on 1)

The system of claim 1, wherein a signal-to-noise ratio (SNR) of the enhanced image is greater than a signal-to-noise ratio (SNR) of the raw image.

### Claim 7 (depends on 1)

The system of claim 1, wherein the temporal filter is configured to perform a noise reduction function by smoothing random fluctuations of noise of the enhanced image under an under exposure condition.

### Claim 8 (depends on 1)

The system of claim 1, wherein the blending device is configured to improve a signal-to-noise ratio (SNR) of the processed image by fusing information of the enhanced image buffered in the second memory with information of the processed image according to the processed image and the enhanced image.

### Claim 9 (depends on 8)

The system of claim 8, wherein after the information of the enhanced image buffered in the second memory is fused with the information of the processed image, the blending device outputs the snapshot image with the first image resolution, and a signal-to-noise ratio (SNR) of the snapshot image is greater than a signal-to-noise ratio (SNR) of the raw image.

### Claim 10 (depends on 1)

The system of claim 1, wherein the first image resolution is an original resolution of a photo sensor of the image capturing device, the second resolution is a thumbnail resolution, the first image resolution is substantially equal to 24M (Mega) pixels, and the second resolution is substantially equal to 6M (Mega) pixels.

### Claim 11 (depends on 1)

The system of claim 1, wherein the re-mosaic device and the blending device support a zero-shutter-lag (ZSL) snapshot operation.

### Claim 12 (depends on 1)

The system of claim 1, wherein the enhanced image and the preview image are generated according to the raw image in real time.

### Claim 13 (depends on 1)

The system of claim 1, wherein after the re-mosaic device and the blending device are idled for a predetermined time length, the re-mosaic device and the blending device enter a standby mode, after the snapshot signal triggers the re-mosaic device and the blending device, the re-mosaic device and the blending device enter an activation mode.

### Claim 14 (depends on 1)

The system of claim 1, wherein the pixel binning device is a hardware core module, and the re-mosaic device is a programmable core module.

### Claim 15 (depends on 1)

The system of claim 1, wherein the snapshot image has a Joint Photographic Experts Group (JPEG) format, and the blending device is configured to fuse direct current (DC) JPEG coefficients of the enhanced image with the alternating current (AC) JPEG coefficients of the processed image to generate the snapshot image.

### Claim 16 (depends on 1)

The system of claim 1, further comprising:
an artificial intelligence controller coupled to the pixel binning device, the temporal filter, the re-mosaic device, the blending device, and the second memory and configured to control the pixel binning device, the temporal filter, the re-mosaic device, and the blending device.

### Claim 17 (depends on 16)

The system of claim 16, wherein the artificial intelligence controller is configured to optimize operations of the pixel binning device, the temporal filter, the re-mosaic device, and the blending device according to the enhanced image buffered in the second memory.

### Claim 18 (depends on 16)

The system of claim 16, wherein the blending device is configured to improve a signal-to-noise ratio (SNR) of the processed image by fusing information of the enhanced image buffered in the second memory with information of the processed image according to the processed image and the enhanced image.

### Claim 19 (depends on 18)

The system of claim 18, wherein after the information of the enhanced image buffered in the second memory is fused with the information of the processed image, the blending device outputs the snapshot image with the first image resolution, and a signal-to-noise ratio (SNR) of the snapshot image is greater than a signal-to-noise ratio (SNR) of the raw image.

### Claim 20 (depends on 16)

The system of claim 16, wherein after the re-mosaic device and the blending device are idled for a predetermined time length, the re-mosaic device and the blending device enter a standby mode, after the snapshot signal triggers the re-mosaic device and the blending device, the re-mosaic device and the blending device enter an activation mode.