# US 2024/0005448 A1 · Imaging device and image processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005448 A1
- Document ID: US20240005448A1
- Status: Published application
- Kind: A1
- Application number: 17/901,023
- Filed: 2022-09-01
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hunsoo LEE; Soyoun PARK; Byungwook JUNG
- Web page: https://patentgrep.com/patent/US20240005448A1

## CPC

- G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
- G06T 3/4046: using neural networks
- G06T 2200/32: involving image mosaicing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing device is provided. The imaging device includes: an image sensor configured to generate an original image; and a processor including: a remosaic processing circuit configured to generate a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method, and generate a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and a merging circuit configured to generate a converted image having a Bayer pattern based on the first remosaic image and the second remosaic image.

## Claims

### Claim 1

An imaging device comprising:
an image sensor configured to generate an original image; and
a processor comprising:
a remosaic processing circuit configured to generate a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method, and generate a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and
a merging circuit configured to generate a converted image having a Bayer pattern based on the first remosaic image and the second remosaic image.

### Claim 2 (depends on 1)

The imaging device of claim 1, wherein the first remosaic method comprises a method of performing the remosaic processing by using a machine learning model, and
wherein the second remosaic method comprises a method of performing the remosaic processing based on a pre-set reference equation.

### Claim 3 (depends on 2)

The imaging device of claim 2, wherein the machine learning model comprises a neural network model.

### Claim 4 (depends on 2)

The imaging device of claim 2, wherein the remosaic processing circuit is further configured to:
select one or more machine learning models from among a plurality of machine learning models, based on quality of the first remosaic image and a generation time of the first remosaic image; and
perform the remosaic processing on the first region to generate the first remosaic image using the one or more machine learning models.

### Claim 5 (depends on 1)

The imaging device of claim 1, wherein the first region is a pre-set central region of the original image, and
wherein the second region is an entire region of the original image.

### Claim 6 (depends on 1)

The imaging device of claim 1, wherein the remosaic processing circuit is further configured to detect a seam in the original image and determine the first region based on the seam.

### Claim 7 (depends on 1)

The imaging device of claim 1, wherein the merging circuit is further configured to generate the converted image by merging the first remosaic image to a corresponding region of the second remosaic image.

### Claim 8 (depends on 7)

The imaging device of claim 7, wherein the merging circuit is further configured to generate the converted image by performing correction processing on boundary regions of the first remosaic image and the second remosaic image after merging the first remosaic image and the second remosaic image.

### Claim 9 (depends on 1)

The imaging device of claim 1, wherein the processor further comprises a selecting circuit configured to output one of the second remosaic image and the converted image based on a selection signal generated by the image sensor.

### Claim 10 (depends on 9)

The imaging device of claim 9, wherein according to the selection signal, the selecting circuit is configured to output the converted image based on illuminance of the original image being equal to or greater than a pre-set reference illuminance, and output the second remosaic image based on the illuminance of the original image being less than the pre-set reference illuminance.

### Claim 11 (depends on 9)

The imaging device of claim 9, wherein according to the selection signal, the selecting circuit is configured to output the converted image based on the original image being an image captured with a zoom ratio less than a pre-set reference zoom ratio, and output the second remosaic image based on the original image being an image captured with a zoom ratio equal to or greater than the pre-set reference zoom ratio.

### Claim 12

An image processing method comprising:
generating an original image through an image sensor;
generating, through a processor, a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method;
generating, through the processor, a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and
generating a converted image having a Bayer pattern based on the first remosaic image and the second remosaic image.

### Claim 13 (depends on 12)

The image processing method of claim 12, wherein the first remosaic method comprises a method of performing the remosaic processing by using a machine learning model, and
wherein the second remosaic method comprises a method of performing the remosaic processing based on a pre-set reference equation.

### Claim 14 (depends on 12)

The image processing method of claim 12, wherein the generating of the first remosaic image comprises:
selecting one or more machine learning models from among a plurality of machine learning models, based on quality of the first remosaic image and a generation time of the first remosaic image; and
generating the first remosaic image by performing the remosaic processing on the first region of the original image by using the one or more machine learning models.

### Claim 15 (depends on 12)

The image processing method of claim 12, further comprising outputting, through the processor, one of the second remosaic image and the converted image.

### Claim 16 (depends on 15)

The image processing method of claim 15, wherein the outputting comprises:
outputting the converted image based on illuminance of the original image being equal to or greater than a pre-set reference illuminance; and
outputting the second remosaic image based on the illuminance of the original image being less than the pre-set reference illuminance.

### Claim 17 (depends on 15)

The image processing method of claim 15,
wherein the outputting comprises:
outputting the converted image when a zoom ratio of the original image being less than a pre-set reference zoom ratio; and
outputting the second remosaic image based on the zoom ratio being equal to or greater than the pre-set reference zoom ratio.

### Claim 18

An imaging device comprising:
an image sensor configured to generate an original image; and
an image processor configured to perform remosaic processing by using first remosaic method and second remosaic method on the original image to generate a converted image having a Bayer pattern.

### Claim 19 (depends on 18)

The imaging device of claim 18, wherein the image processor is further configured to:
generate a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method;
generate a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and
generate the converted image based on the first remosaic image and the second remosaic image.

### Claim 20 (depends on 18)

The imaging device of claim 18, wherein the first remosaic method comprises a method of performing the remosaic processing by using a machine learning model, and
wherein the second remosaic method comprises a method of performing the remosaic processing based on a pre-set reference equation.
