# US 10,375,321 B2 · Imaging processing method and electronic device supporting the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,375,321 B2
- Document ID: US10375321B2
- Status: Granted
- Kind: B2
- Application number: 15/484,601
- Filed: 2017-04-11
- Published: 2017-10-12
- Granted: 2019-08-06
- Earliest claimed: 2016-04-11
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Youngkwon Yoon; Jin Hee Na; Pyojae Kim; Jonghoon Won; Jongmin Hyun
- Web page: https://patentgrep.com/patent/US10375321B2

## CPC

- H04N 5/265: Mixing
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation}
- H04N 23/10: for generating image signals from different wavelengths
- H04N 23/632: {for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters}
- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- H04N 23/6811: {based on the image signal}
- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- H04N 25/40: Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N 25/441: by reading contiguous pixels from selected rows or columns of the array, e.g. interlaced scanning
- G06T 2207/20208: High dynamic range [HDR] image processing
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An electronic device and a method thereof are provided for processing stored images to reconstruct a high-resolution image. The electronic device includes an image sensor configured to acquire an image through a plurality of pixels including a first pixel set and a second pixel set; and a processor configured to acquire at least one first image through the first pixel set; acquire at least one second image through the second pixel set; and generate at least one third image according to a selected mode for generating the at least one third image, by using the at least one first image and the at least one second image.

## Claims

### Claim 1

An electronic device, comprising:
an image sensor configured to acquire an image through a plurality of pixels including a first pixel set and a second pixel set;
a memory; and
a processor configured to:
acquire a first image at a first time through the first pixel set;
acquire a second image at a second time through the second pixel set;
acquire a third image at a third time through the first pixel set;
acquire a fourth image at a fourth time through the second pixel set;
store the first image, the second image, the third image, and the fourth image in the memory;
reconstruct the stored third image based on the stored first image;
reconstruct the stored fourth image based on the stored second image; and
generate a fifth image corresponding to a camera function by using at least one of the reconstructed third image and the reconstructed fourth image.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein at least one pixel of the first pixel set and at least one pixel of the second pixel set are adjacent to each other.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein at least one of the first pixel set and the second pixel set comprises a sub-pixel set.

### Claim 4 (depends on 3)

The electronic device of claim 3, wherein the sub-pixel set comprises at least one RGB pixel of a Bayer pattern.

### Claim 5 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to reconstruct the stored third image based on the stored first image by:
detecting a first motion between the stored first image and the stored third image,
estimating first data of first pixels of among the plurality of pixels based on the detected first motion, and
reconstructing the stored third image by deinterlacing the first pixels with the estimated first data, wherein the first pixels are not sampled when the stored third image is acquired, and
wherein the processor is configured to reconstruct the stored fourth image based on the stored second image by:
detecting a second motion between the stored second image and the stored fourth image,
estimating second data of second pixels among the plurality of pixels based on the detected second motion, and
reconstructing the stored fourth image by deinterlacing the second pixels with the estimated second data, wherein the second pixels are not sampled when the stored fourth image is acquired.

### Claim 6 (depends on 5)

The electronic device of claim 5, wherein, when the camera function is a still image capturing function, the processor is further configured to generate the fifth image by:
selecting a best photo of the reconstructed third image and the reconstructed fourth image based on an analysis of the reconstructed third image and the reconstructed fourth image, and
generate the fifth image with the best photo as a still image.

### Claim 7 (depends on 5)

The electronic device of claim 5, wherein, when the camera function is a video capturing function, the processor is further configured to generate the fifth image by:
identifying at least one of a set resolution and a set frame rate, and
generating the fifth image by using the reconstructed third image and the reconstructed fourth image according to the at least one of the set resolution and the set frame rate as a video.

### Claim 8 (depends on 5)

The electronic device of claim 5, wherein, when the camera function is a burst shot function, the processor is further configured to generate the fifth image by:
identifying at least one of a set image size and a set interval, and
generating the fifth image by using the reconstructed third image and the reconstructed fourth image according to the at least one of the set image size and the set interval as a burst shot image.

### Claim 9 (depends on 5)

The electronic device of claim 5, wherein, when the camera function is a high dynamic range (HDR) capturing function, the processor is further configured to generate the fifth image by:
analyzing a first image grayscale of the reconstructed third image and a second image grayscale of the reconstructed fourth image, correcting the first image grayscale and the second image grayscale, and
generating the fifth image by using the reconstructed third image having the corrected first image grayscale and the reconstructed fourth image having the corrected second image grayscale.

### Claim 10

An electronic device, comprising:
an image sensor configured to acquire an image through a plurality of pixels including a first pixel set and a second pixel set;
a memory; and
a processor configured to:
acquire a plurality of first images through the first pixel set;
acquire a plurality of second images through the second pixel set;
store the plurality of first images and the plurality of second images;
reconstruct at least one first image among the plurality of stored first images based on a first reference image among the plurality of stored first images, the first reference image being acquired in advance of the at least one first image;
reconstruct at least one second image among the plurality of stored second images based on a second reference image among the plurality of stored second images, the second reference image being acquired in advance of the at least one second image; and
generate at least one third image according to a camera function, by using at least one of the at least one reconstructed first image and the at least one reconstructed second image.

### Claim 11

An image processing method of an electronic device, the image processing method comprising:
acquiring a first image at a first time through a first pixel set among a plurality of pixels of an image sensor of the electronic device;
acquiring a second image at a second time through a second pixel set among the plurality of pixels of the image sensor;
acquiring a third image at a third time through the first pixel set;
acquiring a fourth image at a fourth time through the second pixel set;
storing the first image, the second image, the third image, and the fourth image in a memory of the electronic device;
reconstructing the stored third image based on the stored first image;
reconstructing the stored fourth image based on the stored second image; and
generating a fifth image corresponding to a camera function by using at least one of the reconstructed third image and the reconstructed fourth image.

### Claim 12 (depends on 11)

The image processing method of claim 11, wherein at least one pixel of the first pixel set and at least one pixel of the second pixel set are adjacent to each other.

### Claim 13 (depends on 11)

The image processing method of claim 11, wherein at least one of the first pixel set and the second pixel set comprises a sub-pixel set.

### Claim 14 (depends on 13)

The image processing method of claim 13, wherein the sub-pixel set comprises at least one RGB pixel of a Bayer pattern.

### Claim 15 (depends on 11)

The image processing method of claim 11, wherein reconstructing the stored third image comprises:
detecting a first motion between the stored first image and the stored third image;
estimating first data of first pixels among the plurality of pixels based on the detected first motion; and
reconstructing the stored third image by deinterlacing the first pixels with the estimated first data, wherein the first pixels are not sampled when the stored third image is acquired, and
wherein reconstructing the stored fourth image comprises:
detecting a second motion between the stored second image and the stored fourth image;
estimating second data of second pixels among the plurality of pixels based on the detected second motion; and
reconstructing the stored fourth image by deinterlacing the second pixels with the estimated second data, wherein the second pixels are not sampled when the stored fourth image is acquired.

### Claim 16 (depends on 15)

The image processing method of claim 15, wherein, when the camera function is a still image capturing function, generating the fifth image comprises:
selecting a best photo of the reconstructed third image and the reconstructed fourth image based on an analysis of the reconstructed third image and the reconstructed fourth image; and
generating the fifth image with the best photo as a still image.

### Claim 17 (depends on 15)

The image processing method of claim 15, wherein, when the camera function is a video capturing function, generating the fifth image comprises:
identifying at least one of a set resolution and a set frame rate; and
generating the fifth image by using the reconstructed third image and the reconstructed fourth image according to the at least one of the set resolution and the set frame rate as a video.

### Claim 18 (depends on 15)

The image processing method of claim 15, wherein, when the camera function is a burst shot function, generating the fifth image comprises:
identifying at least one of a set image size and a set interval; and
generating the fifth image by using the reconstructed third image and the reconstructed fourth image according to the at least one of the set image size and the set interval as a burst shot image.

### Claim 19 (depends on 15)

The image processing method of claim 15, wherein, when the camera function is a high dynamic range (HDR) capturing function, generating the fifth image comprises:
analyzing a first image grayscale of the reconstructed third image and a second image grayscale of the reconstructed fourth image;
correcting the first image grayscale and the second image grayscale; and
generating the fifth image by using the reconstructed third image having the corrected first image grayscale and the reconstructed fourth image having the corrected second image grayscale.