# US 11,445,122 B2 · Image processing method and apparatus, and device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,445,122 B2
- Document ID: US11445122B2
- Status: Granted
- Kind: B2
- Application number: 16/847,178
- Filed: 2020-04-13
- Published: 2020-07-30
- Granted: 2022-09-13
- Earliest claimed: 2017-10-13
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Yinting Wang; Biying Hu; Xi Zhang
- Web page: https://patentgrep.com/patent/US11445122B2

## CPC

- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- H04N 23/73: by influencing the exposure time
- G06T 3/14: Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/73: Deblurring; Sharpening
- G06T 5/80: Geometric correction
- G06T 5/92: based on global image properties
- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- H04N 23/71: Circuitry for evaluating the brightness variation
- H04N 23/72: Combination of two or more compensation controls
- G06T 2207/20216: Image averaging
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/20224: Image subtraction
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing method includes obtaining N frames of images, determining one of the N frames of images as a reference image, where the other N−1 frames of images are to-be-processed images, obtaining N−1 frames of ghost-free images based on the N−1 frames of to-be-processed images, and performing an averaging operation on the reference image and the N−1 frames of ghost-free images, to obtain a first target image.

## Claims

### Claim 1

An image processing method, comprising:
photographing one frame of a first new image based on a first International Organization of Standardization (ISO) sensitivity and a first exposure time of a camera;
increasing the first ISO sensitivity to a second ISO sensitivity;
decreasing the first exposure time to a second exposure time, wherein a first product of the first ISO sensitivity and the first exposure time is equal to a second product of the second ISO sensitivity and the second exposure time;
photographing N frames of images using the second ISO sensitivity and the second exposure time; and
obtaining a first target image based on the N frames of the images and the first new image,
wherein obtaining the first target image based on the N frames of the images and the first new image comprises:
determining one of the N frames of the images as a reference image, wherein the other N−1 frames of the images are to-be-processed images;
obtaining N−1 frames of ghost-free images based on the to-be-processed images;
performing an averaging operation on the reference image and the N−1 frames of ghost-free images to obtain a second target image;
performing registration on the first new image and the reference image to obtain a first registered image; and
either:
obtaining a first difference image based on the first registered image and the second target image, obtaining a first ghost weight image based on the first difference image, performing fusion on the first registered image and the second target image to obtain a first ghost-free image, and performing pixel-value weighted fusion to obtain the first target image based on the first ghost-free image; or
performing luminance correction on the second target image to obtain a third target image, obtaining a second difference image based on the first registered image and the third target image, obtaining a second ghost weight image based on the second difference image, performing fusion on the first registered image and the third target image to obtain a second ghost-free image, performing pixel-value weighted fusion on the second ghost-free image and the third target image to obtain a fourth target image, and performing pyramid fusion on the fourth target image and the second target image to obtain the first target image.

### Claim 2 (depends on 1)

The image processing method of claim 1, wherein before photographing the N frames of the images, the image processing method further comprises generating a control signal when a to-be-photographed image of the camera is a moving image, the first exposure time exceeds a safe time, or the first ISO sensitivity is in a first preset threshold interval and the first exposure time is in a second preset threshold interval, and wherein the control signal instructs the camera to photograph the N frames of the images.

### Claim 3 (depends on 1)

The image processing method of claim 1, wherein performing fusion on the first registered image and the second target image to obtain the first ghost-free image, and performing pixel-value weighted fusion to obtain the first target image based on the first ghost-free image comprises:
performing fusion on the first registered image and the second target image to obtain the first ghost-free image based on the first ghost weight image; and
performing pixel-value weighted fusion to obtain the first target image based on the first ghost-free image and the first target image.

### Claim 4 (depends on 1)

The image processing method of claim 1, wherein performing pixel-value weighted fusion to obtain the first target image based on the first ghost-free image comprises:
performing pixel-value weighted fusion on the second ghost-free image and the second target image to obtain the first target image.

### Claim 5

An image processing apparatus, comprising:
a processor; and
a memory coupled to the processor and storing instructions that, when executed by the processor, cause the image processing apparatus to be configured to:
photograph one frame of a first new image based on a first International Organization of Standardization (ISO) sensitivity and a first exposure time of a camera;
increase the first ISO sensitivity to a second ISO sensitivity;
decrease the first exposure time to obtain a second exposure time, wherein a first product of the first ISO sensitivity and the first exposure time is equal to a second product of the second ISO sensitivity and the second exposure time;
photograph N frames of images using the second ISO sensitivity and the second exposure time; and
obtain a first target image based on the N frames of the images and the first new image at least in part by:
determining one of the N frames of the images as a reference image, wherein the other N−1 frames of the images are to-be-processed images;
obtaining N−1 frames of ghost-free images based on the to-be-processed images;
performing an averaging operation on the reference image and the N−1 frames of ghost-free images to obtain a second target image;
performing registration on the first new image and the reference image to obtain a first registered image; and
either:
obtaining a first difference image based on the first registered image and the second target image, obtaining a first ghost weight image based on the first difference image, performing fusion on the first registered image and the second target image to obtain a first ghost-free image, and performing pixel-value weighted fusion to obtain the first target image; or
performing luminance correction on the second target image to obtain a third target image, obtaining a second difference image based on the first registered image and the third target image, obtaining a second ghost weight image based on the second difference image, performing fusion on the first registered image and the third target image to obtain a second ghost-free image, performing pixel-value weighted fusion on the second ghost-free image and the third target image to obtain a fourth target image, and performing pyramid fusion on the fourth target image and the second target image to obtain the first target image.

### Claim 6 (depends on 5)

The image processing apparatus of claim 5, wherein the instructions further cause the image processing apparatus to be configured to control obtaining of the N frames of the images when a to-be-photographed image of the camera is a moving image, the first exposure time exceeds a safe time, or the first ISO sensitivity is in a first preset threshold interval and the first exposure time is in a second preset threshold interval.

### Claim 7 (depends on 5)

The image processing apparatus of claim 5, wherein the instructions further cause the image processing apparatus to be configured to:
perform fusion on the first registered image and the second target image to obtain the first ghost-free image based on the first ghost weight image; and
perform pixel-value weighted fusion to obtain the first target image based on the first ghost-free image and the second target image.

### Claim 8 (depends on 5)

The image processing apparatus of claim 5, wherein the instructions further cause the image processing apparatus to be configured to:
perform pixel-value weighted fusion on the second ghost-free image and the second target image to obtain the first target image.

### Claim 9

A terminal device, comprising:
an image processing apparatus comprising:
a processor; and
a memory coupled to the processor and storing instructions that, when executed by the processor, cause the terminal device to be configured to:
photograph one frame of a first new image based on current International Organization of Standardization (ISO) sensitivity and current exposure time of a camera;
reduce the current exposure time and increasing the current ISO sensitivity at a preset ratio to obtain a second exposure time and a second ISO sensitivity with a product of the current ISO sensitivity and the current exposure time unchanged;
set the current exposure time and the current ISO sensitivity to the second exposure time and the second ISO sensitivity respectively;
photograph N frames of a plurality of images; and
obtain a first target image based on the N frames of the images and the first new image at least in part by:
determining one of the N frames of the images as a reference image, wherein the other N−1 frames of the images are to-be-processed images;
obtaining N−1 frames of ghost-free images based on the to-be-processed images;
performing an averaging operation on the reference image and the N−1 frames of ghost-free images to obtain a second target image;
performing registration on the first new image and the reference image to obtain a first registered image; and
either:
obtaining a first difference image based on the first registered image and the second target image, obtaining a first ghost weight image based on the first difference image, performing fusion on the first registered image and the second target image to obtain a first ghost-free image, and performing pixel-value weighted fusion to obtain the first target image; or
performing luminance correction on the second target image to obtain a third target image, obtaining a second difference image based on the first registered image and the third target image, obtaining a second ghost weight image based on the second difference image, performing fusion on the first registered image and the third target image to obtain a second ghost-free image, performing pixel-value weighted fusion on the second ghost-free image and the third target image to obtain a fourth target image, and performing pyramid fusion on the fourth target image and the second target image to obtain the first target image.

### Claim 10 (depends on 9)

The terminal device of claim 9, wherein the terminal device further comprises an antenna coupled to the image processing apparatus and is configured to:
send a radio communication signal to implement radio communication with a mobile communications network that comprises one or more of a Global System for Mobile Communications (GSM) network, a code division multiple access (CDMA) network, a third generation (3G) network, a fourth generation (4G) network, frequency division multiple access (FDMA), time division multiple access (TDMA), Personal Digital Cellular (PDC), Total Access Communication System (TACS), Advanced Mobile Phone System (AMPS), wideband CDMA (WCDMA), time division-synchronous CDMA (TDSCDMA), WI-FI, or a Long Term Evolution (LTE) network; and
receive the communication signal.

### Claim 11 (depends on 9)

The terminal device of claim 9, wherein the instructions further cause the terminal device to be configured to generate a control signal when one or more of:
a to-be-photographed image of the camera is a moving image;
the current exposure time exceeds a safe time; or
the current ISO sensitivity is in a first preset threshold interval and the current exposure time is in a second preset threshold interval, and
wherein the control signal instructs to obtain the N frames of the images.

### Claim 12 (depends on 9)

The terminal device of claim 9, wherein the instructions further cause the terminal device to be configured to:
perform fusion on the first registered image and the second target image to obtain the first ghost-free image based on the first ghost weight image; and
perform pixel-value weighted fusion to obtain the first target image based on the first ghost-free image and the second target image.

### Claim 13 (depends on 9)

The terminal device of claim 9, wherein the instructions cause the terminal device to be configured to:
perform pixel-value weighted fusion on the second ghost-free image and the second target image to obtain the first target image.

### Claim 14 (depends on 9)

The terminal device of claim 9, wherein the instructions further cause the terminal device to be configured to enter into a snapshot mode.

### Claim 15 (depends on 11)

The terminal device of claim 11, wherein the instructions further cause the terminal device to be configured to determine that the to-be-photographed image of the camera is the moving image by performing motion detection on a preview data stream to analyze a photographing preview stream.

### Claim 16 (depends on 9)

The terminal device of claim 9, wherein the instructions further cause the terminal device to be configured to enter a first snapshot mode when a to-be-photographed image is a moving image, a current exposure time of the camera exceeds a safe time, and an environment of the to-be-photographed scene is an extreme high-light environment.

### Claim 17 (depends on 16)

The terminal device of claim 16, wherein the instructions further cause the terminal device to be configured to enter a second snapshot mode when the to-be-photographed image is the moving image, and the current exposure time of the camera exceeds the safe time, and the environment of the to-be-photographed scene is a moderate high-light environment.