# US 2021/0342986 A1 · Method for generating images with high dynamic range during multiple exposures and times of capture, device employing method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342986 A1
- Document ID: US20210342986A1
- Status: Published application
- Kind: A1
- Application number: 17/243,903
- Filed: 2021-04-29
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: Chiun Mai Communication Systems, Inc.
- Applicants: Chiun Mai Communication Systems, Inc.
- Inventors: Sheng-Yeh Chen; Yung-Yu Chuang; TZU-KUEI HUANG; NAI-SHENG SYU; YU-CHING WANG; TING-HAO CHUNG; CHUN-HSIANG HUANG
- Web page: https://patentgrep.com/patent/US20210342986A1

## CPC

- G06T 5/92: based on global image properties
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06N 20/00: Machine learning
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10144: Varying exposure
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for generating images with high dynamic range (HDR) based on multiple images captured at different aperture values, under different conditions, or at different shutter speeds is applied in a device. The method inputs the original multiple images into a predetermined model and aligns the multiple images. The method further confirms object images that need to be attended among multiple aligned images and obtains a merge weighting for each of the object images, and merges the images for a generated HDR according to the merge weighting of each image. The device utilizing the method is also disclosed.

## Claims

### Claim 1

A high dynamic range (HDR) image generating method comprising:
inputting a plurality of first images of a scene into a predetermined model, wherein the plurality of first images is captured at different aperture values or different shutter speeds;
aligning the plurality of first images by the predetermined model to obtain a plurality of second images;
confirming a plurality of third images that need to be attended among the plurality of second images;
obtaining a merge weighting of each of the plurality of third images; and
merging a generated HDR image according to the merge weighting of each of the plurality of third images.

### Claim 2 (depends on 1)

The HDR image generating method of claim 1, further comprising:
determining whether a merging result of the generated HDR image meets a predetermined requirement based on a set of real HDR images and fake HDR images.

### Claim 3 (depends on 2)

The HDR image generating method of claim 2, wherein determining whether the merging result of the generated HDR image meets the predetermined requirement based on the set of real HDR images and fake HDR images comprises:
determining the merging result of the generated HDR image meets the predetermined requirement in response to the generated HDR image is regarded as the real HDR image.

### Claim 4 (depends on 1)

The HDR image generating method of claim 1, wherein aligning the plurality of first images by the predetermined model to obtain the plurality of second images comprises:
aligning the plurality of first images by a deep learning architecture to obtain the plurality of second images.

### Claim 5 (depends on 1)

The HDR image generating method of claim 1, wherein confirming the plurality of third images that need to be attended among the plurality of second images comprises:
performing feature extractions on the plurality of second images;
defining attention levels of image regions of each of the plurality of second images according to feature extraction results of the plurality of second images;
generating a weighting distribution map according to the attention levels of the image regions of each of the plurality of second images; and
confirming the plurality of third images that need to be attended among the plurality of second images according to the weighting distribution map.

### Claim 6 (depends on 1)

The HDR image generating method of claim 1, wherein obtaining the merge weighting of each of the plurality of third images comprises:
obtaining the merge weighting of pixels of each of the plurality of third images.

### Claim 7 (depends on 1)

The HDR image generating method of claim 1, wherein the predetermined model is trained based on a predetermined sample image dataset, and the predetermined sample image dataset comprises multiple sample images and multiple fourth images that are generated by performing a random perspective transformation on the multiple sample images.

### Claim 8 (depends on 1)

The HDR image generating method of claim 1, wherein obtaining the merge weighting of each of the plurality of third images comprises:
obtaining the merge weighting of each of the plurality of third images according to brightness of image regions of each of the plurality of third images.

### Claim 9

A high dynamic range (HDR) image generating device comprising:
at least one processor; and
a data storage storing one or more programs which when executed by the at least one processor, cause the at least one processor to:
input a plurality of first images of a scene into a predetermined model, wherein the plurality of first images is captured at different aperture values or different shutter speeds;
align the plurality of first images by the predetermined model to obtain a plurality of second images;
confirm a plurality of third images that need to be attended among the plurality of second images;
obtain a merge weighting of each of the plurality of third images; and
merge a generated HDR image according to the merge weighting of each of the plurality of third images.

### Claim 10 (depends on 9)

The HDR image generating device of claim 9, wherein the at least one processor is further configured to:
determining whether a merging result of the generated HDR image meets a predetermined requirement based on a set of real HDR images and fake HDR images.

### Claim 11 (depends on 10)

The HDR image generating device of claim 10, wherein the at least one processor to determine whether the merging result of the generated HDR image meets the predetermined requirement based on the set of real HDR images and fake HDR images comprises:
determine the merging result of the generated HDR image meets the predetermined requirement in response to the generated HDR image is regarded as the real HDR image.

### Claim 12 (depends on 9)

The HDR image generating device of claim 9, wherein the at least one processor to align the plurality of first images by the predetermined model to obtain the plurality of second images comprises:
align the plurality of first images by a deep learning architecture to obtain the plurality of second images.

### Claim 13 (depends on 9)

The HDR image generating device of claim 9, wherein the at least one processor to confirm the plurality of third images that need to be attended among the plurality of second images comprises:
perform feature extractions on the plurality of second images;
define attention levels of image regions of each of the plurality of second images according to feature extraction results of the plurality of second images;
generate a weighting distribution map according to the attention levels of the image regions of each of the plurality of second images; and
confirm the plurality of third images that need to be attended among the plurality of second images according to the weighting distribution map.

### Claim 14 (depends on 9)

The HDR image generating device of claim 9, wherein the at least one processor to obtain the merge weighting of each of the plurality of third images comprises:
obtain the merge weighting of pixels of each of the plurality of third images.

### Claim 15 (depends on 9)

The HDR image generating device of claim 9, wherein the predetermined model is trained based on a predetermined sample image dataset, and the predetermined sample image dataset comprises multiple sample images and multiple fourth images that are generated by performing a random perspective transformation on the multiple sample images.

### Claim 16 (depends on 9)

The HDR image generating device of claim 9, wherein the at least one processor to obtain the merge weighting of each of the third images comprises:
obtain the merge weighting of each of the plurality of third images according to brightness of image regions of each of the plurality of third images.

### Claim 17

A non-transitory storage medium having stored thereon instructions that, when executed by a processor of an electronic device, causes the electronic device to perform a high dynamic range (HDR) image generating method, the HDR image generating method comprising:
inputting a plurality of first images of a scene into a predetermined model, wherein the plurality of first images is captured at different aperture values or different shutter speeds;
aligning the plurality of first images by the predetermined model to obtain a plurality of second images;
confirming a plurality of third images that need to be attended among the plurality of second images;
obtaining a merge weighting of each of the plurality of third images; and
merging a generated HDR image according to the merge weighting of each of the plurality of third images.
