# US 2026/0237023 A1 · System and method for generating an all-in-focus image from multi-focus images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0237023 A1
- Document ID: US20260237023A1
- Status: Published application
- Kind: A1
- Application number: 19/051,711
- Filed: 2025-02-12
- Published: 2026-08-13
- Earliest claimed: 2025-02-12
- Assignees: Hong Kong Applied Science and Technology Research Institute Company Limited
- Applicants: Hong Kong Applied Science and Technology Research Institute Co., Ltd.
- Inventors: Shangping LIU; Tianxiao XU; Lu WANG
- Web page: https://patentgrep.com/patent/US20260237023A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/571: from focus
- G06T 5/20: using local operators
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- G06T 7/55: from multiple images
- G06T 2200/21: involving computational photography
- G06T 2207/10024: Color image
- G06T 2207/10052: Images from lightfield camera
- G06T 2207/10056: Microscopic image
- G06T 2207/10148: Varying focus
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A computer-implemented method of fusing multiple images, where the multiple images are taken at different focal planes on a same field of view (FOV). The method includes the steps of a) to each one of the multiple images, performing Laplacian pyramid decomposition to generate a first pyramid; b) fusing the first pyramids of the multiple images to obtain a fused pyramid under guidance of energy maps respectively corresponding to each one of the multiple mages; c) performing a dynamic weighting of the fused pyramid to generate a weighted pyramid; and d) reconstructing a fusion image of the FOV from the weighted pyramid. The invention provides further a computer-implemented method of generating a depth map based on multiple images.

## Claims

### Claim 1

A computer-implemented method of fusing multiple images; the multiple images taken at different focal planes on a same field of view; the method comprising steps of:
a) to each color channel of each one of the multiple images, performing Laplacian pyramid decomposition to generate a first pyramid;
b) for each said color channel, fusing the first pyramids of the color channel of the multiple images to obtain a fused pyramid for the color channel under guidance of energy maps respectively corresponding to each one of the multiple mages;
c) performing a dynamic weighting of the fused pyramid to generate a weighted pyramid for each said color channel;
d) reconstructing a channel fusion image for each said color channel from the weighted pyramid; and
e) merging the channel fusion images of all the color channels to obtain a final fused image.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising a step of:
f) for each one of the multiple images, calculating the energy map using a linked Laplacian pyramid decomposition.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein Step f) comprises:
g) for each one of the multiple images, performing the linked Laplacian pyramid decomposition to generate a second pyramid, during which information of a bottom layer of the second pyramid is gradually introduced to a top layer of the corresponding pyramid by applying weighted averages.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, further comprises, before Step f), steps of converting the one of the multiple images to a grayscale image, and using the grayscale image for performing the linked Laplacian pyramid decomposition.

### Claim 5 (depends on 3)

The computer-implemented method of claim 3 wherein Step f) further comprises:
h) conducting Gaussian smoothing to the second pyramid to obtain the energy map for each one of the multiple images.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein in Step b), a pixel that has a highest energy among all similar pixels in the multiple images is chosen as a pixel in the fused pyramid.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein in Step c), a linear function is applied to the fused pyramid, such that weights of a first plurality of layers of the fused pyramid are enhanced, and weights of a second plurality of layers of the fused pyramid are reduced.

### Claim 8

A computer-implemented method of generating a depth map based on multiple images; the multiple images taken at different focal planes on a same field of view; the method comprising steps of:
a) for each one of the multiple images, conducting focus measurement to generate an initial focus map;
b) estimating an uncertainty map for the multiple images based on the initial focus maps;
c) refining the initial focus maps according to the uncertain map and a fusion image of the multiple images to obtain filtered focus maps; and
d) conducting depth acquisition on the filtered focus maps to generate the depth map.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein Step b) comprises steps of:
e) calculating peak signal-to-noise ratio (PSNR) of each pixel in the field of view based on the initial focus maps; and
f) generating the uncertain map based on results of comparison between the PSNR of each said pixel and a threshold.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein in Step e) the PSNR of each said pixel is calculated based on a mean-square-error (MSE) of the pixel and a maximum focus value of the pixel.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, wherein Step c) comprises steps of:
g) masking out uncertainty areas in the initial focus maps based on the uncertainty map; and
h) applying a guided filter based on the fusion image to the initial fusion maps to obtain the filtered focus maps.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, further comprising a step of:
i) performing a topmost focus detection on the filtered focus maps; and
j) based on a result of the topmost focus detection and the depth map, generating a final depth map.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein Step i) further comprises:
k) finding a topmost local peak for each pixel in the field of view in order to generate a topmost depth map; and
wherein Step j) further comprises:
l) selecting a preferable topmost depth for each said pixel in the field of view based on the uncertainty map; and
m) generating the final depth map based on the preferable topmost depths of the pixels.

### Claim 14 (depends on 1)

A computing system comprising:
a) one or more processors; and
b) memory containing instructions that, when executed by the one or more processors, cause the computing system to perform the method according to claim 1.
