# US 2023/0419437 A1 · Systems and methods for fusing images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0419437 A1
- Document ID: US20230419437A1
- Status: Published application
- Kind: A1
- Application number: 18/463,179
- Filed: 2023-09-07
- Published: 2023-12-28
- Earliest claimed: 2016-09-30
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: James Wilson NASH; Kalin Mitkov ATANASSOV; Sergiu Radu GOMA
- Web page: https://patentgrep.com/patent/US20230419437A1

## CPC

- G06T 3/053: Detail-in-context presentations (fisheye or wide-angle transformations G06T3/047)
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06T 5/20: using local operators
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/70: Denoising; Smoothing
- G06T 7/11: Region-based segmentation
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- H04N 5/2621: {Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability}
- H04N 9/76: for mixing of colour signals (H04N9/75 takes precedence)
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N 23/81: for suppressing or minimising disturbance in the image signal generation
- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- H04N 25/60: Noise processing, e.g. detecting, correcting, reducing or removing noise
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20004: Adaptive image processing
- G06T 2207/20221: Image fusion; Image merging
- H04N 23/58: Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method performed by an electronic device is described. The method includes obtaining a first image from a first camera, the first camera having a first focal length and a first field of view. The method also includes obtaining a second image from a second camera, the second camera having a second focal length and a second field of view disposed within the first field of view. The method further includes aligning at least a portion of the first image and at least a portion of the second image to produce aligned images. The method additionally includes fusing the aligned images based on a diffusion kernel to produce a fused image. The diffusion kernel indicates a threshold level over a gray level range. The method also includes outputting the fused image. The method may be performed for each of a plurality of frames of a video feed.

## Claims

### Claim 1

An apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory and configured to:
select, for a fused image, pixels from a first area of a first image;
select, for the fused image, pixels from a first area of a second image;
blend pixels in a second area of the first image and pixels in a second area of the second image to obtain blended pixels; and
include the pixels from the first area of the first image, the pixels from the first area of the second image, and the blended pixels in the fused image.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to:
output the fused image.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the second area of the first image is within the first area of the first image, wherein the second area of the second image is within the first area of the second image.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the second area of the first image overlaps with the second area of the second image.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the first area of the first image comprises a first composite region from the first image, and wherein the first area of the second image comprises a second composite region from the second image.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein, to blend the pixels in the second area of the first image and the pixels in the second area of the second image, the at least one processor is configured to:
perform seam blending between the first composite region and the second composite region.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the first area of the first image and the first area of the second image are within a region of interest.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the first area of the second image is used to fill in a portion of the region of interest not captured in the first area of the first image.

### Claim 9 (depends on 7)

The apparatus of claim 7, wherein the at least one processor is configured to blend the pixels in the second area of the first image and the pixels in the second area of the second image to recover the region of interest based on replacing a portion of the region of interest that does not exist in the first image with at least a part of the second image.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the first image is captured using a telephoto lens, and wherein the second image is captured using a wide-angle lens.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the first image and the second image are captured concurrently.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the first image and the second image are captured at different times.

### Claim 13

A method performed by an electronic device, the method comprising:
selecting, for a fused image, pixels from a first area of a first image;
selecting, for the fused image, pixels from a first area of a second image;
blending pixels in a second area of the first image and pixels in a second area of the second image to obtain blended pixels; and
including the pixels from the first area of the first image, the pixels from the first area of the second image, and the blended pixels in the fused image.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
outputting the fused image.

### Claim 15 (depends on 13)

The method of claim 13, wherein the second area of the first image is within the first area of the first image, wherein the second area of the second image is within the first area of the second image.

### Claim 16 (depends on 15)

The method of claim 15, wherein the second area of the first image overlaps with the second area of the second image.

### Claim 17 (depends on 13)

The method of claim 13, wherein the first area of the first image comprises a first composite region from the first image, and wherein the first area of the second image comprises a second composite region from the second image.

### Claim 18 (depends on 17)

The method of claim 17, wherein blending the pixels in the second area of the first image and the pixels in the second area of the second image comprises:
performing seam blending between the first composite region and the second composite region.

### Claim 19 (depends on 13)

The method of claim 13, wherein the first area of the first image and the first area of the second image are within a region of interest.

### Claim 20 (depends on 19)

The method of claim 19, wherein the first area of the second image is used to fill in a portion of the region of interest not captured in the first area of the first image.

### Claim 21 (depends on 19)

The method of claim 19, wherein blending the pixels in the second area of the first image and the pixels in the second area of the second image is performed in order to recover the region of interest based on replacing a portion of the region of interest that does not exist in the first image with at least a part of the second image.

### Claim 22 (depends on 13)

The method of claim 13, wherein the first image is captured using a telephoto lens, and wherein the second image is captured using a wide-angle lens.

### Claim 23 (depends on 13)

The method of claim 13, wherein the first image and the second image are captured concurrently.

### Claim 24 (depends on 13)

The method of claim 13, wherein the first image and the second image are captured at different times.
