# US 2019/0318463 A1 · Systems and methods for fusing infrared image and visible light image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318463 A1 - Document ID: US20190318463A1 - Status: Published application - Kind: A1 - Application number: 16/454,210 - Filed: 2019-06-27 - Published: 2019-10-17 - Earliest claimed: 2016-12-27 - Assignees: Zhejiang Dahua Technology Co., Ltd. - Applicants: Zhejiang Dahua Technology Co., Ltd. - Inventors: Dong ZHANG; Song WANG - Web page: https://patentgrep.com/patent/US20190318463A1 ## CPC - G06T 5/10: using non-spatial domain filtering - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/70: Denoising; Smoothing - G06T 5/73: Deblurring; Sharpening - G06T 7/90: Determination of colour characteristics - G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) - G06T 2207/10004: Still image; Photographic image - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/10048: Infrared image - G06T 2207/20064: Wavelet transform [DWT] - G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering - G06T 2207/20221: Image fusion; Image merging - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A system and a method for image fusion are provided. The method may comprise: obtaining a visible light image and an infrared image relating to a same scene; performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image; performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image; fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image. ## Claims ### Claim 1 An image fusion system, comprising: a processor coupled to a storage; and the storage configured to store instructions, the instructions, when executed by the processor, causing the image fusion system to effectuate a method comprising: obtaining a visible light image and an infrared image relating to a same scene; performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image; performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image; fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image. ### Claim 2 (depends on 1) The system of claim 1, wherein the image fusion system is further caused to effectuate the method comprising: preprocessing the visible light image and the infrared image, wherein the preprocessing comprises: registering the visible light image and the infrared image; performing haze removal on the visible light image; and performing color filtering on the visible light image. ### Claim 3 (depends on 2) The system of claim 2, wherein the performing haze removal on the visible light image comprises: determining a dark channel matrix; determining an atmospheric light; determining an initial transmittance based on the dark channel matrix and the atmospheric light; determining a bright area to obtain an optimized transmittance based on the initial transmittance; and performing the haze removal based on the optimized transmittance. ### Claim 4 (depends on 1) The system of claim 1, wherein the first decomposition is pyramid decomposition. ### Claim 5 (depends on 4) The system of claim 4, wherein the pyramid decomposition comprises: performing low-pass filtering on a subject image to obtain a filtered image; downsampling the filtered image to obtain a downsampled image; upsampling the downsampled image to obtain an upsampled image; and performing band-pass filtering on the upsampled image to obtain a decomposed subject image. ### Claim 6 (depends on 5) The system of claim 5, the subject image comprising the first high-frequency component of the visible light image, the first low-frequency component of the visible light image, the first high-frequency component of the infrared image, or the first low-frequency component of the infrared image. ### Claim 7 (depends on 5) The system of claim 5, wherein the image fusion system is further caused to effectuate the method comprising fusing the first low-frequency component of the visible light image and the first low-frequency component of the infrared image, wherein the fusing comprises: determining a difference between the first low-frequency component of the visible light image and the first low-frequency component of the infrared image; determining whether the difference exceeds a threshold; designating, in response to the determination that the difference exceeds the threshold, the low-frequency component of the visible light image as a first fused low-frequency component; and determining, in response to the determination that the difference is lower than the threshold, the first fused low-frequency component based on a second algorithm. ### Claim 8 (depends on 7) The system of claim 7, the fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image comprises: determining the first fused high-frequency component based on a third algorithm. ### Claim 9 (depends on 8) The system of claim 8, wherein the performing the reconstruction comprises: performing an inverse pyramid decomposition based on the first fused low-frequency component and the first fused high-frequency component. ### Claim 10 (depends on 1) The system of claim 1, wherein the image fusion system is further caused to effectuate the method comprising: filtering the first high-frequency component of the visible light image by performing one or both of mapping and median filtering. ### Claim 11 (depends on 10) The system of claim 10, wherein the image fusion system is further caused to effectuate the method comprising: amplifying the first high-frequency component of the infrared image. ### Claim 12 (depends on 7) The system of claim 7, wherein the image fusion system is further caused to effectuate the method comprising: performing a second decomposition to the first low-frequency component of the visible light image to obtain a second high-frequency component of the visible light image and a second low-frequency component of the visible light image; performing a second decomposition to the first low-frequency component of the infrared image to obtain a second high-frequency component of the infrared image and a second low-frequency component of the infrared image; and fusing the second high-frequency component of the visible light image and the second high-frequency component of the infrared image based on the first algorithm to generate a second fused high-frequency component. ### Claim 13 (depends on 12) The system of claim 12, the performing the reconstruction further including: fusing the second low-frequency component of the visible light image and the second low-frequency component of the infrared image based on the second algorithm to generate a second fused low-frequency component; and generating the fused image based on the first fused high-frequency component, the second fused high-frequency component, and the second fused low-frequency component. ### Claim 14 An image fusion method, comprising: obtaining a visible light image and an infrared image relating to a same scene; performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image; performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image; fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image. ### Claim 15 (depends on 14) The method of claim 14, wherein the first decomposition is pyramid decomposition. ### Claim 16 (depends on 15) The method of claim 15, wherein the pyramid decomposition comprises performing low-pass filtering on a subject image to obtain a filtered image; downsampling the filtered image to obtain a downsampled image; upsampling the downsampled image to obtain an upsampled image; and performing band-pass filtering on the upsampled image to obtain a decomposed subject image. ### Claim 17 (depends on 14) The method of claim 14, the method further comprising: filtering the first high-frequency component of the visible light image by performing mapping and median filtering. ### Claim 18 (depends on 14) The method of claim 14, the method further comprising: amplifying the first high-frequency component of the infrared image. ### Claim 19 (depends on 14) The method of claim 14, the method further comprising: performing a second decomposition to the first low-frequency component of the visible light image to obtain a second high-frequency component of the visible light image and a second low-frequency component of the visible light image; performing a second decomposition to the first low-frequency component of the infrared image to obtain a second high-frequency component of the infrared image and a second low-frequency component of the infrared image; and fusing the second high-frequency component of the visible light image and the second high-frequency component of the infrared image based on the first algorithm to generate a second fused high-frequency component. ### Claim 20 A non-transitory computer readable medium storing instructions, the instructions, when executed by a computing device, causing the computing device to implement a method, comprising: obtaining a visible light image and an infrared image relating to a same scene; performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image; performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image; fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image. ### Claim at position 21 21-24. (canceled)