# US 2021/0279530 A1 · Systems and methods for image fusion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279530 A1 - Document ID: US20210279530A1 - Status: Published application - Kind: A1 - Application number: 17/330,359 - Filed: 2021-05-25 - Published: 2021-09-09 - Earliest claimed: 2018-12-27 - Assignees: Zhejiang Dahua Technology Co., Ltd. - Applicants: Zhejiang Dahua Technology Co., Ltd. - Inventors: Erping QU - Web page: https://patentgrep.com/patent/US20210279530A1 ## CPC - G06T 7/33: using feature-based methods - G06F 18/2431: Multiple classes - G06F 18/25: Fusion techniques - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06V 10/143: Sensing or illuminating at different wavelengths - G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10) - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06T 2207/10048: Infrared image - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30232: Surveillance - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for image fusion may include obtaining a first image including a plurality of first pixels and a second image including a plurality of second pixels. Each of the plurality of second pixels may correspond to one of the plurality of first pixels. The method may further include classifying the plurality of first pixels into different categories which at least includes a first category and a second category. The method may further include fusing the first image with the second image based on a fusion operation between each of the plurality of first pixels and its corresponding second pixel to generate a fused image. The fusion operation associated with a first pixel belonging to the first category and the fusion operation associated with a first pixel belonging to the second category are performed according to different fusion rules. ## Claims ### Claim 1 A system, comprising: at least one storage medium including a set of instructions; and at least one processor in communication with the at least one storage medium, wherein when executing the set of instructions, the at least one processor is directed to cause the system to: obtain a first image including a plurality of first pixels, the first image being generated by a first imaging channel; obtain a second image including a plurality of second pixels, the second image being generated by a second imaging channel distinct from the first imaging channel, each of the plurality of second pixels corresponding to one of the plurality of first pixels; classify the plurality of first pixels into different categories, the different categories at least including a first category and a second category; and fuse the first image with the second image based on a fusion operation between each of the plurality of first pixels and its corresponding second pixel to generate a fused image, wherein the fusion operation associated with a first pixel belonging to the first category and the fusion operation associated with a first pixel belonging to the second category are performed according to different fusion rules. ### Claim 2 (depends on 1) The system of claim 1, wherein the first imaging channel is configured to capture a visible light image, and the second imaging channel is configured to capture an infrared light image. ### Claim 3 (depends on 1) The system of claim 1, wherein the first category at least includes a white point or a point approximate to the white point. ### Claim 4 (depends on 3) The system of claim 3, wherein the second category at least includes a non-white point. ### Claim 5 (depends on 4) The system of claim 4, wherein to perform the fusion operation associated with the first pixel belonging to the first category, the at least one processor is directed to cause the system to: determine a chrominance component of the first pixel belonging to the first category; and designate the chrominance component of the first pixel belonging to the first category as a chrominance component of a third pixel in the fused image. ### Claim 6 (depends on 5) The system of claim 5, wherein the at least one processor is directed to cause the system to: generate a first binary image based on the first pixels belonging to the first category and the first pixels belonging to the second category; and perform a convolution operation on the first image and the first binary image such that the chrominance component of the first pixel belonging to the first category remains unchanged. ### Claim 7 (depends on 4) The system of claim 4, wherein to perform the fusion operation associated with the first pixel belonging to the second category, the at least one processor is directed to cause the system to: determine a chrominance component of the first pixel belonging to the second category; determine a chrominance enhancement coefficient; and determine a chrominance component of a fourth pixel in the fused image based on the chrominance component of the first pixel belonging to the second category and the chrominance enhancement coefficient. ### Claim 8 (depends on 7) The system of claim 7, wherein the chrominance enhancement coefficient is determined based on a luminance component of the first pixel belonging to the second category and a luminance component of a second pixel that corresponds to the first pixel belonging to the second category. ### Claim 9 (depends on 7) The system of claim 7, wherein the at least one processor is directed to cause the system to: generate a second binary image based on the first pixels belonging to the first category and the first pixels belonging to the second category; and perform a convolution operation on the first image and the second binary image such that the chrominance component of the first pixel belonging to the second category remains unchanged. ### Claim 10 (depends on 4) The system of claim 4, wherein to perform the fusion operation associated with the first pixel belonging to the first category, the at least one processor is directed to cause the system to: determine a luminance of the first pixel belonging to the first category and a luminance of a second pixel that corresponds to the first pixel belonging to the first category; and determine a luminance component of a fifth pixel in the fused image based on a comparison result between the luminance of the first pixel belonging to the first category and the luminance of the second pixel that corresponds to the first pixel belonging to the first category. ### Claim 11 A method for image fusion, implemented on a computing device including at least one processor and at least one storage medium, the method comprising: obtaining a first image including a plurality of first pixels, the first image being generated by a first imaging channel; obtaining a second image including a plurality of second pixels, the second image being generated by a second imaging channel distinct from the first imaging channel, each of the plurality of second pixels corresponding to one of the plurality of first pixels; classifying the plurality of first pixels into different categories, the different categories at least including a first category and a second category; and fusing the first image with the second image based on a fusion operation between each of the plurality of first pixels and its corresponding second pixel to generate a fused image, wherein the fusion operation associated with a first pixel belonging to the first category and the fusion operation associated with a first pixel belonging to the second category are performed according to different fusion rules. ### Claim 12 (canceled) ### Claim 13 (depends on 11) The method of claim 11, wherein the first category at least includes a white point or a point approximate to the white point. ### Claim 14 (depends on 13) The method of claim 13, wherein the second category at least includes a non-white point. ### Claim 15 (depends on 14) The method of claim 14, wherein the performing the fusion operation associated with the first pixel belonging to the first category, comprising: determining a chrominance component of the first pixel belonging to the first category; and designating the chrominance component of the first pixel belonging to the first category as a chrominance component of a third pixel in the fused image. ### Claim 16 (depends on 15) The method of claim 15, further comprising: generating a first binary image based on the first pixels belonging to the first category and the first pixels belonging to the second category; and performing a convolution operation on the first image and the first binary image such that the chrominance component of the first pixel belonging to the first category remains unchanged. ### Claim 17 (depends on 14) The method of claim 14, wherein the performing the fusion operation associated with the first pixel belonging to the second category, comprising: determining a chrominance component of the first pixel belonging to the second category; determining a chrominance enhancement coefficient; and determining a chrominance component of a fourth pixel in the fused image based on the chrominance component of the first pixel belonging to the second category and the chrominance enhancement coefficient. ### Claim 18 (depends on 17) The method of claim 17, wherein the chrominance enhancement coefficient is determined based on a luminance component of the first pixel belonging to the second category and a luminance component of a second pixel that corresponds to the first pixel belonging to the second category. ### Claim 19 (depends on 17) The method of claim 17, further comprising: generating a second binary image based on the first pixels belonging to the first category and the first pixels belonging to the second category; and performing a convolution operation on the first image and the second binary image such that the chrominance component of the first pixel belonging to the second category remains unchanged. ### Claim 20 (depends on 14) The method of claim 14, wherein the performing the fusion operation associated with the first pixel belonging to the first category, comprising: determining a luminance of the first pixel belonging to the first category and a luminance of a second pixel that corresponds to the first pixel belonging to the first category; and determining a luminance component of a fifth pixel in the fused image based on a comparison result between the luminance of the first pixel belonging to the first category and the luminance of the second pixel that corresponds to the first pixel belonging to the first category. ### Claim 21 A non-transitory computer readable medium, comprising executable instructions that, when executed by at least one processor, directs the at least one processor to perform a method for image fusion, the method comprising: obtaining a first image including a plurality of first pixels, the first image being generated by a first imaging channel; obtaining a second image including a plurality of second pixels, the second image being generated by a second imaging channel distinct from the first imaging channel, each of the plurality of second pixels corresponding to one of the plurality of first pixels; classifying the plurality of first pixels into different categories, the different categories at least including a first category and a second category; and fusing the first image with the second image based on a fusion operation between each of the plurality of first pixels and its corresponding second pixel to generate a fused image, wherein the fusion operation associated with a first pixel belonging to the first category and the fusion operation associated with a first pixel belonging to the second category are performed according to different fusion rules.