# US 12,003,860 B2 · Image processing method and electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,860 B2 - Document ID: US12003860B2 - Status: Granted - Kind: B2 - Application number: 17/690,859 - Filed: 2022-03-09 - Published: 2022-06-23 - Granted: 2024-06-04 - Earliest claimed: 2019-09-27 - Assignees: Vivo Mobile Communication Co., Ltd. - Applicants: Vivo Mobile Communication Co., Ltd. - Inventors: Zhuojian Yang - Web page: https://patentgrep.com/patent/US12003860B2 ## CPC - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/71: Circuitry for evaluating the brightness variation - H04N 23/13: with multiple sensors - H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing method includes: obtaining a first image photographed by each of N1 first cameras in the M1 first cameras and a second image photographed by each of N2 second cameras in the M2 second cameras; obtaining luminance values of at least a part of pixels in the first image and luminance values of at least a part of pixels in the second image for each first image and each second image; and increasing luminance of a target image based on obtained luminance values, and using, as a photographed image of the under-display camera assembly, a target image obtained after the luminance is increased. M1, N1, M2, and N2 are all integers greater than or equal to 1. ## Claims ### Claim 1 An image processing method, applied to an electronic device, wherein the electronic device comprises an under-display camera assembly, the under-display camera assembly comprises M1 first cameras and M2 second cameras, and the method comprises: obtaining a first image photographed by each of N1 first cameras in the M1 first cameras and a second image photographed by each of N2 second cameras in the M2 second cameras; obtaining luminance values of at least a part of pixels in the first image and luminance values of at least a part of pixels in the second image for each first image and each second image; and increasing luminance of a target image based on obtained luminance values, and using, as a photographed image of the under-display camera assembly, the target image obtained after the luminance is increased, wherein the target image is a first image photographed by one of the N1 first cameras, wherein M1, N1, M2, and N2 are all integers greater than or equal to 1, N1 is less than or equal to M1, and N2 is less than or equal to M2; wherein the obtaining luminance values of at least a part of pixels in the first image and luminance values of at least a part of pixels in the second image for each first image and each second image comprises: obtaining a target photographing distance between the under-display camera assembly and a target object photographed by the under-display camera assembly; obtaining a target area of each first image at the target photographing distance and a target area of each second image at the target photographing distance, based on a predetermined target area in an image photographed by each first camera and a predetermined target area in an image photographed by each second camera at prestored different photographing distances; and for each target area, obtaining a luminance value of each pixel in the target area, wherein the increasing luminance of a target image based on obtained luminance values comprises: using each pixel in a target area of the target image as a first target pixel; and for each first target pixel, adding a luminance value of the first target pixel and a first luminance value to obtain a first target luminance value, and increasing the luminance value of the first target pixel to the first target luminance value, and wherein the first luminance value is a sum of a luminance value of a pixel corresponding to the first target pixel in a target area of each first image other than the target image, and a luminance value of a pixel corresponding to the first target pixel in the target area of each second image. ### Claim 2 (depends on 1) The method according to claim 1, wherein the first camera is a color camera, and the second camera is a monochrome camera. ### Claim 3 (depends on 1) The method according to claim 1, wherein a predetermined target area in an image photographed by the first camera or the second camera at any prestored photographing distance is a position area in which the target object is located in the image. ### Claim 4 (depends on 1) The method according to claim 1, wherein the increasing luminance of a target image based on obtained luminance values further comprises: increasing an obtained luminance value of each pixel in a non-target area of the target image by a preset multiple, wherein the preset multiple is N1−1 times. ### Claim 5 (depends on 1) The method according to claim 1, wherein the obtaining luminance values of at least a part of pixels in the first image and luminance values of at least a part of pixels in the second image for each first image and each second image comprises: obtaining luminance values of all pixels in the first image and luminance values of all pixels in the second image for each first image and each second image. ### Claim 6 (depends on 5) The method according to claim 5, wherein the increasing luminance of a target image based on obtained luminance values comprises: using each pixel in the target image as a second target pixel; and for each second target pixel, adding a luminance value of the second target pixel and a second luminance value to obtain a second target luminance value, and increasing the luminance value of the second target pixel to the second target luminance value, wherein the second luminance value is a sum of a luminance value of a pixel corresponding to the second target pixel in each first image other than the target image, and a luminance value of a pixel corresponding to the second target pixel in each second image. ### Claim 7 An electronic device, comprising: an under-display camera assembly, wherein the under-display camera assembly comprises M1 first cameras and M2 second cameras; and an image processing apparatus, configured to: obtain a first image photographed by each of N1 first cameras in the M1 first cameras and a second image photographed by each of N2 second cameras in the M2 second cameras; obtain luminance values of at least a part of pixels in the first image and luminance values of at least a part of pixels in the second image for each first image and each second image; and increase luminance of a target image based on obtained luminance values, and use, as a photographed image of the under-display camera assembly, the target image obtained after the luminance is increased, wherein the target image is a first image photographed by one of the N1 first cameras; and M1, N1, M2, and N2 are all integers greater than or equal to 1, N1 is less than or equal to M1, and N2 is less than or equal to M2; wherein the image processing apparatus is configured to: obtain a target photographing distance between the under-display camera assembly and a target object photographed by the under-display camera assembly; obtain a target area of each first image at the target photographing distance and a target area of each second image at the target photographing distance, based on a predetermined target area in an image photographed by each first camera and a predetermined target area in an image photographed by each second camera at prestored different photographing distances; and for each target area, obtain a luminance value of each pixel in the target area; wherein the image processing apparatus is configured to: use each pixel in a target area of the target image as a first target pixel; and for each first target pixel, add a luminance value of the first target pixel and a first luminance value to obtain a first target luminance value, and increase the luminance value of the first target pixel to the first target luminance value, wherein the first luminance value is a sum of a luminance value of a pixel corresponding to the first target pixel in a target area of each first image other than the target image, and a luminance value of a pixel corresponding to the first target pixel in the target area of each second image. ### Claim 8 (depends on 7) The electronic device according to claim 7, wherein the first camera is a color camera, and the second camera is a monochrome camera. ### Claim 9 (depends on 7) The electronic device according to claim 7, wherein fields of view of all cameras in the under-display camera assembly have a same overlapping area. ### Claim 10 (depends on 7) The electronic device according to claim 7, wherein the M1 first cameras and the M2 second cameras are arranged in arrays. ### Claim 11 (depends on 7) The electronic device according to claim 7, wherein the first camera comprises a lens, a lens holder, a filter assembly, a photosensitive chip, and a circuit board connected to the photosensitive chip; and the lens holder is disposed on the circuit board, the lens and the filter assembly are disposed on the lens holder, and the filter assembly is located between the lens and the photosensitive chip. ### Claim 12 An electronic device, comprising a processor and a memory that stores a computer program instruction, wherein the electronic device comprises an under-display camera assembly, the under-display camera assembly comprises M1 first cameras and M2 second cameras; and the computer program instruction, when executed by the processor, causes the electronic device to perform: obtaining a first image photographed by each of N1 first cameras in the M1 first cameras and a second image photographed by each of N2 second cameras in the M2 second cameras; obtaining luminance values of at least a part of pixels in the first image and luminance values of at least a part of pixels in the second image for each first image and each second image; and increasing luminance of a target image based on obtained luminance values, and using, as a photographed image of the under-display camera assembly, the target image obtained after the luminance is increased, wherein the target image is a first image photographed by one of the N1 first cameras, wherein M1, N1, M2, and N2 are all integers greater than or equal to 1, N1 is less than or equal to M1, and N2 is less than or equal to M2; wherein the computer program instruction, when executed by the processor, causes the electronic device to perform: obtaining a target photographing distance between the under-display camera assembly and a target object photographed by the under-display camera assembly; obtaining a target area of each first image at the target photographing distance and a target area of each second image at the target photographing distance, based on a predetermined target area in an image photographed by each first camera and a predetermined target area in an image photographed by each second camera at prestored different photographing distances; and for each target area, obtaining a luminance value of each pixel in the target area; wherein the computer program instruction, when executed by the processor, causes the electronic device to perform: using each pixel in a target area of the target image as a first target pixel; and for each first target pixel, adding a luminance value of the first target pixel and a first luminance value to obtain a first target luminance value, and increasing the luminance value of the first target pixel to the first target luminance value, and wherein the first luminance value is a sum of a luminance value of a pixel corresponding to the first target pixel in a target area of each first image other than the target image, and a luminance value of a pixel corresponding to the first target pixel in the target area of each second image. ### Claim 13 (depends on 12) The electronic device according to claim 12, wherein a predetermined target area in an image photographed by the first camera or the second camera at any prestored photographing distance is a position area in which the target object is located in the image. ### Claim 14 (depends on 12) The electronic device according to claim 12, wherein the computer program instruction, when executed by the processor, causes the electronic device to further perform: increasing an obtained luminance value of each pixel in a non-target area of the target image by a preset multiple, wherein the preset multiple is N1-1 times. ### Claim 15 (depends on 12) The electronic device according to claim 12, wherein the computer program instruction, when executed by the processor, causes the electronic device to perform: obtaining luminance values of all pixels in the first image and luminance values of all pixels in the second image for each first image and each second image. ### Claim 16 (depends on 15) The electronic device according to claim 15, wherein the computer program instruction, when executed by the processor, causes the electronic device to perform: using each pixel in the target image as a second target pixel; and for each second target pixel, adding a luminance value of the second target pixel and a second luminance value to obtain a second target luminance value, and increasing the luminance value of the second target pixel to the second target luminance value, wherein the second luminance value is a sum of a luminance value of a pixel corresponding to the second target pixel in each first image other than the target image, and a luminance value of a pixel corresponding to the second target pixel in each second image.