Face recognition monitoring system based on spectrum and multi-band fusion and recognition method using same

A face recognition monitoring system based on spectrum and multi-band fusion, comprising:

• a spectral camera;
• a first module for acquiring a face spectral image;
• a second module for preprocessing data of the face spectral image;
• a face spectral image database;
• and
• a third module for recognizing the face spectral image;
• wherein the spectral camera comprises an optical lens and a silicon-based detector;
• the silicon-based detector comprises a photoelectric conversion substrate and a filter film arranged on the photoelectric conversion substrate;
• the filter film comprises N units, and each of the N units comprises a visible spectral sensing area, a near-infrared spectral image sensing area and a red-green-green-blue (RGGB) image acquisition area;
• the N units cover all pixels on the photoelectric conversion substrate;
• the visible spectral sensing area is configured to distinguish an incident visible light spectrum;
• the visible spectral sensing area is formed by splicing n kinds of materials with known and different light transmittance through coating and etching one by one;
• the visible spectral sensing area has a single-layer structure, comprising n sub-units T1, T2 . . . Tn;
• each of the n sub-units covers M pixels on the photoelectric conversion substrate, wherein M is greater than or equal to 1;
• the visible spectral sensing area, the near-infrared spectral image sensing area and the RGGB image acquisition area together constitute a periodic structure;
• a transmission band of the near-infrared spectral image sensing area is divided into 780-850 nm, 850-950 nm, 950-1050 nm and 1050-1200 nm, and is configured to detect a deep-layer facial feature image in a near-infrared light band with a wavelength greater than 780 nm;
• the near-infrared spectral image sensing area is formed by splicing four materials with known and different near-infrared transmittance through coating and etching one by one;
• the near-infrared spectral image sensing area has a single-layer structure, comprising four sub-units, respectively Tir1, Tire, Tir2, and Tir4;
• each of the four sub-units covers M pixels on the photoelectric conversion substrate, wherein M is greater than or equal to 1;
• the RGGB image acquisition area is configured to collect a face RGB image in a monitoring area;
• the RGGB image acquisition area comprises four sub-units respectively transmitting red, green, green and blue;
• each of the four sub-units of the RGGB image acquisition area covers M pixels on the photoelectric conversion substrate, wherein M is greater than or equal to 1;
• the photoelectric conversion substrate is configured to convert a light signal passing through the visible spectral sensing area into an electrical signal, and convert the electrical signal to form a digital signal or a code for output through amplification and analog-to-digital conversion;
• the photoelectric conversion substrate is also configured to convert a light signal passing through the near-infrared spectral image sensing area into an electrical signal, and convert the electrical signal to form a digital signal or a code for output through amplification and analog-to-digital conversion;
• and the photoelectric conversion substrate is also configured to convert a light signal passing through the RGGB image acquisition area into an electrical signal, and convert the electrical signal to form a digital signal or a code for output through amplification and analog-to-digital conversion;
• the first module is configured to inverse a face spectrum and image data according to a signal intensity information of an incident light and a position information of a corresponding pixel on the photoelectric conversion substrate, wherein the face spectrum and image data comprises a visible-near infrared face reflection multi-spectral data, a near-infrared face image and a RGGB face image;
• the near-infrared face image and the RGGB face image are acquired by image inversion;
• and the visible-near infrared face reflection multi-spectral data is inversed according to formula (1):
• Si=∫I(λ)Ti(λ)η(λ)dλ, (1);
• wherein S is an intensity value of a light signal output by the photoelectric conversion substrate;
• I is an incident spectrum, which is a signal to be solved;
• T is a spectral transmittance of the filter film, comprising a spectral transmittance of a visible light filter film and a spectral transmittance of a near-infrared light filter film;
• η is a quantum efficiency of the photoelectric conversion substrate;
• and λ is an incident wavelength;
• the second module is configured to denoise the near-infrared face image acquired in the near-infrared spectral image sensing area and the RGGB face image acquired in the RGGB image acquisition area;
• to perform feature extraction on a denoised near-infrared face image and a denoised RGGB face image;
• to extract a face multi-spectral data from a multi-spectral image acquired in the visible spectral sensing area and a multi-spectral image acquired in the near-infrared spectral image sensing area;
• and to compare a multi-dimensional facial feature with a face spectral image and a multi-spectral data in the face spectral image database;
• the face spectral image database is configured to store an image feature value and spectrum data of Nx eigenfaces to be searched, and multi-spectral data of multiple mask materials, so as to facilitate a comparison with a face image and a spectrum thereof collected in the monitoring area;
• and
• the third module is configured to compare the visible-near infrared face reflection multi-spectral data, a preprocessed near-infrared face image and a preprocessed RGGB face image data with information in the face spectral image database.