# US 12,002,294 B2 · RGB-NIR dual camera face anti-spoofing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,294 B2
- Document ID: US12002294B2
- Status: Granted
- Kind: B2
- Application number: 17/192,605
- Filed: 2021-03-04
- Published: 2022-09-08
- Granted: 2024-06-04
- Earliest claimed: 2021-03-04
- Assignees: Black Sesame Technologies Inc.
- Applicants: Black Sesame International Holding Limited
- Inventors: Xiaomin Liu; Lei Zhang; Qun Gu; Zilong Hu
- Web page: https://patentgrep.com/patent/US12002294B2

## CPC

- G06V 40/164: {using holistic features}
- G06V 40/40: Spoof detection, e.g. liveness detection
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/82: using neural networks
- G06V 40/166: {using acquisition arrangements}
- G06V 40/168: {Feature extraction; Face representation}
- H04N 23/11: for generating image signals from visible and infrared light wavelengths
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of face anti-spoofing, comprising, receiving a near infra-red facial image, having a near infrared channel, receiving a red-green-blue facial image, having a red channel, a green channel and a blue channel, generating a synthetic three channel image based on the near infrared channel, the red channel, the green channel and the blue channel and training a deep neural network based on the synthetic three channel image.

## Claims

### Claim 1

A method of facial recognition, comprising:
receiving a near infrared facial image, having a near infrared channel;
receiving a red-green-blue facial image, having a red channel, a green channel and a blue channel;
generating a synthetic three channel image, based on the near infrared channel, the red channel, the green channel and the blue channel, having a first new channel, a second new channel, and a third new channel; and
processing a deep neural network based on the synthetic three channel image wherein the near infrared facial image and the red-green-blue facial image are fed into the deep neural network.

### Claim 2 (depends on 1)

The method of facial recognition of claim 1, wherein the first new channel comprises the near infrared channel, the second new channel comprises the near infrared channel and the third new channel comprises an average of the red channel, the green channel and the blue channel.

### Claim 3 (depends on 1)

The method of facial recognition of claim 1, wherein the first new channel comprises the near infrared channel, the second new channel comprises an average of the green channel and the blue channel and the third new channel comprises the red channel.

### Claim 4 (depends on 1)

The method of facial recognition of claim 1, further comprising:
converting the red-green-blue facial image to a luma-blue-red facial image having a luma channel, the blue channel and the red channel;
replacing the luma channel with the near infrared channel; and
converting the luma-blue-red facial image to a second red-green-blue facial image.

### Claim 5 (depends on 1)

The method of facial recognition of claim 1, further comprising:
converting the red-green-blue facial image to a luma-blue-red image having a luma channel, the blue channel and the red channel;
replacing the luma channel with an average of the near infrared channel and the luma channel; and
converting the luma-blue-red facial image to a second red-green-blue facial image.

### Claim 6 (depends on 1)

The method of facial recognition of claim 1, further comprising:
converting the red-green-blue facial image to a luma-blue-red facial image having a luma channel, the blue channel and the red channel;
decomposing the luma channel in a frequency domain;
decomposing the near infrared channel in the frequency domain;
combining high frequency components of the decomposed luma channel and high frequency components of the decomposed near infrared channel;
combining low frequency components of the decomposed luma channel and low frequency components of the decomposed near infrared channel; and
reconstructing a new synthetic image from the combined high frequency components and the combined low frequency components.

### Claim 7 (depends on 1)

The method of facial recognition of claim 1, wherein processing the deep neural network based on the synthetic three channel image wherein the near infrared facial image and the red-green-blue facial image are fed into the deep neural network comprises
fusing at an intermediate layer a first feature map based on the near infrared channel and a second feature map based on the red channel, the green channel and the blue channel to form a fused feature map,
wherein the fused feature map is fused at a convolutional layer at an intermediate level of the deep neural network.

### Claim 8 (depends on 7)

The method of facial recognition of claim 7, wherein the convolutional layer fusing is performed on a depth-wise and point-wise basis.

### Claim 9 (depends on 7)

The method of facial recognition of claim 7, wherein the first feature map and the second feature map are processed in parallel.