# US 2020/0349372 A1 · Method and apparatus with liveness detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349372 A1
- Document ID: US20200349372A1
- Status: Published application
- Kind: A1
- Application number: 16/807,565
- Filed: 2020-03-03
- Published: 2020-11-05
- Earliest claimed: 2019-05-02
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hana LEE; Solae LEE; Minsu KO; Jiwon BAEK; Seungju HAN
- Web page: https://patentgrep.com/patent/US20200349372A1

## CPC

- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06V 10/20: Image preprocessing
- G06V 10/243: {by compensating for image skew or non-uniform image deformations}
- G06V 40/162: {using pixel segmentation or colour matching}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/45: {Detection of the body part being alive}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A liveness detection method and apparatus, and a facial verification method and apparatus are disclosed. The liveness detection method includes detecting a face region in an input image, measuring characteristic information of the face region, adjusting the measured characteristic information in response to the characteristic information not satisfying a condition, and performing a liveness detection on the face region with the adjusted characteristic information upon the measured characteristic information not satisfying the condition.

## Claims

### Claim 1

A processor-implemented liveness detection method, comprising:
detecting a face region in an input image;
measuring characteristic information of the detected face region;
in response to the measured characteristic information being determined as not satisfying a condition, generating an adjusted face region by adjusting the characteristic information of the face region; and
performing a liveness detection on the adjusted face region.

### Claim 2 (depends on 1)

The method of claim 1, wherein the measuring of the characteristic information of the face region comprises measuring a hue value of the face region,
wherein the generating of the adjusted face region comprises:
in response to a determination that the measured hue value is not included in a preset hue value range, performing the generating of the adjusted face region by adjusting the hue value of the face region to a hue value included in the preset hue value range.

### Claim 3 (depends on 2)

The method of claim 2, wherein the adjusting of the hue value of the face region comprises:
adjusting the measured hue value of the face region by applying a hue adjustment model to the face region.

### Claim 4 (depends on 1)

The method of claim 1, wherein the measuring of the characteristic information of the face region comprises measuring a face tilt value indicating a degree of a tilt of a face included in the face region,
wherein the generating of the adjusted face region comprises:
in response to a determination that the measured face tilt value is not included in a preset face tilt value range, performing the generating of the adjusted face region by adjusting the face tilt value.

### Claim 5 (depends on 4)

The method of claim 4, wherein the measuring of the face tilt value comprises:
detecting feature points corresponding to a left eye, a right eye, and both mouth ends in the face region; and
measuring the face tilt value based on the detected feature points.

### Claim 6 (depends on 4)

The method of claim 4, wherein the adjusting of the face tilt value comprises:
adjusting the face tilt value by rotating the face region based on the measured face tilt value.

### Claim 7 (depends on 1)

The method of claim 1, wherein the measuring of the characteristic information of the face region comprises measuring a white balance value of the face region,
wherein the generating of the face adjusted region comprises:
in response to a determination that the measured white balance value is not included in a preset white balance value range, performing the generating of the adjusted face region by adjusting the white balance value of the face region to a white balance value included in the preset white balance value range.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
in response to a determination that the measured characteristic information satisfies the condition, performing the liveness detection based on the detected face region without performing the generating of the adjusted face region.

### Claim 9 (depends on 1)

The method of claim 1, wherein the performing of the liveness detection comprises:
performing the liveness detection using a neural network-based liveness detection model.

### Claim 10 (depends on 1)

The method of claim 1, wherein the measuring of the characteristic information of the face region comprises:
measuring any one or any combination of any two or more of a hue value, a face tilt value, and a white balance value of the face region; and
in response to any one or any combination of any two or more of the measured hue value, the measured face tilt value, and the measured white balance value being respectively determined to be out of range of a preset hue value range, a preset face tilt value range, and a preset white balance value range, generating the adjusted face region by respectively adjusting the hue value, the face tilt value, and the white balance value to respectively fall within the preset hue value range, the preset face tilt value range, and the preset white balance value range.

### Claim 11 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1.

### Claim 12 (depends on 1)

The method of claim 1, wherein a neural network is used to perform a facial verification on the adjusted face region.

### Claim 13

A processor-implemented facial verification method using a neural network, the facial verification method comprising:
extracting a current facial image of a face region from an input image;
calculating a current image feature value of the current facial image;
comparing the current image feature value to a range of respective image feature values of training facial images;
in response to a determination that the current image feature value is out of the range of the respective image feature values, generating an adjusted current facial image by adjusting the current image feature value of the current facial image to be included in the range of the respective image feature values; and
inputting of the adjusted current facial image to the neural network.

### Claim 14 (depends on 13)

The method of claim 13, wherein the range of the respective image feature values is an absolute range determined based on a minimum image feature value and a maximum image feature value among the image feature values of the training facial images.

### Claim 15 (depends on 13)

The method of claim 13, wherein the range of the respective image feature values is a statistical range determined based on a distribution characteristic of the image feature values of the training facial images, and
wherein the distribution characteristic includes a mean and a standard deviation of the image feature values.

### Claim 16 (depends on 15)

The method of claim 15, wherein the statistical range is one of a range of 1 standard deviation, a range of 2 standard deviations, a range of 3 standard deviations, and a range of 4 standard deviations, on both sides from the mean as a center.

### Claim 17 (depends on 13)

The method of claim 13, wherein the neural network is configured to perform a liveness detection on a face object in the input image.

### Claim 18 (depends on 13)

The method of claim 13, wherein the neural network is configured to verify a face object in the input image.

### Claim 19 (depends on 13)

The method of claim 13, wherein the calculating of the current image feature value comprises:
calculating a hue value of the current facial image,
wherein the adjusting of the current facial image comprises:
in response to a determination that the calculated hue value is out of the range of the respective image feature values, generating the adjusted current facial image by adjusting the hue value of the current facial image to be included in the range of the respective image feature values.

### Claim 20 (depends on 13)

The method of claim 13, wherein the calculating of the current image feature value comprises:
calculating a face tilt value indicating a degree of a tilt of a face in the current facial image,
wherein the adjusting of the current facial image comprises:
in response to a determination that the calculated face tilt value is out of the range of the respective image feature values, generating the adjusted current facial image by rotating the face region of the current facial image so a tilt value of the adjusted current facial image to be included in the range of the respective image feature values.

### Claim 21 (depends on 13)

The method of claim 13, wherein the calculating of the current image feature value comprises:
calculating a white balance value of the face region in the current facial image,
wherein the adjusting of the current facial image comprises:
in response to a determination that the calculated white balance value is out of the range of the respective image feature values, generating the adjusted current facial image by adjusting the white balance value of the current facial image so a while balance value of the adjusted current facial image to be included in the range of the respective image feature values.

### Claim 22

An apparatus, comprising:
one or more processors configured to:
detect a face region in an input image;
measure characteristic information of the detected face region;
in response to the measured characteristic information being determined as not satisfying a condition, generate an adjusted face region by adjusting the characteristic information of the face region; and
perform a liveness detection on the adjusted face region.

### Claim 23 (depends on 22)

The apparatus of claim 22, wherein the one or more processors are further configured to:
measure a hue value of the face region; and
in response to a determination that the measured hue value is not included in a preset hue value range, perform the generating of the adjusted face region by adjusting the hue value of the face region to a hue value included in the preset hue value range.

### Claim 24 (depends on 22)

The apparatus of claim 22, wherein the one or more processors are further configured to:
measure a face tilt value indicating a degree of a tilt of a face included in the face region; and
in response to a determination that the measured face tilt value is not included in a preset face tilt value range, perform the generating of the adjusted face region by adjusting the face tilt value.

### Claim 25 (depends on 22)

The apparatus of claim 22, wherein the one or more processors are further configured to:
measure a white balance value of the face region; and
in response to a determination that the measured white balance value is not included in a preset white balance value range, perform the generating of the adjusted face region by adjusting the white balance value of the face region.

### Claim 26 (depends on 22)

The apparatus of claim 22, wherein the one or more processors are further configured to:
in response to a determination that the measured characteristic information satisfies the condition, perform the liveness detection based on the detected face region without performing the generating of the adjusted face region.

### Claim 27 (depends on 22)

The apparatus of claim 22, wherein the one or more processors are further configured to:
perform the liveness detection using a neural network-based liveness detection model.

### Claim 28 (depends on 22)

The apparatus of claim 22, wherein the apparatus is any of a smartphone, a wearable device, a tablet computer, a netbook, a laptop computer, a desktop computer, a personal digital assistant (PDA), a set-top box, a home appliance, a biometric door lock, a security device, and a vehicle start device.

### Claim 29

An apparatus using a neural network, the apparatus comprising:
one or more processors configured to:
extract a current facial image of a face region from an input image;
calculate a current image feature value of the current facial image;
compare the current image feature value to a range of respective image feature values of training facial images;
in response to a determination that the current image feature value is out of the range of the respective image feature values, generate an adjusted current face region by adjusting the current facial image such that the current image feature value of the current facial image is included in the range of the respective image feature values; and
input the adjusted current facial image to the neural network.

### Claim 30 (depends on 29)

The apparatus of claim 29, wherein the range of the respective image feature values is an absolute range to be determined based on a minimum image feature value and a maximum image feature value among the image feature values of the facial images used to train the neural network.

### Claim 31 (depends on 29)

The apparatus of claim 29, wherein the range of the respective image feature values is a statistical range to be determined based on a distribution characteristic of the image feature values of the facial images used to train the neural network,
wherein the distribution characteristic includes a mean and a standard deviation of the image feature values.

### Claim 32 (depends on 29)

The apparatus of claim 29, wherein the neural network is used to verify a face object in the input image.

### Claim 33 (depends on 29)

The apparatus of claim 29, wherein the one or more processors are configured to:
calculate a hue value of the current facial image; and
in response to a determination that the calculated hue value is out of the range of the respective image feature values, generate the adjusted current facial image by adjusting the hue value of the current facial image to be included in the range of the respective image feature values.

### Claim 34 (depends on 29)

The apparatus of claim 29, wherein the one or more processors are configured to:
calculate a face tilt value indicating a degree of a tilt of a face in the current facial image; and
in response to a determination that the calculated face tilt value is out of the range of the respective image feature values, generate the adjusted current facial image by rotating the face region of the current facial image so a tilt value of the adjusted current facial image is included in the range of the respective image feature values.

### Claim 35 (depends on 29)

The apparatus of claim 29, wherein the one or more processors are configured to:
calculate a white balance value of the face region in the current facial image; and
in response to a determination that the calculated white balance value is out of the range of the respective image feature values, generate the adjusted current facial image by adjusting the white balance value of the current facial image to be included in the range of the respective image feature values.

### Claim 36 (depends on 29)

The apparatus of claim 29, wherein the one or more processors are configured to:
calculate any one or any combination of any two or more of a hue value, a face tilt value, and a white balance value of the current facial image; and
in response to any one or any combination of any two or more of the calculated hue value, the calculated face tilt value, and the calculated white balance value being respectively determined to be out of the range of respective image feature values, generating the adjusted face region by respectively adjusting the out of range image feature values to be within the respective range of image feature values.
