# US 2020/0394289 A1 · Biometric verification framework that utilizes a convolutional neural network for feature matching

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394289 A1
- Document ID: US20200394289A1
- Status: Published application
- Kind: A1
- Application number: 16/583,599
- Filed: 2019-09-26
- Published: 2020-12-17
- Earliest claimed: 2019-06-14
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Ivan RAZUMENIC; Radim SPETLÍK
- Web page: https://patentgrep.com/patent/US20200394289A1

## CPC

- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G02B 27/017: {Head mounted}
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/193: {Preprocessing; Feature extraction}
- G06V 40/197: {Matching; Classification}
- G06V 40/45: {Detection of the body part being alive}
- G06V 40/70: Multimodal biometrics, e.g. combining information from different biometric modalities
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for biometric verification includes obtaining a verification image and extracting a set of verification image features from the verification image. The method also includes processing the set of verification image features and a set of enrollment image features using a convolutional neural network to determine a metric. A determination may then be made about whether the verification image matches an enrollment image based on the metric.

## Claims

### Claim 1

A computer-readable medium comprising instructions that are executable by one or more processors to cause a computing device to:
obtain a verification image;
extract a set of verification image features from the verification image;
process the set of verification image features and a set of enrollment image features using a convolutional neural network to determine a metric; and
determine whether the verification image matches an enrollment image based on the metric.

### Claim 2 (depends on 1)

The computer-readable medium of claim 1, wherein the enrollment image and the verification image both comprise a human iris.

### Claim 3 (depends on 1)

The computer-readable medium of claim 1, wherein the enrollment image and the verification image both comprise a human face.

### Claim 4 (depends on 1)

The computer-readable medium of claim 1, wherein:
the set of verification image features are extracted from the verification image using a set of verification complex-response layers; and
the computer-readable medium further comprises additional instructions that are executable by the one or more processors to obtain the enrollment image and extract the set of enrollment image features from the enrollment image using a set of enrollment complex-response layers.

### Claim 5 (depends on 1)

The computer-readable medium of claim 1, further comprising additional instructions that are executable by the one or more processors to process a plurality of sets of enrollment image features with the set of verification image features using the convolutional neural network to determine the metric.

### Claim 6 (depends on 1)

The computer-readable medium of claim 1, wherein the convolutional neural network is included in a recurrent neural network, and further comprising additional instructions that are executable by the one or more processors to:
obtain a plurality of verification images;
extract a plurality of sets of verification image features from the plurality of verification images; and
process each set of verification image features with the set of enrollment image features to determine a plurality of metrics, wherein the metric that is determined in connection with processing a particular set of verification image features depends on information obtained in connection with processing one or more previous sets of verification image features.

### Claim 7 (depends on 6)

The computer-readable medium of claim 6, further comprising additional instructions that are executable by the one or more processors to determine an additional metric that indicates a likelihood that the plurality of verification images correspond to a live human being.

### Claim 8 (depends on 1)

The computer-readable medium of claim 1, wherein the convolutional neural network is included in a recurrent neural network, and further comprising additional instructions that are executable by the one or more processors to:
obtain a plurality of sets of enrollment image features corresponding to a plurality of enrollment images;
obtain a plurality of verification images;
extract a plurality of sets of verification image features from the plurality of verification images; and
process each set of verification image features with the plurality of sets of enrollment image features to determine a plurality of metrics, wherein the metric that is determined in connection with processing a particular set of verification image features depends on information obtained in connection with processing one or more previous sets of verification image features.

### Claim 9 (depends on 8)

The computer-readable medium of claim 8, further comprising additional instructions that are executable by the one or more processors to determine an additional metric that indicates a likelihood that the plurality of verification images correspond to a live human being.

### Claim 10 (depends on 1)

The computer-readable medium of claim 1, wherein the enrollment image comprises a left-eye enrollment image, wherein the verification image comprises a left-eye verification image, wherein the convolutional neural network comprises a left-eye convolutional neural network, and further comprising additional instructions that are executable by the one or more processors to:
obtain right-eye enrollment image features that are extracted from a right-eye enrollment image;
obtain right-eye verification image features that are extracted from a right-eye verification image; and
process the right-eye enrollment image features and the right-eye verification image features using a right-eye convolutional neural network, wherein the metric depends on output from the left-eye convolutional neural network and the right-eye convolutional neural network.

### Claim 11

A computing device, comprising:
a camera;
one or more processors;
memory in electronic communication with the one or more processors;
a set of enrollment image features stored in the memory, the set of enrollment image features corresponding to an enrollment image;
instructions stored in the memory, the instructions being executable by the one or more processors to:
cause the camera to capture a verification image;
extract a set of verification image features from the verification image;
process the set of verification image features and the set of enrollment image features using a convolutional neural network to determine a metric; and
determine whether the verification image matches the enrollment image based on the metric.

### Claim 12 (depends on 11)

The computing device of claim 11, further comprising additional instructions that are executable by the one or more processors to:
receive a user request to perform an action; and
perform the action in response to determining that the metric exceeds a pre-defined threshold value.

### Claim 13 (depends on 12)

The computing device of claim 12, wherein the computing device comprises a head-mounted mixed reality device, and wherein the action comprises loading a user model corresponding to a user of the computing device.

### Claim 14 (depends on 11)

The computing device of claim 11, further comprising:
a plurality of sets of enrollment image features stored in the memory; and
additional instructions that are executable by the one or more processors to process the plurality of sets of enrollment image features with the set of verification image features using the convolutional neural network to determine the metric.

### Claim 15 (depends on 11)

The computing device of claim 11, further comprising additional instructions that are executable by the one or more processors to:
cause the camera to capture a plurality of verification images;
extract a plurality of sets of verification image features from the plurality of verification images; and
process each set of verification image features with the set of enrollment image features to determine a plurality of metrics, wherein the metric that is determined in connection with processing a particular set of verification image features depends on information obtained in connection with processing one or more previous sets of verification image features.

### Claim 16 (depends on 15)

The computing device of claim 15, further comprising additional instructions that are executable by the one or more processors to determine an additional metric that indicates a likelihood that the plurality of verification images correspond to a live human being.

### Claim 17 (depends on 11)

The computing device of claim 11, wherein the convolutional neural network is included in a recurrent neural network, and further comprising additional instructions that are executable by the one or more processors to:
obtain a plurality of sets of enrollment image features corresponding to a plurality of enrollment images;
cause the camera to capture a plurality of verification images;
extract a plurality of sets of verification image features from the plurality of verification images; and
process each set of verification image features with the plurality of sets of enrollment image features to determine a plurality of metrics, wherein the metric that is determined in connection with processing a particular set of verification image features depends on information obtained in connection with processing one or more previous sets of verification image features.

### Claim 18 (depends on 17)

The computing device of claim 17, further comprising additional instructions that are executable by the one or more processors to determine an additional metric that indicates a likelihood that the plurality of verification images correspond to a live human being.

### Claim 19

A system, comprising:
one or more processors;
memory in electronic communication with the one or more processors;
instructions stored in the memory, the instructions being executable by the one or more processors to:
receive a request from a client device to perform biometric verification;
receive a verification image from the client device;
process a set of verification image features and a set of enrollment image features using a convolutional neural network to determine a metric;
determine a verification result based on the metric; and
send the verification result to the client device.

### Claim 20 (depends on 19)

The system of claim 19, further comprising additional instructions that are executable by the one or more processors to:
extract the set of verification image features from the verification image;
obtain an enrollment image; and
extract the set of enrollment image features from the enrollment image.
