US 12,711,812 B1Granted

Uncertainty aware fusion of biometric data

AssigneeAmazon Technologies, Inc.

InventorsRui Zhao, Rajeev Ranjan, Manoj Aggarwal, Gerard Guy Medioni

Life of this patent

Estimating the term…

Loading patent…

Abstract

Biometric data may be subject to internal and external factors that complicate assessment. Internal factors include variations due to physiological changes, such as dilation or constriction of blood vessels. External factors include variations in pose, illumination, cosmetics, and so forth. One or more machine learning networks are trained to accept as input images and their modality information and provide as output parameters of conditional distributions characterizing the uncertainty due to these internal and external factors. The input images are processed by embedding modules to determine input embedding data. Based on the associated output parameters, weight values may be calculated that are associated with distances in embedding space between input embedding data from multiple modalities and other embedding data corresponding to those modalities, such as from a previously enrolled user. The weight values and distances may then be used to determine if the input images correspond to the other embedding data.

Claims20

Dependent claims

A system comprising:

one or more memories, storing first computer-executable instructions;
and
one or more hardware processors to execute the first computer-executable instructions to:
determine first input image data associated with a first modality, wherein the first input image data associated with the first modality is further associated with internal factors of a hand of a user;
determine first embedding data representative of the internal factors of the hand of the user as depicted in the first input image data;
determine second input image data associated with a second modality, wherein the second input image data associated with the second modality is further associated with external factors of the hand of the user;
determine second embedding data representative of the external factors of the hand of the user as depicted in the second input image data;
determine first uncertainty data indicative of uncertainties associated with the internal factors of the hand of the user;
determine second uncertainty data indicative of uncertainties associated with the external factors of the hand of the user;
determine first XOR embedding data that is representative of features that are not associated with both the first embedding data and the second embedding data;
determine third embedding data associated with the first modality, wherein the third embedding data associated with the first modality is further associated with internal factors of an enrolled hand of a corresponding user;
determine fourth embedding data associated with the second modality, wherein the fourth embedding data associated with the second modality is further associated with external factors of the enrolled hand of the corresponding user;
determine third uncertainty data indicative of uncertainties associated with the internal factors of the enrolled hand of the corresponding user;
determine fourth uncertainty data indicative of uncertainties associated with the external factors of the enrolled hand of the corresponding user;
determine second XOR embedding data that is representative of features that are not associated with both the third embedding data and the fourth embedding data;
determine a first distance in a first embedding space based on the first embedding data and the third embedding data;
determine a second distance in a second embedding space based on the second embedding data and the fourth embedding data;
determine a third distance in a third embedding space based on the first XOR embedding data and the second XOR embedding data;
determine a first weight value based on:
the first distance,
the second distance,
the third distance,
the first uncertainty data,
the second uncertainty data,
the third uncertainty data, and
the fourth uncertainty data;
determine comparison data based on:
the first distance,
the second distance,
the third distance, and
the first weight value;
and
determine identification data based at least on the comparison data.

30 more elements

A computer-implemented method comprising:

determining first biometric data associated with a first modality, wherein the first biometric data associated with the first modality is further associated with internal factors of a portion of a user;
determining first embedding data representative of the internal factors of the portion of the user depicted in the first biometric data;
determining second biometric data associated with a second modality, wherein the second biometric data associated with the second modality is further associated with external factors of the portion of the user;
determining second embedding data representative of the external factors of the portion of the user depicted in the second biometric data;
determining first uncertainty data indicative of uncertainties associated with the internal factors of the portion of the user;
determining second uncertainty data indicative of uncertainties associated with the external factors of the portion of the user;
determining third embedding data associated with the first modality, wherein the third embedding data associated with the first modality is further associated with internal factors of an enrolled portion of a corresponding user;
determining fourth embedding data associated with the second modality, wherein the fourth embedding data associated with the second modality is further associated with external factors of the enrolled portion of the corresponding user;
determining third uncertainty data indicative of uncertainties associated with the internal factors of the enrolled portion of the corresponding user;
determining fourth uncertainty data indicative of uncertainties associated with the external factors of the enrolled portion of the corresponding user, wherein one or more of the first uncertainty data, the second uncertainty data, the third uncertainty data, or the fourth uncertainty data is indicative of:
uncertainty associated with physiological processes of the portion of the user depicted in the first or second biometric data;
determining a first distance in a first embedding space based on the first embedding data and the third embedding data;
determining a second distance in a second embedding space based on the second embedding data and the fourth embedding data;
determining a first weight value based on:
the first distance,
the second distance;
and
one or more of:
the first uncertainty data,
the second uncertainty data,
the third uncertainty data, or
the fourth uncertainty data;
determining comparison data based on:
the first distance,
the second distance, and
the first weight value;
and
determining identification data based at least on the comparison data.

25 more elements

A system comprising:

one or more memories, storing first computer-executable instructions;
and
one or more hardware processors to execute the first computer-executable instructions to:
determine first biometric data associated with a first modality, wherein the first biometric data associated with the first modality is further associated with internal factors of a hand of a user;
determine first embedding data representative of the internal factors of the hand of the user depicted in the first biometric data;
determine second biometric data associated with a second modality, wherein the second biometric data associated with the second modality is further associated with external factors of the hand of the user;
determine second embedding data representative of the external factors of the hand of the user depicted in the second biometric data;
determine first uncertainty data indicative of uncertainties associated with the internal factors of the hand of the user;
determine second uncertainty data indicative of uncertainties associated with the external factors of the hand of the user;
determine third embedding data associated with the first modality, wherein the third embedding data associated with the first modality is further associated with internal factors of an enrolled hand of a corresponding user;
determine fourth embedding data associated with the second modality, wherein the fourth embedding data associated with the second modality is further associated with external factors of the enrolled hand of the corresponding user;
determine third uncertainty data indicative of uncertainties associated with the internal factors of the enrolled hand of the corresponding user;
determine fourth uncertainty data indicative of uncertainties associated with the external factors of the enrolled hand of the corresponding user;
determine a first distance in a first embedding space based on the first embedding data and the third embedding data;
determine a second distance in a second embedding space based on the second embedding data and the fourth embedding data;
determine a first weight value based on:
the first distance,
the second distance;
and
one or more of:
the first uncertainty data,
the second uncertainty data,
the third uncertainty data, or
the fourth uncertainty data;
and
determine comparison data based on:
the first distance,
the second distance, and
the first weight value.

26 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/193,080Filed 30 Mar 2023
Granted: 18 Aug 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06V 40/70
Recognition of biometric, human-related or animal-related patterns in image or video data: multimodal biometrics, e.g. combining information from…
G06V 10/82
Using pattern recognition or machine learning: using neural networks
G06V 40/1353
Preprocessing; Feature extraction: extracting features related to minutiae or pores
G06V 40/1359
Preprocessing; Feature extraction: extracting features related to ridge properties; Determining the fingerprint type, e.g. whorl or loop