# US 2021/0174069 A1 · System for verifying the identity of a user

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174069 A1
- Document ID: US20210174069A1
- Status: Published application
- Kind: A1
- Application number: 17/172,852
- Filed: 2021-02-10
- Published: 2021-06-10
- Earliest claimed: 2018-08-06
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Abdelkader M'Hamed Benkreira; Joshua Edwards; Michael Mossoba
- Web page: https://patentgrep.com/patent/US20210174069A1

## CPC

- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images
- G06V 30/1444: {Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields}
- G06V 30/153: {using recognition of characters or words}
- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/40: Spoof detection, e.g. liveness detection
- G06V 40/45: {Detection of the body part being alive}
- H04W 12/06: Authentication
- G06V 30/10: Character recognition
- G06V 2201/09: Recognition of logos
- H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A system receives an image including a live facial image of the user and an identity document including a photograph of the user. Moreover, the system calculates a facial match score by comparing facial features in the live facial image to facial features in the photograph. The system recognizes data objects and characters in the identity document using optical character recognition (OCR) and computer vision, and then identifies, based on the recognized data objects and characters, a type of the identity document. Further, the system calculates a document validity score by comparing the recognized characters and data objects to character strings and data objects known to be present in the identified type of the identity document. Additionally, the system determines and outputs the user's identity verification status based on comparing the facial match score to a facial match threshold and comparing the document validity score to a document validity threshold.

## Claims

### Claim 1

A method comprising:
receiving, by a computer processor, image data from an electronic source comprising:
i) a facial image of a user, and
ii) an identity document, comprising a photograph of the user;
calculating, by the computer processor, a facial match score by comparing a plurality of first facial features extracted from the facial image to a plurality of second facial features extracted from the photograph;
recognizing, by the computer processor, at least one data object and at least one character in the identity document to identify a type of the identity document;
determining, by the computer processor, a document validity score based on:
i) the at least one data object,
ii) the at least one recognized character,
iii) at least one known data object that is known to be present in the identified type of the identity document, and
iv) at least one known character that is known to be present in the identified type of the identity document; and
determining, by the computer processor, an identity verification status of the user based on:
i) a weighted combination of the facial match score and the document validity score, and
ii) a predetermined threshold value; and
outputting, by the computer processor, the identity verification status so as to verify the identify of the user with a user request.

### Claim 2 (depends on 1)

The method of claim 1, wherein calculating the facial match score comprises:
extracting a first set of facial features from the live facial image;
extracting a second set of facial features from the photograph;
comparing, using a facial recognition algorithm, the first set of facial features to the second set of facial features; and
calculating the facial match score as a percentage of facial features in common between the live facial image and the photograph.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
outputting the identity verification status comprises presenting, on a display device, the identity verification status; and
the identity verification status indicates that the identity of the user is verified in response to determining that:
the facial match score exceeds the facial match threshold; and
the document validity score exceeds the document validity threshold.

### Claim 4 (depends on 1)

The method of claim 1, wherein calculating the document validity score comprises:
comparing the recognized data objects to a watermark security feature present in the identified type of the identity document.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
extracting, based on comparing the recognized characters to characteristics present in the identified type of the identity document, one or more secondary characteristics of the user from the image; and
comparing the one or more secondary characteristics to the facial features in the live facial image and the facial features in the photograph.

### Claim 6 (depends on 5)

The method of claim 5, wherein the characteristics present in the identified type of the identity document include one or more of: hair color; eye color; gender, weight; height; date of birth; and ethnicity.

### Claim 7 (depends on 5)

The method of claim 5, wherein determining the identity verification status of the user is further based on:
comparing the one or more secondary characteristics to user profile data for the user retrieved from a data store.

### Claim 8 (depends on 1)

The method of claim 1, wherein outputting the identity verification status comprises prompting the user to submit an additional image including another live facial image of the user holding the identity document in response to determining that the facial match score is below the facial match threshold.

### Claim 9 (depends on 1)

The method of claim 1, wherein outputting the identity verification status comprises prompting the user to submit an additional image including another live facial image of the user holding another identity document in response to determining that the document validity score is below the document validity threshold.

### Claim 10 (depends on 1)

The method of claim 1, wherein the recognized data objects include one or more of: a watermark; a hologram; a bar code; a serial number; a thumbnail version of the photograph; a negative image of the photograph; and a Quick Response (QR) code.

### Claim 11 (depends on 1)

The method of claim 1, wherein determining the identity verification status of the user is further based on comparing the recognized data objects to user profile data for the user retrieved from a data store.

### Claim 12

A method comprising:
receiving an image, the image including a live facial image of the user and an identity document that includes a photograph of the user;
receiving, by a computer processor, image data from an electronic source comprising:
i) a facial image of a user, and
ii) an identity document, comprising a photograph of the user;
calculating, by the computer processor, a facial match score by comparing a plurality of first facial features extracted from the facial image to a plurality of second facial features extracted from the photograph;
recognizing, by the computer processor, at least one data object and at least one character in the identity document to identify a type of the identity document;
determining, by the computer processor, a document validity score based on:
i) the at least one data object,
ii) the at least one recognized character,
iii) at least one known data object that is known to be present in the identified type of the identity document, and
iv) at least one known character that is known to be present in the identified type of the identity document; and
determining, by the computer processor, an identity verification status of the user based on:
i) a weighted combination of the facial match score and the document validity score, and
ii) a predetermined threshold value; and
outputting the identity verification status, wherein outputting the identity verification status comprises prompting the user to submit an additional image including another live facial image of the user holding the identity document in response to determining that the facial match score is below the facial match threshold.

### Claim 13 (depends on 12)

The method of claim 12, wherein calculating the facial match score comprises:
extracting a first set of facial features from the live facial image;
extracting a second set of facial features from the photograph;
comparing, using a facial recognition algorithm, the first set of facial features to the second set of facial features; and
calculating the facial match score as a percentage of facial features in common between the live facial image and the photograph.

### Claim 14 (depends on 12)

The method of claim 12, wherein:
outputting the identity verification status comprises presenting, on a display device, the identity verification status; and
the identity verification status indicates that the identity of the user is verified in response to determining that:
the facial match score exceeds the facial match threshold; and
the document validity score exceeds the document validity threshold.

### Claim 15 (depends on 12)

The method of claim 12, wherein calculating the document validity score comprises:
comparing the recognized data objects to a watermark security feature present in the identified type of the identity document.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
extracting, based on comparing the recognized characters to characteristics present in the identified type of the identity document, one or more secondary characteristics of the user from the image; and
comparing the one or more secondary characteristics to the facial features in the live facial image and the facial features in the photograph.

### Claim 17 (depends on 16)

The method of claim 16, wherein the characteristics present in the identified type of the identity document include one or more of: hair color; eye color; gender, weight; height; date of birth; and ethnicity.

### Claim 18 (depends on 16)

The method of claim 16, wherein determining the identity verification status of the user is further based on:
comparing the one or more secondary characteristics to user profile data for the user retrieved from a data store.

### Claim 19 (depends on 12)

The method of claim 12, wherein receiving the image comprises receiving the image from a camera, the image including the user holding the identity document that includes the photograph of the user.

### Claim 20 (depends on 19)

The method of claim 19, wherein the camera is part of a mobile device.

### Claim 21 (depends on 12)

The method of claim 12, wherein outputting the identity verification status further comprises prompting the user to submit another additional image including another live facial image of the user holding another identity document in response to determining that the document validity score is below the document validity threshold.

### Claim 22 (depends on 12)

The method of claim 12, wherein the recognized data objects include one or more of: a watermark; a hologram; a bar code; a serial number; a thumbnail version of the photograph; a negative image of the photograph; and a Quick Response (QR) code.

### Claim 23 (depends on 12)

The method of claim 12, wherein determining the identity verification status of the user is further based on comparing the recognized data objects to user profile data for the user retrieved from a data store.

### Claim 24

A method comprising:
receiving an image, the image including a live facial image of the user and an identity document that includes a photograph of the user;
receiving, by a computer processor, image data from an electronic source comprising:
i) a facial image of a user, and
ii) an identity document, comprising a photograph of the user;
calculating, by the computer processor, a facial match score by comparing a plurality of first facial features extracted from the facial image to a plurality of second facial features extracted from the photograph;
recognizing, by the computer processor, at least one data object and at least one character in the identity document to identify a type of the identity document;
determining, by the computer processor, a document validity score based on:
i) the at least one data object,
ii) the at least one recognized character,
iii) at least one known data object that is known to be present in the identified type of the identity document, and
iv) at least one known character that is known to be present in the identified type of the identity document; and
determining, by the computer processor, an identity verification status of the user based on:
i) a weighted combination of the facial match score and the document validity score, and
ii) a predetermined threshold value; and
outputting the identity verification status, wherein outputting the identity verification status comprises prompting the user to submit an additional image including another live facial image of the user holding another identity document in response to determining that the document validity score is below the document validity threshold.

### Claim 25 (depends on 24)

The method of claim 24, wherein calculating the facial match score comprises:
extracting a first set of facial features from the live facial image;
extracting a second set of facial features from the photograph;
comparing, using a facial recognition algorithm, the first set of facial features to the second set of facial features; and
calculating the facial match score as a percentage of facial features in common between the live facial image and the photograph.

### Claim 26 (depends on 24)

The method of claim 24, wherein:
outputting the identity verification status comprises presenting, on a display device, the identity verification status; and
the identity verification status indicates that the identity of the user is verified in response to determining that:
the facial match score exceeds the facial match threshold; and
the document validity score exceeds the document validity threshold.

### Claim 27 (depends on 24)

The method of claim 24, wherein the identity verification status indicates that an identity of the user is not verified in response to determining that a weighted combination of the facial match score and the document validity score is less than or equal to a weighted combination of the facial match threshold and the document validity threshold.

### Claim 28 (depends on 24)

The method of claim 24, wherein calculating the document validity score comprises:
comparing the recognized data objects to a watermark security feature present in the identified type of the identity document.

### Claim 29 (depends on 24)

The method of claim 24, further comprising:
extracting, based on comparing the recognized characters to characteristics present in the identified type of the identity document, one or more secondary characteristics of the user from the image; and
comparing the one or more secondary characteristics to the facial features in the live facial image and the facial features in the photograph.

### Claim 30 (depends on 29)

The method of claim 29, wherein the characteristics present in the identified type of the identity document include one or more of: hair color; eye color; gender, weight; height; date of birth; and ethnicity.

### Claim 31 (depends on 29)

The method of claim 29, wherein determining the identity verification status of the user is further based on:
comparing the one or more secondary characteristics to user profile data for the user retrieved from a data store.

### Claim 32 (depends on 24)

The method of claim 24, wherein receiving the image comprises receiving the image from a camera, the image including the user holding the identity document that includes the photograph of the user.

### Claim 33 (depends on 32)

The method of claim 32, wherein the camera is part of a mobile device.

### Claim 34 (depends on 24)

The method of claim 24, wherein the recognized data objects include one or more of: a watermark; a hologram; a bar code; a serial number; a thumbnail version of the photograph; a negative image of the photograph; and a Quick Response (QR) code.

### Claim 35 (depends on 24)

The method of claim 24, wherein determining the identity verification status of the user is further based on comparing the recognized data objects to user profile data for the user retrieved from a data store.
