# US 11,908,238 B2 · Methods and systems for facial point-of-recognition (POR) provisioning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,238 B2
- Document ID: US11908238B2
- Status: Granted
- Kind: B2
- Application number: 17/176,896
- Filed: 2021-02-16
- Published: 2021-07-22
- Granted: 2024-02-20
- Earliest claimed: 2015-10-01
- Assignees: Nortek Security & Control LLC
- Applicants: Nortek Security & Control
- Inventors: Chandan Gope; Gagan Gupta; Nitin Jindal; Krishna Khadloya; Vaidhi Nathan
- Web page: https://patentgrep.com/patent/US11908238B2

## CPC

- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/172: {Classification, e.g. identification}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20072: Graph-based image processing
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30201: Face
- G06V 10/467: {Encoded features or binary features, e.g. local binary patterns [LBP]}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disclosed is a facial recognition system/method, utilizing modules to perform the following routine: extracting a feature descriptor from a detected feature point of a detected face in an input image frame; and matching the extracted feature descriptor with at least one of a pre-stored facial image that is index-mapped, comprising at least a first and second round of matching, wherein the second round of matching only selects the index-mapped facial images that matched above a pre-defined threshold from the first round of matching. Optionally, the above described steps may be coupled to a Point-of-Recognition (POR) provisioning, enabling an on-demand gate-keeping and/or payment processing for an end-user at an event/venue entry or point-of-sale.

## Claims

### Claim 1

A facial recognition method, said method comprising the steps of:
extracting a feature descriptor from a detected feature point of a detected face in an input image frame from a video/image capturing device; and
matching the extracted feature descriptor with at least one of a pre-stored facial image that is index-mapped using a constrained deformable part-based model (CDPBM) that uses Histogram-of-Gradient (HoG) as part-based filters and trained by latent Support Vector Machine (SVM) to determine face feature point location and geometry, comprising at least a first and second round of matching, wherein the second round of matching only selects the index-mapped facial images that matched above a pre-defined threshold from the first round of matching.

### Claim 2 (depends on 1)

The method of claim 1, wherein the index mapping generating an index map, is in at least one of an array, hash table, lookup table, or a k-dimensional (k-d) tree format in order to speed up identifying a matching image among the pre-stored images for the input image for facial recognition.

### Claim 3 (depends on 2)

The method of claim 2, further comprising the step of pruning the k-dimensional (k-d) tree.

### Claim 4 (depends on 1)

The method of claim 1, further comprising the step of comparing a signature of the input image with the one or more feature descriptors of the pre-stored image using at least one of spatial consistency or a temporal verification check on the matching image.

### Claim 5 (depends on 1)

The method of claim 1, further comprising the step of computing a matching score between the input image and the pre-stored image.

### Claim 6 (depends on 1)

The method of claim 1, further comprising the step of clustering a set of training images into Similar Image Clusters (SICs).

### Claim 7 (depends on 1)

The method of claim 1, further comprising the step of receiving the input image frame from at least one of an access control point or surveillance point, wherein the access control point or surveillance point further comprises at least one of the following provisioning: allow access to a venue/event; deny access to a venue/event; process a purchasing transaction; or provide admission to a mass transit.

### Claim 8

A facial recognition system comprising:
an image input unit;
a feature extraction unit;
an index mapping unit;
a matching unit;
a processor, a non-transitory storage element coupled to the processor, encoded instructions stored in the non-transitory storage element, wherein the encoded instructions when implemented by the processor, configure the system to:
receive a video and/or image from a surveillance or access-control point for parsing into image frames or segments by the image input unit;
extract a feature descriptor using a constrained deformable part-based model (CDPM) that uses Histogram of Gradient (HoG) as part-based filters and trained by latent Support Vector Machine (SVM) to determine face feature point location and geometry by the feature extraction unit; and
match the extracted feature descriptor with at least one of a pre-stored facial image that is index-mapped, comprising at least a first and second round of matching, wherein the second round of matching only selects the index-mapped facial images that matched above a pre-defined threshold from the first round of matching by the matching unit.

### Claim 9 (depends on 8)

The system of claim 8, wherein the image input unit receives input from at least one of a real-time streaming system, video/image archive, computer system, video capture device, remote server, or hand-held device.

### Claim 10 (depends on 8)

The system of claim 8, wherein the image input unit or feature extraction unit selects the best appearance of the face across the frames based on at least one of a sharpness, size, noise, motion blur, or pose variation.

### Claim 11 (depends on 8)

The system of claim 8, further comprising a face alignment and normalization unit further configured to normalize the detected feature point by at least one of a Gamma Correction, Difference of Gaussian (DoG) filtering, Masking, or Contrast Equalization.

### Claim 12 (depends on 11)

The system of claim 11, wherein the masking further comprises detecting an upper and lower edge of glasses and applying a filter to remove masking effects of the glasses on the face.

### Claim 13 (depends on 11)

The system of claim 11, wherein the masking further comprises detecting outer edges of facial hair and applying a filter to remove masking effects of the facial hair on the face.

### Claim 14 (depends on 8)

The system of claim 8, wherein the feature extraction unit is further configured to
transform the feature descriptors to a lower dimensional space.

### Claim 15 (depends on 14)

The system of claim 14, wherein the transformation is performed using at least one
of a discriminate analysis approach such as Fisher Discriminant Analysis, Principle Component Analysis and Independent Component Analysis (ICA).

### Claim 16 (depends on 8)

The system of claim 8, further comprising the step of computing a matching score
between the input image and the pre-stored image and determining the threshold based on the surveillance or access control point by the matching unit.

### Claim 17 (depends on 8)

The system of claim 8, wherein the access control point further comprises at least one of the following provisioning: allow access to a venue/event, deny access to a venue/event, or process a purchasing transaction.

### Claim 18

A facial recognition system comprising:
a processor, a non-transitory storage element coupled to the processor, encoded instructions stored in the non-transitory storage element, wherein the encoded instructions when implemented by the processor, configure the system to:
receive a video and/or image from a surveillance or access-control point for parsing into image frames or segments;
extract a feature descriptor from a detected feature point of a detected face in the input image frame or segment; and
match the extracted feature descriptor with at least one of a pre-stored facial image using a constrained deformable part-based model (CDPM) that uses Histogram of Gradient (HoG) as part-based filters and trained by latent Support Vector Machine (SVM) to determine face feature point location and geometry, comprising at least a first and second round of matching, wherein the second round of matching only selects the facial images that matched above a pre-defined threshold from the first round of matching wherein the access control point further comprises at least one of the following provisioning: allow access to a venue/event, deny access to a venue/event, or process a purchasing transaction.