US 9,852,364 B2Granted

Face track recognition with multi-sample multi-view weighting

AssigneeHulu, LLC

InventorsCailiang Liu, Zhibing Wang, Chenguang Zhang, Tao Xiong

Life of this patent

Estimating the term…

Loading patent…

Abstract

In one embodiment, a method determines known features for existing face tracks that have identity labels and builds a database using these features. The face tracks may have multiple different views of a face. Multiple features from the multiple faces may be taken to build the face models. For an unlabeled face track without identity information, the method determines its sampled features and finds labeled nearest neighbor features with respect to multiple feature spaces from the face models. For each face in the unlabeled face track, the method decomposes the face as a linear combination of its neighbors from the known features from the face models. Then, the method determines weights for the known features to weight the coefficients of the known features. Particular embodiments use a non-linear weighting function to learn the weights that provides more accurate labels.

Claims20

Dependent claims

A method comprising:

determining, by a computing device, a plurality of face tracks for detected face samples in a video;
receiving, by the computing device, a set of labels for a portion of the plurality of face tracks from a set of users to form a set of labeled face tracks, wherein a label in the set of labels identifies an identity for a face track in the set of labeled face tracks;
extracting, by the computing device, a first set of features for a first plurality of face samples in an unlabeled face track in the plurality of face tracks, wherein an identity for the unlabeled face track is not known;
correlating, by the computing device, the extracted first set of features for the unlabeled face track to a second set of features from a second plurality of face samples in the set of labeled face tracks;
generating, by the computing device, feature weights for the first set of features based on a set of confidence scores for the second set of features using a weighting function that magnifies feature weights for second features from the second plurality of face samples in the second set of features with higher confidence scores more than feature weights for second features from the second plurality of face samples in the second set of features with lower confidence scores in a non-linear manner, wherein a confidence score in the set of confidence scores for a feature in the second plurality of features is based on a quality of a face track in the set of labeled face tracks that includes the feature in the second plurality of features;
and
using, by the computing device, the generated feature weights to determine a label for the unlabeled face track by applying the generated feature weights to the second set of features.

4 more elements

A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be configured for:

determining a plurality of face tracks for detected face samples in a video;
receiving a set of labels for a portion of the plurality of face tracks from a set of users to form a set of labeled face tracks, wherein a label in the set of labels identifies an identity for a face track in the set of labeled face tracks;
extracting a first set of features for a first plurality of face samples in an unlabeled face track in the plurality of face tracks, wherein an identity for the unlabeled face track is not known;
correlating the extracted first set of features for the unlabeled face track to a second set of features from a second plurality of face samples in the set of labeled face tracks;
generating feature weights for the first set of features based on a set of confidence scores for the second set of features using a weighting function that magnifies feature weights for second features from the second plurality of face samples in the second set of features with higher confidence scores more than feature weights for second features from the second plurality of face samples in the second set of features with lower confidence scores in a non-linear manner, wherein a confidence score in the set of confidence scores for a feature in the second plurality of features is based on a quality of a face track in the set of labeled face tracks that includes the feature in the second plurality of features;
and
using the generated feature weights to determine a label for the unlabeled face track by applying the generated feature weights to the second set of features.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/662,889Filed 19 Mar 2015 · us provisional application
Priority: 19 Mar 2014
Published: US 2015/0269421 A124 Sep 2015
Granted: 26 Dec 2017
Term ends: Not estimatedThe source records don’t support an estimate.

Classification5 codes

G06V 40/173
Classification, e.g. identification: face re-identification, e.g. recognising unknown faces across different face tracks
G06F 18/253
Fusion techniques: of extracted features
G06V 10/806
Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level: of…
G06V
Image and video recognition