Back to results
US 12,217,549 B1Granted
System and method for matching a voice sample to a facial image based on voice and image metaproperties
AssigneeCorsound AI Ltd
InventorsNir Schwartz, Arkady Krishtul
Life of this patent
Estimating the term…
Abstract
System and method for matching a voice sample to a facial image may include, using a processor: obtaining a voice sample and a facial image; calculating a plurality of voice metaproperties from the voice sample; calculating a plurality of image metaproperties from image sample, where each of the image metaproperties corresponds to one of the voice metaproperties; and determining a level of match between the voice sample and the facial image, based on the plurality of voice metaproperties and the plurality of image metaproperties.
Claims12
A method for matching a voice sample to a facial image, the method comprising, using a processor:
- obtaining a voice sample and a facial image;
- calculating a plurality of voice metaproperties from the voice sample;
- calculating a plurality of image metaproperties from the facial image, wherein each of the image metaproperties corresponds to one of the voice metaproperties, wherein each of the voice metaproperties and the image metaproperties comprises a probability distribution providing the probabilities that the voice metaproperty or the image metaproperty equals certain values of the metaproperty;
- and
- determining a level of match between the voice sample and the facial image, based on the plurality of voice metaproperties and the plurality of image metaproperties, wherein determining whether the voice sample matches the facial image is performed by:
- calculating a distance between each of the voice metaproperties and the corresponding image metaproperty;
- calculating weights for a weighted sum operation by training a classifier and deriving the weights from the parameters of the classifier;
- calculating the weighted sum of the distances;
- and
- determining that the voice sample matches the facial image if the weighted sum satisfies a threshold condition, and that the voice sample does not match the facial image otherwise,
- wherein the classifier is trained by:
- obtaining a labelled dataset comprising a plurality of matching pairs labelled as matching pairs, and a plurality of unmatching pairs, labelled as unmatching pairs, wherein each of the matching pairs comprises a matching labelled voice sample and labelled facial image, and each of the unmatching pairs comprises an unmatching labelled voice sample and labelled facial image;
- calculating, for each of the labelled voice samples, the plurality of voice metaproperties from the labelled voice sample;
- calculating, for each of the labelled facial images, the plurality of image metaproperties from the labelled facial image;
- and
- using the plurality of voice metaproperties and the plurality of image metaproperties of the plurality of matching pairs and the plurality of unmatching pairs, and the associated labels, to train the classifier.
13 more elements
A system for matching a voice sample to a facial image, the system comprising:
- a memory;
- and
- a processor configured to:
- obtain a voice sample and a facial image;
- calculate a plurality of voice metaproperties from the voice sample;
- calculate a plurality of image metaproperties from the facial image, wherein each of the image metaproperties corresponds to one of the voice metaproperties, wherein each of the voice metaproperties and the image metaproperties comprises a probability distribution providing the probabilities that the voice metaproperty or the image metaproperty equals certain values of the metaproperty;
- and
- determine a level of match between the voice sample and the facial image, based on the plurality of voice metaproperties and the plurality of image metaproperties, by:
- calculating a distance between each of the voice metaproperties and the corresponding image metaproperty;
- calculating a weighted sum of the distances;
- and
- determining that the voice sample matches the facial image if the weighted sum satisfies a threshold condition, and that the voice sample does not match the facial image otherwise,
- wherein the processor is configured to calculate weights for the weighted sum operation by training a classifier and deriving the weights from the parameters of the classifier, and
- wherein the processor is configured to train the classifier by:
- obtaining a labelled dataset comprising a plurality of matching pairs labelled as matching pairs, and a plurality of unmatching pairs, labelled as unmatching pairs, wherein each of the matching pairs comprises a matching labelled voice sample and labelled facial image, and each of the unmatching pairs comprises an unmatching labelled voice sample and labelled facial image;
- calculating, for each of the labelled voice samples, the plurality of voice metaproperties from the labelled voice sample;
- calculating, for each of the labelled facial images, the plurality of image metaproperties from the labelled facial image;
- and
- using the plurality of voice metaproperties and the plurality of image metaproperties of the plurality of matching pairs and the plurality of unmatching pairs, and the associated labels, to train the classifier.
16 more elements
Dependencies are inferred from the claim text.