Back to results
US 10,950,222 B2Granted
Multimodal video system for generating a personality assessment of a user
AssigneeYobs Technologies, Inc.
InventorsRaphael Louis Jean Danilo, Prathmesh A. Gat, Aamir K. Goriawala
Life of this patent
Estimating the term…
Abstract
The present disclosure is directed to a system for generating a personality assessment that uses multimodal behavioral signal processing technology and machine learning prediction technology. This system takes a video as input, processes it through an artificial intelligence software built for extracting hundreds of behavioral features, and consequently generates an accurate and reliable personality assessment with its machine-learning predictive software. The personality assessment is based on the five-factor model (FFM), also known as the big 5 personality traits.
Claims20
A method for generating a personality assessment for a user from video input thereof, via a computing system, wherein the computing system comprises:
- (a) a processor operable to control the computing system,
- (b) a data storage operatively coupled to the processor, wherein the data storage is configured to store a plurality of personality data associated with a user,
- (c) an input/output device operatively coupled to the processor, wherein the input/output device is configured to receive a plurality of data for transmission to the processor, wherein the input/output device is configured to transmit a plurality of data generated by the processor,
- (d) a feature extraction component operatively coupled to the processor and controlled in part by the processor, wherein the feature extraction component is configured to extract a plurality of feature data from a video of the user,
- (e) a training component operatively coupled to the processor and controlled in part by the processor, wherein the training component is configured to generate a plurality of trained feature data, and
- (f) a prediction component operatively coupled to the processor and controlled in part by the processor, wherein the prediction component is configured to generate a personality assessment for the user, the method comprising:
- receiving, via the input/output device, video input associated with the user and transmitting the video input to the feature extraction component, wherein the video input comprises video data and audio data;
- generating, via the feature extraction component, a text transcript of at least a portion of the audio data of the received video input;
- extracting, via the feature extraction component, a plurality of parts of speech (POS) feature data, emotion feature data, and linguistic inquiry word count (LIWC) feature data from the text transcript of the audio data;
- extracting, via the feature extraction component, audio feature data from the audio data of the video input;
- extracting, via the feature extraction component, video feature data from the video data of the video input;
- transmitting, via the input/output device, at least a portion of each of the extracted POS feature data, extracted emotion feature data, extracted LIWC feature data, extracted audio feature data, and extracted video feature data to the training component;
- subjecting each of the POS feature data, emotion feature data, LIWC feature data, audio feature data, and video feature data to at least one training function by the training component to generate trained feature data therefrom;
- transmitting, via the input/output device, at least a portion of the trained POS feature data, trained emotion feature data, trained LIWC feature data, trained audio feature data, and trained video feature data to the prediction component;
- and
- selectively integrating at least a portion of the trained POS feature data, trained emotion feature data, trained LIWC feature data, trained audio feature data, and trained video feature data by the prediction component to generate a personality assessment of the user.
13 more elements
A system for generating a personality assessment for a user from video input thereof, the system comprising
- a processor operable to control the computing system;
- a data storage operatively coupled to the processor, wherein the data storage is configured to store a plurality of personality data associated with a user;
- an input/output device operatively coupled to the processor, wherein the input/output device is configured to receive a plurality of data for transmission to the processor, wherein the input/output device is configured to transmit a plurality of data generated by the processor;
- a feature extraction component operatively coupled to the processor and controlled in part by the processor, wherein the feature extraction component is configured to extract a plurality of feature data from a video of the user;
- a training component operatively coupled to the processor and controlled in part by the processor, wherein the training component is configured to generate a plurality of trained feature data;
- and
- a prediction component operatively coupled to the processor and controlled in part by the processor, wherein the prediction component is configured to generate a personality assessment for the user;
- wherein the input/output device is operable to:
- receive video input associated with the user and transmit the video input to the feature extraction component, wherein the video input comprises video data and audio data,
- receive a plurality of extracted feature data from the feature extraction component and transmit the plurality of extracted feature data to the training component,
- receive a plurality of trained feature data from the training component and transmit the plurality of trained feature data to the prediction component;
- wherein the feature extraction component is operable to:
- receive the video input from the input/output device,
- generate a text transcript of at least a portion of the audio data of the received video input,
- extract a plurality of parts of speech (POS) feature data, emotion feature data, and linguistic inquiry word count (LIWC) feature data from the text transcript of the audio data,
- extract audio feature data from the audio data of the video input, and
- extract video feature data from the video data of the video input;
- wherein the training component is operable to:
- receive each of the extracted POS feature data, extracted emotion feature data, extracted LIWC feature data, extracted audio feature data, and extracted video feature data from the input/output device, and
- subject each of the POS feature data, emotion feature data, LIWC feature data, audio feature data, and video feature data to at least one training function to generate trained feature data therefrom;
- wherein the prediction component is operable to:
- receive the trained POS feature data, trained emotion feature data, trained LIWC feature data, trained audio feature data, and trained video feature data from the input/output device, and
- selectively integrate at least a portion of the trained POS feature data, trained emotion feature data, trained LIWC feature data, trained audio feature data, and trained video feature data to generate a personality assessment of the user.
20 more elements
The system of 14, wherein the trained POS feature data, trained emotion feature data, trained LIWC feature data, trained audio feature data, and trained video feature data are subjected to a classification prediction function by the prediction component to generate the personality assessment of the user.
Dependencies are inferred from the claim text.