Robotic System For Synchronously Reproducing Facial Expression And Speech And Related Method Thereof
OwnerChyi-Yeu Lininventor
Life of this patent
Estimating the term…
Abstract
A robotic system and a related method for reproducing a real person's facial expression and speech simultaneously and synchronously is provided herein. The robotic system comprises at least a robotic head which in turn comprises a speaker, a plurality of face actuators, and a computing engine. The robotic head drives the speaker and the face actuators synchronously based on a speech segment and a sequence of time-stamped control vectors so that the robotic system could mimic a real person's facial expression and speech. The speech segment and the sequence of time-stamped control vectors are retrieved from a storage device of the robotic system, or from an external source via an appropriate communication mechanism.
Claims22
A robotic system for reproducing a person's facial expression and speech comprising:
- a robotic head comprising a speaker, a plurality of face actuators, and a computing engine connected to said speaker and said face actuators, said computing engine driving said face actuators based on a sequence of sets of control parameters while delivering speech from said speaker based on a speech segment;
- and
- at least one of a camera and a microphone for capturing a person's performance over a period of time to produce at least one of a recorded video and a recorded speech, wherein said sequence of sets of control parameters is generated from said recorded video, and said speech segment is generated from said recorded speech;
- wherein each of said sets of control parameters includes a time stamp and a control parameter corresponding to each of said face actuators;
- and said computing engine drives said face actuators at appropriate time specified by the time stamps of said sets of control parameters so that said face actuators are activated according to the corresponding control parameters to reproduce said person's facial expression.
A method for reproducing a person's facial expression and speech on a robotic head, said robotic head comprising a speaker and a plurality of face actuators, said method comprising the steps of:
- (1) operating at least a camera and a microphone to capture said person's performance over a period of time into a recorded speech and a recorded video having synchronized timing information for said person's facial expression and speech;
- (2) processing said recorded speech and said recorded video to produce a speech segment and a sequence of sets of control parameters, each of said sets of control parameters including a time stamp derived from said timing information and a control parameter corresponding to each of said face actuators, each of said control parameters controlling one of said face actuators respectively;
- and
- (3) delivering said person's speech based on said speech segment via said speaker while driving said face actuators at appropriate time specified by the time stamps of said sets of control parameters so that said face actuators are activated according to the corresponding control parameters to reproduce said person's facial expression.
Dependencies are inferred from the claim text.