US 5,630,017 A · Advanced tools for speech synchronized animation

Abstract

A random access animation user interface environment referred to as interFACE enabling a user to create and control animated lip-synchronized images or objects utilizing a personal computer for use in the users programs and products. A real-time random-access interface driver (RAVE) together with a descriptive authoring language (RAVEL) is used to provide synthesized actors ("synactors"). The synactors may represent real or imaginary persons or animated characters, objects or scenes. The synactors may be created and programmed to perform actions including speech which are not sequentially pre-stored records of previously enacted events. Furthermore, animation and sound synchronization may be produced automatically and in real-time. Sounds and visual images of a real or imaginary person or animated character associated with those sounds are input to a system and may be decomposed into constituent parts to produce fragmentary images and sounds. A set of characteristics is utilized to define a digital model of the motions and sounds of a particular synactor. The general purpose system is provided for random access and display of synactor images on a frame-by-frame basis, which is organized and synchronized with sound. Both synthetic speech and digitized recording may provide the speech for synactors.

Claims16

Dependent claims

Apparatus for generating and displaying user created animated objects having synchronized visual and audio characteristics, said apparatus comprising:

a program-controlled microprocessor;
first means coupled to said microprocessor and responsive to user input text for segmenting said user input text in accordance with predefined algorithms to generate a set of vector signals, each said vector signal associated with a sound segment of said user input text;
second means coupled to said microprocessor and to said first means and responsive to said user input text and said set of vector signals for generating a sequence of phonetic codes, each of said phonetic codes associated with a corresponding vector signal and its associated sound segment, each of said phonetic codes identifying a predefined visual image associated with said phonetic code;
display means coupled to said first and second means responsive to said set of vector signals for displaying a dendrogram defined by said set of vector signals and representative of an acoustical relationship among said sound segments of said user input text, each of said phonetic codes displayed on said dendrogram disposed in relative relationship with said associated sound segment;
and
controller means coupled to said first and second means, to said display means and to said microprocessor and having editing means for generating a set of instructions synchronizing said sequence of visual images with said associated sound segments corresponding to said user input text and for editing said set of instructions hereby defining an animated object having synchronized visual and audio characteristics.

3 more elements

An apparatus for generating and displaying a user created animated object having synchronized visual and audio characteristics, the apparatus comprising:

a programmed computer including storage means for storing signals representing sound, a display device, an input means, a real-time random access speech synchronization animation means for controlling the animated object, editing means and speech segmentation generation means for displaying speech synchronization;
the input means for providing signals to the speech segmentation generation means, the signals representing text;
the speech segmentation generation means including first generation means, second generation means and third generation means;
the first generation means for generating acoustic information representing the text, the acoustic information having component parts;
the second generation means for generating visual information associated with the acoustic information, the visual information defining facial expressions associated with the acoustic information;
the third generation means for generating timing information, the timing information capable of being manipulated for synchronization between the visual information and the acoustic information;
and
the display device for providing an aligned display of the component parts of the acoustic information with the visual information and the timing information, the aligned display representative of a synchronization between the component parts and the visual and timing information associated therewith;
the editing means for editing the aligned display to adjust the timing values to alter the synchronization;
and
the programmed computer for synchronized integration of the acoustic, visual and timing information with the visual and audio characteristics to provide the animated object, the audio characteristics being retrieved from the storage means.

8 more elements

In a system having a programmed computer including a display device, an input device and a real-time random access speech synchronization animation means for controlling the animated object, a method for generating and displaying a user created animated object having synchronized visual and audio characteristics, the method comprising the steps of:

providing speech segmentation means for displaying speech synchronization;
providing signals representing text with the input device to the speech segmentation generation means;
generating acoustic information representing the text, the acoustic information having component parts;
generating visual information associated with the acoustic information, the visual information defining facial expressions associated with the acoustic information;
generating timing information capable of being manipulated for synchronization between the visual information and the acoustic information;
providing on the display device an aligned display of the component parts of the acoustic information with the visual information and the timing information, the aligned display representative of a synchronization between the component parts and the visual and timing information associated therewith;
editing the aligned display to adjust the timing values to manipulate the synchronization;
and
synchronized integration of the acoustic, visual and timing information with the visual and audio characteristics to provide the animated object.

6 more elements

Dependencies are inferred from the claim text.

Advanced tools for speech synchronized animation

Life of this patent

Advanced tools for speech synchronized animation

Life of this patent

Abstract

Claims16