Character animations in a virtual environment based on reconstructed three-dimensional motion data

A method comprising:

• receiving a search query in a virtual reality (VR) environment;
• retrieving a set of editable keyframe-based animation data relevant to the search query from a database comprising editable keyframe-based animation data, associated text descriptions, and two-dimensional videos correspondingly associated with movements related to the editable keyframe-based animation data, wherein the editable keyframe-based animation data has been generated from three-dimensional motion data constructed from the two-dimensional videos and represents the movements in the two-dimensional videos, and wherein the editable keyframe-based animation data and the associated text descriptions have been generated from the two-dimensional videos;
• presenting a user interface in the VR environment for use in prototyping VR character animations using the set of the editable keyframe-based animation data of the database, wherein the user interface provides interfaces for searching and displaying two-dimensional videos from the database to select a motion from a corresponding two-dimensional video to be applied to a character during VR character animation, and wherein the user interface allows editing of the selected motion when applied to the character in the VR environment;
• receiving input to select the two-dimensional video from the two-dimensional videos associated with the set of editable keyframe-based animation data to select the motion to be applied to the character;
• in response to the received input, displaying at least a portion of the two-dimensional video to preview the selected motion;
• and
• applying the selected motion to animate the character by extracting corresponding editable keyframe-based animation data that is applied to the character, thereby animating the character in a three-dimensional space by synchronizing the applied motion to the character with at least the portion of the displayed two-dimensional video.

A system comprising:

• a computing device;
• and
• a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to:
• construct three-dimensional motion data from two-dimensional videos, the three-dimensional motion data representing movement of people in the two-dimensional videos and including, for each person, a root of a three-dimensional skeleton of the person, wherein the three-dimensional skeleton comprises multiple three-dimensional poses of the person during at least a portion of frames of a video from the two-dimensional videos;
• convert the three-dimensional motion data into editable keyframe-based animation data in three-dimensional space;
• provide the editable keyframe-based animation data for applying to a character to animate motion of the character in three-dimensional space;
• and
• associate text descriptions, which have been generated from the two-dimensional videos, with the movements from the editable keyframe-based animation data, wherein the instructions cause the computing device to associate the text descriptions with the movements by causing the computing device to
• represent the text descriptions as feature vectors,
• store respective feature vectors with respective movements detected in the two-dimensional videos to pair a text description with a movement, and
• generate an index object that encapsulates the feature vectors for use for comparison with query vectors provided during searching for a movement to animate the motion of the character.