Video content generation system, video content generation device, and storage media

A video content generation device comprising:

• a capture unit which captures a motion graph including nodes, edges, and weights, which are calculated based on motion characteristics extracted from motion data, as well as metadata indicating a synchronization probability for each node between the motion graph and a musical tune used to generate video content;
• a music data storage unit which retrieves a predetermined number of beats, each having a plurality of nodes, from the musical tune so as to store a predetermined amount of music data with their musical features;
• an optimum path search unit that searches an optimum path connecting between nodes, each of which is selected for each beat with a high synchronization probability, on the motion graph with motion characteristics matching musical characteristics based on the predetermined amount of music data by way of a dynamic programming update method;
• a synchronization unit which generates synchronization information used to correlate music data to motion data along with the optimum path;
• and
• a video data generation unit which generates video data synchronized with music data based on the synchronization information, thus simultaneously reproducing video data and music data
• wherein when the music data storage unit newly stores the predetermined number of beats of music data, the optimum path search unit searches a new optimum path in response to a ratio between a first cost, calculated between a last node of a previous optimum path and a newly added node representing one of the plurality of nodes for each node included in the newly stored music data, and a second cost calculated between a candidate node associated to the last node of the previous optimum path and the newly added node, and wherein the new optimum path is made by adding concatenation between the last node and the newly added node to the foregoing optimum path.

A non-transitory computer-readable storage medium, storing one or more programs for implementation by a computer, a video content generation method comprising:

• capturing a motion graph including nodes, edges, and weights, which are calculated based on motion characteristics extracted from motion data, as well as metadata indicating a synchronization probability for each node between the motion graph and a musical tune used to generate video content;
• retrieving a predetermined number of beats, each having a plurality of nodes, from the musical tune so as to store a predetermined amount of music data with their musical features;
• searching an optimum path connecting between nodes, each of which is selected for each beat with a high synchronization probability, on the motion graph with motion characteristics matching musical characteristics based on the predetermined amount of music data by way of a dynamic programming update method;
• generating synchronization information used to correlate music data to motion data along with the optimum path;
• and
• generating video data synchronized with music data based on the synchronization information, thus simultaneously reproducing video data and music data,
• wherein upon newly storing the predetermined number of beats of music data, a new optimum path is searched in response to a ratio between a first cost, calculated between a last node of a previous optimum path and a newly added node representing one of the plurality of nodes included in the newly stored music data, and a second cost calculated between a candidate node associated to of the last node of the previous optimum path and the newly added node, and
• wherein the new optimum path is made by adding concatenation between the last node and the newly added node to the foregoing optimum path.