Non-linear media segment capture and edit platform

#The inventions relate generally to capture and/or processing of audiovisual performances and, in particular, to user interface techniques suitable for capturing and manipulating media segments encoding audio and/or visual performances for non-linear capture, recapture, overdub or lip-sync, with or without collaboration.

#The installed base of mobile phones, personal media players, and portable computing devices, together with media streamers and television set-top boxes, grows in sheer number and computational power each day. Hyper-ubiquitous and deeply entrenched in the lifestyles of people around the world, many of these devices transcend cultural and economic barriers. Computationally, these computing devices offer speed and storage capabilities comparable to engineering workstation or workgroup computers from less than ten years ago, and typically include powerful media processors, rendering them suitable for real-time sound synthesis and other musical applications. Indeed, some modern devices, such as iPhone®, iPad®, iPod Touch® and other iOS® or Android devices, support audio and video processing quite capably, while at the same time providing platforms suitable for advanced user interfaces.

#Applications such as the Smule Ocarina™, Leaf Trombone®, I Am T-Pain™, AutoRap®, Smule (fka Sing! Karaoke™), Guitar! By Smule®, and Magic Piano® apps available from Smule, Inc. have shown that advanced digital acoustic techniques may be delivered using such devices in ways that provide compelling musical experiences. As researchers seek to transition their innovations to commercial applications deployable to modern handheld devices and media application platforms within the real-world constraints imposed by processor, memory and other limited computational resources thereof and/or within communications bandwidth and transmission latency constraints typical of wireless networks, significant practical challenges continue to present. Improved techniques and functional capabilities are desired, particularly relative to audiovisual content and user interfaces.

#The present invention(s) are illustrated by way of examples and not limitation with reference to the accompanying figures, in which like references generally indicate similar elements or features.

#FIG. 1 depicts information flows amongst illustrative mobile phone-type portable computing devices employed for non-linear audiovisual capture and/or edit in preparation of a group audiovisual performance in accordance with some embodiments of the present invention(s).

#FIG. 2 depicts in somewhat greater detail an exemplary user interface with visually synchronized presentation of lyrics, pitch cues and a scrubber in connection with a vocal capture session on portable computing device.

#Skilled artisans will appreciate that elements or features in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions or prominence of some of the illustrated elements or features may be exaggerated relative to other elements or features in an effort to improve understanding of embodiments of the present invention.

#Techniques have been developed to facilitate the capture, pitch correction, compositing, encoding and rendering of audiovisual performances. Vocal audio together with performance synchronized video may be captured and coordinated with audiovisual contributions of other users to form multi-performer, duet-style or glee club-style audiovisual performances. Nonlinear capture and/or edit of individual segments or portions of a performance timeline allows freeform collaboration of multiple contributors, typically with independent and geographically-distributed audio and/or video capture. In some cases, audio and video may be separately captured and associated after capture. In some cases, the performances of individual users (audio, video or, in some cases, audio together with performance synchronized video) are captured on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track or vocal performance. Captured audio, video or audiovisual content of one contributor may serve as a seed for a group performance.

#FIG. 1 depicts information flows amongst illustrative mobile phone-type portable computing devices (101A, 101B) and a content server 110 in accordance with some embodiments of the present invention(s). In the illustrated flows, lyrics 102, pitch cues 105 and a backing track 107 are supplied to one or more of the portable computing devices (101A, 101B) to facilitate vocal (and in some cases, audiovisual) capture. User interfaces of the respective devices provide a scrubber (103A, 103B), whereby a given user-vocalist is able to move forward and backward through temporally synchronized content (e.g., audio, lyrics, pitch cues, etc.) using gesture control on a touchscreen. In some cases, scrubber control also allows forward and backward movement through performance-synchronized video.