# US 2020/0286457 A1 · Automated generation of coordinated audiovisual work based on content captured geographically distributed performers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0286457 A1
- Document ID: US20200286457A1
- Status: Published application
- Kind: A1
- Application number: 16/574,971
- Filed: 2019-09-18
- Published: 2020-09-10
- Earliest claimed: 2015-06-03
- Assignees: Smule, Inc.
- Applicants: Smule, Inc.
- Inventors: Kevin Sung; Bona Kim; Jon Moldover; John Shimmin; Jeannie Yang; Perry R. Cook
- Web page: https://patentgrep.com/patent/US20200286457A1

## CPC

- G10H 1/368: {displaying animated or moving pictures synchronized with the music or audio part}
- G10H 1/366: {with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice}
- G11B 27/02: Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- G10H 2210/331: Note pitch correction, i.e. modifying a note pitch or replacing it by the closest one in a given scale
- G10H 2220/005: Non-interactive screen display of musical or status data
- G10H 2220/011: Lyrics displays, e.g. for karaoke applications
- G10H 2220/355: Geolocation input, i.e. control of musical parameters based on location or geographic position, e.g. provided by GPS, Wi-Fi® network location databases or mobile phone base station position databases
- G10H 2230/015: PDA [personal digital assistant] or palmtop computing devices used for musical purposes, e.g. portable music players, tablet computers, e-readers or smart phones in which mobile telephony functions need not be used
- G10H 2240/175: for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
- G10H 2240/251: Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analogue or digital, e.g. DECT, GSM, UMTS
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

Vocal audio of a user together with performance synchronized video is captured and coordinated with audiovisual contributions of other users to form composite duet-style or glee club-style or window-paned music video-style audiovisual performances. In some cases, the vocal performances of individual users are captured (together with performance synchronized video) on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track. Contributions of multiple vocalists are coordinated and mixed in a manner that selects for presentation, at any given time along a given performance timeline, performance synchronized video of one or more of the contributors. Selections are in accord with a visual progression that codes a sequence of visual layouts in correspondence with other coded aspects of a performance score such as pitch tracks, backing audio, lyrics, sections and/or vocal parts.

## Claims

### Claim 1

A method of preparing a coordinated audiovisual work from geographically distributed performer contributions, the method comprising:
receiving via a communication network, plural audiovisual encodings of performances captured at respective remote devices in temporal correspondence with respective audible renderings of a seed, the received audiovisual encodings each including respective performer vocals and temporally synchronized video;
retrieving a visual progression that encodes, in temporal correspondence with the seed, a succession of templated screen layouts each specifying a number and arrangement of visual cells in which respective of the videos are visually renderable;
associating individual ones of the captured performances, including the respective performer vocals and coordinated videos, to respective ones of the visual cells; and
rendering the coordinated audiovisual work, in accordance with the visual progression and the associations, as an audio mix and coordinated visual presentation of the captured performances.
