# US 10,771,521 B2 · Methods and systems for overlaying and playback of audio data received from distinct sources > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,521 B2 - Document ID: US10771521B2 - Status: Granted - Kind: B2 - Application number: 14/979,225 - Filed: 2015-12-22 - Published: 2017-06-22 - Granted: 2020-09-08 - Earliest claimed: 2015-12-22 - Assignees: Spotify AB - Applicants: Spotify AB - Inventors: Pontus Persson - Web page: https://patentgrep.com/patent/US10771521B2 ## CPC - H04L 65/612: for unicast - G10H 1/366: {with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice} - G10L 21/013: Adapting to target pitch - G10L 21/055: for synchronising with other signals, e.g. video signals - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - G11B 27/11: by using information not detectable on the record carrier - H04N 21/439: Processing of audio elementary streams - H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)} - H04N 21/8113: {comprising music, e.g. song in MP3 format} - H04N 21/8547: involving timestamps for synchronizing content - H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication - G10H 2240/211: Wireless transmission, e.g. of music parameters or control data by radio, infrared or ultrasound - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Methods, systems, and/or devices for overlaying and playing back audio data for user vocals and media content received from distinct devices and systems at a media presentation system are described herein. In one aspect, a media presentation system receives, from a remote server, a first data stream for a first media item, and plays the first media item. While playing the first media item, a second data stream for audio data that includes vocals is received from an application at a client device distinct from the remote server. The vocals are overlaid with the first media item to generate a composite data stream, which includes combining the first and second data streams, and the media presentation system plays the overlaid vocals. ## Claims ### Claim 1 A method, comprising: at a media presentation system having a network connection, one or more processors, and memory storing instructions for execution by the one or more processors, wherein the media presentation system is distinct from a remote server and distinct from a client device: receiving, over a first communications channel, from the remote server via the network connection, a first data stream for a first media item; playing the first media item; and while playing the first media item: receiving, over a second communications channel distinct from the first communications channel, from an application at the client device distinct from the remote server and distinct from the media presentation system, a second data stream for audio data that includes vocals provided by a user as the first media item plays; measuring a latency of the second communications channel; overlaying, with the first media item, the vocals provided by the user as the first media item plays to generate a composite data stream, the overlaying comprising: offsetting the first data stream from the second data stream in accordance with the measured latency of the second communications channel; and combining the first and second data streams in accordance with the offset of the data streams; and playing the composite data stream, including the overlaid vocals provided by the user and the first media item. ### Claim 2 (depends on 1) The method of claim 1, wherein the first media item includes a song, and the first data stream for the first media item includes audio data for the song. ### Claim 3 (depends on 2) The method of claim 2, wherein the first data stream further comprises textual data for lyrics of the song. ### Claim 4 (depends on 1) The method of claim 1, wherein the application is associated with the remote server. ### Claim 5 (depends on 1) The method of claim 1, wherein the first data stream is received in accordance with a media item request by the application at the client device. ### Claim 6 (depends on 1) The method of claim 1, wherein the client device is a mobile phone that includes a microphone. ### Claim 7 (depends on 1) The method of claim 1, wherein receiving the second data stream; overlaying, with the first media item, the vocals; and playing the overlaid vocals overlap with receiving the first data stream. ### Claim 8 (depends on 7) The method of claim 7, wherein the first data stream corresponds to a portion of data for the first media item. ### Claim 9 (depends on 1) The method of claim 1, wherein playing the first media item is performed after receiving the entire first media item. ### Claim 10 (depends on 1) The method of claim 1, wherein the second communications channel is based on a close-range wireless communications protocol. ### Claim 11 (depends on 1) The method of claim 1, wherein the second communications channel is a wired connection. ### Claim 12 (depends on 1) The method of claim 1, wherein overlaying, with the first media item, the vocals comprises: auto-tuning the vocals using data received from the remote server indicating pitch for the first media item; and overlaying, with the first media item, the auto-tuned vocals to generate the composite data stream. ### Claim 13 (depends on 1) The method of claim 1, wherein the client device is a first client device and the composite data stream further includes vocals provided by a second user at a second client device distinct from the first client device. ### Claim 14 (depends on 1) The method of claim 1, wherein the latency of the second communications channel is a latency measured between the client device and the media presentation system. ### Claim 15 A media presentation system distinct from a remote server and distinct from a client device, comprising: one or more processors; and memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for: receiving, over a first communications channel, from the remote server via the network connection, a first data stream for a first media item; playing the first media item; and while playing the first media item: receiving, over a second communications channel distinct from the first communications channel, from an application at the client device distinct from the remote server and distinct from the media presentation system, a second data stream for audio data that includes vocals provided by a user as the first media item plays; measuring a latency of the second communications channel; overlaying, with the first media item, the vocals provided by the user as the first media item plays to generate a composite data stream, the overlaying comprising: offsetting the first data stream from the second data stream in accordance with the measured latency of the second communications channel; and combining the first and second data streams in accordance with the offset of the data streams; and playing the composite data stream, including the overlaid vocals provided by the user and the first media item. ### Claim 16 A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of a media presentation system having a network connection, wherein the media presentation system is distinct from a remote server and distinct from a client device, the one or more programs including instructions for: receiving, over a first communications channel, from the remote server via the network connection, a first data stream for a first media item; playing the first media item; and while playing the first media item: receiving, over a second communications channel distinct from the first communications channel, from an application at the client device distinct from the remote server and distinct from the media presentation system, a second data stream for audio data that includes vocals provided by a user as the first media item plays; measuring a latency of the second communications channel; overlaying, with the first media item, the vocals provided by the user as the first media item plays to generate a composite data stream, the overlaying comprising: offsetting the first data stream from the second data stream in accordance with the measured latency of the second communications channel; and combining the first and second data streams in accordance with the offset of the data streams; and playing the composite data stream, including the overlaid vocals provided by the user and the first media item.