# US 10,484,735 B2 · Method and system for synchronizing media streams > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,735 B2 - Document ID: US10484735B2 - Status: Granted - Kind: B2 - Application number: 16/019,046 - Filed: 2018-06-26 - Published: 2018-11-01 - Granted: 2019-11-19 - Earliest claimed: 2018-06-26 - Assignees: Telefonaktiebolaget LM Ericsson - Applicants: Telefonaktiebolaget LM Ericsson (publ) - Inventors: Dennis Erik Bouwknegt; Henk Den Bok - Web page: https://patentgrep.com/patent/US10484735B2 ## CPC - H04N 21/43079: {of additional data with content streams on multiple devices} - H04N 21/41265: {having a remote control device for bidirectional communication between the remote control device and client device (unidirectional remote control device H04N21/42204)} - H04N 21/43072: {of multiple content streams on the same device} - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)} - H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)} - H04N 21/4722: for requesting additional data associated with the content - H04N 21/6543: for forcing some client operations, e.g. recording {(remote booting in general G06F9/4416)} - H04N 21/6581: {Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application} - H04N 21/8352: involving content or source identification data, e.g. Unique Material Identifier [UMID] - H04N 21/8586: {by using a URL (processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method and system for synchronizing two or more media streams, generated by different sources. An identifier generator generates identifiers with unique sequence numbers and a unique media identifier in relation to a second media stream generated by second media server. The generated identifiers are supplied to an encoder and an identifier storage. In the identifier storage a broadcaster stores an associated URL (first media server address & content) with a particular identifier. The identifiers and the second media stream are encoded by the encoder into a carrier which is recorded or broadcasted. Play-out of the recorded or broadcasted is accomplished by feeding the encoded carrier to a decoder, which decodes the identifiers and the second media stream. The decoded identifier is received by a first device that retrieves the stored associated URL for the particular identifier and subsequently request first media server for the first media stream indicated by the associated URL. The first media stream is provided to the first device, which is also provided with the decoded second media stream. ## Claims ### Claim 1 A method of synchronizing two different media streams, the method comprising: decoding a primary media stream to obtain, from the primary media stream, a primary media stream identifier identifying the primary media steam; decoding the primary media stream to obtain, from the primary media stream, a plurality of sequence numbers indicating respective locations in the primary media stream, the plurality of sequence numbers comprising at least a first sequence number and a second sequence number different from the first sequence number; determining, based on the primary media stream identifier: a first action type and a first action indicator, each associated with the first sequence number, wherein the first action type is a command for executing any action indicators associated with the first action type; and a second action type and a second action indicator, each associated with the second sequence number, wherein the second action type is a command different than the first action type for executing any action indicators associated with the second action type; and synchronizing, with the primary media stream, a secondary media stream identified by at least one of the first action indicator and the second action indicator by: retrieving the secondary media stream in accordance with executing the first action type and the second action type; and synchronizing the retrieved secondary media stream with the primary media stream. ### Claim 2 (depends on 1) The method of claim 1, where the plurality of sequence numbers are encoded at equidistant points in the primary media stream and some of the plurality of sequence numbers of the plurality of sequence numbers are associated with zero action indicators. ### Claim 3 (depends on 1) The method of claim 1, wherein the first action type is a cache action for retrieving and storing media content of the secondary media stream and associating the stored media content with the second sequence number. ### Claim 4 (depends on 1) The method of claim 1, wherein the first action type is a cache action for delaying retrieving and storing media content of the secondary media stream. ### Claim 5 (depends on 1) The method of claim 1, wherein the first action type is further associated with a third action indicator. ### Claim 6 (depends on 5) The method of claim 5, wherein the first action indicator is a third sequence number of the plurality of sequence numbers; wherein the third action type is a location for retrieving or storing media content of the secondary media stream; and wherein retrieving the secondary media stream comprises setting a device to detect the third sequence number for playing the media content of the secondary media stream at the device synchronized with the primary media stream. ### Claim 7 (depends on 6) The method of claim 6, wherein the location is a uniform resource locator (URL). ### Claim 8 (depends on 5) The method of claim 5, wherein the executing the first action type or the second action type comprises retrieving media content associated only with the first action indicator and not with the third action indicator based on a predetermined characteristic of the user of the device. ### Claim 9 (depends on 8) The method of claim 8, wherein the predetermined characteristic is an age of the user of a device synchronizing the retrieved secondary media stream with the primary media stream. ### Claim 10 (depends on 8) The method of claim 8, wherein the predetermined characteristic is a gender of the user of a device synchronizing the retrieved secondary media stream with the primary media stream. ### Claim 11 (depends on 8) The method of claim 8, wherein the predetermined characteristic is a language of the user of a device synchronizing the retrieved secondary media stream with the primary media stream. ### Claim 12 (depends on 1) The method of claim 1, wherein the first action type or the second action type is a setting to be downloaded and executed in a device synchronizing the primary media stream and the secondary media stream, wherein the setting is related to synchronizing the primary media stream and the secondary media stream. ### Claim 13 (depends on 1) The method of claim 1, wherein the first action type or the second action type is a control action for controlling a decoder of a device synchronizing the primary media stream and the secondary media stream. ### Claim 14 A device for synchronized provision of at least two different media streams to the device, the device comprising: a processor; and a memory, the memory containing instructions executable by the processor whereby the device is configured to: decode a primary media stream to obtain, from the primary media stream, a primary media stream identifier identifying the primary media steam; decode the primary media stream, to obtain, from the primary media stream, a plurality of sequence numbers indicating respective locations in the primary media stream, the plurality of sequence numbers comprising at least a first sequence number and a second sequence number different from the first sequence number; determine, based on the primary media stream identifier: a first action type and a first action indicator, each associated with the first sequence number, wherein the first action type is a command for executing any action indicators associated with the first action type; and a second action type and a second action indicator, each associated with the second sequence number, wherein the second action type is a command different than the first action type for executing any action indicators associated with the second action type; and synchronize, with the primary media stream, a secondary media stream identified by at least one of the first action indicator and the second action indicator by: retrieving the secondary media stream in accordance with executing the first action type and the second action type; and synchronizing the retrieved secondary media stream with the primary media stream. ### Claim 15 A computer program product stored on a non-transitory, computer readable medium and comprising program instructions, which when executed by at least one processor, causes the at least one processor to: decode a primary media stream to obtain, from the primary media stream, a primary media stream identifier identifying the primary media steam; decode the primary media stream, to obtain, from the primary media stream a plurality of sequence numbers indicating respective locations in the primary media stream, the plurality of sequence numbers comprising at least a first sequence number and a second sequence number different from the first sequence number; determine, based on the media stream identifier: a first action type and a first action indicator, each associated with the first sequence number, wherein the first action type is a command for executing any action indicators associated with the first action type; and a second action type and a second action indicator, each associated with the second sequence number, wherein the second action type is a command different than the first action type for executing any action indicators associated with the second action type; and synchronize, with the primary media stream, a secondary media stream identified by at least one of the first action indicator and the second action indicator by: retrieving the secondary media stream in accordance with executing the first action type and the second action type; and synchronizing the retrieved secondary media stream with the primary media stream. ### Claim 16 (depends on 1) The method of claim 1, wherein the respective locations in the primary media stream indicate a time point in the playing of the primary media stream, wherein the first sequence number and the second sequence number are generated to indicate a separation of time points in the primary media stream of 0.5-1 seconds apart, and the synchronizing the retrieved secondary media stream with the primary media stream comprises playing the secondary media stream at a time point within 0.5-1 seconds of an intended time point in the primary media stream.