# US 9,883,204 B2 · Systems and methods for encoding source media in matroska container files for adaptive bitrate streaming using hypertext transfer protocol > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,204 B2 - Document ID: US9883204B2 - Status: Granted - Kind: B2 - Application number: 15/005,990 - Filed: 2016-01-25 - Published: 2016-07-28 - Granted: 2018-01-30 - Earliest claimed: 2011-01-05 - Assignees: Sonic IP, Inc. - Applicants: Sonic IP, Inc. - Inventors: Jason Braness; Auke Sjoerd van der Schaar; Kourosh Soroushian - Web page: https://patentgrep.com/patent/US9883204B2 ## CPC - H04N 19/593: involving spatial prediction techniques - G11B 27/005: {Reproducing at a different information rate from the information rate of recording (for television signals H04N5/783)} - G11B 27/11: by using information not detectable on the record carrier - G11B 27/322: {used signal is digitally coded} - H04L 65/612: for unicast - H04L 65/613: for the control of the source by the destination (control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637) - H04L 65/70: Media network packetisation - H04N 19/172: the region being a picture, frame or field - H04N 19/177: the unit being a group of pictures [GOP] - H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/23439: {for generating different versions} - H04N 21/2387: Stream processing in response to a playback request from an end-user, e.g. for trick-play - H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)} - H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities - H04N 21/42607: {for processing the incoming bitstream} - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/44004: {involving video buffer management, e.g. video decoder buffer or video display buffer} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)} - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N 21/8543: using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] {(information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80)} - H04N 21/85406: {involving a specific file format, e.g. MP4 format} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Systems and methods for encoding source media in Matroska container files for adaptive bitrate streaming utilizing Hypertext Transfer Potocol (HTTP) in accordance with embodiments of the invention are disclosed. One embodiment of the invention includes a processor configured via a source encoding application to ingest at least one multimedia file containing a source video. In addition, the source encoding application further configures the processor to select a portion of the source video, transcode the selected portion of the source video into a plurality of alternative portions of encoded video, where each alternative portion is encoded using a different set of encoding parameters and commences with an intra frame starting a closed Group of Pictures (GOP), write each of the alternative portions of encoded video to an element of a different EBML container file, where each element is located within an EBML container file that also includes another element that indicates the encoding parameters used to encode the alternative portion of encoded video, and add an entry to at least one index that identifies the location of the element containing one of the alternative portions of encoded video within each of the EBML container files. ## Claims ### Claim 1 A playback device configured to decode video from a plurality of alternative video streams with different maximum bitrates and packed in container files, wherein the container files are extensible binary markup language (EBML) files, the playback device comprising: a processor configured via a playback application to decode at least one multimedia file containing an encoded source video; wherein the playback application further configures the processor to: parse a top level index to obtain information that identifies a plurality of EBML container files, wherein each EBML container file of the plurality of EBML container files comprises a set of alternative portions of a source video, wherein the set of alternative portions for each EBML container file is encoded using a different maximum bitrate that is associated with a particular alternative video stream using a different set of encoding parameters, wherein each alternative portion of a set of alternative portions is stored in a Cluster element of the EBML container and commences with an intra frame starting a closed Group of Pictures (GOP), wherein corresponding alternative portions in the sets of alternative portions of different EBML container files share a same time code, wherein the top level index describes at least the maximum bitrate of the particular alternative video stream associated with each EBML container file and is separate from the EBML container files; request a portion of at least one of the plurality of EBML container files based on the maximum bitrate of the particular alternative stream associated with the at least one EBML container file, wherein the requested portion comprises an element that specifies the set of encoding parameters used to encode the set of alternative portions of the source video contained within the EBML container file; retrieve an index from a Cues element within the at least one EBML container file, wherein a first index entry of the index references an initial byte for a first Cluster element comprising a first alternative portion and a second index entry of the index references an initial byte for a next Cluster element comprising a second alternative portion; infer a size of the given Cluster element from at least the first and second index entries of the Cues element within the at least one EBML container file; request at least a portion of the first alternative portion from the Cluster element of the at least one EBML container file by retrieving a byte range of the at least one EBML container file based on the inferred size and the initial byte of the first Cluster; receive and buffer the requested portion; and decode the encoded video contained within the buffered Cluster elements utilizing the set of encoding parameters. ### Claim 2 (depends on 1) The playback device of claim 1, wherein the alternative portions in each of the Cluster elements have a same duration. ### Claim 3 (depends on 1) The playback device of claim 1, wherein the alternative portions in each of the Cluster elements have a two-second duration. ### Claim 4 (depends on 1) The playback device of claim 1, wherein each Cluster element contains a time code and each encoded frame of the alternative portion contained within the Cluster element is contained within a separate BlockGroup element. ### Claim 5 (depends on 4) The playback device of claim 4, wherein a first BlockGroup element in the Cluster element contains the intra frame. ### Claim 6 (depends on 5) The playback device of claim 5, wherein the first BlockGroup element contains a Block element which specifies a time code attribute of the intra frame relative to the time code of the Cluster element. ### Claim 7 (depends on 1) The playback device of claim 1, wherein at least two of the alternative streams of encoded video are encoded at a same display aspect ratio, but with different resolutions using different sample aspect ratios. ### Claim 8 (depends on 7) The playback device of claim 7, wherein the set of encoding parameters used to encode the set of alternative portions of the source video contained within the EBML container file specifies the frame height, frame width, and sample aspect ratio of the video stream contained within the EBML container file. ### Claim 9 (depends on 8) The playback device of claim 8, wherein the playback application further configures the processor to set the frame height, frame width, and sample aspect ratio prior to commencing the decoding of encoded video contained within buffered elements from one of the EBML container files. ### Claim 10 (depends on 1) The playback device of claim 1, wherein at least two of the alternative streams of encoded video are encoded at different frame rates. ### Claim 11 (depends on 10) The playback device of claim 10, where the set of encoding parameters used to encode the set of alternative portions of the source video contained within the EBML container file specifies the frame rate of the video stream contained within the EBML container file. ### Claim 12 (depends on 11) The playback device of claim 11, wherein the playback application further configures the processor to set the frame rate prior to commencing the decoding of encoded video contained within buffered Cluster elements from one of the EBML container files. ### Claim 13 (depends on 1) The playback device of claim 1, wherein the client application further configures the processor to retrieve the index that references an initial byte for each Cluster element within an EBML container file by requesting the portion of the EBML container file that includes the Cues element containing the index. ### Claim 14 (depends on 1) The playback device of claim 1, wherein the top level index comprises SMIL data and the information that identifies the plurality of EBML container files is a plurality of Universal Resource Indicators (URIs) that identify the location of each of the EBML container files. ### Claim 15 (depends on 1) The playback device of claim 1, wherein the Cues element comprises a plurality of Cue Point elements that each includes a time attribute and points to the location of each Cluster element containing an alternative portion within the EBML file. ### Claim 16 (depends on 15) The playback device of claim 15, wherein the playback application further configures the processor to: search for the location of a Cluster element containing a specific portion of encoded video within the EBML file using the time attributes of the CuePoint elements within the index; and request a portion of the EBML file including at least the Cluster element identified using the CuePoint elements. ### Claim 17 (depends on 1) The playback device of claim 1, wherein the set of encoding parameters utilized to decode the encoded video include at least one encoding parameter selected from the group consisting of frame rate, frame height, frame width, sample aspect ratio, maximum bitrate, and minimum buffer size. ### Claim 18 (depends on 1) The playback device of claim 1, wherein the playback application further configures the processor to: measure current streaming conditions by measuring the time taken to receive requested Cluster elements from the time at which the Cluster elements were requested; and retrieve content from a second EBML container file for a second alternative video stream encoded at a different, second maximum bitrate based on the measured current streaming conditions. ### Claim 19 (depends on 1) The playback device of claim 1, wherein: the playback application further configures the processor to initially request a portion of a first EBML container file containing the element that specifies the set of encoding parameters of the alternative video stream contained within the first EBML container file; and the playback application further configures the processor to request a portion of a second EBML container file containing the element that specifies the set of encoding parameters of the alternative video stream contained within the second EBML container file, wherein the playback application selects to retrieve Cluster elements containing portions of encoded video for decoding from the second EBML container file based upon measured streaming conditions and the description of the bitrate of the alternative stream contained within the top level index. ### Claim 20 (depends on 1) The playback device of claim 1, wherein the playback application further configures the processor to request portions of EBML container files from remote servers via Hypertext Transfer Protocol (HTTP) byte range requests.