# US 7,613,727 B2 · Method and apparatus for supporting advanced coding formats in media files > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,613,727 B2 - Document ID: US7613727B2 - Status: Granted - Kind: B2 - Application number: 10/371,434 - Filed: 2003-02-21 - Published: 2003-08-28 - Granted: 2009-11-03 - Earliest claimed: 2003-02-21 - Assignees: Sont Corporation; Sont Electronics Inc. - Inventors: Mohammed Zubair Visharam; Ali Tabatabai; Toby Walker - Web page: https://patentgrep.com/patent/US7613727B2 ## CPC - H04N 21/8451: {using Advanced Video Coding [AVC]} - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 21/23439: {for generating different versions} - H04N 21/4621: {Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen (arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10)} - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/85406: {involving a specific file format, e.g. MP4 format} - Y10S 707/99933: Query processing, i.e. searching - Y10S 707/99942: Manipulating data structure, e.g. compression, compaction, compilation - Y10S 707/99943: Generating database or data structure, e.g. via user interface - Y10S 707/99945: Object-oriented database structure processing - Y10S 707/99948: Application of database or data structure, e.g. distributed, multimedia, or image - Y10S 707/99953: Recoverability - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract Switch sample metadata defining switch sample sets associated with multimedia data is created. Each switch sample set includes samples that have identical decoding values. Further, a file associated with the multimedia data is formed. This file includes the switch sample metadata, as well as other information pertaining to the multimedia data. ## Claims ### Claim 1 A computerized method for creating metadata for multimedia data, the method comprising: receiving a file with encoded multimedia data; examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data; identifying samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; creating, by a computer including a processor, switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences; and forming a file associated with the encoded multimedia data, the formed file comprising the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 2 (depends on 1) The method of claim 1 wherein the samples in each switch sample set of the plurality of switch sample sets use different reference samples. ### Claim 3 (depends on 1) The method of claim 1, wherein the predefined data structure is a table box containing a set of nested tables. ### Claim 4 (depends on 1) The method of claim 1 further comprising: sending a file associated with the encoded multimedia data to a decoding system; receiving the file associated with the encoded multimedia data at the decoding system; and extracting, at the decoding system, the switch sample metadata from the file associated with the encoded multimedia data, the extracted switch sample metadata being subsequently used to find a replacement for a specific sample. ### Claim 5 A computerized method for creating metadata for multimedia data, the method comprising: receiving a file with encoded multimedia data; examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data; identifying samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; creating sub-sample metadata defining a plurality of sub-samples within each sample of the encoded multimedia data; creating parameter set metadata identifying one or more parameter sets for a plurality of portions of the encoded multimedia data; creating, by a computer including a processor, switch sample metadata defining a plurality of switch sample sets, each sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences; and forming a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata, the parameter set metadata, and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 6 (depends on 5) The method of claim 5 wherein the creating sub-sample metadata comprises: organizing the sub-sample metadata into a set of predefined data structures comprising a first data structure containing information about sub-sample sizes, a second data structure containing information about a number of sub-samples in each sample, and a third data structure containing information describing each sub-sample. ### Claim 7 (depends on 5) The method of claim 5 wherein each portion of the plurality of portions of multimedia data is any one of a sample and a sub-sample within the multimedia data. ### Claim 8 (depends on 5) The method of claim 5 wherein the creating parameter set metadata comprises: organizing the parameter set metadata into a set of predefined data structures comprising a first data structure containing descriptive information about the one or more parameter sets and a second data structure containing information that defines associations between the one or more parameter sets and the plurality of portions of multimedia data. ### Claim 9 (depends on 5) The method of claim 5 wherein the samples in each switch sample set of the plurality of switch sample sets use different reference samples. ### Claim 10 (depends on 5) The method of claim 5 wherein the predefined data structure is a table box containing a set of nested tables. ### Claim 11 (depends on 5) The method of claim 5 further comprising: sending a file associated with the encoded multimedia data to a decoding system; receiving the file associated with the encoded multimedia data at the decoding system; and extracting, at the decoding system, the sub-sample metadata, the parameter set metadata, and the switch sample metadata from the file associated with the encoded multimedia data. ### Claim 12 A computerized method for creating metadata for multimedia data, the method comprising: receiving a file with encoded multimedia data; examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data; identifying samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; creating sub-sample metadata defining a plurality of sub-samples within each sample of multimedia data; creating, by a computer including a processor, switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences; and forming a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs a decoding system on finding a replacement for a specific sample. ### Claim 13 (depends on 12) The method of claim 12 wherein the creating sub-sample metadata comprises: organizing the sub-sample metadata into a set of predefined data structures comprising a first data structure containing information about sub-sample sizes, a second data structure containing information about a number of sub-samples in each sample, and a third data structure containing information describing each sub-sample. ### Claim 14 (depends on 12) The method of claim 12 wherein the samples in each switch sample set of the plurality of switch sample sets uses different reference samples. ### Claim 15 (depends on 12) The method of claim 12 wherein the creating switch sample metadata comprises: organizing the switch sample metadata into a predefined data structure represented as a table box containing a set of nested tables. ### Claim 16 (depends on 12) The method of claim 12 further comprising: sending a file associated with the encoded multimedia data to a decoding system; receiving the file associated with the encoded multimedia data at the decoding system; and extracting, at the decoding system, the sub-sample metadata and the switch sample metadata from the file associated with the encoded multimedia data. ### Claim 17 An apparatus for creating metadata for multimedia data, the apparatus comprising: a memory to store the multimedia data; a processor coupled to the memory; a file creator to receive a file with encoded multimedia data, examine the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and a metadata generator to create switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and the file creator further to form a file associated with the encoded multimedia data, the formed file comprising the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 18 (depends on 17) The apparatus of claim 17 wherein the samples in each switch sample set of the plurality of switch sample sets use different reference samples. ### Claim 19 (depends on 17) The apparatus of claim 17 further comprising: a metadata extractor to receive the file associated with the encoded multimedia data at a decoding system and to extract the switch sample metadata from the file associated with the encoded multimedia data; and a media data stream processor to use the extracted switch sample metadata finding a replacement for a specific sample. ### Claim 20 An apparatus for creating metadata for multimedia data, the apparatus comprising: a memory to store the multimedia data; a processor coupled to the memory; a file creator to receive a file with encoded multimedia data, examine the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and a metadata generator to create sub-sample metadata defining a plurality of sub-samples within each sample of the multimedia data, to create parameter set metadata identifying one or more parameter sets for a plurality of portions of the encoded multimedia data, and to create switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and the file creator further to form a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata, the parameter set metadata, and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 21 An apparatus for creating metadata for multimedia data, the apparatus comprising: a memory to store the multimedia data; a processor coupled to the memory; a file creator to receive a file with encoded multimedia data, examine the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and a metadata generator to create sub-sample metadata defining a plurality of sub-samples within each sample of multimedia data, and to create switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and the file creator further to form a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 22 An apparatus for creating metadata for multimedia data, the apparatus comprising: means for receiving a file with encoded multimedia data, examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and a memory to store instructions; a processor to execute the instructions for performing steps; creating switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and forming a file associated with the encoded multimedia data, the formed file comprising the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 23 An apparatus for creating metadata for multimedia data, the apparatus comprising: means for creating sub-sample metadata defining a plurality of sub-samples within each sample of the multimedia data; means for creating parameter set metadata identifying one or more parameter sets for a plurality of portions of the multimedia data; a memory to store instructions; a processor to execute the instructions for performing steps; forming a file associated with the multimedia data, the forming is to receive a file with encoded multimedia data, examine the multimedia data to identify switch sample sets associated with the multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and creating switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure, and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, wherein the formed file comprises the sub-sample metadata, the parameter set metadata, and the switch sample metadata, and wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample. ### Claim 24 An apparatus for creating metadata for multimedia data, the apparatus comprising: means for creating sub-sample metadata defining a plurality of sub-samples within each sample of the multimedia data; a memory to store instructions; a processor to execute the instructions for performing steps: forming a file associated with multimedia data, wherein the forming is to receive a file with encoded multimedia data, examine the multimedia data to identify switch sample sets associated with the multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and creating switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, wherein the formed file comprises the sub-sample metadata and the switch sample metadata, and wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.