# US 12,695,940 B2 · File format concepts for video coding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,695,940 B2 - Document ID: US12695940B2 - Status: Granted - Kind: B2 - Application number: 18/819,584 - Filed: 2024-08-29 - Published: 2025-02-20 - Granted: 2026-07-28 - Earliest claimed: 2020-04-03 - Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Inventors: Yago Sánchez De La Fuente; Dimitri Podborski; Karsten Grueneberg; Cornelius Hellge; Thomas Schierl; Robert Skupin; Thomas Wiegand - Web page: https://patentgrep.com/patent/US12695940B2 ## CPC - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/234327: {by decomposing into layers, e.g. base layer and one or more enhancement layers} - H04N 21/23439: {for generating different versions} - H04N 21/4402: involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} - H04N 21/440227: {by decomposing into layers, e.g. base layer and one or more enhancement layers} - H04N 21/44029: {for generating different versions} - H04N 21/85406: {involving a specific file format, e.g. MP4 format} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract File format concepts for video coding are described. Embodiments allow for an efficient extraction of sub-streams from a video file. Further embodiments allow for a flexible switching between sub-streams of a video file. Further embodiments allow for a flexible handling of decoder initializations. ## Claims ### Claim 1 File parser, comprising one or more processors, wherein the one or more processors are configured to: derive from a file switching information for a set of tracks, the switching information indicating samples of a first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that one or more preceding samples in the second track are available to a video decoder fed by the file parser from the file; and derive, from the switching information, information on drift-affected samples of the first track which are affected from decoding drift by the video decoder decoding the drift-affected samples of the first track based on previous samples of the second track which are temporally aligned to reference samples of the first track referred to by the drift-affected samples. ### Claim 2 (depends on 1) File parser of claim 1, wherein the one or more processors are configured to: provide the switching information to an external device configured to decide on the switching. ### Claim 3 (depends on 1) File parser of claim 1, wherein the one or more processors are configured to: derive, from the switching information, information on dependent samples of the first track, which follow one of the samples of the first track at which switching from the second track is allowed, and which depend on one or more preceding reference samples of the first track, the one or more preceding reference samples preceding the one of the samples of the first track at which switching from the second track is allowed. ### Claim 4 (depends on 3) File parser of claim 3, wherein the information on the dependent sample is indicative of an upper bound of a number of, or a temporal duration of, the dependent samples. ### Claim 5 (depends on 3) File parser of claim 3, wherein the information on the dependent samples is indicative of a quality of the dependent samples meeting a predetermined criterion. ### Claim 6 (depends on 3) File parser of claim 3, wherein the information on the dependent samples is indicative of the dependent samples being coded without temporal syntax prediction. ### Claim 7 (depends on 3) File parser of claim 3, wherein the information on the dependent samples identifies the dependent samples. ### Claim 8 (depends on 3) File parser of claim 3, wherein the one or more processors are configured to: derive, for the first track, RAP information from the file which is indicative of RAP samples of the first track, the RAP information being separate from the switching information. ### Claim 9 (depends on 3) File parser of claim 3, wherein the one or more processors are configured to: derive, for the second track, further RAP information from the file which is indicative of RAP samples of the second track, the switching information indicating the samples of the first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that those samples in the second track are available to the video decoder which precede the samples of the first track upstream to the closest preceding RAP sample of the second track. ### Claim 10 (depends on 1) File parser of claim 1, wherein the information on the drift-affected samples is indicative of an upper bound of a number of, or a temporal duration of, the drift-affected samples. ### Claim 11 (depends on 1) File parser of claim 1, wherein the information on the drift-affected samples is indicative of a quality of the drift-affected samples meeting a predetermined criterion. ### Claim 12 (depends on 1) File parser of claim 1, wherein the information on the drift-affected samples is indicative of the drift-affected samples being coded without temporal syntax prediction. ### Claim 13 (depends on 1) File parser of claim 1, wherein the information on the drift-affected samples identifies the drift-affected samples. ### Claim 14 (depends on 1) File parser of claim 1, wherein the one or more processors are configured to: derive, for the first track, RAP information from the file which is indicative of RAP samples of the first track, the RAP information being separate from the switching information. ### Claim 15 (depends on 1) File parser of claim 1, wherein the one or more processors are configured to: derive, for the second track, further RAP information from the file which is indicative of RAP samples of the second track, the switching information indicating the samples of the first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that those samples in the second track are available to the video decoder which precede the samples of the first track upstream to the closest preceding RAP sample of the second track. ### Claim 16 Non-transitory digital storage medium having stored thereon a file of a video, the file comprising: a set of tracks; and switching information for the set of tracks, the switching information indicating samples of a first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that one or more preceding samples in the second track are available to a video decoder fed from the file, and the switching information indicating information on drift-affected samples of the first track which are affected from decoding drift by the video decoder decoding the drift-affected samples of the first track based on previous samples of the second track which are temporally aligned to reference samples of the first track referred to by the drift-affected samples. ### Claim 17 File generator, comprising one or more processors, wherein the one or more processors are configured to: provide a file with information for a set of tracks, the switching information indicating samples of a first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that one or more preceding samples in the second track are available to a video decoder fed from the file, and the switching information indicating information on drift-affected samples of the first track which are affected from decoding drift by the video decoder decoding the drift-affected samples of the first track based on previous samples of the second track which are temporally aligned to reference samples of the first track referred to by the drift-affected samples. ### Claim 18 Method for processing a file, comprising: derive from the file switching information for a set of tracks, the switching information indicating samples of a first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that one or more preceding samples in the second track are available to a video decoder to which the file is provided, and the switching information indicating information on drift-affected samples of the first track which are affected from decoding drift by the video decoder decoding the drift-affected samples of the first track based on previous samples of the second track which are temporally aligned to reference samples of the first track referred to by the drift-affected samples. ### Claim 19 Method for generating a file, comprising: provide the file with information for a set of tracks, the switching information indicating samples of a first track of the set of tracks at which switching from a second track of the set of tracks is allowed provided that one or more preceding samples in the second track are available to a video decoder fed from the file, and the switching information indicating information on drift-affected samples of the first track which are affected from decoding drift by the video decoder decoding the drift-affected samples of the first track based on previous samples of the second track which are temporally aligned to reference samples of the first track referred to by the drift-affected samples.