# US 2019/0320155 A1 · Methods and apparatus for encoding and decoding virtual reality content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0320155 A1 - Document ID: US20190320155A1 - Status: Published application - Kind: A1 - Application number: 16/381,846 - Filed: 2019-04-11 - Published: 2019-10-17 - Earliest claimed: 2018-04-12 - Assignees: MediaTek Singapore Pte. Ltd. - Applicants: MEDIATEK Singapore Pte. Ltd. - Inventors: Xin Wang; Lulin Chen - Web page: https://patentgrep.com/patent/US20190320155A1 ## CPC - H04N 13/178: Metadata, e.g. disparity information - H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking - H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The techniques described herein relate to methods, apparatus, and computer readable media configured to process multi-view multimedia data. The multi-view multimedia data includes a hierarchical track structure with at least a first track, wherein the first track is at a first level in the hierarchical track structure, and includes data for both a first view and a second view of the multi-view multimedia data. Metadata contained within the first track can be determined and used to perform an extraction operation on the first track to generate first media data of a second track, wherein the first media data is for the first view, and second media data of a third track, wherein the second media data is for the second view, wherein the second track and the third track are at a second level in the hierarchical track structure above the first level of the first track. ## Claims ### Claim 1 A decoding method for decoding video data, the method comprising: receiving multi-view multimedia data comprising a hierarchical track structure comprising at least a first track, wherein the first track: is at a first level in the hierarchical track structure; and comprises data for both a first view and a second view of the multi-view multimedia data; determining metadata contained within the first track; and performing, based on the determined metadata, an extraction operation on the first track to generate: first media data of a second track, wherein the first media data is for the first view; and second media data of a third track, wherein the second media data is for the second view, wherein the second track and the third track are at a second level in the hierarchical track structure above the first level of the first track. ### Claim 2 (depends on 1) The decoding method of claim 1, wherein: determining the metadata comprises determining stereoscopic frame packing metadata, wherein the stereoscopic frame packing metadata comprises data indicative of one or more parameters used to generate the first media data, the second media data, or both. ### Claim 3 (depends on 1) The decoding method of claim 1, wherein: the first track comprises a projected two-dimensional track; the second track comprises a first projected video track for the first view; and the third track comprises a second projected video track for the second view. ### Claim 4 (depends on 1) The decoding method of claim 1, further comprising performing a track derivation operation on multimedia content associated with the first track to generate media data of the first track. ### Claim 5 (depends on 4) The decoding method of claim 4, wherein: the first track is associated with a plurality of variant tracks, each variant track including multi-view multimedia data; and performing the track derivation operation comprises performing the track derivation operation on the plurality of variant tracks to generate the media data of the first track. ### Claim 6 (depends on 1) The decoding method of claim 1, wherein: receiving the multi-view multimedia data comprises receiving stereoscopic multimedia data, wherein the stereoscopic multimedia data comprises data for both a left view and a right view of the multi-view multimedia data. ### Claim 7 A method for encoding video data, the method comprising: encoding multi-view multimedia data comprising a hierarchical track structure comprising at least a first track, wherein the first track: is at a first level in the hierarchical track structure; and comprises data for both a first view and a second view of the multi-view multimedia data; and encoding metadata within the first track, such that the metadata can be used to determine an extraction operation on the first track to generate: first media data of a second track, wherein the first media data is for the first view; and second media data of a third track, wherein the second media data is for the second view, wherein the second track and the third track are at a second level in the hierarchical track structure above the first level of the first track. ### Claim 8 (depends on 7) The encoding method of claim 7, wherein encoding the metadata comprises encoding stereoscopic frame packing metadata comprising data indicative of one or more parameters used to generate the first media data, the second media data, or both. ### Claim 9 (depends on 7) The encoding method of claim 7, wherein: the first track comprises a projected two-dimensional track; the second track comprises a first projected video track for the first view; and the third track comprises a second projected video track for the second view. ### Claim 10 (depends on 7) The encoding method of claim 7, wherein encoding the multi-view multimedia data comprises encoding stereoscopic multimedia data, wherein the stereoscopic multimedia data comprises data for both a left view and a right view of the multi-view multimedia data. ### Claim 11 An apparatus configured to decode video data, the apparatus comprising a processor in communication with memory, the processor being configured to execute instructions stored in the memory that cause the processor to: receive multi-view multimedia data comprising a hierarchical track structure comprising at least a first track, wherein the first track: is at a first level in the hierarchical track structure; and comprises data for both a first view and a second view of the multi-view multimedia data; determine metadata contained within the first track; and perform, based on the determined metadata, an extraction operation on the first track to generate: first media data of a second track, wherein the first media data is for the first view; and second media data of a third track, wherein the second media data is for the second view, wherein the second track and the third track are at a second level in the hierarchical track structure above the first level of the first track. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein determining the metadata comprises determining stereoscopic frame packing metadata, wherein the stereoscopic frame packing metadata comprises data indicative of one or more parameters used to generate the first media data, the second media data, or both. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein: the first track comprises a projected two-dimensional track; the second track comprises a first projected video track for the first view; and the third track comprises a second projected video track for the second view. ### Claim 14 (depends on 11) The apparatus of claim 11, wherein the instructions are further configured to cause the processor to perform a track derivation operation on multimedia content associated with the first track to generate media data of the first track. ### Claim 15 (depends on 14) The apparatus of claim 14, wherein: the first track is associated with a plurality of variant tracks, each variant track including multi-view multimedia data; and performing the track derivation operation comprises performing the track derivation operation on the plurality of variant tracks to generate the media data of the first track. ### Claim 16 (depends on 11) The apparatus of claim 11, wherein: receiving the multi-view multimedia data comprises receiving stereoscopic multimedia data, wherein the stereoscopic multimedia data comprises data for both a left view and a right view of the multi-view multimedia data. ### Claim 17 An apparatus for encoding video data, the apparatus comprising a processor in communication with memory, the processor being configured to execute instructions stored in the memory that cause the processor to: encode multi-view multimedia data comprising a hierarchical track structure comprising at least a first track, wherein the first track: is at a first level in the hierarchical track structure; and comprises data for both a first view and a second view of the multi-view multimedia data; and encode metadata within the first track, such that the metadata can be used to determine an extraction operation on the first track to generate: first media data of a second track, wherein the first media data is for the first view; and second media data of a third track, wherein the second media data is for the second view, wherein the second track and the third track are at a second level in the hierarchical track structure above the first level of the first track. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein encoding the metadata comprises encoding stereoscopic frame packing metadata comprising data indicative of one or more parameters used to generate the first media data, the second media data, or both. ### Claim 19 (depends on 17) The apparatus of claim 17, wherein: the first track comprises a projected two-dimensional track; the second track comprises a first projected video track for the first view; and the third track comprises a second projected video track for the second view. ### Claim 20 (depends on 17) The apparatus of claim 17, wherein encoding the multi-view multimedia data comprises encoding stereoscopic multimedia data, wherein the stereoscopic multimedia data comprises data for both a left view and a right view of the multi-view multimedia data.