# US 11,115,641 B2 · Method of transmitting omnidirectional video, method of receiving omnidirectional video, device for transmitting omnidirectional video, and device for receiving omnidirectional video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,115,641 B2 - Document ID: US11115641B2 - Status: Granted - Kind: B2 - Application number: 16/877,248 - Filed: 2020-05-18 - Published: 2020-09-03 - Granted: 2021-09-07 - Earliest claimed: 2016-08-25 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Jangwon Lee; Sejin Oh; Jongyeul Suh - Web page: https://patentgrep.com/patent/US11115641B2 ## CPC - H04N 13/194: Transmission of image signals - G06T 3/047: Fisheye or wide-angle transformations - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - H04N 13/106: Processing image signals (for multi-view video sequence encoding H04N19/597) - H04N 13/178: Metadata, e.g. disparity information - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)} - G06T 15/04: Texture mapping - G06T 15/20: Perspective computation - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method of transmitting omnidirectional video is provided according to one aspect of the present invention. The method of transmitting omnidirectional video according to an embodiment of the present invention includes: acquiring an image for the omnidirectional video; projecting the image for the omnidirectional video onto a 3D projection structure; packing the image projected on the 3D projection structure into a 2D frame; encoding the image packed into the 2D frame; and transmitting a data signal including the encoded image and metadata about the omnidirectional video. ## Claims ### Claim 1 A method of generating 360 degree video in a server, the method comprising: obtaining image data captured by at least one camera; stitching the image data; mapping the image data onto a packed picture; encoding the packed picture; generating one or more media tracks for the packed picture and audio data; generating timed metadata track for specifying a sphere region, wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis; wherein the timed metadata track is linked to the one or more media tracks; and transmitting at least one segment including the one or more media tracks and the timed metadata track. ### Claim 2 (depends on 1) The method according to claim 1, wherein the timed metadata track is linked to the one or more media tracks based on a track reference box having at least one track ID (identification). ### Claim 3 (depends on 1) The method according to claim 1, wherein the one or more media tracks and the timed metadata track are included in an adaptation set of DASH (Dynamic Adaptive Streaming over HTTP). ### Claim 4 (depends on 1) The method according to claim 1, wherein the one or more media tracks and the timed metadata track are included in a PES (Packetized Elementary Stream) packet of an MPEG-2 TS (Transport Stream) or an adaptation field of a TS. ### Claim 5 (depends on 1) The method according to claim 1, wherein the one or more media tracks and the timed metadata track are included in an SEI (Supplemental Enhancement Layer) message of a VCL (Video Coding layer). ### Claim 6 A device for transmitting 360 degree video, comprising: an image acquisition unit for obtaining image data captured by at least one camera; a projection unit for stitching the image data and mapping the image data onto a packed picture; a video encoder for encoding the packed picture; a controller for generating one or more media tracks for the packed picture and audio data, and generating timed metadata track for specifying a sphere region, wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis, wherein the timed metadata track is linked to the one or more media tracks; and a transmitter for transmitting at least one segment including the one or more media tracks and the timed metadata track. ### Claim 7 (depends on 6) The device according to claim 6, wherein the timed metadata track is linked to the one or more media tracks based on a track reference box having at least one track ID (identification). ### Claim 8 (depends on 6) The device according to claim 6, wherein the one or more media tracks and the timed metadata track are included in an adaptation set of DASH (Dynamic Adaptive Streaming over HTTP). ### Claim 9 (depends on 6) The device according to claim 6, wherein the one or more media tracks and the timed metadata track are included in a PES (Packetized Elementary Stream) packet of an MPEG-2 TS (Transport Stream) or an adaptation field of a TS. ### Claim 10 (depends on 6) The device according to claim 6, wherein the one or more media tracks and the timed metadata track are included in an SEI (Supplemental Enhancement Layer) message of a VCL (Video Coding layer). ### Claim 11 A digital receiver for processing 360 degree video, comprising: a receiving module configured to receive at least one segment including one or more media tracks related to the 360 degree video and audio data, and timed metadata track for specifying a sphere region, wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis, wherein the timed metadata track is linked to the one or more media tracks; a video decoder configured to decode the 360 degree video; a rendering module configured to render the 360 degree video; and a display module configured to display the 360 degree video. ### Claim 12 (depends on 11) The digital receiver according to claim 11, wherein the timed metadata track is linked to the one or more media tracks based on a track reference box having at least one track ID (identification). ### Claim 13 (depends on 11) The digital receiver according to claim 11, wherein the one or more media tracks and the timed metadata track are included in an adaptation set of DASH (Dynamic Adaptive Streaming over HTTP). ### Claim 14 (depends on 11) The digital receiver according to claim 11, wherein the one or more media tracks and the timed metadata track are included in a PES (Packetized Elementary Stream) packet of an MPEG-2 TS (Transport Stream) or an adaptation field of a TS. ### Claim 15 (depends on 11) The digital receiver according to claim 11, wherein the one or more media tracks and the timed metadata track are included in an SEI (Supplemental Enhancement Layer) message of a VCL (Video Coding layer). ### Claim 16 A method for processing 360 degree video in a digital receiver, comprising: receiving at least one segment including one or more media tracks related to the 360 degree video and audio data, and timed metadata track for specifying a sphere region, wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis, wherein the timed metadata track is linked to the one or more media tracks; decoding the 360 degree video; rendering the 360 degree video; and displaying the 360 degree video. ### Claim 17 (depends on 16) The method according to claim 16, wherein the timed metadata track is linked to the one or more media tracks based on a track reference box having at least one track ID (identification). ### Claim 18 (depends on 16) The method according to claim 16, wherein the one or more media tracks and the timed metadata track are included in an adaptation set of DASH (Dynamic Adaptive Streaming over HTTP). ### Claim 19 (depends on 16) The method according to claim 16, wherein the one or more media tracks and the timed metadata track are included in a PES (Packetized Elementary Stream) packet of an MPEG-2 TS (Transport Stream) or an adaptation field of a TS. ### Claim 20 (depends on 16) The method according to claim 16, wherein the one or more media tracks and the timed metadata track are included in an SEI (Supplemental Enhancement Layer) message of a VCL (Video Coding layer).