# US 2019/0379884 A1 · Method and apparatus for processing overlay media in 360 degree video system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379884 A1
- Document ID: US20190379884A1
- Status: Published application
- Kind: A1
- Application number: 16/432,657
- Filed: 2019-06-05
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Sejin Oh; Hyejung Hur; Jangwon Lee
- Web page: https://patentgrep.com/patent/US20190379884A1

## CPC

- H04N 13/293: Generating mixed stereoscopic images; Generating mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 13/178: Metadata, e.g. disparity information
- H04N 13/194: Transmission of image signals
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A 360-degree image data processing method performed by a 360-degree video reception apparatus according to the present invention includes: receiving 360-degree image data; obtaining information on an encoded picture and metadata from the 360-degree image data; decoding a picture based on the information on the encoded picture; and rendering the decoded picture and an overlay based on the metadata, wherein the metadata includes group information, the group information includes group type information indicating a group including a main media and the overlay that can be rendered together, the decoded picture includes the main media, and the group information includes information indicating whether a track belonging to the group includes the main media or the overlay.

## Claims

### Claim 1

A 360-degree image data processing method performed by a 360-degree video reception apparatus, the method comprising:
receiving 360-degree image data;
obtaining information on an encoded picture and metadata from the 360-degree image data;
decoding a picture based on the information on the encoded picture; and
rendering the decoded picture and an overlay based on the metadata,
wherein the metadata comprises group information,
the group information comprises group type information indicating a group comprising a main media and the overlay that can be rendered together,
the decoded picture comprises the main media, and
the group information comprises information indicating whether a track belonging to the group comprises the main media or the overlay.

### Claim 2 (depends on 1)

The method of claim 1, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises information indicating whether the overlay is to be essentially displayed, and
neither the overlay nor the main media is displayed when the overlay is to be essentially displayed and a player comprised in or connected to the 360-degree video reception apparatus does not support processing of the overlay.

### Claim 3 (depends on 1)

The method of claim 1, wherein the metadata comprises overlay-related metadata, and
the overlay-related metadata comprises overlay identifier information representing an identifier of the overlay, overlay essential flag information representing whether the overlay is to be essentially processed, and overlay priority information representing a priority of the overlay.

### Claim 4 (depends on 3)

The method of claim 3, wherein, when a player comprised in or connected to the 360-degree video reception apparatus lacks a capacity to decode all overlays, a decoding order of the overlays is derived on the basis of the priority indicated by the overlay priority information.

### Claim 5 (depends on 1)

The method of claim 1, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises region type information representing a region type of the overlay and region information of the overlay according to the region type, and
the overlay-related metadata comprises at least one of depth information and order information of the overlay.

### Claim 6 (depends on 5)

The method of claim 5, wherein the region type information represents one of a type overlaid on a viewport, a type overlaid on a spherical region, and a type overlaid in a 3D space.

### Claim 7 (depends on 6)

The method of claim 6, wherein an index value of 0 of the region type information indicates the type overlaid on the viewport, an index value of 0 of the region type information indicates the type overlaid on the spherical region, and an index value of 0 of the region type information indicates the type overlaid in the 3D space.

### Claim 8 (depends on 6)

The method of claim 6, wherein the depth information is comprised in the overlay-related metadata when the type of the overlay is the type overlaid on the spherical region or the type overlaid in the 3D space,
the depth information represents a depth value of a region in which the overlay is rendered, and
the order information represents a layering order of overlays which correspond to the type overlaid on the viewport or a layering order of overlays having the same depth value.

### Claim 9 (depends on 1)

The method of claim 1, wherein the metadata comprises overlay-related metadata, and
the overlay-related metadata comprises opacity information representing an opacity applied to the overlay and blending type information about the overlay.

### Claim 10 (depends on 1)

The method of claim 1, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises information representing whether the overlay is for a recommended viewport,
the overlay-related metadata comprises reference track indication information indicating a metadata track related to a recommended viewport as a reference track, and
it is represented whether the overlay is for the recommended viewport based on the reference track indication information.

### Claim 11 (depends on 1)

The method of claim 1, wherein the 360-degree video reception apparatus is connected to a self-driving vehicle and provides a user with an internal or external image of the self-driving vehicle.

### Claim 12

A 360-degree image data processing method performed by a 360-degree video transmission apparatus, the method comprising:
obtaining a 360-degree image;
deriving a picture by processing the 360-degree image;
generating metadata on the 360-degree image;
encoding the picture; and
processing the encoded picture and metadata for storage or transmission,
wherein the metadata comprises group information,
the group information comprises group type information indicating a group comprising a main media and an overlay that can be rendered together,
the encoded picture comprises the main media, and
the group information comprises information indicating whether a track belonging to the group comprises the main media or the overlay.

### Claim 13 (depends on 12)

The method of claim 12, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises information indicating whether the overlay is to be essentially displayed, and
neither the overlay nor the main media is displayed when the overlay is to be essentially displayed and a player comprised in or connected to a 360-degree video reception apparatus does not support processing of the overlay.

### Claim 14 (depends on 12)

The method of claim 12, wherein the metadata comprises overlay-related metadata, and
the overlay-related metadata comprises overlay identifier information representing an identifier of the overlay, overlay essential flag information representing whether the overlay is to be essentially processed, and overlay priority information representing a priority of the overlay.

### Claim 15 (depends on 12)

The method of claim 12, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises region type information representing a region type of the overlay and region information of the overlay according to the region type, and
the overlay-related metadata comprises at least one of depth information and order information of the overlay.

### Claim 16 (depends on 12)

The method of claim 12, wherein the metadata comprises overlay-related metadata, and
the overlay-related metadata comprises opacity information representing an opacity applied to the overlay and blending type information about the overlay.

### Claim 17

A 360-degree video reception apparatus comprising:
a reception processor configured to receive 360-degree image data and to obtain information on an encoded picture and metadata from the 360-degree image data;
a data decoder configured to decode a picture based on the information on the encoded picture; and
a renderer configured to render the decoded picture and an overlay based on the metadata,
wherein the metadata comprises group information,
the group information comprises group type information indicating a group comprising a main media and the overlay that can be rendered together,
the decoded picture comprises the main media, and
the group information comprises information indicating whether a track belonging to the group comprises the main media or the overlay.

### Claim 18 (depends on 17)

The 360-degree video reception apparatus of claim 17, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises information indicating whether the overlay is to be essentially displayed, and
neither the overlay nor the main media is displayed when the overlay is to be essentially displayed and a player comprised in or connected to the 360-degree video reception apparatus does not support processing of the overlay.

### Claim 19 (depends on 17)

The 360-degree video reception apparatus of claim 17, wherein the metadata comprises overlay-related metadata, and
the overlay-related metadata comprises overlay identifier information representing an identifier of the overlay, overlay essential flag information representing whether the overlay is to be essentially processed, and overlay priority information representing a priority of the overlay.

### Claim 20 (depends on 17)

The 360-degree video reception apparatus of claim 17, wherein the metadata comprises overlay-related metadata,
the overlay-related metadata comprises region type information representing a region type of the overlay and region information of the overlay according to the region type, and
the overlay-related metadata comprises at least one of depth information and order information of the overlay.
