# US 2019/0379876 A1 · Overlay processing method in 360 video system, and device thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379876 A1
- Document ID: US20190379876A1
- Status: Published application
- Kind: A1
- Application number: 16/490,049
- Filed: 2018-09-21
- Published: 2019-12-12
- Earliest claimed: 2017-09-26
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Hyejung HUR; Sejin OH; Soojin HWANG; Jangwon LEE
- Web page: https://patentgrep.com/patent/US20190379876A1

## CPC

- H04N 13/178: Metadata, e.g. disparity information
- G06T 3/12: Panospheric to cylindrical image transformations
- H04N 5/278: Subtitling
- H04N 13/139: Format conversion, e.g. of frame-rate or size
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 13/183: On-screen display [OSD] information, e.g. subtitles or menus
- H04N 13/194: Transmission of image signals
- H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment}
- H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)}
- H04N 21/2368: Multiplexing of audio and video streams
- H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)}
- H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
- H04N 21/4884: {for displaying subtitles}
- H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection
- H04N 21/816: {involving special video data, e.g 3D video}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A 360 image data processing method performed by a 360 video receiving device, according to the present invention, comprises the steps of: receiving 360 image data; acquiring information and metadata on an encoded picture from the 360 image data; decoding the picture on the basis of the information on the encoded picture; and rendering the decoded picture and an overlay on the basis of the metadata, wherein the metadata includes overlay-related metadata, the overlay is rendered on the basis of the overlay-related metadata, and the overlay-related metadata includes information on a region of the overlay.

## Claims

### Claim 1

A 360-degree image data processing method performed by a 360-degree video receiving device, the method comprising:
receiving 360-degree image data;
acquiring information on an encoded picture and metadata from the 360-degree image data;
decoding a picture based on the information on the encoded picture; and
rendering the decoded picture and an overlay based on the metadata,
wherein the metadata includes overlay related metadata,
wherein the overlay is rendered based on the overlay related metadata, and
wherein the overlay related metadata includes information on a region of the overlay.

### Claim 2 (depends on 1)

The method of claim 1, wherein information on a region of the overlay indicates the region of the overlay based on azimuth and elevation.

### Claim 3 (depends on 1)

The method of claim 1, wherein information on a region of the overlay includes at least one of position information and size information of the overlay.

### Claim 4 (depends on 3)

The method of claim 3, wherein the position information of the overlay indicates at least one of x coordinate, y coordinate, and z coordinate of a center point of the overlay within a 3D coordinate space to which the decoded picture is rendered, and
wherein the size information of the overlay indicates at least one of width and height of the overlay.

### Claim 5 (depends on 1)

The method of claim 1, wherein the overlay related metadata includes information on a type of the overlay, and
wherein the information on the type of the overlay indicates that the type of the overlay is an overlay type being rendered in a 3D space, an overlay type being rendered in a viewport, or an overlay type being rendered in a sphere surface.

### Claim 6 (depends on 5)

The method of claim 5, wherein, when the information on the type of the overlay indicates that the type of the overlay is an overlay type being rendered in a 3D space, the information on the region of the overlay indicates the region of the overlay based on azimuth and elevation.

### Claim 7 (depends on 5)

The method of claim 5, wherein, when the information on the type of the overlay indicates that the type of the overlay is an overlay type being rendered in a 3D space, the overlay related metadata includes information on a rotation of the overlay.

### Claim 8 (depends on 7)

The method of claim 7, wherein the information on the rotation of the overlay indicates at least one of yaw value, pitch value, and roll value for the rotation of the overlay.

### Claim 9 (depends on 5)

The method of claim 5, wherein, when the information on the type of the overlay indicates that the type of the overlay is an overlay type being rendered in a 3D space, the overlay related metadata further includes flag information indicating whether or not a layer or plane of the overlay is rotated based on a viewing orientation.

### Claim 10 (depends on 5)

The method of claim 5, wherein, when the information on the type of the overlay indicates that the type of the overlay is an overlay type being rendered in a 3D space, the overlay related metadata further include distance information indicating a distance from a center of a unit sphere.

### Claim 11 (depends on 5)

The method of claim 5, wherein, when the information on the type of the overlay indicates that the type of the overlay is an overlay type being rendered in a sphere surface overlay type, the overlay related metadata includes information on a depth of the region of the overlay.

### Claim 12 (depends on 11)

The method of claim 11, wherein information indicating a depth of the region of the overlay indicates rendering priority of the overlay.

### Claim 13 (depends on 12)

The method of claim 12, wherein, in case multiple overlays are overlaid, for an overlaying region, among the multiple overlays, an overlay being associated with a smaller depth value is rendered.

### Claim 14 (depends on 1)

The method of claim 1, wherein the overlay related metadata further includes target flag information indicating whether or not the overlay is associated with a reference point.

### Claim 15 (depends on 14)

The method of claim 14, wherein an image, text, sound or supplementary information indicated by the reference point is provided through the overlay, or wherein an image of a position indicated by the reference point is rendered.

### Claim 16 (depends on 1)

The method of claim 1, wherein the overlay related metadata includes information on a number of overlays, and
wherein overlay related information corresponding to each of the overlays are included in the overlay related metadata.

### Claim 17 (depends on 1)

The method of claim 1, wherein the overlay related metadata is included in an overlay timed metadata track.

### Claim 18 (depends on 1)

The method of claim 1, wherein the overlay related metadata includes information on an order of overlays each having an overlay type being rendered in a viewport.

### Claim 19

A 360-degree image data processing method performed by a 360-degree video transmitting device, the method comprising:
acquiring a 360-degree image;
deriving a picture by processing the 360-degree image;
generating metadata related to the 360-degree image;
encoding the picture; and
performing processing for storing or transmission of the encoded picture and the metadata,
wherein the metadata includes overlay related metadata, and
wherein the overlay related metadata includes information on a region of the overlay.

### Claim 20

A 360-degree video receiving device, the device comprising:
a reception processor receiving 360-degree image data and acquiring information on an encoded picture and metadata from the 360-degree image data;
a data decoder decoding a picture based on the information on the encoded picture; and
a renderer rendering the decoded picture and an overlay based on the metadata,
wherein the metadata includes overlay related metadata,
wherein the renderer renders the overlay based on the overlay related metadata, and
wherein the overlay related metadata includes information on a region of the overlay.
