# US 10,873,737 B1 · VR device and control method for the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,737 B1 - Document ID: US10873737B1 - Status: Granted - Kind: B1 - Application number: 16/550,759 - Filed: 2019-08-26 - Granted: 2020-12-22 - Earliest claimed: 2019-08-16 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Kyungyong Kim; Hongsuk Jeong - Web page: https://patentgrep.com/patent/US10873737B1 ## CPC - H04N 13/194: Transmission of image signals - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 7/0102: {involving the resampling of the incoming video signal} - H04N 7/025: Systems for the transmission of digital non-picture data, e.g. of text during the active part of a television frame {(transmission of digital non-picture data during the vertical blanking interval only H04N7/088)} - H04N 7/08: Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band {, e.g. by time division (H04N7/007 takes precedence)} - H04N 13/156: Mixing image signals - H04N 13/172: image signals comprising non-image signal components, e.g. headers or format information - H04N 13/178: Metadata, e.g. disparity information - H04N 19/167: Position within a video image, e.g. region of interest [ROI] - H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks - H04N 19/33: in the spatial domain - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras} - H04N 21/234327: {by decomposing into layers, e.g. base layer and one or more enhancement layers} - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A control method for a VR device is disclosed, which comprises the steps of receiving 360-degree video bitstreams, extracting sub-bitstreams included in the 360-degree video bitstreams, the sub-bitstreams including a Region of Interest (ROI) for the 360-degree video bitstreams, generating restoration data for the ROI from sub-bitstreams, and generating thumbnail data by remapping the restoration data for the ROI based on signaling information. ## Claims ### Claim 1 A method of controlling a Virtual Reality (VR) device, the method comprising receiving a 360 video bitstream; extracting a sub-bitstream included in the 360 video bitstream, wherein the sub-bitstream includes Region of Interest (ROI) for the 360 video bitstream; decoding the 360 video bitstream in accordance with a scalable coding scheme; performing inter-layer prediction (ILP) for the 360 video bitstream; performing inter-prediction (IP) for blocks of the 360 video bitstream; generating restoration data for the ROI from the sub-bitstream; and generating thumbnail data by remapping the restoration data for the ROI based on signaling information; wherein the ROI is a region for which down-sampling is not performed or a region which is less down-sampled during region-wise packing, wherein the signaling information includes a region-wise packing parameter representing information related to the down-sampling, wherein the restoration data for the ROI is generated based on the region-wise packing parameter, and wherein the region-wise packing parameter packs high-priority part of the ROI into a larger image and rest of the ROI into a smaller image, wherein the ILP indicates prediction of higher spatial resolution or quality signal by means of low spatial resolution or quality signal, wherein the IP indicates prediction from decoded samples of a current decoded picture and reference pictures, wherein the IP includes temporal inter-prediction, wherein Motion Constrained Tile Set (MCTS) restricts a range of temporal inter-prediction to a current tile internal range. ### Claim 2 (depends on 1) The method of claim 1, wherein when the signaling information is included in the 360 video bitstream, the signaling information includes information related to unpacking for the 360 video bitstream. ### Claim 3 (depends on 1) The method of claim 1, wherein when the signaling information is received separately from the 360 video bitstream, the controlling the VR device includes parsing the separately received signaling information, wherein the signaling information includes first information for extracting the sub-bitstream for the ROI from the 360 video bitstream and second information for extracting thumbnail data from the restoration data for the ROI. ### Claim 4 (depends on 1) The method of claim 1, wherein when the 360 video bitstream only includes the sub-bitstream for the ROI, the VR device generates the thumbnail data by remapping the restoration data for the ROI without extracting the sub-bitstream. ### Claim 5 (depends on 1) The method of claim 1, wherein when the 360 video bitstream includes an overlay region, the overlay region is used as the ROI. ### Claim 6 (depends on 1) The method of claim 1, wherein when the 360 video bitstream is classified to a first tile and a second tile and a number of bits for the first tile is greater than a number of bits for the second tile, the first tile is used as the ROI. ### Claim 7 (depends on 1) The method of claim 1, wherein when the 360 video bitstream is received based on a Dynamic Adaptive Streaming over HTTP (DASH) protocol, a Media Presentation Description (MPD) delivered by the DASH protocol includes information for the ROI, wherein the ROI is determined based on the information for the ROI included in the MPD. ### Claim 8 (depends on 1) The method of claim 1, wherein when the 360 video bitstream is received based on a scalable coding scheme, the 360 video bitstream includes a base layer including data of a first resolution and an enhancement layer including data of a second resolution, the first resolution is lower than the second resolution, wherein the ROI is determined based on the base layer. ### Claim 9 (depends on 1) The method of claim 1, wherein the 360 video bitstream is data stored based on one of file formats, the file formats include MP4 file format, MKV file format and TS file format. ### Claim 10 A Virtual Reality (VR) device, the device comprising a receiver configured to receive a 360 video bitstream; a bitstream extractor configured to extract a sub-bitstream included in the 360 video bitstream, wherein the sub-bitstream includes Region of Interest (ROI) for the 360 video bitstream; a decoder configured to decode the 360 video bitstream in accordance with a scalable coding scheme; an inter-layer predictor configured to perform inter-layer prediction (ILP) for the 360 video bitstream; an inter-predictor to perform inter-prediction (IP) for blocks of the 360 video bitstream; a thumbnail generator configured to generate restoration data for the ROI from the sub-bitstream; and a remapper configured to generate thumbnail data by remapping the restoration data for the ROI based on signaling information; wherein the ROI is a region for which down-sampling is not performed or a region which is less down-sampled during region-wise packing, wherein the signaling information includes a region-wise packing parameter representing information related to the down-sampling, wherein the restoration data for the ROI is generated based on the region-wise packing parameter, and wherein the region-wise packing parameter packs high-priority part of the ROI into a larger image and rest of the ROI into a smaller image, wherein the ILP indicates prediction of higher spatial resolution or quality signal by means of low spatial resolution or quality signal, wherein the IP indicates prediction from decoded samples of a current decoded picture and reference pictures, wherein the IP includes temporal inter-prediction, wherein Motion Constrained Tile Set (MCTS) restricts a range of temporal inter-prediction to a current tile internal range. ### Claim 11 (depends on 10) The device of claim 10, wherein when the signaling information is included in the 360 video bitstream, the signaling information includes information related to unpacking for the 360 video bitstream. ### Claim 12 (depends on 11) The device of claim 11, wherein when the signaling information is received separately from the 360 video bitstream, the VR device further includes a parser configured to parse the separately received signaling information, wherein the signaling information includes first information for extracting the sub-bitstream for the ROI from the 360 video bitstream and second information for extracting thumbnail data from the restoration data for the ROI. ### Claim 13 (depends on 11) The device of claim 11, wherein when the 360 video bitstream only includes the sub-bitstream for the ROI, the VR device generates the thumbnail data by remapping the restoration data for the ROI without extracting the sub-bitstream. ### Claim 14 (depends on 11) The device of claim 11, wherein when the 360 video bitstream includes an overlay region, the overlay region is used as the ROI. ### Claim 15 (depends on 11) The device of claim 11, wherein when the 360 video bitstream is classified to a first tile and a second tile and a number of bits for the first tile is greater than a number of bits for the second tile, the first tile is used as the ROI. ### Claim 16 (depends on 10) The device of claim 10, wherein when the 360 video bitstream is received based on a Dynamic Adaptive Streaming over HTTP (DASH) protocol, a Media Presentation Description (MPD) delivered by the DASH protocol includes information for the ROI, wherein the ROI is determined based on the information for the ROI included in the MPD. ### Claim 17 (depends on 11) The device of claim 11, wherein when the 360 video bitstream is received based on a scalable coding scheme, the 360 video bitstream includes a base layer including data of a first resolution and an enhancement layer including data of a second resolution, the first resolution is lower than the second resolution, wherein the ROI is determined based on the base layer. ### Claim 18 (depends on 11) The device of claim 11, wherein the 360 video bitstream is data stored based on one of file formats, the file formats include MP4 file format, MKV file format and TS file format.