# US 10,484,717 B2 · Codec architecture for multiple layer video coding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,717 B2 - Document ID: US10484717B2 - Status: Granted - Kind: B2 - Application number: 15/863,088 - Filed: 2018-01-05 - Published: 2018-05-10 - Granted: 2019-11-19 - Earliest claimed: 2012-07-09 - Assignees: Vid Scale, Inc. - Applicants: Vid Scale, Inc. - Inventors: Yan Ye; George W. McClellan; Yong He; Xiaoyu Xiu; Yuwen He; Jie Dong; Can Bal; Eun Seok Ryu - Web page: https://patentgrep.com/patent/US10484717B2 ## CPC - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) - H04N 19/157: Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter - H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) - H04N 19/31: in the temporal domain - H04N 19/33: in the spatial domain - H04N 19/36: Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems, methods, and instrumentalities are provided to implement video coding system (VCS). The VCS may be configured to receive a video signal, which may include one or more layers (e.g., a base layer (BL) and/or one or more enhancement layers (ELs)). The VCS may be configured to process a BL picture into an inter-layer reference (ILR) picture, e.g., using picture level inter-layer prediction process. The VCS may be configured to select one or both of the processed ILR picture or an enhancement layer (EL) reference picture. The selected reference picture(s) may comprise one of the EL reference picture, or the ILR picture. The VCS may be configured to predict a current EL picture using one or more of the selected ILR picture or the EL reference picture. The VCS may be configured to store the processed ILR picture in an EL decoded picture buffer (DPB). ## Claims ### Claim 1 A method comprising: receiving base layer (BL) coded data, enhancement layer (EL) coded data, and inter-layer prediction (ILP) information, wherein the ILP information is packetized in a network abstraction unit separate from network abstraction units containing the BL coded data and the EL coded data; reconstructing a BL picture based on the BL coded data; performing a picture level inter-layer process on the reconstructed BL picture to generate an inter-layer reference (ILR) picture based on the packetized ILP information; and reconstructing an EL picture based on the ILR picture and the EL coded data. ### Claim 2 (depends on 1) The method of claim 1, wherein the packetized ILP information comprises upsampling filter information. ### Claim 3 (depends on 1) The method of claim 1, wherein the ILP information is packetized in a plurality of network abstraction units, each of the network abstraction units being associated with a unit type. ### Claim 4 (depends on 1) The method of claim 1, wherein the packetized ILP information comprises at least one of upsampling filter information packetized in a network abstraction unit of a first unit type, one or more of disparity compensation parameters packetized in a network abstraction unit of a second unit type, or one or more of inverse tone mapping parameters packetized in a network abstraction unit of a third unit type. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining a first scalability type that relates the EL to the BL and a second scalability type that relates the EL to the BL based on one or more of the BL coded data, the EL coded data, or the ILP information; and determining, based on the first scalability type and the second scalability type, a combined picture level inter-layer process for processing the reconstructed BL picture into the ILR picture, wherein the picture level inter-layer process comprises the combined picture level inter-layer process. ### Claim 6 (depends on 5) The method of claim 5, wherein the first scalability type is spatial scalability, and the combined picture level inter-layer process comprises upsampling. ### Claim 7 (depends on 5) The method of claim 5, wherein the first scalability type is spatial scalability and the second scalability type is bit-depth scalability, and the combined picture level inter-layer process comprises upsampling and inverse tone mapping. ### Claim 8 (depends on 1) The method of claim 1, further comprising demuxing a video signal to separate the network abstraction unit containing the ILP information from the network abstraction units containing the BL coded data and the EL coded data. ### Claim 9 (depends on 1) The method of claim 1, wherein the reconstructed EL picture and the ILR picture correspond to a same time instance. ### Claim 10 (depends on 1) The method of claim 1, wherein the ILR picture is comprised in a set of reference pictures that further comprises a reference EL picture, and reconstructing the EL picture comprises: predicting a first portion of the EL picture using the ILR picture; and predicting a second portion of the EL picture using the reference EL picture. ### Claim 11 (depends on 1) The method of claim 1, wherein the packetized ILP information comprises one or more of inverse tone mapping parameters. ### Claim 12 A system comprising: a processor configured to: receive a video signal comprising base layer (BL) coded data, enhancement layer (EL) coded data, and inter-layer prediction (ILP) information, wherein the ILP information is packetized in a network abstraction unit separate from network abstraction units containing the BL coded data and the EL coded data; reconstruct a BL picture based on the BL coded data; perform a picture level inter-layer process on the reconstructed BL picture to generate an inter-layer reference (ILR) picture based on the packetized ILP information; and reconstruct an EL picture based on the ILR picture and the EL coded data. ### Claim 13 (depends on 12) The system of claim 12, wherein the packetized ILP information comprises upsampling filter information. ### Claim 14 (depends on 12) The system of claim 12, wherein the processor is further configured to: determine a first scalability type that relates the EL to the BL and a second scalability type that relates the EL to the BL based on one or more of the BL coded data, the EL coded data, or the ILP information; and determine, based on the first scalability type and the second scalability type, a combined picture level inter-layer process for processing the reconstructed BL picture into the ILR picture, wherein the picture level inter-layer process comprises the combined picture level inter-layer process. ### Claim 15 (depends on 14) The system of claim 14, wherein the first scalability type is spatial scalability, and the combined picture level inter-layer process comprises upsampling. ### Claim 16 (depends on 14) The system of claim 14, wherein the first scalability type is bit-depth scalability, and the combined picture level inter-layer process comprises inverse tone mapping. ### Claim 17 (depends on 14) The system of claim 14, wherein the first scalability type is spatial scalability and the second scalability type is bit-depth scalability, and the combined picture level inter-layer process comprises upsampling and inverse tone mapping. ### Claim 18 (depends on 12) The system of claim 12, wherein the reconstructed EL picture and the ILR picture correspond to a same time instance. ### Claim 19 (depends on 12) The system of claim 12, wherein the network abstraction unit containing the ILP information is in a first mode, and the network abstraction units containing the BL coded data and the EL coded data are in a second mode. ### Claim 20 (depends on 12) The system of claim 12, wherein the packetized ILP information comprises one or more of inverse tone mapping parameters.