# US 2014/0198182 A1 · Representation and Coding of Multi-View Images Using Tapestry Encoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0198182 A1 - Document ID: US20140198182A1 - Status: Published application - Kind: A1 - Application number: 14/237,548 - Filed: 2012-09-27 - Published: 2014-07-17 - Earliest claimed: 2011-09-29 - Assignees: Dolby Laboratories Licensing Corporation - Inventors: Gregory John Ward; Scott Daly - Web page: https://patentgrep.com/patent/US20140198182A1 ## CPC - H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation - H04N 13/139: Format conversion, e.g. of frame-rate or size - H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Representation and coding of multi-view images using tapestry encoding are described. A tapestry comprises information on a tapestry image and a world coordinates map associated with the tapestry image, each of which may contain information from one or more perspectives of a scene. Perspective images of a scene can be generated from the tapestry image and the world coordinates map. ## Claims ### Claim at position 1 1-59. (canceled) ### Claim 60 A method for generating at least one view of a scene from a tapestry image, each of the at least one view being associated with one desired viewpoint of the scene, the method comprising: providing the tapestry image; wherein the tapestry image comprises a 2 D array of pixels comprising information from a plurality of views associated with the scene; providing a coordinates map associated with the tapestry image; wherein the position data associated with the pixels of the tapestry image comprise depth data for each pixel in the tapestry image; and wherein the position data associated with the pixels of the tapestry image further comprise horizontal disparity data and/or vertical disparity data for each pixel in the tapestry image; deriving one or more views of the scene based on the tapestry image and the coordinates map. ### Claim 61 (depends on 60) The method according to claim 60, further comprising: providing reference images of the scene, each reference image being associated with one desired viewpoint; and generating one or more correction layers, each correction layer being associated with one desired viewpoint, wherein a correction layer associated with a particular desired viewpoint is generated based on the reference image associated with the particular desired viewpoint and the derived view associated with the same particular desired viewpoint, wherein the generating of the at least one view is further based on the one or more correction layers. ### Claim 62 (depends on 61) The method according to claim 61, wherein each of the one or more correction layers is based on a difference between the reference image associated with the particular desired viewpoint and the derived view associated with the same particular desired viewpoint. ### Claim 63 (depends on 62) The method according to claim 62, wherein the difference comprises subtracting each pixel in the reference image associated with the particular desired viewpoint from a corresponding pixel in the derived view associated with the same particular desired viewpoint. ### Claim 64 (depends on 61) The method according to claim 61, wherein the deriving comprises adding, for each view in the at least one view, a view from the at least one view with a correction layer from the one or more correction layers, and wherein the view and the correction layer are associated with the same desired viewpoint. ### Claim 65 (depends on 60) The method according to claim 60, wherein the deriving comprises performing textured area filling based on the tapestry image and the coordinates map. ### Claim 66 (depends on 60) The method according to claim 60, wherein the providing the tapestry image comprises: taking pictures of the scene from a plurality of viewpoints of the scene; and processing the taken pictures to generate the tapestry image. ### Claim 67 (depends on 60) The method according to claim 60, wherein the providing the tapestry image comprises: generating pictures, using a computer, of the scene for a plurality of viewpoints of the scene; and processing the computer generated pictures to generate the tapestry image. ### Claim 68 (depends on 65) The method according to claim 65, wherein the processing the tapestry image comprises: selecting views from the plurality of views of the scene; and performing a combination based on the selected views to generate the tapestry image. ### Claim 69 (depends on 60) The method according to claim 60, wherein the providing the tapestry image comprises: performing an analysis of the scene to obtain a plurality of features associated with the scene; prioritizing each feature in the plurality of features from most important to least important; and generating the tapestry image such that higher resolution is allocated to features of higher importance than to features of lower importance. ### Claim 70 (depends on 69) The method according claim 69, wherein the performing an analysis is based on one or more of manual allocation of importance, frequency analysis, and saliency analysis. ### Claim 71 (depends on 60) The method according to claim 60, further comprising generating a camera grid, wherein the camera grid comprises identification information of each image in the at least one image of a scene from a tapestry image. ### Claim 72 (depends on 71) The method according to claim 71, wherein the identification information of a particular image comprises at least one of view position, view angle, view direction, orientation, and horizontal and vertical size of the particular image. ### Claim 73 An encoding system adapted to receive a plurality of input views of a scene, comprising: a tapestry generator adapted to generate a tapestry image and a coordinates map based on the plurality of input views; wherein the tapestry image comprises a 2 D array of pixels comprising information from the plurality of input views of the scene; wherein the position data associated with the pixels of the tapestry image comprise depth data for each pixel in the tapestry image; and wherein the position data associated with the pixels of the tapestry image further comprise horizontal disparity data and/or vertical disparity data for each pixel in the tapestry image; and an encoder adapted to encode the tapestry image and the coordinates map into a bitstream. ### Claim 74 (depends on 73) The encoding system according to claim 73, further comprising: a view generation unit adapted to derive one or more views of the scene based on the tapestry image and the coordinates map; and a subtractor coupled with the view generation unit and adapted to generate a difference between the one or more derived views and the one or more input views, each difference being between a derived view of a particular viewpoint and an input view of the same particular viewpoint, thus generating one or more correction layers, wherein the encoder is adapted to encode the tapestry image, the coordinates map, and the one or more correction layers into a bitstream. ### Claim 75 (depends on 73) The encoding system according to claim 73, wherein: the encoding system is further adapted to receive a camera grid, each point in the camera grid being associated with one input view of the scene, and the view generation unit is adapted to derive the one or more views of the scene based further on the camera grid. ### Claim 76 (depends on 75) The encoding system according to claim 75, wherein each point in the camera grid comprises, for each input view of the scene, information on one or more of view position, view angle, view direction, orientation, and horizontal and vertical size associated with the input view. ### Claim 77 (depends on 75) The encoding system according to claim 75, wherein each point in the camera grid comprises, for each input view of the scene, information on one or more of view position, view angle, view direction, orientation, horizontal and vertical size, and correction layer associated with the input view. ### Claim 78 A decoding system adapted to receive a bitstream associated with a scene and adapted to generate one or more output images, comprising: a decoder adapted to decode a tapestry image and a coordinates map from the bitstream, wherein the tapestry image and the coordinates map are associated with a plurality of input views of the scene; wherein the tapestry image comprises a 2 D array of pixels comprising information from the plurality of input views of the scene; wherein the position data associated with the pixels of the tapestry image comprise depth data for each pixel in the tapestry image; and wherein the position data associated with the pixels of the tapestry image further comprise horizontal disparity data and/or vertical disparity data for each pixel in the tapestry image; and a view generation unit adapted to derive the one or more output images of the scene based on the tapestry image and the coordinates map. ### Claim 79 (depends on 78) The decoding system according to claim 78, wherein: the decoder is further adapted to decode one or more correction layers, each correction layer being associated with one input view of the scene, and the view generation unit is adapted to derive the one or more output images of the scene based further on the one or more correction layers. ### Claim 80 (depends on 78) The decoding system according to claim 78, wherein: the decoder is further adapted to decode a camera grid, and the view generation unit is adapted to derive the one or more output images of the scene based further on the camera grid, wherein each point in the camera grid is associated with one input view of the scene. ### Claim 81 (depends on 79) The decoding system according to claim 79, wherein the view generation unit is adapted to derive a first set of images based on the tapestry image and the coordinates map, the decoding system further comprising: a combining unit adapted to combine an image from among the first set of images with a corresponding correction layer to generate an output image in the one or more output images, wherein the image from among the first set of images, the correction layer, and the generated output image are associated with the same input view of the scene. ### Claim 82 (depends on 80) The decoding system according to claim 80, wherein each point in the camera grid comprises, for each input view of the scene, information on one or more of view position, view angle, view direction, orientation, and horizontal and vertical size associated with the input view. ### Claim 83 (depends on 80) The decoding system according to claim 80, wherein each point in the camera grid comprises, for each input view of the scene, information on one or more of view position, view angle, view direction, orientation, horizontal and vertical size, and correction layer associated with the input view. ### Claim 84 (depends on 78) The decoding system according to claim 78, wherein the view generation unit is adapted to derive the one or more output images of the scene by performing textured area filling based on the tapestry image and the coordinates map.