Lightfield compression using disparity predicted replacement

A system comprising:

• a first storage device to store a first lightfield, the first lightfield comprising a two-dimensional array of images, each image comprising a corresponding two-dimensional array of image tiles;
• a second storage device to store a second lightfield, the second lightfield comprising a compressed representation of the first lightfield;
• and
• at least one first processor configured to:
• selectively eliminate image tiles of images of the first lightfield from inclusion in the second lightfield based on identified redundancies between the eliminated image tiles and image tiles in the same tile positions of other images of the lightfield, wherein the at least one first processor is to selectively eliminate image tiles by:
• organizing the two-dimensional array of images of the first lightfield into a binary tree structure, with each node of the binary tree structure representing a corresponding image of the two-dimensional array of images;
• and
• for each non-leaf node of the binary tree structure and for each image tile of the image at the non-leaf node, selectively eliminating a candidate image tile at the same tile position in an image at a descendant node of the non-leaf node from the second lightfield responsive to a disparity between the candidate image tile and the image tile of the image at the non-leaf node;
• determine a disparity derivative vector for the image tile of the image at the non-leaf node, the disparity derivative vector comprising one or more disparity derivatives specifying how to reconstruct an eliminated image tile of an image at a descendant node of the non-leaf node through corresponding shifting of the image at the non-leaf node in one or more dimensions as a function of a positional relationship between the image at the non-leaf node and the image at the descendant node in the two-dimensional array;
• and
• store the disparity derivative vector for the image tile to the second lightfield.

A system comprising:

• a storage device to store a compressed lightfield, the compressed lightfield comprising a compressed representation of a lightfield having a two-dimensional array of images, each image comprising a corresponding two-dimensional array of image tiles, and wherein at least a subset of the image tiles has been eliminated from the compressed lightfield;
• and
• at least one processor coupled to the storage device and configured to:
• identify a set of one or more image tiles of one or more images of the two-dimensional array for use in rendering imagery representing a view of a scene represented by the compressed lightfield;
• and
• responsive to determining the set of one or more image tiles comprises a first image tile eliminated from a first image in the compressed lightfield, reconstructing the first image tile using a second image in the compressed lightfield identified as reference image for the first image and using a first disparity derivative vector accessed from the compressed lightfield for a second image tile of the second image that has a same tile position as the first image tile, the first disparity derivative vector comprising one or more disparity derivatives specifying how to reconstruct the first image tile through corresponding shifting of the second image in one more dimensions as a function of a positional relationship between first image and the second image in the two-dimensional array, wherein the at least one processor is configured to reconstruct the first image tile by:
• shifting a portion of the second image including the second image tile and at least one other image tile of the second image in one or more dimensions relative to a reconstruction window representing a boundary of the tile position of the first image tile based on the first disparity derivative vector;
• and
• setting pixels contained within the reconstruction window after shifting the portion of the second image as a reconstructed representation of the first image tile.

A method comprising:

• storing, at a first storage device, a first lightfield, the first lightfield comprising a two-dimensional array of images, each image comprising a corresponding two-dimensional array of image tiles;
• and
• compressing, by at least one first processor, the first lightfield to generate a second lightfield, the second lightfield comprising a compressed representation of the first lightfield, wherein compressing the first lightfield comprises selectively eliminating image tiles of images of the first lightfield from inclusion in the second lightfield based on identified redundancies between the eliminated image tiles and image tiles in the same tile positions of other images of the lightfield;
• wherein selectively eliminating image tiles includes:
• organizing the two-dimensional array of images of the first lightfield into a binary tree structure, with each node of the binary tree structure representing a corresponding image of the two-dimensional array of images;
• and
• for each non-leaf node of the binary tree structure:
• for each image tile of the image at the non-leaf node, selectively eliminating a candidate image tile at the same tile position in an image at a descendant node of the non-leaf node from the second lightfield responsive to a disparity between the candidate image tile and the image tile of the image at the non-leaf node;
• and
• wherein determining the disparity includes:
• determining a first rate distortion resulting from replacement of the candidate image tile with the image tile of the image at the non-leaf node;
• determining a second rate distortion resulting from encoding the candidate image tile;
• including the candidate image tile in the second lightfield when the first rate distortion is not greater than the second rate distortion;
• and
• eliminating the candidate image tile from the second lightfield when the first rate distortion is greater than the second rate distortion.

A method comprising:

• storing, at a storage device, a compressed lightfield, the compressed lightfield comprising a compressed representation of a lightfield having a two-dimensional array of images, each image comprising a corresponding two-dimensional array of image tiles, and wherein at least a subset of the image tiles has been eliminated from the compressed lightfield;
• and
• identifying, by at least one processor, a set of one or more image tiles of one or more images of the two-dimensional array for use in rendering imagery representing a view of a scene represented by the compressed lightfield;
• and
• responsive to determining the set of one or more image tiles comprises a first image tile eliminated from a first image in the compressed lightfield, reconstructing, by the at least one processor, the first image tile using a second image in the compressed lightfield identified as reference image for the first image and using a first disparity derivative vector accessed from the compressed lightfield for a second image tile of the second image that has a same tile position as the first image tile, the first disparity derivative vector comprising one or more disparity derivatives specifying how to reconstruct the first image tile through corresponding shifting of the second image in one more dimensions as a function of a positional relationship between first image and the second image in the two-dimensional array, wherein reconstructing the first image tile comprises:
• shifting a portion of the second image including the second image tile and at least one other image tile of the second image in one or more dimensions relative to a reconstruction window representing a boundary of the tile position of the first image tile based on the first disparity derivative vector;
• and
• setting pixels contained within the reconstruction window after shifting the portion of the second image as a reconstructed representation of the first image tile.