System for tile coding of moving images
AssigneeNational Semiconductor Corporation
InventorsVivek Bhargava, Andrew Jue, Michael A. Van Scherrenburg, Les J. Wilson
Life of this patent
Estimating the term…
Abstract
A video image frame consisting of a two-dimensional array of picture elements (pixels) is decomposed into a set of rectangular image portions (tiles). Within each tile, variance between pixel intensity values is less than a predetermined value. A tile is encoded by a value set identifying the tile and including a single intensity value for all pixels in the tile. Frame-to-frame variation of the video image is encoded by inter-frame tile comparison and encoding of sub-tiles representative of change from a previous frame to a current frame.
Claims30
In a system for encoding a scanned representation of an image, the scanned representation including ordered, two-dimensional arrays of pixels, each pixel including an intensity value, a combination comprising:
- a buffer for storing a succession of two-dimensional arrays of pixels, each two-dimensional array representing a temporal instantiation of an image;
- a first tiling unit coupled to the buffer for:
- partitioning pixels of a first two-dimensional array into a plurality of tiles, wherein within each of the tiles, a variance among pixel intensity values is less than a first predetermined value;
- and
- generating a single pixel intensity value for each tile;
- and
- a second tiling unit coupled to the buffer and to the first tiling unit for producing a representation of change in the image occurring between the first two-dimensional array and a second two-dimensional array following the first two-dimensional array, the representation of change including a plurality of sub-tiles in the second two-dimensional array, each subtile corresponding to at least a portion of a tile in the first two-dimensional array, wherein within each sub-tile a variance among pixel intensity values is less than the first predetermined value, and for at least one sub-tile in the second two-dimensional array, the variance between an intensity value for a pixel in the sub-tile and a single pixel intensity value for a corresponding tile in the first two-dimensional array is greater than a second predetermined value.
A method for encoding change in a scanned visual representation of an image, the scanned representation including a sequence of ordered, two-dimensional arrays of pixels, each pixel including an intensity value, the method including the steps of:
- receiving a succession of two-dimensional arrays of pixels, each two-dimensional array representing a temporal instantiation of a changing image;
- operating an encoder in a still tile coding mode to produce a representation a first two-dimensional array from among the succession of two-dimensional arrays by:
- partitioning pixels of the first two-dimensional array into a plurality of tiles, wherein within each of the tiles, variance among pixel intensity values is less than a first predetermined value;
- and
- producing a single pixel intensity value for each tile;
- and
- operating the encoder in a motion tile coding mode to produce a representation of change in the image occurring between the first two-dimensional array and a second two-dimensional array which follows the first two-dimensional array in the succession of two-dimensional arrays, the representation of change including a plurality of sub-tiles in the second two-dimensional array, each sub-tile corresponding to at least a portion of a tile in the first two-dimensional array, wherein within each sub-tile a variance among pixel intensity values is less than the first predetermined value, and for at least one of the sub-tiles, the variance between an intensity value for a pixel in the sub-tile of the second two-dimensional array and the single pixel intensity value for the corresponding tile in the first two-dimensional array is greater than a second predetermined value.
In a system for encoding image frames, each frame including an ordered, two-dimensional array of pixels, each pixel including an intensity value, a combination comprising:
- a camera;
- a frame buffer connected to the camera for storing a succession of pixel arrays;
- a still tile generator coupled to the frame buffer for producing a compressed representation of a first frame, the compressed representation including a plurality of tiles, each tile corresponding to a rectangular region of the first frame, each tile represented by a coded representation which identifies the tile and includes an assigned single pixel intensity value for all pixels in the tile;
- a motion tile generator coupled to the frame buffer and to the still tile generator for producing a compressed representation of change occurring between the first frame and a second frame, the compressed representation of change including one or more sub-tiles, each sub-tile corresponding to at least a portion of a tile of the first frame which is changed in the second frame, each sub-tile represented by a coded representation which identifies the sub-tile and includes an assigned single pixel intensity value for all pixels in the sub-tile;
- a transmission encoder coupled to the still tile generator and to the motion tile generator for generating encoded representations of tiles and sub-tile;
- a transmission path connected to the transmission encoder;
- a transmission decoder connected to the transmission path for decoding tiles and sub-tiles from the encoded transmission representations;
- and
- a tile decoder coupled to the transmission decoder for reproducing the first frame in response to coded representations of tiles and for reproducing the second frame in response to coded representations of tiles and coded representations of subtiles.
In an imaging system including a display for displaying a scanned video image in response to a succession of frames in which each frame includes a scanned array of pixels which have respective pixel intensity values, and a decoder coupled to the display for generating the succession of frames in response to encoded frame representations, an improvement comprising a means for providing:
- a first encoded frame representation which includes a first sequence of tuples of the form {dx.sub.i,dy.sub.i,dz.sub.i,} in which each tuple represents a rectangular region of pixels in a first frame, wherein:
- dx.sub.i represents an x-dimension of the ith rectangular region;
- dy.sub.i represents a y-dimension of the ith rectangular region;
- and
- dz.sub.i is a pixel intensity value assigned to all pixels of the ith rectangular region;
- and
- one or more second encoded frame representations, each representing a respective second frame following the first frame and including a second sequence including coding symbols and tuples, wherein:
- a first coding symbol occupies locations in the second sequence which correspond to locations of tuples in the first sequence, the first symbol signifying that a rectangular region in the first frame represented by a tuple in the first sequence is identical to the corresponding rectangular region in the second frame;
- a second coding symbol occupies locations in the second sequence which correspond to locations of tuples in the first sequence, the second symbol signifying that a rectangular region in the first frame represented by a tuple in the first sequence differs from the corresponding rectangular region in the second frame;
- and
- at least one tuple follows each second coding symbol, each tuple representing a rectangular region of pixels in the second frame.
A system for encoding a moving image which is represented by a succession of arrays of pixels values, the system comprising:
- a dictionary for storing a plurality of value sets;
- a first tiling unit for partitioning pixel values of a first two-dimensional array into tiles, wherein within each tile, a variance among pixel values is less than a first predetermined value, and the first tiling unit writes into the dictionary a value set for each tile of the first array, each value set indicating a region which corresponds to the tile and an intensity value for the tile;
- and
- a second tiling unit for producing a representation of change occurring between the first array and a second array of pixels, the second tiling unit comprising:
- means for identifying in the second array a changed region which contains a pixel value that differs by more than a second predetermined value from the intensity value indicated by a value set which is stored in the dictionary and indicates the region;
- and
- means for partitioning a changed region of the second array into one or more sub-tiles, wherein within each sub-tile, a variance among pixel values is less than a third predetermined value.
Dependencies are inferred from the claim text.