Low-bandwidth image streaming
AssigneeSeiko Epson Corporation
InventorsAnoop K Bhattacharjya, Victor Ivashin, Kar-Han Tan
Life of this patent
Estimating the term…
Abstract
Methods and systems are disclosed for processing image frames to reduce the bandwidth requirements. Embodiment of the present invention may include mode-specific image frame rendering in photorealistic and non-photorealistic modes, such as outline and cartoon modes. In embodiments, update regions may be identified and reduced by an edge position mask. In embodiments, update regions may be bounded by rectangles and such regions may be reduced in number by merging regions together using various no-cost or cost approaches. To improve compressibility, regions to be transmitted that do not require updating at the receiver may be encoded as transparent.
Claims20
A method for improving bandwidth required to transmit at least a portion of a video stream comprising a plurality of image frames, each image frame comprising a set of pixels, the method comprising:
- using an update regions finder for:
- identifying a set of update pixels in a rendered image frame derived from an image frame of the video stream by comparing pixels in the rendered image frame with corresponding pixels in a stored image frame, the stored image frame representing a current state of an image frame at a receiver;
- using an edge position mask derived from edge pixels in the stored image frame to generate a reduced set of update pixels by removing from the set of update pixels any pixels that exist in the edge position mask;
- forming a plurality of update rectangles wherein each update rectangle comprises at least a portion of the reduced set of update pixels;
- and
- responsive to being able to improve the bandwidth required to transmit the plurality of update rectangles by merging at least two update rectangles.
A method for improving bandwidth required to transmit at least a portion of a video stream comprising a plurality of image frames, each image frame comprising a set of pixels, the method comprising:
- using a temporal filter for temporally filtering the set of pixels of an image frame of the video stream across at least some of the plurality of image frames to create a temporally filtered image frame;
- applying a palettizer to an image frame derived from the temporally filtered image frame to obtain a rendered image frame;
- using an update regions for:
- identifying a set of update pixels in the rendered image frame by comparing pixels in the rendered image frame with corresponding pixels in a stored image frame, the stored image frame representing a current state of an image frame at a receiver;
- using an edge position mask derived from edge pixels in the stored image frame to generate a reduced set of update pixels by removing from the set of update pixels any pixels that exist in the edge position mask;
- forming a plurality of update rectangles wherein each update rectangle comprises at least a portion of the reduced set of update pixels;
- responsive to being able to improve the bandwidth required to transmit the plurality of update rectangles by merging at least two update rectangles;
- and
- responsive to an update rectangle comprising a set of non-update pixels, encoding each pixel in the set of non-update pixels with a transparent color value.
A system for improving bandwidth required to transmit at least a portion of a video stream comprising a plurality of image frames, each image frame comprising a set of pixels, the system comprising:
- a temporal filter, communicatively coupled to receive the video stream, that temporally filters an image frame in the video stream into a temporally filtered image frame;
- a palettizer, communicatively coupled to receive a rendered image frame derived from the temporally filtered image frame, that maps color values of pixels in the rendered image frame to a discrete number of colors to form a palettized image frame;
- a palettized image frame buffer, communicatively coupled to the palettizer, that receives the palettized image frame from the palettizer;
- a received image frame buffer that contains a stored image frame representing a current state of an image frame at a receiver;
- a position mask computer, communicatively coupled to the received image frame buffer, that derives an edge position mask from edge pixel positions in the stored image frame;
- an update regions finder, communicatively coupled with the received image frame buffer and the position mask computer, that identifies a set of update pixels in the palettized image frame by comparing pixels in the palettized image frame with corresponding pixels in a stored image frame, that uses the edge position mask to generate a reduced set of update pixels by removing from the set of update pixels any pixels that exist in the edge position mask, and that forms a plurality of update rectangles wherein each update rectangle comprises at least a portion of the reduced set of update pixels;
- and
- a rectangle packer, coupled to update regions finder, that, responsive to being able to improve the bandwidth required to transmit the plurality of update rectangles by merging at least two update rectangles.
Dependencies are inferred from the claim text.