Back to results
US 8,804,821 B2Granted
Adaptive video processing of an interactive environment
AssigneeMicrosoft Corporation
InventorWalid Ali
Life of this patent
Estimating the term…
Abstract
Embodiments are configured to provide interactive communication functionality including adaptive video processing functionality that can be used to process aspects of a video signal, but the embodiments are not so limited. In an embodiment, components of a video conferencing system can operate to provide a video signal based in part on the use of adaptive processing features which include scaling and/or other pixel processing features. In one embodiment, components of an interactive video system can operate to adaptively manage and control video payload parameters to adapt to various communication conditions associated with a real-time or near-real time interactive video environment.
Claims20
A video communication system comprising communication components operable to:
- communicate a communication signal over a communication path, wherein the communication signal includes pixel data associated with an interactive video environment;
- determine a condition of a communication architecture associated with the interactive video environment, including determining if a packet loss ratio is acceptable;
- adjust one or more processing operations, including adjustment of a rate control operation and a scaling operation that includes using rate control feedback to determine a scaling factor parameter to use as part of a scaling operation, wherein the system is further operable to increase a video processing bandwidth, decrease the scaling factor parameter, and increase a spatial resolution if the packet loss ratio is acceptable, or reduce the video processing bandwidth, increase the scaling factor parameter, and reduce the spatial resolution if the packet loss ratio is not acceptable, if the condition of the communication architecture warrants an adjustment;
- provide adjustment information to one or more components of the video communication system;
- use aspects of pixel data associated with a first frame of pixel data in conjunction with aspects of pixel data associated with a second frame of pixel data to reconstruct frames of pixel data including using motion vector information to determine weighting factors to apply when calculating pixel values for a reconstructed frame, wherein the weighting factors are applied to a first number of pixel values associated with the first frame and a second number of pixel values associated with the second frame;
- and,
- continue to communicate the communication signal.
4 more elements
A video conferencing device comprising:
- a video capture device operable to provide a captured signal associated with a video conferencing environment, the captured signal including a frame of pixel data representative of a video scene;
- a pre-processor component operable to process the captured signal to provide a pre-processed signal, the pre-processed signal including scaled pixel data associated with a first scaling operation;
- a post-processor component operable to process a decoded signal to provide a post-processed signal, the post-processed signal including scaled pixel data associated with a second scaling operation, wherein the post-processor component operates to reconstruct a frame of pixel data to provide a reconstructed frame using aspects of pixel data associated with a first frame of pixel data in conjunction with aspects of pixel data associated with a second frame of pixel data including using motion vector information to determine weighting factors to apply when calculating pixel values for a reconstructed frame, wherein the weighting factors are applied to a first number of pixel values associated with the first frame and a second number of pixel values associated with the second frame;
- a feedback loop to communicate information between the pre-processor component and an encoder component, wherein scaling operations and rate control operations can be controlled using the information communicated using the feedback loop including operations to increase a video processing bandwidth, decrease a scaling factor parameter, and increase a spatial resolution if a packet loss ratio is acceptable, or reduce the video processing bandwidth, increase the scaling factor parameter, and reduce the spatial resolution if the packet loss ratio is not acceptable, wherein the feedback loop uses rate control feedback to determine the scaling factor parameter to use as part of a scaling operation;
- and,
- a display to display a post-processed frame of pixel data.
3 more elements
A method of providing a video signal comprising:
- determining a packet loss ratio associated with a real-time communication of pixel data;
- using rate control feedback to determine a scaling factor to use as part of a scaling operation;
- increasing a scaling factor to scale the pixel data if the packet loss ratio is not within an acceptable range of packet loss ratios;
- adjusting a number of pixel processing operations if the packet loss ratio is not within an acceptable range of packet loss ratios including reducing a video processing bandwidth and reducing a spatial resolution;
- increasing the video processing bandwidth, decreasing the scaling factor, and increasing a spatial resolution if the packet loss ratio is within the acceptable range of packet loss ratios;
- communicating adjustment information to one or more components of a video conferencing system;
- appending a number of protection packets to a video payload that includes the pixel data to maintain a QOS level;
- and,
- using aspects of pixel data associated with a first frame of pixel data in conjunction with aspects of pixel data associated with a second frame of pixel data to reconstruct a video frame including using motion vector information to determine weighting factors to apply when calculating pixel values for a reconstructed frame, wherein the weighting factors are applied to a first number of pixel values associated with the first frame and a second number of pixel values associated with the second frame.
6 more elements
Dependencies are inferred from the claim text.