# US 8,570,441 B2 · One pass video processing and composition for high-definition video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,570,441 B2 - Document ID: US8570441B2 - Status: Granted - Kind: B2 - Application number: 12/136,771 - Filed: 2008-06-11 - Published: 2009-12-17 - Granted: 2013-10-29 - Earliest claimed: 2008-06-11 - Assignees: Microsoft Corporation - Applicants: Kazuyuki Nakayama - Inventors: Kazuyuki Nakayama - Web page: https://patentgrep.com/patent/US8570441B2 ## CPC - H04N 9/76: for mixing of colour signals (H04N9/75 takes precedence) - H04N 7/013: {the incoming video signal comprising different parts having originally different frame rate, e.g. video and graphics} - G09G 5/006: {Details of the interface to the display terminal (specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04)} - H04N 7/0112: {one of the standards corresponding to a cinematograph film standard} - H04N 7/012: {Conversion between an interlaced and a progressive signal (for graphics images G09G2310/0229)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract A video composition model that provides a set of application programming interfaces (APIs) to set device contexts, and determine capabilities of graphics hardware from a device driver. After the model determines a configuration, the model determines input video stream states applicable to frame rates, color-spaces and alpha indexing of input video streams, interactive graphics, and background images. The model prepares the input video frames and reference frames, as well as a frame format and input/output frame index information. The input video streams, interactive graphics and background images are processed individually and mixed to generate an output video stream. ## Claims ### Claim 1 A method comprising: receiving a plurality of input video streams; establishing, by a computing device, a frame rate converter based on input/output frame index information for the plurality of input video streams that is based on capabilities of a graphics device; processing the plurality of input video streams resulting in a processed plurality of input video streams, the processing including: de-interlacing, by the established frame rate converter, any interlaced frames of the plurality of input video streams, and non-linearly scaling a resolution and aspect-ratio of each of the plurality of input video streams to an output size supported by a display device; blending, in a single stage and by per-plane and per-pixel alpha blending, the processed plurality of input video streams resulting in an output video stream; linearly scaling the output video stream to the resolution supported by the display device resulting in a scaled output video stream; and outputting the scaled output video stream to the display device. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving alpha channel data on a first video stream; receiving background color data on a second video stream; and receiving video content on third video stream. ### Claim 3 (depends on 2) The method of claim 2, further comprising: receiving a specification of an output surface from a video application; and performing the processing on a per-plane and per-pixel alpha basis for the second video stream and third video stream in accordance with the specification. ### Claim 4 (depends on 1) The method of claim 1, further comprising reducing a quality of the output video stream. ### Claim 5 (depends on 1) The method of claim 1, the processing further comprising performing a filtering operation, a de-interlace operation, a frame rate conversion operation, a color-space conversion operation, or a scaling operation. ### Claim 6 (depends on 1) The method of claim 1, further comprising applying a common color-space to the plurality of input video streams. ### Claim 7 (depends on 1) The method of claim 1, further comprising applying a common color-space to input interactive graphics. ### Claim 8 A system, comprising: a computing device; a graphics kernel configured for exposing application programming interfaces (APIs) of a graphics device; a video input stage configured for receiving a plurality of input video streams; a pre-processing stage implemented at least in part by the computing device and configured for establishing a frame rate converter based on input/output frame index information for the plurality of input video streams that is based on capabilities of the graphics device; a processing stage implemented at least in part by the computing device and configured for performing processing on the plurality of input video streams resulting in a processed plurality of input video streams, the processing including: de-interlacing, by the established frame rate converter, any interlaced frames of the plurality of input video streams, and non-linearly scaling a resolution and aspect-ratio of each of the plurality of input video streams to an output size supported by a display device; a mixing stage implemented at least in part by the computing device and configured for performing blending, in a single stage and by per-plane and per-pixel alpha blending, the processed plurality of input video streams resulting in an output video stream; a final scaling stage implemented at least in part by the computing device and configured for performing linear scaling of the output video stream to a resolution supported by the display device resulting in a scaled output video stream; and an output stage implemented at least in part by the computing device and configured for performing outputting of the scaled output video stream to the display device. ### Claim 9 (depends on 8) The system of claim 8, further comprising reducing a quality of the output video stream. ### Claim 10 (depends on 8) The system of claim 8, wherein at least two of the plurality of input video streams have differing frame rates. ### Claim 11 (depends on 8) The system of claim 8, wherein the output video stream has a frame rate determined in accordance with the display device operatively connected to the output stage. ### Claim 12 (depends on 8) The system of claim 8, wherein at least two of the plurality of input video streams have differing color-space definitions. ### Claim 13 (depends on 8) The system of claim 8, wherein each of the plurality of input stream is processed in accordance with API calls made by a video application to the graphics kernel independently of other input video streams. ### Claim 14 (depends on 8) The system of claim 8, wherein the processing stage comprises at least one stage configured for performing a filtering operation, a de-interlace operation, a frame rate conversion operation, a color-space conversion operation, or a scaling operation. ### Claim 15 At least one computer storage device storing computer-executable instructions that, when executed by a computing device, cause the computing device to perform a method for individually processing a plurality of input video streams, the method comprising: receiving the plurality of input video streams; establishing a frame rate converter based on input/output frame index information for the plurality of input video streams that is based on capabilities of a graphics device; processing the plurality of input video streams resulting in a processed plurality of input video streams, the processing including: de-interlacing, by the established frame rate converter, any interlaced frames of the plurality of input video streams, and non-linearly scaling a resolution and aspect-ratio of each of the plurality of input video streams to an output size supported by a display device; blending, in a single stage and by per-plane and per-pixel alpha blending, the processed plurality of input video streams resulting in an output video stream; linearly scaling the output video stream to a resolution supported by the display device resulting in a scaled output video stream; and outputting the scaled output video stream to the display device. ### Claim 16 (depends on 15) The at least one computer storage device of claim 15, the method further comprising: receiving a specification of the display device from a video application; and performing the processing on a per-plane and per-pixel alpha basis for each of a first input video stream, a second input video stream, and a third input video stream in accordance with the specification. ### Claim 17 (depends on 15) The at least one computer storage device of claim 15, the method further comprising reducing a quality of the output video stream. ### Claim 18 (depends on 15) The at least one computer storage device of claim 15, the method further comprising performing a filtering operation, a de-interlace operation, a frame rate conversion operation, a color-space conversion operation, or a scaling operation.