Encoding for remoting graphics to decoder device

In a server computer, a computer-readable storage medium having instructions that, when executed on the server computer, performs acts comprising:

• connecting a graphics encoder with the one or more protocol decoder devices through a communication protocol;
• informing the application program to begin sending higher-level graphics commands from the application program to the graphics encoder;
• sending the higher-level graphics commands from the application program to the graphics encoder;
• using an application program interface (API) to provide information to and access the higher-level graphics commands from the application program, wherein the high level graphics commands from the application program are used to generate and display graphics to a local display device, and the higher-level graphics commands are particularly used by a local graphics driver to generate graphics displayed on the local display device;
• extracting the higher-level graphics commands using a screen scraping engine and a protocol encoder, wherein the screen scraping engine and protocol encoder encode the higher-level graphics commands into a format as defined by the communication protocol
• receiving input commands from the one or more protocol decoder devices, wherein the input commands represent an action or instruction to be performed on the application program from which higher-level graphics commands are received;
• and
• sending the higher-level graphics commands to the protocol decoder devices, wherein the higher-level graphics commands are formatted to support the communication protocol and sent through a stream-oriented connection by protocol compliant state machine(s) supporting each of the protocol decoder devices.