Systems and methods for using screen sampling to detect display changes
AssigneeVMware, Inc.
InventorJonathan Clark
Life of this patent
Estimating the term…
Abstract
A graphical user interface (GUI) is polled to identify updates for transmission to a client device. A generalized poll of the GUI is conducted at a first frequency, wherein the poll compares color values of each pixel in a general region of the GUI with a previously known color value for that pixel to detect changed pixel color values. A local region that is a subset of the general region is determined to have a greater likelihood of being updated, e.g., regions encompassing a location of mouse and/or keyboard activity. A localized poll is conducted at a second frequency of only the local region of the GUI.
Claims17
A method, comprising:
- generating, by a server device, a graphical user interface (GUI) for remoting to a client device over a network connection;
- polling, at the server device, a first portion of the GUI at a first frequency to detect one or more display updates to the GUI that are to be transmitted to the client device;
- and
- polling, at the server device, a second portion of the GUI at a second frequency to detect the one or more display updates to the GUI that are to be transmitted to the client device, wherein the second frequency is different from the first frequency;
- wherein polling the first portion of the GUI and polling the second portion of the GUI is performed by a sampling agent configured to:
- read a display buffer containing a representation of the GUI by accessing one or more application programming interfaces (APIs) of a guest operating system (GOS) on a virtual machine (VM) executing on the server device;
- and
- compare a color value of a pixel in the first portion of the GUI of a current frame in the display buffer with the color value of said pixel of a previous frame in the display buffer to detect changed pixel color values, wherein the changed pixel values indicate the one or more display updates to the GUI.
A server computing device, comprising:
- at least one processor;
- and
- memory including instructions that, when executed by the at least one processor, cause the server computing device to perform the operations of:
- generating a graphical user interface (GUI) for remoting to a client device over a network connection;
- polling a first portion of the GUI at a first frequency to detect one or more display updates to the GUI that are to be transmitted to the client device;
- and
- polling a second portion of the GUI at a second frequency to detect the one or more display updates to the GUI that are to be transmitted to the client device, wherein the second frequency is different from the first frequency;
- wherein polling the first portion of the GUI and polling the second portion of the GUI is performed by a sampling agent configured to:
- read a display buffer containing a representation of the GUI by accessing one or more application programming interfaces (APIs) of a guest operating system (GOS) on a virtual machine (VM) executing on the server computing device;
- and
- compare a color value of a pixel in the first portion of the GUI of a current frame in the display buffer with the color value of said pixel of a previous frame in the display buffer to detect changed pixel color values, wherein the changed pixel values indicate the one or more display updates to the GUI.
A non-transitory computer readable storage medium comprising one or more sequences of instructions, the instructions when executed by one or more processors causing the one or more processors to execute the operations of:
- generating, by a server device, a graphical user interface (GUI) for remoting to a client device over a network connection;
- polling, at the server device, a first portion of the GUI at a first frequency to detect one or more display updates to the GUI that are to be transmitted to the client device;
- and
- polling, at the server device, a second portion of the GUI at a second frequency to detect the one or more display updates to the GUI that are to be transmitted to the client device, wherein the second frequency is different from the first frequency;
- wherein polling the first portion of the GUI and polling the second portion of the GUI is performed by a sampling agent configured to:
- read a display buffer containing a representation of the GUI by accessing one or more application programming interfaces (APIs) of a guest operating system (GOS) on a virtual machine (VM) executing on the server device;
- and
- compare a color value of a pixel in the first portion of the GUI of a current frame in the display buffer with the color value of said pixel of a previous frame in the display buffer to detect changed pixel color values, wherein the changed pixel values indicate the one or more display updates to the GUI.
Dependencies are inferred from the claim text.