# US 2020/0004488 A1 · On-gpu compositioning of graphical data from multiple domains

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0004488 A1
- Document ID: US20200004488A1
- Status: Published application
- Kind: A1
- Application number: 16/019,978
- Filed: 2018-06-27
- Published: 2020-01-02
- Earliest claimed: 2018-06-27
- Assignees: Assured Information Security, Inc.
- Applicants: Assured Information Security, Inc.
- Inventors: Brendan Kerrigan; Maurice Gale; Rian Quinn; Rodney Forbes; Sandy Stutsman; Jonathan Farrell
- Web page: https://patentgrep.com/patent/US20200004488A1

## CPC

- G06F 9/451: Execution arrangements for user interfaces
- G06F 3/1431: {using a single graphics controller}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G06T 11/60: Creating or editing images; Combining images with text
- G09G 5/14: Display of multiple viewports
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06F 2009/45587: {Isolation or security of virtual machine instances}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A method provides a graphical interface for a computer system and includes receiving window information from each domain of multiple domains in which applications execute. Based on the received window information, the method builds the graphical interface on a graphics device of the computer system from graphics data provided from the multiple domains to the graphics device. The graphics device includes a GPU and graphics memory having multiple graphics memory portions, where each domain is dedicated a respective different graphics memory portion and is given write access thereto. The building issues commands to the graphics device that instruct the GPU to composition together graphics data from graphics memory portion(s) to thereby composition together graphics data from each of two of more domains of the multiple domains. The method also includes issuing commands to the graphics device to output the graphical interface to a set of one or more display devices.

## Claims

### Claim 1

A computer-implemented method of providing a graphical interface for a computer system, the method comprising:
receiving window information from each domain of multiple domains in which applications execute, the multiple domains executing as separate execution environments for executing their respective applications, the window information from each domain informing of window position and size for a respective one or more applications executing in the domain;
based at least on the received window information, building the graphical interface on a graphics device of the computer system from graphics data provided from the multiple domains to the graphics device, the graphics device comprising a graphics processing unit and graphics memory having multiple graphics memory portions, wherein each domain of the multiple domains is dedicated a respective different graphics memory portion of the multiple graphics memory portions and is given write access thereto, wherein the building comprises issuing commands to the graphics device that instruct the graphics processing unit to composition together at least some graphics data from each of two or more graphics memory portions of the multiple graphics memory portions to thereby composition together graphics data from each of two of more domains of the multiple domains, wherein the receiving and the building are performed by a display domain executing as another execution environment separate from the execution environments of the multiple domains, wherein the display domain is also dedicated a graphics memory portion of the graphics device memory, and wherein each domain of the multiple domains is restricted from writing data to every graphics memory portion, of the multiple graphics memory portions, that is other than the graphics memory portion dedicated to the domain; and
issuing commands to the graphics device to output the graphical interface to a set of one or more display devices.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The method of claim 1, wherein the commands issued as part of the building are issued by the display domain and instruct the graphics processing unit to copy the at least some graphics data from the two or more graphics memory portions to the graphics memory portion dedicated to the display domain and arrange the at least some graphics data based on the received window information, the copying providing the graphical interface in the graphics memory portion dedicated to the display domain, and wherein the graphical interface is rendered for output to the set of one or more display devices from the graphics memory portion dedicated to the display domain.

### Claim 4 (depends on 3)

The method of claim 3, wherein each of the multiple graphics memory portions comprises a texture memory, wherein the graphics data provided from each domain of the multiple domains to the graphics device comprises texture data provided to the texture memory of the graphics memory portion dedicated to the domain, wherein the copied at least some graphics data is texture data copied from the texture memories of the two or more graphics memory portions to a texture memory of the graphics memory portion dedicated to the display domain, and wherein the graphical interface comprises the texture data rendered from the texture memory of the graphics memory portion dedicated to the display domain.

### Claim 5 (depends on 4)

The method of claim 4, wherein the building the graphics interface further comprises the display domain accessing, with read-only access, the texture memories of the two or more graphics memory portions.

### Claim 6 (depends on 1)

The method of claim 1, wherein the graphics data provided from the multiple domains to the graphics device is provided absent the multiple domains providing that graphics data to the display domain.

### Claim 7 (depends on 1)

The method of claim 1, wherein the graphics device is a single graphics card of the computer system.

### Claim 8 (depends on 1)

The method of claim 1, wherein the at least some graphics data from each domain of the two or more domains comprises graphical user interface (GUI) windows of applications executing on the domain, and wherein the built graphics interface comprises a collection of GUI windows from the two of more domains, at least a portion of at least one desktop background of a desktop of a domain of the two or more domains, a taskbar of the domain of the two or more domains, and added graphics data added by the building to graphically correlate which GUI windows of the collection of GUI windows are from which domains of the two or more domains.

### Claim 9 (depends on 1)

The method of claim 1, wherein the at least two domains comprise at least three domains, wherein the set of one or more display devices comprises a first display device and a group or two or more display devices, and wherein the commands build the graphical interface to produce a first graphics output, to the first display device, comprising graphics data from a first domain of the at least three domains, and a second graphics output, to the group of two or more display devices, comprising graphics data from second and third domains of the at least three domains.

### Claim 10 (depends on 1)

The method of claim 1, wherein the building is in accordance with a fingerprint for the one or more display devices, the fingerprint identifying a configuration of respective portions of the graphical interface to display on each display device of the one or more display devices, wherein the output of the graphical interface to the set of one or more display devices outputs, to each display device of the one or more display devices, a respective graphics output that corresponds to the portion of the graphical interface to display on that display as indicated by the configuration identified by the fingerprint.

### Claim 11

A computer system configured to perform a method of providing a graphical interface, the method comprising:
receiving window information from each domain of multiple domains in which applications execute, the multiple domains executing as separate execution environments for executing their respective applications, the window information from each domain informing of window position and size for a respective one or more applications executing in the domain;
based at least on the received window information, building the graphical interface on a graphics device of the computer system from graphics data provided from the multiple domains to the graphics device, the graphics device comprising a graphics processing unit and graphics memory having multiple graphics memory portions, wherein each domain of the multiple domains is dedicated a respective different graphics memory portion of the multiple graphics memory portions and is given write access thereto, wherein the building comprises issuing commands to the graphics device that instruct the graphics processing unit to composition together at least some graphics data from each of two or more graphics memory portions of the multiple graphics memory portions to thereby composition together graphics data from each of two of more domains of the multiple domains, wherein the receiving and the building are performed by a display domain executing as another execution environment separate from the execution environments of the multiple domains, wherein the display domain is also dedicated a graphics memory portion of the graphics device memory, and wherein each domain of the multiple domains is restricted from writing data to every graphics memory portion, of the multiple graphics memory portions, that is other than the graphics memory portion dedicated to the domain; and
issuing commands to the graphics device to output the graphical interface to a set of one or more display devices.

### Claim 12 (depends on 11)

The computer system of claim 11, wherein the commands issued as part of the building are issued by the display domain and instruct the graphics processing unit to copy the at least some graphics data from the two or more graphics memory portions to the graphics memory portion dedicated to the display domain and arrange the at least some graphics data based on the received window information, the copying providing the graphical interface in the graphics memory portion dedicated to the display domain, and wherein the graphical interface is rendered for output to the set of one or more display devices from the graphics memory portion dedicated to the display domain.

### Claim 13 (depends on 12)

The computer system of claim 12, wherein each of the multiple graphics memory portions comprises a texture memory, wherein the graphics data provided from each domain of the multiple domains to the graphics device comprises texture data provided to the texture memory of the graphics memory portion dedicated to the domain, wherein the copied at least some graphics data is texture data copied from the texture memories of the two or more graphics memory portions to a texture memory of the graphics memory portion dedicated to the display domain, wherein the graphical interface comprises the texture data rendered from the texture memory of the graphics memory portion dedicated to the display domain, and wherein the building the graphics interface further comprises the display domain accessing, with read-only access, the texture memories of the two or more graphics memory portions.

### Claim 14 (depends on 11)

The computer system of claim 11, wherein the at least two domains comprise at least three domains, wherein the set of one or more display devices comprises a first display device and a group or two or more display devices, and wherein the commands build the graphical interface to produce a first graphics output, to the first display device, comprising graphics data from a first domain of the at least three domains, and a second graphics output, to the group of two or more display devices, comprising graphics data from second and third domains of the at least three domains.

### Claim 15 (depends on 11)

The computer system of claim 11, wherein the building is in accordance with a fingerprint for the one or more display devices, the fingerprint identifying a configuration of respective portions of the graphical interface to display on each display device of the one or more display devices, wherein the output of the graphical interface to the set of one or more display devices outputs, to each display device of the one or more display devices, a respective graphics output that corresponds to the portion of the graphical interface to display on that display as indicated by the configuration identified by the fingerprint.

### Claim 16

A computer program product comprising:
a non-transitory computer readable storage medium storing instructions for execution to perform a method of providing a graphical interface for a computer system, the method comprising:
receiving window information from each domain of multiple domains in which applications execute, the multiple domains executing as separate execution environments for executing their respective applications, the window information from each domain informing of window position and size for a respective one or more applications executing in the domain;
based at least on the received window information, building the graphical interface on a graphics device of the computer system from graphics data provided from the multiple domains to the graphics device, the graphics device comprising a graphics processing unit and graphics memory having multiple graphics memory portions, wherein each domain of the multiple domains is dedicated a respective different graphics memory portion of the multiple graphics memory portions and is given write access thereto, wherein the building comprises issuing commands to the graphics device that instruct the graphics processing unit to composition together at least some graphics data from each of two or more graphics memory portions of the multiple graphics memory portions to thereby composition together graphics data from each of two of more domains of the multiple domains, wherein the receiving and the building are performed by a display domain executing as another execution environment separate from the execution environments of the multiple domains, wherein the display domain is also dedicated a graphics memory portion of the graphics device memory, and wherein each domain of the multiple domains is restricted from writing data to every graphics memory portion, of the multiple graphics memory portions, that is other than the graphics memory portion dedicated to the domain; and
issuing commands to the graphics device to output the graphical interface to a set of one or more display devices.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the commands issued as part of the building are issued by the display domain and instruct the graphics processing unit to copy the at least some graphics data from the two or more graphics memory portions to the graphics memory portion dedicated to the display domain and arrange the at least some graphics data based on the received window information, the copying providing the graphical interface in the graphics memory portion dedicated to the display domain, and wherein the graphical interface is rendered for output to the set of one or more display devices from the graphics memory portion dedicated to the display domain.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein each of the multiple graphics memory portions comprises a texture memory, wherein the graphics data provided from each domain of the multiple domains to the graphics device comprises texture data provided to the texture memory of the graphics memory portion dedicated to the domain, wherein the copied at least some graphics data is texture data copied from the texture memories of the two or more graphics memory portions to a texture memory of the graphics memory portion dedicated to the display domain, wherein the graphical interface comprises the texture data rendered from the texture memory of the graphics memory portion dedicated to the display domain, and wherein the building the graphics interface further comprises the display domain accessing, with read-only access, the texture memories of the two or more graphics memory portions.

### Claim 19 (depends on 16)

The computer program product of claim 16, wherein the at least two domains comprise at least three domains, wherein the set of one or more display devices comprises a first display device and a group or two or more display devices, and wherein the commands build the graphical interface to produce a first graphics output, to the first display device, comprising graphics data from a first domain of the at least three domains, and a second graphics output, to the group of two or more display devices, comprising graphics data from second and third domains of the at least three domains.

### Claim 20 (depends on 16)

The computer program product of claim 16, wherein the building is in accordance with a fingerprint for the one or more display devices, the fingerprint identifying a configuration of respective portions of the graphical interface to display on each display device of the one or more display devices, wherein the output of the graphical interface to the set of one or more display devices outputs, to each display device of the one or more display devices, a respective graphics output that corresponds to the portion of the graphical interface to display on that display as indicated by the configuration identified by the fingerprint.
