# US 11,110,348 B2 · Memory management in gaming rendering > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,110,348 B2 - Document ID: US11110348B2 - Status: Granted - Kind: B2 - Application number: 16/380,831 - Filed: 2019-04-10 - Published: 2019-10-10 - Granted: 2021-09-07 - Earliest claimed: 2018-04-10 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Paul Lalonde; Paul Leventis; Jean-Francois Roy - Web page: https://patentgrep.com/patent/US11110348B2 ## CPC - A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client - A63F 13/352: involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world - A63F 13/48: Starting a game, e.g. activating a game device or waiting for other players to join a multiplayer session - A63F 13/52: involving aspects of the displayed game scene - G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)} - G06F 12/0811: with multilevel cache hierarchies - G06F 12/0888: using selective caching, e.g. bypass - G06F 12/123: with age lists, e.g. queue, most recently used [MRU] list or least recently used [LRU] list - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 1/60: Memory management - G06T 15/005: {General purpose rendering architectures} - H04L 67/131: Protocols for games, networked simulations or virtual reality - A63F 2300/203: Image generating hardware - A63F 2300/538: for performing operations on behalf of the game client, e.g. rendering - A63F 2300/5593: involving scheduling aspects - G06F 12/023: {Free address space management} - G06F 12/08: in hierarchically structured memory systems, e.g. virtual memory systems - G06F 2212/1024: Latency reduction - G06F 2212/1028: Power efficiency - G06F 2212/1048: Scalability - G06F 2212/154: Networked environment - G06F 2212/205: Hybrid memory, e.g. using both volatile and non-volatile memory - G06F 2212/455: Image or video data - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract Some implementations of this application are directed to a server system including one or more CPUs, a plurality of GPUs, main dynamic memory storing programs and data for use by the CPUs and/or GPUs during program execution, a static memory pool stored in a non-volatile memory, and a memory controller configured to manage the static memory pool. Each of the GPUs includes a local cache and is configured to access the static memory pool via the memory controller. The server system executes a plurality of gaming sessions for a gaming title in parallel on the one or more CPUs. Each of the plurality of gaming sessions is associated with a static data item stored in the static memory pool, and requires a graphics operation executable by a respective GPU using the static data item. ## Claims ### Claim 1 A method of rendering image frames for online gaming sessions, comprising: executing on one or more CPUs a plurality of online gaming sessions in parallel for a gaming title; rendering by one or more GPUs a plurality of image frames for each of a plurality of online gaming sessions using a first sequence of compiled shaders; identifying a performance criterion for the plurality of online gaming sessions associated with the gaming title; collecting real-time usage statistics characterizing execution of each of the online gaming sessions; and based on the real-time usage statistics, modifying a first shader in the first sequence of compiled shaders to improve performance of the first sequence of compiled shaders with respect to the performance criterion for the plurality of online gaming sessions, thereby creating a modified first sequence of compiled shaders; establishing a shader hash table that associates each of a plurality of performance criteria with a respective sequence of compiled shaders, including associating the performance criterion and the first sequence of compiled shaders in the shader hash table; and storing the shader hash table in a shader cache of a memory. ### Claim 2 (depends on 1) The method of claim 1, further comprising: replacing the first sequence of compiled shaders stored in a shader cache of the memory with the modified first sequence of compiled shaders. ### Claim 3 (depends on 1) The method of claim 1, further comprising: storing in a shader cache of the memory the first sequence of compiled shaders as a first version of the first sequence of compiled shaders; storing in the shader cache the modified first sequence of compiled shaders as a second version of the first sequence of compiled shaders; and collecting subsequent usage statistics characterizing execution of each of the online gaming sessions; and in accordance with the subsequent usage statistics, selecting one of the first and second versions of the first sequence of compiled shaders to render the plurality of image frames for each of the plurality of online gaming sessions. ### Claim 4 (depends on 1) The method of claim 1, wherein modifying the first shader in the first sequence of compiled shaders further comprises: selecting an alternative shader from a group of alternative shader options to replace the first shader in the first sequence of compiled shaders, such that performance of the plurality of online gaming sessions satisfies the performance criterion, wherein the modified first sequence of compiled shaders includes the alternative shader. ### Claim 5 (depends on 1) The method of claim 1, further comprising: receiving a user instruction to modify the first shader in the first sequence of compiled shaders, wherein the first shader is modified in the first sequence of compiled shaders in response to the user instruction. ### Claim 6 (depends on 1) The method of claim 1, wherein the first shader in the first sequence of compiled shaders is modified automatically and without user intervention to improve performance of the first sequence of compiled shaders with respect to the performance criterion. ### Claim 7 (depends on 1) The method of claim 1, wherein creating the modified first sequence of compiled shaders further comprises: modifying a second shader in the first sequence of compiled shaders to improve performance of the first sequence of compiled shaders with respect to the performance criterion. ### Claim 8 (depends on 1) The method of claim 1, further comprising: determining that the plurality of online gaming sessions need to be rendered in accordance with the performance criterion; and identifying the first sequence of compiled shaders for rending the plurality of image frames for each of the plurality of online gaming sessions. ### Claim 9 (depends on 1) The method of claim 1, wherein the performance criterion includes a first performance criterion, and the plurality of online gaming sessions are rendered and executed in accordance with a second performance criterion distinct from the first criterion, further comprising: associating the first performance criterion with the modified first sequence of compiled shaders; and associating the second performance criterion with the first sequence of compiled shaders. ### Claim 10 A server system, comprising: one or more CPUs; one or more GPUs; and memory storing one or more programs for execution by the one or more CPUs and/or GPUs, the one or more programs comprising instructions for: executing on the one or more CPUs a plurality of online gaming sessions in parallel for a gaming title; rendering by the one or more GPUs a plurality of image frames for each of the plurality of online gaming sessions using a first sequence of compiled shaders; identifying a performance criterion for the plurality of online gaming sessions associated with the gaming title; collecting real-time usage statistics characterizing execution of each of the online gaming sessions; and based on the real-time usage statistics, modifying a first shader in the first sequence of compiled shaders to improve performance of the first sequence of compiled shaders with respect to the performance criterion for the plurality of online gaming sessions, thereby creating a modified first sequence of compiled shaders; establishing a shader hash table that associates each of a plurality of performance criteria with a respective sequence of compiled shaders, including associating the performance criterion and the first sequence of compiled shaders in the shader hash table; and storing the shader hash table in a shader cache of the memory. ### Claim 11 (depends on 10) The server system of claim 10, wherein the performance criterion includes a first performance criterion, further comprising: storing the modified first sequence of compiled shaders in association with the first performance criterion in a shader cache of the memory. ### Claim 12 (depends on 11) The server system of claim 11, further comprising: while executing the plurality of online gaming sessions: identifying a second performance criterion for the plurality of online gaming sessions associated with the gaming title; determining a second sequence of compiled shaders to improve performance of the first sequence of compiled shaders with respect to the second performance criterion; and storing in the shader cache the second sequence of compiled shaders in association with the second performance criterion. ### Claim 13 (depends on 12) The server system of claim 12, wherein the usage statistics includes a total power consumption of the plurality of online gaming sessions, and the first performance criterion and the second performance criterion optimize response times and power usage of the plurality of online gaming sessions, respectively, further comprising: in accordance with a determination that the total power consumption of the plurality of online gaming sessions exceeds a predefined power threshold, selecting the second sequence of compiled shaders and rendering the plurality of image frames for each of the plurality of online gaming sessions using the second sequence of compiled shaders. ### Claim 14 (depends on 10) The server system of claim 10, wherein the usage statistics include information of user inputs, further comprising: associating in the shader hash table a plurality of versions of the first sequence of compiled shaders with the information of user inputs. ### Claim 15 (depends on 10) The server system of claim 10, further comprising: dynamically updating the shader hash table according to the usage statistics, including replacing the first sequence of compiled shaders in the shader cache with the modified first sequence of compiled shaders. ### Claim 16 A non-transitory computer readable storage medium storing one or more programs configured for execution by one or more processors of a server system, the one or more programs comprising instructions for: executing on one or more CPUs a plurality of online gaming sessions in parallel for a gaming title; rendering by one or more GPUs a plurality of image frames for each of the plurality of online gaming sessions using a first sequence of compiled shaders; identifying a performance criterion for the plurality of online gaming sessions associated with the gaming title; collecting real-time usage statistics characterizing execution of each of the online gaming sessions; and based on the real-time usage statistics, modifying a first shader in the first sequence of compiled shaders to improve performance of the first sequence of compiled shaders with respect to the performance criterion for the plurality of online gaming sessions, thereby creating a modified first sequence of compiled shaders; establishing a shader hash table that associates each of a plurality of performance criteria with a respective sequence of compiled shaders, including associating the performance criterion and the first sequence of compiled shaders in the shader hash table; and storing the shader hash table in a shader cache of a memory. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein the usage statistics include at least one or all of user inputs, power consumption, one or more gaming latencies, code size, memory usage and cache usage of the first sequence of compiled shaders corresponding to each of the plurality of online gaming sessions. ### Claim 18 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein the performance criterion corresponds to one or more of total power consumption, a code size, an average latency, total cache usage, and total memory usage of the first sequence of compiled shaders during the course of executing the plurality of online gaming sessions. ### Claim 19 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein the plurality of online gaming sessions is sampled from all online gaming sessions that are processed by the CPUs and GPUs for the gaming title.