# US 10,369,465 B2 · System and method for streaming game video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,369,465 B2 - Document ID: US10369465B2 - Status: Granted - Kind: B2 - Application number: 15/614,586 - Filed: 2017-06-05 - Published: 2017-09-21 - Granted: 2019-08-06 - Earliest claimed: 2002-12-10 - Assignees: Sony Interactive Entertainment America LLC - Applicants: Sony Interactive Entertainment America LLC - Inventors: Stephen G. Perlman; Roger van der Laan; Timothy Cotter; Scott Furman; Robert McCool; Ian Buckley - Web page: https://patentgrep.com/patent/US10369465B2 ## CPC - A63F 13/12 - A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client - A63F 13/30: Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers - A63F 13/33: using wide area network [WAN] connections - A63F 13/35: Details of game servers - A63F 13/358: Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients - A63F 13/49: Saving the game status; Pausing or ending the game - A63F 13/493: Resuming a game, e.g. after pausing, malfunction or power failure - A63F 13/50: Controlling the output signals based on the game progress - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06) - H04L 65/752: adapting media to network capabilities - H04L 65/765: {intermediate} - H04L 67/131: Protocols for games, networked simulations or virtual reality - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/146: Data rate or code amount at the encoder output - H04N 19/169: characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding - H04N 19/172: the region being a picture, frame or field - H04N 19/188: {the unit being a video data packet, e.g. a network abstraction layer [NAL] unit} - H04N 19/436: using parallelised computational arrangements - H04N 19/61: in combination with predictive coding - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)} - H04N 21/2381: Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network {(transmission of MPEG streams over ATM H04L12/5601)} - H04N 21/4781: {Games} - H04N 21/6125: {involving transmission via Internet (transmission by internet of broadcast information H04H60/82)} - H04N 21/6377: directed to server {(one-way streaming services wherein the source is controlled by the destination H04L65/613)} - H04N 21/6405: Multicasting {(data broadcast and multicast in packet switching networks H04L12/18)} - H04N 21/658: Transmission by the client directed to the server - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - A63F 2300/402: Communication between platforms, i.e. physical link to protocol - A63F 2300/407: Data transfer via internet - A63F 2300/534: for network load management, e.g. bandwidth optimization, latency reduction - A63F 2300/552: for downloading to client devices, e.g. using OS version, hardware or software profile of the client device - A63F 2300/554: by saving game or status data - A63F 2300/556: Player lists, e.g. online players, buddy list, black list - A63F 2300/572: Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video - A63F 2300/577: for watching a game played by other players - A63F 2300/636: involving process of starting or resuming a game - A63F 2300/69: Involving elements of the real world in the game world, e.g. measurement in live races, real video - H04N 7/106: {for domestic distribution} - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A system and method are described for managing the state of an online video game. A method includes initiating a new online video game in response to user input from a client device, the online video game being in a first state on a first server when initiated; executing the online video game on the server, thereby causing the online video game to enter a second state; pausing or terminating the online video game; determining differences between the first state and the second state and generating difference data containing the differences; transmitting the difference data over a network to a second server; and recreating the second state from the difference data and the first state in response to user input indicating that the user wishes to resume the online video game and in response to the second server being selected as the server on which to execute the video game. ## Claims ### Claim 1 A server system, comprising: a processor configured to execute a game application for simultaneous play by multiple users in accordance with signals received from multiple client computing devices respectively corresponding to the multiple users to generate multiple video streams of graphical output of the executed game application, the multiple video streams of graphical output of the executed game application respectively corresponding to play of the game application by the multiple users, wherein the processor is configured to generate multiple user interfaces for respective transmission to the multiple client computing devices, each of the multiple user interfaces including separate video windows, the separate video windows in any one of the multiple user interfaces respectively displaying the multiple video streams of graphical output of the executed game application, each of the multiple user interfaces configured for transmission as a single user interface video stream to a respective one of the multiple client computing devices in lieu of separate transmission of the multiple video streams of graphical output of the executed game application to the respective one of the multiple client computing devices, wherein at least one of the separate video windows in each of the multiple user interfaces is an interaction-selectable video window that is selectable to enable user interaction within the game application executing to generate the video stream of graphical output displayed within the interaction-selectable video window; a network connection configured to connect the server system to a network over which data is transmittable to the multiple client computing devices; and an encoder configured to compress each user interface video stream into one or more encoded formats for transmission through the network connection to corresponding ones of the multiple client computing devices. ### Claim 2 (depends on 1) The server system as recited in claim 1, wherein at least one of the separate video windows in a given user interface video stream includes display of a video capture of a user of a client computing device to which the given user interface video stream is transmitted. ### Claim 3 (depends on 1) The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to live play of the game application. ### Claim 4 (depends on 1) The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to a time-delayed presentation of live play of the game application. ### Claim 5 (depends on 1) The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to a view of a user playing the game application. ### Claim 6 (depends on 1) The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to a view that is different from a view of a user playing the game application. ### Claim 7 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices. ### Claim 8 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of a ranking to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices. ### Claim 9 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of a level of play to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices. ### Claim 10 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of a relationship among users to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices. ### Claim 11 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to display the multiple video streams of graphical output of the executed game application within the separate video windows in accordance with display settings received from the multiple users. ### Claim 12 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to scale the multiple video streams of graphical output of the executed game application to fit within the separate video windows of each of the multiple user interfaces. ### Claim 13 (depends on 1) The server system as recited in claim 1, wherein at least one of the separate video windows in each of the multiple user interfaces is an audio-selectable video window that is selectable to enable transmission of audio corresponding to the video stream of graphical output displayed within the audio-selectable video window. ### Claim 14 (depends on 1) The server system as recited in claim 1, wherein the processor is configured to modify a given one of the multiple user interfaces to show a zoomed-up view of a given video window upon selection of the given video window at one of the multiple client computing devices corresponding to the given one of the multiple user interfaces. ### Claim 15 (depends on 1) The server system as recited in claim 1, wherein upon selection of a given video window in a given user interface at a given one of the multiple client computing devices, the processor is configured to substitute a scaled-up version of the given video window in place of the given user interface transmitted in the single user interface video stream to the given one of the multiple client computing devices. ### Claim 16 A method, comprising: executing a game application for simultaneous play by multiple users in accordance with signals received from multiple client computing devices respectively corresponding to the multiple users to generate multiple video streams of graphical output of the executed game application, the multiple video streams of graphical output of the executed game application respectively corresponding to play of the game application by the multiple users; generating multiple user interfaces for respective transmission to the multiple client computing devices, each of the multiple user interfaces including separate video windows, the separate video windows in any one of the multiple user interfaces respectively displaying the multiple video streams of graphical output of the executed game application, each of the multiple user interfaces configured for transmission as a single user interface video stream to a respective one of the multiple client computing devices in lieu of separate transmission of the multiple video streams of graphical output of the executed game application to the respective one of the multiple client computing devices, wherein at least one of the separate video windows in each of the multiple user interfaces is an interaction-selectable video window that is selectable to enable user interaction within the game application executing to generate the video stream of graphical output displayed within the interaction-selectable video window; compressing each user interface video stream into one or more encoded formats for transmission through a network connection to corresponding ones of the multiple client computing devices; and transmitting each user interface video stream through the network connection to corresponding ones of the multiple client computing devices. ### Claim 17 (depends on 16) The method as recited in claim 16, wherein at least one of the separate video windows in a given user interface video stream includes display of a video capture of a user of a client computing device to which the given user interface video stream is transmitted. ### Claim 18 (depends on 16) The method as recited in claim 16, further comprising: generating a selection interface to enable selection of the multiple users corresponding to the multiple video streams of graphical output of the executed game application; configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices; compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices. ### Claim 19 (depends on 16) The method as recited in claim 16, further comprising: generating a selection interface to enable selection of a ranking to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application; configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices; compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices. ### Claim 20 (depends on 16) The method as recited in claim 16, further comprising: generating a selection interface to enable selection of a level of play to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application; configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices; compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices. ### Claim 21 (depends on 16) The method as recited in claim 16, further comprising: generating a selection interface to enable selection of a relationship among users to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application; configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices; compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices.