Method and device for generating video frames

A non-transitory, computer readable storage medium having computer executable instructions stored thereon, which when executed by a computer system acting as a client device, cause the computer system to perform a method of generating video frames for a cloud-based video game, by carrying out actions, comprising:

• obtaining, via a communications network, a plurality of video frames of a video game rendered at a cloud device;
• obtaining a player input received at an input device, the input device being used to play the video game, the player input being used to play the video game and being received in response to the display of at least one of the plurality of video frames;
• inputting at least one of the plurality of video frames and the player input, which is being used to play the video game, into a trained machine learning model comprised by the client device;
• and
• predicting, via the trained machine learning model, a subsequent video frame for output at a display based on the at least one of the obtained video frames and the player input, which is being used to play the video game, and outputting the subsequent video frame at the display, wherein:
• at least some of the plurality of video frames obtained from the cloud device correspond to video frames having a wider field of view than a field of view of the subsequent video frame that is to be output at the display, and
• inputting the at least one of the plurality of video frames into the trained by machine learning model comprises inputting a wider field of view frame to the trained machine learning model, the wider field of view frame including an object coming into the player's view.

A client computing device comprising:

• a network interface operable to receive, via a communications network, a plurality of video frames of a video game rendered at a cloud device;
• an input unit operable to receive player inputs input at an input device, the input device being used to play the video game, the player input being used to play the video game and being received in response to the display of at least one of the plurality of video frames;
• a prediction unit operable to receive an input from the network interface and the input unit, the input comprising at least one of the obtained video frames and the player input, which is being used to play the video game, and comprising a trained machine learning model to receive the input comprising at least one of the plurality of video frames and the player input and to predict a subsequent video frame for output at a display;
• and
• an output unit operable to transmit the subsequent video frame to a display, wherein:
• at least some of the plurality of video frames obtained from the cloud device correspond to video frames having a wider field of view than a field of view of the subsequent video frame that is to be output at the display, and
• inputting the at least one of the plurality of video frames into the trained by machine learning model comprises inputting a wider field of view frame to the trained machine learning model, the wider field of view frame including an object coming into the player's view.