Generating corrected simulations using video generation models

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• obtaining an input video comprising a sequence of frames depicting a state transition of an environment that does not meet a state transition criterion;
• generating, based on a frame from a sequence of frames that depicts an incorrect end state of the state transition, a synthetic ending frame depicting a corrected end state of the state transition;
• and
• processing an input comprising one or more key frames from sequence of frames of the input video and the synthetic ending frame depicting the corrected end state of the state transition using a video generation model to generate an output video depicting a synthetic state transition of the environment that meets the state transition criterion.

One or more non-transitory storage media encoded with instructions that when executed by a computing device cause the computing device to perform operations comprising:

• obtaining an input video comprising a sequence of frames depicting a state transition of an environment that does not meet a state transition criterion;
• generating, based on a frame from a sequence of frames that depicts an incorrect end state of the state transition, a synthetic ending frame depicting a corrected end state of the state transition;
• and
• processing an input comprising one or more key frames from sequence of frames of the input video and the synthetic ending frame depicting the corrected end state of the state transition using a video generation model to generate an output video depicting a synthetic state transition of the environment that meets the state transition criterion.