Systems and methods for generating trajectories from broadcast footage implementing diffusion

A system for generating trajectories for one or more players during a sporting event, the system comprising:

• a memory configured to store processor-readable instructions;
• and
• a processor operatively connected to the memory, and configured to execute the instructions to perform operations comprising:
• receiving, as an input, broadcast footage of a sporting event;
• determining tracking data of one or more players in the sporting event from the broadcast footage, the tracking data including one or more vectors;
• receiving event data of the sporting event;
• inputting the one or more vectors and event data into a multimodal model, the multimodal model including:
• an event encoder;
• and
• an tracking decoder;
• applying a linear layer of the multimodal model to the one or more vectors and event data to tokenize the event data and one or more vectors;
• determining, by the multimodal model, a tensor, the tensor representing a representation of sequence of the event data and tracking data;
• receiving perturbed tracking data of the sporting event;
• inputting the perturbed tracking data and tensor into a diffusion model, wherein the diffusion model includes a decoder;
• and
• generating, by the diffusion model, one or more trajectories for the one or more players in the sporting event.

A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:

• receiving, as an input, broadcast footage of a sporting event;
• determining tracking data of one or more players in the sporting event from the broadcast footage, the tracking data including one or more vectors;
• receiving event data of the sporting event;
• inputting the one or more vectors and event data into a multimodal model, the multimodal model including:
• an event encoder;
• and
• an tracking decoder;
• applying a linear layer of the multimodal model to the one or more vectors and event data to tokenize the event data and one or more vectors;
• determining, by the multimodal model, a tensor, the tensor representing a representation of sequence of the event data and tracking data;
• receiving perturbed tracking data of the sporting event;
• inputting the perturbed tracking data and tensor into a diffusion model, wherein the diffusion model includes a decoder;
• and
• generating, by the diffusion model, one or more trajectories for the one or more players in the sporting event.