Enhanced training of machine learning systems based on automatically generated realistic gameplay information

Non-transitory computer storage media storing instructions that when executed by a system of one or more computers, cause the one or more computers to perform operations comprising:

• obtaining electronic game data comprising a plurality of rendered images and associated annotation information, the annotation information identifying features included in the rendered images to be learned, and the electronic game data being generated by a video game associated with a particular sport;
• training, based on the obtained electronic game data, one or more machine learning models, wherein training comprises causing the machine learning models to output annotation information based on associated input of a rendered image;
• obtaining real-world gameplay data, the real-world gameplay data being a plurality of images of real-world gameplay of the particular sport;
• and
• analyzing the obtained real-world gameplay data, wherein analyzing comprises:
• adjusting the obtained images of real-world gameplay based on application of the machine learning models;
• extracting features from the adjusted images;
• providing, to the machine learning models, a particular image of real-world gameplay and adjusting the particular image based on application of the machine learning models;
• extracting, via a first machine learning model, a contour of a real-world player included in the real-world gameplay;
• identifying, via a second machine learning model, portions of the real-world player that correspond to portions of a skeletal model utilized in the video game, the second machine learning model being trained to extract skeletal models of respective video game characters;
• and
• based on the identified portions, determining pose information associated with the real world player.

A system comprising one or more computers and computer storage media storing instructions that when executed by the one or more computers, cause the one or more computers to perform operations comprising:

• obtaining electronic game data comprising a plurality of rendered images and associated annotation information, the annotation information identifying features included in the rendered images to be learned, and the electronic game data being generated by a video game associated with a particular sport;
• training, based on the obtained electronic game data, one or more machine learning models, wherein training comprises causing the machine learning models to output annotation information based on associated input of a rendered image;
• obtaining real-world gameplay data, the real-world gameplay data being a plurality of images of real-world gameplay of the particular sport;
• and
• analyzing the obtained real-world gameplay data, wherein analyzing comprises:
• adjusting the obtained images of real-world gameplay based on application of the machine learning models;
• extracting features from the adjusted images;
• providing, to the machine learning models, a particular image of real-world gameplay and adjusting the particular image based on application of the machine learning models;
• extracting, via a first machine learning model, a contour of a real-world player included in the real-world gameplay;
• identifying, via a second machine learning model, portions of the real-world player that correspond to portions of a skeletal model utilized in the video game, the second machine learning model being trained to extract skeletal models of respective video game characters;
• and
• based on the identified portions, determining pose information associated with the real world player.