Simulating multiple robots in virtual environments

A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:

• simulate a three-dimensional virtual environment that includes a plurality of robot avatars that are controlled independently and contemporaneously by a corresponding plurality of robot controllers that are external from the virtual environment;
• operate the plurality of robot avatars at frequencies that correspond to real-world frequencies of the respective robot controllers that control the plurality of robot avatars;
• receive, from each robot controller of the plurality of robot controllers, joint commands that cause actuation of one or more joints of the respective robot avatar of the plurality of robot avatars that is controlled by the robot controller;
• actuate one or more joints of each robot avatar of the plurality of robot avatars pursuant to corresponding joint commands;
• to account for a deviation of a lagging robot avatar of the plurality of robot avatars from an expected operational frequency of the lagging robot avatar, temporarily pause operations of one or more other robot avatars of the plurality of robot avatars;
• and
• generate and store training data for a machine learning model based on interactions between the plurality of robot avatars and the virtual environment.

At least one non-transitory computer-readable medium comprising instructions that, in response to execution by one or more processors, cause the one or more processors to:

• simulate a three-dimensional virtual environment that includes a plurality of robot avatars that are controlled independently and contemporaneously by a corresponding plurality of robot controllers that are external from the virtual environment;
• operate the plurality of robot avatars at frequencies that correspond to real-world frequencies of the respective robot controllers that control the plurality of robot avatars;
• receive, from each robot controller of the plurality of robot controllers, joint commands that cause actuation of one or more joints of the respective robot avatar of the plurality of robot avatars that is controlled by the robot controller;
• actuate one or more joints of each robot avatar of the plurality of robot avatars pursuant to corresponding joint commands;
• to account for a deviation of a lagging robot avatar of the plurality of robot avatars from an expected operational frequency of the lagging robot avatar, temporarily pause operations of one or more other robot avatars of the plurality of robot avatars;
• and
• generate and store training data for a machine learning model based on interactions between the plurality of robot avatars and the virtual environment.