System and method for controlling an autonomous device

A method for controlling an autonomous device to perform a task, wherein the device has a plurality of parts whose movement can be controlled by actuators, by

• generating, by a motion machine-learning model, from a movement trajectory of each part of a proper subset of the plurality of parts, motion information specifying a movement trajectory of each part of the plurality of parts;
• and
• generating, by a tracking machine-learning model, actuator control signals to control the robot device to follow, with each of its parts, the movement trajectory of the part specified by the motion information.

A non-transitory computer-readable medium, comprising instructions stored thereon, that when executed on a processor, perform a method for controlling an autonomous device to perform a task, wherein the device has a plurality of parts whose movement can be controlled by actuators, by

• generating, by a motion machine-learning model, from a movement trajectory of each part of a proper subset of the plurality of parts, motion information specifying a movement trajectory of each part of the plurality of parts;
• and
• generating, by a tracking machine-learning model, actuator control signals to control the robot device to follow, with each of its parts, the movement trajectory of the part specified by the motion information.