Object-centric scene representation for robotic learning

A non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to:

• collect sensor data about an environment associated with a robotic manipulator, the sensor data including at least one image;
• pre-process the sensor data to form a multi-tiered representation of the environment including image features and global features;
• generate an action sequence for the robotic manipulator to perform a task according to the multi-tiered representation based, at least in part, on a diffusion policy;
• and
• control the robotic manipulator using the action sequence.

A method, comprising:

• collecting sensor data about an environment associated with a robotic manipulator, the sensor data including at least one image;
• pre-processing the sensor data to form a multi-tiered representation of the environment including image features and global features;
• generating an action sequence for the robotic manipulator to perform a task according to the multi-tiered representation based, at least in part, on a diffusion policy;
• and
• controlling the robotic manipulator using the action sequence.