Guided uncertainty-aware policy optimization: combining model-free and model-based strategies for sample-efficient learning

A computer system comprising:

• one or more processors;
• and
• computer-readable memory storing executable instructions that, as a result of being executed by the one or more processors, cause the computer system to:
• divide at least a portion of a physical model created based at least in part on information from a perception system into a plurality of regions;
• generate estimates of uncertainty for the plurality of regions based at least in part on at least one uncertainty estimation provided by the perception system, a first set of the plurality of regions to comprise any of the plurality of regions associated with any of the estimates of uncertainty that indicate a robot is unlikely to interact with its environment, a second set of the plurality of regions to comprise any of the plurality of regions associated with any of the estimates of uncertainty that indicate the robot is likely to interact with its environment;
• determine if the robot is positioned in at least one of the first set of regions or at least one of the second set of regions;
• use the physical model to control the robot if it is determined that the robot is in at least one of the first set of regions;
• and
• use at least one reinforcement learning process instead of using the physical model to control the robot if it is determined that the robot is in at least one of the second set of regions.

A non-transitory computer-readable medium having stored thereon instructions that, which if performed by one or more processors, cause the one or more processors to at least:

• partition at least a portion of a physical model into a plurality of regions;
• generate estimates of uncertainty for the plurality of regions;
• identifying a first set of the plurality of regions based at least in part on any of the estimates of uncertainty associated with the first set of regions;
• identifying a second set of the plurality of regions based at least in part on any of the estimates of uncertainty associated with the second set of regions;
• use the physical model to control a robot in any of the first set of regions;
• and
• use at least one reinforcement learning process instead of using the physical model to control the robot in any of the second set of regions.

A processor comprising:

• one or more arithmetic logic units (ALUs) to perform a task by at least:
• dividing at least a portion of a physical model into a plurality of regions;
• identifying a first set of the plurality of regions in which to use the physical model to control a robot;
• identifying a second set of the plurality of regions in which to use at least one reinforcement learning process to control the robot;
• using the physical model to control of the robot's if the robot is in the first set of regions;
• and
• using the at least one reinforcement learning process instead of using the physical model to control the robot if the robot is in the second set of regions.