Systems, apparatuses, and methods for robotic learning and execution of skills including navigation and manipulation functions

A robotic device, comprising:

• a base supported on a transport element;
• a manipulating element coupled to the base and including an end effector;
• a set of sensors;
• and
• a memory storing a plan cache including a plurality of plans for executing a skill, each plan from the plurality of plans being associated with a starting pose of the base and the end effector relative to a physical object in an environment near the robotic device;
• a processor operatively coupled to the base, the manipulating element, the set of sensors, and the memory, the processor configured to:
• scan, using at least one sensor from the set of sensors, the environment near the robotic device;
• identify, based on the scan of the environment near the robotic device, a location of the physical object in the environment near the robotic device;
• determine a current pose of the base and the end effector relative to the location of the physical object;
• select a first plan from the plurality of plans having a first starting pose of the base and the end effector that is substantially the same as the current pose of the base and the end effector;
• execute the first plan;
• in response to determining that the skill has not been successfully executed,
• select a second plan from the plan cache based on the current pose of the base and the end effector, the second plan having a second starting pose of the base and the end effector that is different from the first starting pose;
• and
• execute the second plan.

A robotic device, comprising:

• a base supported on a transport element;
• a manipulating element coupled to the base and including an end effector;
• a set of sensors;
• and
• a memory storing a plan cache including a plurality of plans for executing a skill, each plan from the plurality of plans being associated with a starting pose of the base and the end effector relative to a physical object in an environment near the robotic device;
• a processor operatively coupled to the base, the manipulating element, the set of sensors, and the memory, the processor configured to:
• scan, using at least one sensor from the set of sensors, the environment near the robotic device;
• identify, based on the scan of the environment near the robotic device, a location of the physical object in the environment near the robotic device;
• determine a current pose of the base and the end effector relative to the location of the physical object;
• select the first plan from the plurality of plans having a starting pose of the base and the end effector that is substantially the same as the current pose of the base and the end effector;
• execute the plan;
• in response to detecting a collision with an obstacle during the execution of the plan, terminate the execution of the plan and move the end effector into a predefined safe position;
• and
• execute a second plan that avoids a collision with the obstacle.

A robotic device, comprising:

• a memory;
• a base supported on a transport element;
• a manipulating element coupled to the base and including an end effector;
• a set of one or more sensors;
• and
• a processor operatively coupled to the memory, the base, the manipulating element, and the set of sensors, the processor configured to:
• in response to receiving a first user input indicating a starting pose for executing a skill, register the starting pose of the base and the end effector relative to a with a physical object in an environment;
• obtain, using the set of sensors and during a demonstration of a manipulation required to execute the skill, sensor data at a plurality of keyframes of the demonstration;
• in response to receiving a second user input indicating a target pose for interacting with the physical object to execute the skill, register the target pose of the end effector relative to the first activation device;
• generate a plan for executing the skill based at least in part on the starting pose, the sensor data collected at the plurality of keyframes, and the target pose;
• identify a plurality of additional starting poses for executing the skill;
• generate a plurality of additional plans for executing the skill based at least in part on the plurality of additional starting poses, the sensor data collected at the plurality of keyframes, and the target pose;
• and
• store the plan and the plurality of additional plans for executing the skill as a plan cache in the memory.