Optimization of motion paths of a robot using vision data

A non-transitory computer readable medium having stored thereon instructions, that when executed by one or more processors of a computing device, cause the computing device to perform functions comprising:

• receiving, from one or more vision components in an environment, vision data that captures features of the environment, including object features of an object that is located in a first area of the environment;
• prior to a robot manipulating the object:
• determining based on the vision data, at least one first adjustment to a programmed trajectory of movement of the robot operating in the environment to perform a task of transporting the object from the first area to a second area, wherein the programmed trajectory of movement of the robot comprises a plurality of waypoints with movement between the plurality of waypoints along substantially straight lines, wherein determining the at least one first adjustment to the programmed trajectory of movement of the robot involves removing at least one of the plurality of waypoints and adjusting at least one of the substantially straight lines to a non-straight line, and wherein the at least one first adjustment causes movement of the robot to reach the object to deviate from the programmed trajectory based on an optimization parameter;
• determining based on the object features of the object, at least one second adjustment to the programmed trajectory of movement of the robot operating in the environment to perform the task, wherein the at least one second adjustment causes movement of the robot after manipulating the object to deviate from the programmed trajectory based on the optimization parameter;
• and
• causing the robot to perform the task, in accordance with the at least one first adjustment and the at least one second adjustment to the programmed trajectory of movement of the robot.

A system comprising:

• one or more processors;
• and
• non-transitory data storage storing instructions, that when executed by the one or more processors, causes the one or more processors to perform functions comprising:
• receiving, from one or more vision components in an environment, vision data that captures features of the environment, including object features of an object that is located in a first area of the environment;
• prior to a robot manipulating the object:
• determining based on the vision data, at least one first adjustment to a programmed trajectory of movement of the robot operating in the environment to perform a task of transporting the object from the first area to a second area, wherein the programmed trajectory of movement of the robot comprises a plurality of waypoints with movement between the plurality of waypoints along substantially straight lines, wherein determining the at least one first adjustment to the programmed trajectory of movement of the robot involves removing at least one of the plurality of waypoints and adjusting at least one of the substantially straight lines to a non-straight line, and wherein the at least one first adjustment causes movement of the robot to reach the object to deviate from the programmed trajectory based on an optimization parameter;
• determining based on the object features of the object, at least one second adjustment to the programmed trajectory of movement of the robot operating in the environment to perform the task, wherein the at least one second adjustment causes movement of the robot after manipulating the object to deviate from the programmed trajectory based on the optimization parameter;
• and
• causing the robot to perform the task, in accordance with the at least one first adjustment and the at least one second adjustment to the programmed trajectory of movement of the robot.