Generating and utilizing spatial affordances for an object in robotics applications

A method implemented by one or more processors of a robot, the method comprising:

• receiving vision data, the vision data generated based on output from one or more vision sensors of a vision component viewing an object in an environment of a robot;
• applying the vision data as input to at least one trained neural network model;
• processing the vision data using the trained neural network model to generate output defining multiple spatial affordances for the object in the environment,
• wherein processing of the vision data is based on trained parameters of the trained neural network model,
• wherein the output is generated directly by the processing of the vision data using the trained neural network model, and
• wherein the multiple spatial affordances defined by the output include a first spatial affordance for a first spatial region of the object and a second spatial affordance for a second spatial region of the object;
• determining that the first spatial affordance is a target affordance for the object;
• and
• based on the first spatial affordance being the target affordance for the object and being defined for the first spatial region of the object:
• controlling one or more actuators of the robot to cause one or more components of the robot to interact with the first spatial region of the object to perform the first spatial affordance through interaction of the one or more components with the first spatial region of the object.