Learning and applying empirical knowledge of environments by robots

A method implemented by one or more processors, comprising:

• receiving a plurality of training visual frames captured by one or more vision sensors, wherein at least a first subset of the plurality of training visual frames do not directly capture an instance of an object of interest and at least a second subset of the plurality of training visual frames directly capture an instance of the object of interest, and each given training visual frame of the plurality of training visual frames is classified with one or more annotations that are indicative of whether an instance of the object of interest is present in an area captured in the given training visual frame;
• applying the plurality of training visual frames as input across a machine learning model to generate training output;
• comparing the training output to the annotations used to classify the plurality of training visual frames;
• and
• training the machine learning model based on the comparing, wherein output generated based on the trained machine learning model is usable to narrow a search space in which a robot seeks an instance of the object of interest.

A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:

• determine an object of interest;
• receive vision data, the vision data generated based on output from one or more vision sensors of a vision component of a robot, the vision data including one or more visual frames that capture at least a portion of an environment in which a robot operates or will operate;
• apply one or more of the visual frames as input across a machine learning model to generate output, wherein the output identifies one or more probabilities that a plurality of pixels representing one or more surfaces in the portion of the environment captured in the one or more visual frames conceal, from a vantage point of the one or more vision sensors, an instance of the object of interest;
• and
• alter a position or pose of the robot based on the output to relocate one or more of the vision sensors to have a direct view behind one or more of the surfaces;
• wherein the machine learning model was trained using one or more annotated vision frames in which a portion of a surface of the annotated vision frame is annotated as concealing another object of interest of a same type as the object of interest.