Systems, methods, and computer program products for implementing object permanence in a simulated environment

A computer program product comprising a non-transitory processor-readable storage medium storing processor-executable instructions and/or data that, when executed by at least one processor of a robot system comprising a robot body in a physical environment, cause the robot system to implement object permanence in a simulated environment that comprises an environment model representing the physical environment in which the robot body operates, wherein the robot system uses the simulated environment to operate within the physical environment, and wherein the processor executable instructions and/or data that, when executed by the at least one processor, cause the robot system to implement object permanence in the simulated environment, cause the robot system to:

• access, by the at least one processor, the simulated environment;
• capture, by at least one image sensor carried by the robot body, first image data representing the physical environment at a first time, the first image data further representing a first object at a first position;
• include, in the environment model, a first representation of the first object based on the first image data;
• capture, by the at least one image sensor, second image data representing the physical environment at a second time after the first time;
• determine, by the at least one processor, whether the first object is represented in the second image data;
• if the first object is represented in the second image data, maintain representation of the first object in the environment model;
• if the first object is not represented in the second image data:
• determine, by the at least one processor, whether the first object is occluded from the second image data by at least one member of the robot body at the second time;
• if the first object is occluded from the second image data by the at least one member of the robot body at the second time:
• maintain representation of the first object in the environment model;
• and
• if the first object is not occluded from the second image data by the at least one member of the robot body at the second time:
• remove representation of the first object from the environment model;
• wherein the processor-executable instructions and/or data which cause the robot system to determine whether the first object is occluded from the second image data by the at least one member of the robot body at the second time cause the robot system to:
• identify, at a third time before the second time, a motion path of the at least one member of the robot body;
• determine, by the at least one processor before the second time, a predicted position of the at least one member of the robot body at the second time based on the identified motion path;
• and
• determine whether the predicted position of the at least one member of the robot body is between the first position of the first object at the first time and a position of the image sensor.