Spatial mapping and planning with language models using knowledge graphs

One or more processors comprising processing circuitry to:

• query a temporal knowledge graph to predict a next action that a robotic system is to execute in an environment, the temporal knowledge graph including a graph representation of entities and relationships between the entities in the environment that are produced based at least on sensor data captured from the environment;
• receive additional sensor data captured from the environment;
• generate, using a machine learning model, spatial and temporal data associated with the additional sensor data;
• and
• update, using the machine learning model, the temporal knowledge graph to include representations of the spatial and temporal data based at least on a similarity between the spatial and temporal data and the graph representation.

A system comprising one or more processors to:

• query a temporal knowledge graph to predict a next action that a robotic system is to execute in an environment;
• receive sensor data captured from the environment;
• generate, via a machine learning model, spatial and temporal data associated with the additional sensor data;
• and
• update, via the machine learning model, the temporal knowledge graph to include representations of the spatial and temporal data based at least on a similarity between the spatial and temporal data and the graph representation.