Scripted interactive scenarios based on scene graph matching

A device comprising:

• a non-transitory computer-readable storage medium;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
• obtaining sensor data for a physical environment;
• generating a local scene graph for the physical environment based on the sensor data, wherein the local scene graph represents a set of objects and relationships between the objects;
• identifying a principal scene graph from a set of previously obtained principal scene graphs based on matching characteristics and relationships between the set of objects associated with the local scene graph to one or more characteristics and relationships between a set of objects associated with each principal scene graph;
• and
• executing one or more scripted actions involving the objects based on a narrative associated with the identified principal scene graph.

A non-transitory computer-readable storage medium, storing program instructions executable by one or more processors of an electronic device to perform operations comprising:

• obtaining sensor data for a physical environment;
• generating a local scene graph for the physical environment based on the sensor data, wherein the local scene graph represents a set of objects and relationships between the objects;
• identifying a principal scene graph from a set of previously obtained principal scene graphs based on matching characteristics and relationships between the set of objects associated with the local scene graph to one or more characteristics and relationships between a set of objects associated with each principal scene graph;
• and
• executing one or more scripted actions involving the objects based on a narrative associated with the identified principal scene graph.