Dynamic modification of an experience based on scene understanding, user state, and user position

A device comprising:

• a non-transitory computer-readable storage medium;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
• determining a scene understanding of a physical environment, wherein the scene understanding identifies one or more objects and is determined based on obtaining sensor data corresponding to the physical environment;
• determining a state of a user based on physiological data obtained via one or more physiological sensors;
• tracking a position of the user as the user moves within the physical environment;
• generating a sequence of content within an extended reality (XR) environment, wherein the sequence of content comprises a guiding path, wherein the sequence of content and the guiding path are determined based on the scene understanding, the state of the user, and the position of the user as the user moves within the physical environment, and wherein the guiding path is generated based on avoiding the identified one or more objects in the physical environment;
• and
• presenting the sequence of content that includes guiding indicators to guide the user to move the device to a new position associated with the guiding path.

A non-transitory computer-readable storage medium, storing program instructions executable on a device to perform operations comprising:

• determining a scene understanding of a physical environment, wherein the scene understanding identifies one or more objects and is determined based on obtaining sensor data corresponding to the physical environment;
• determining a state of a user based on physiological data obtained via one or more physiological sensors;
• tracking a position of the user as the user moves within the physical environment;
• generating a sequence of content within an extended reality (XR) environment, wherein the sequence of content comprises a guiding path, wherein the sequence of content and the guiding path are determined based on the scene understanding, the state of the user, and the position of the user as the user moves within the physical environment, and wherein the guiding path is generated based on avoiding the identified one or more objects in the physical environment;
• and
• presenting the sequence of content that includes guiding indicators to guide the user to move the device to a new position associated with the guiding path.