Presenting computer-generated content based on extremity tracking

An electronic device comprising:

• one or more processors;
• a non-transitory memory;
• a first input device;
• and
• one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
• detecting, via the first input device, an extremity directed to a first location within a physical environment, wherein the first location is identified by an extremity tracking function;
• identifying a physical feature of the physical environment proximate to the first location;
• determining and updating an interaction event between the extremity and the physical feature by tracking movements of the extremity and based on a semantic identifier that characterizes the physical feature of the physical environment;
• and
• presenting computer-generated content inferred from the interaction event tracking the movements of the extremity and a trajectory of the semantically identified physical feature interacted with the extremity.

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with one or more processors and a first input device, cause the electronic device to:

• detect, via the first input device, an extremity directed to a first location within a physical environment, wherein the first location is identified by an extremity tracking function;
• identify a physical feature of the physical environment proximate to the first location;
• determine and update an interaction event between the extremity and the physical feature by tracking movements of the extremity and based on a semantic identifier that characterizes the physical feature of the physical environment;
• and
• present computer-generated content inferred from the interaction event tracking the movements of the extremity and a trajectory of the semantically identified physical feature interacted with the extremity.