Feedback coordination for a virtual interaction

A device, in a physical environment, comprising:

• one or more processors;
• a non-transitory memory;
• one or more feedback devices;
• one or more input devices;
• and
• one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
• obtain user movement information characterizing a position and trajectory of a hand of a user in the physical environment;
• determining a location of a CGR item in the physical environment;
• generate, prior to the occurrence of a virtual interaction including a collision between the CGR item and at least one of the hand of the user or an object controlled by the hand of the user, a predicted virtual interaction time for the collision based at least in part on the user movement information and the location of the CGR item;
• determine a first initiation time for a first feedback device among the one or more feedback devices based at least in part on a sensation profile associated with the virtual interaction, the predicted virtual interaction time, and a first predetermined latency period associated with the first feedback device based at least in part on a motor latency;
• initiate at the first initiation time, by the device, first feedback from the first feedback device;
• determine whether or not the predicted virtual interaction time is within an acceptable temporal range around one of a plurality of temporal sound markers;
• in response to determining that the predicted virtual interaction time falls within the acceptable temporal range around a particular temporal sound marker of the plurality of the temporal sound markers, quantize the virtual interaction by presenting the virtual interaction to match the particular temporal sound marker of the plurality of sound markers;
• and
• display, after the first initiation time and in association with the physical environment, the CGR item at the location in the physical environment and the virtual interaction.

A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device, in a physical environment, with one or more feedback devices and one or more input devices, cause the device to:

• obtain user movement information characterizing a position and trajectory of a hand of a user in the physical environment;
• determine a location of a CGR item in the physical environment;
• generate, prior to the occurrence of a virtual interaction including a collision between the CGR item and at least one of the hand of the user or an object controlled by a hand of the user, a predicted virtual interaction time for the collision based at least in part on the user movement information, and the location of the CGR item;
• determine a first initiation time for a first feedback device among the one or more feedback devices based at least in part on a sensation profile associated with the virtual interaction, the predicted virtual interaction time, and a first predetermined latency period associated with the first feedback device based at least in part on a motor latency;
• initiate at the first initiation time, by the device, first feedback from the first feedback device;
• determine whether or not the predicted virtual interaction time is within an acceptable temporal range around one of a plurality of temporal sound markers;
• in response to determining that the predicted virtual interaction time falls within the acceptable temporal range around a particular temporal sound marker of the plurality of the temporal sound markers, quantize the virtual interaction by presenting the virtual interaction to match the particular temporal sound marker of the plurality of sound markers;
• and
• display, after the first initiation time and in association with the physical environment, the CGR item at the location in the physical environment and the virtual interaction.