Voice assistance in virtual reality environments and augmented reality environments

A method of providing voice assistance in a virtual or augmented reality environment, comprising:

• receiving, at a device, an indication of a visualization change associated with the virtual or augmented reality environment;
• determining, at the device, that a change value, calculated based on the indication of the visualization change, satisfies a relevance threshold;
• applying, at the device and based on the indicator of change satisfying the relevance threshold, a description model to determine at least one sentence describing the virtual or augmented reality environment;
• applying, at the device, a text-to-speech model to the at least one sentence to generate a digital sound signal;
• transmitting, at the device and to an output component, the digital sound signal;
• receiving, at the device, an additional indication of a visualization change associated with the virtual or augmented reality environment;
• determining, at the device, that an additional change value, calculated based on the additional indication of visualization change, fails to satisfy the relevance threshold;
• receiving, at the device, a command to provide voice assistance;
• applying, at the device and based on the command to provide voice assistance, the description model to determine at least one additional sentence describing the virtual or augmented reality environment;
• applying, at the device, the text-to-speech model to the at least one additional sentence to generate an additional digital sound signal;
• and
• transmitting, at the device and to the output component, the additional digital sound signal.

A non-transitory computer-readable medium storing a set of instructions for providing voice assistance in a virtual or augmented reality environment, the set of instructions comprising:

• one or more instructions that, when executed by one or more processors of a device, cause the device to:
• display a first visualization associated with the virtual or augmented reality environment;
• display a second visualization, subsequent to the first visualization, associated with the virtual or augmented reality environment;
• determine that an indicator of change from the first visualization to the second visualization satisfies a relevance threshold;
• apply a description model, based on the indicator of change satisfying the relevance threshold, to determine at least one sentence describing the virtual or augmented reality environment;
• output, to a user and using an output component, sound based on applying a text-to-speech model to the at least one sentence;
• display a third visualization, subsequent to the second visualization, associated with the virtual or augmented reality environment;
• determine that an additional indicator of change from the second visualization to the third visualization fails to satisfy the relevance threshold;
• receive, from the user and using an input component, a command to provide voice assistance;
• apply the description model, based on the command to provide voice assistance, to determine at least one additional sentence describing the virtual or augmented reality environment;
• and
• output, to the user and using the output component, additional sound based on applying the text-to-speech model to the at least one additional sentence.