Enhanced real-time audio generation via cloud-based virtualized orchestra

A system comprising one or more computers and non-transitory computer storage media storing instructions that when executed by the one or more computers, cause the one or more computers to perform operations comprising:

• receiving, from a user device, a request to generate output audio associated with a musical score;
• simulating, based on one or more machine learning models, actions associated with virtual musicians with respect to respective instruments during performance of the musical score;
• receiving user input comprising conductor cues during the performance of the musical score, the user input being received from an interactive user interface presented via the user device and the interactive user interface presenting a graphical representation of the virtual musicians or the instruments, wherein a particular conductor cue of the conductor cues is directed to a subset of the virtual musicians or to a particular instrument of the subset of the virtual musicians, and wherein the simulated actions associated with the subset of the virtual musicians are generated based, at least in part, on the particular conductor cue, the simulated actions being indicative of the subset of the virtual musicians performing the musical score based, at least in part, on the particular conductor cue;
• and
• generating output audio to be provided to the user device, the output audio being generated based on the simulated actions by the virtual musicians.

Non-transitory computer storage media storing instructions that when executed by a system of one or more computers, cause the computers to perform operations comprising:

• receiving, from a user device, a request to generate output audio associated with a musical score;
• simulating, based on one or more machine learning models, actions associated with virtual musicians with respect to respective instruments during performance of the musical score;
• receiving user input comprising conductor cues during the performance of the musical score, the user input being received from an interactive user interface presented via the user device and the interactive user interface presenting a graphical representation of the virtual musicians or the instruments, wherein a particular conductor cue of the conductor cues is directed to a subset of the virtual musicians or to a particular instrument of the subset of the virtual musicians, and wherein the simulated actions associated with the subset of the virtual musicians are generated based, at least in part, on the particular conductor cue, the simulated actions being indicative of the subset of the virtual musicians performing the musical score based, at least in part, on the particular conductor cue;
• and
• generating output audio to be provided to the user device, the output audio being generated based on the simulated actions by the virtual musicians.