Real time generative audio for brush and canvas interaction in digital drawing

A non-transitory computer-readable storage medium including instructions stored thereon which, executed by at least one processor, cause the at least one processor to perform operations comprising:

• receiving a selection of a drawing tool;
• receiving a user input that represents a digital drawing interaction between a digital canvas and the drawing tool;
• selecting a trained audio transformation model from a plurality of trained audio transformation models, wherein the trained audio transformation model is trained for a combination of the digital canvas and the drawing tool;
• providing the user input to the trained audio transformation model;
• modifying the trained audio transformation model using network bending to insert one or more layers into the trained audio transformation model;
• and
• generating an audio sample using the trained audio transformation model.

A system comprising:

• a memory component;
• and
• a processing device coupled to the memory component, the processing device to perform operations comprising:
• receiving a user input and a selection of a tool;
• generating intermediary audio data based on the user input and the tool selection, wherein the intermediary audio data includes a pitch and a frequency;
• adjusting a trained audio transformation model including a series of one or more layers to include one or more additional layers to produce an adjusted audio transformation model;
• and
• generating, by the adjusted audio transformation model, an audio sample based on the intermediary audio data.