Facilitating model output modifications via physical gesture directed to portion of generative output

A method implemented by one or more processors, the method comprising:

• receiving a first portion of an input gesture at an input interface of a client computing device,
• wherein a user performs the input gesture by motioning at or near a particular portion of a graphical user interface (GUI) of the client computing device without interacting with a GUI element that is separate from a generative model output embodied in the particular portion of the GUI;
• causing gesture data to be processed by one or more generative models to determine modified output data for rendering at the GUI of the client computing device,
• wherein the modified output data indicates a modification to the particular portion of the GUI according to the first portion of the input gesture and the gesture data characterizes the input gesture;
• causing the modification to the particular portion of the GUI to be rendered at a display interface of the client computing device,
• wherein the modification to the particular portion of the GUI is rendered while the input gesture continues to be provided by the user;
• receiving a second portion of the input gesture at the input interface of the client computing device,
• wherein the second portion of the input gesture is performed by motioning at or near an additional portion of the GUI that includes the modification to the particular portion of the GUI;
• determining that the second portion of the input gesture indicates approval of, or a request to modify, the additional portion of the GUI;
• and
• causing the client computing device, or another computing device, to render feedback for the user according to whether the second portion of the input gesture indicates approval of, or the request to modify, the additional portion of the GUI.

A system, comprising:

• memory storing instructions;
• and
• one or more processors operable to execute the instructions to:
• cause an initial generative output to be visually rendered at a display interface of a client computing device in response to user interface input,
• wherein the initial generative output is generated using one or more generative models to process the user interface input received at the client computing device;
• determine, while the initial generative output is visually rendered at the display interface, that an input interface of the client computing device, or a separate computing device, has received an input gesture,
• wherein a user performs the input gesture by motioning, at or near a particular portion of the display interface that includes the visually rendered initial generative output, without directly interacting with a separate graphical user interface (GUI) element for modifying the initial generative output;
• generate, based on the input gesture and using the one or more generative models or a separate generative model, modified generative output,
• wherein the modified generative output includes a modification of, or refinement to, the initial generative output, and
• wherein the modification is determined according to the input gesture;
• cause the modified generative output to be visually rendered at the display interface of the client computing device in lieu of, or in addition to, the initial generative output being rendered at the display interface.