Dynamically adjusting instructions in an augmented-reality experience

A computer-implemented method, the method comprising:

• obtaining, by a computing system comprising one or more processors, image data with one or more image sensors of a user computing device, wherein the image data is descriptive of one or more images, wherein the one or more images are descriptive of one or more pages;
• processing, by the computing system, the image data with an optical character recognition model to generate text data, wherein the text data is descriptive of text on the one or more pages;
• determining, by the computing system, a prompt based on the text data, wherein the prompt is descriptive of a request for a response;
• determining, by the computing system, text data comprises a problem of a particular problem type;
• in response to determining the text data comprises the problem of the particular problem type, obtaining, by the computing system, a problem-specific machine-learned model associated with the particular problem type;
• processing, by the computing system, the prompt with the problem-specific machine-learned model to generate a multi-part response to the prompt, wherein the multi-part response comprises a plurality of individual responses associated with the prompt, wherein the problem-specific machine-learned model comprises a language model trained for multi-part quantitative reasoning, wherein the language model was trained on a plurality of mathematical proofs, wherein the multi-part response is descriptive of a proof for the detected problem;
• obtaining, by the computing system, additional image data, wherein the additional image data is descriptive of one or more additional images, wherein the one or more additional images are descriptive of the one or more pages with user-generated text;
• processing, by the computing system, the additional image data with the optical character recognition model to generate additional text data, wherein the additional text data is descriptive of the user-generated text on the one or more pages;
• determining, by the computing system, the user-generated text deviates from the multi-part response;
• and
• providing, by the computing system, a notification rendered in an augmented-reality experience via the user computing device, wherein the notification is descriptive of the user-generated text having an error.

One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising:

• obtaining image data, wherein the image data is descriptive of one or more images, wherein the one or more images are descriptive of one or more pages, wherein the one or more pages comprise a plurality of characters;
• processing the image data with a machine-learned model to generate semantic data, wherein the semantic data is descriptive of a semantic understanding of at least a portion of the plurality of characters, wherein the machine-learned model comprises a language model trained for multi-part quantitative reasoning, wherein the language model was trained on a plurality of mathematical proofs;
• processing the semantic data with the machine-learned model to generate a multi-part response for a detected problem in the one or more images, wherein the multi-part response is descriptive of a proof for the detected problem;
• determining the plurality of characters comprise an error based at least in part on the multi-part response, wherein the error is descriptive of text that is at least one of counter to the semantic understanding or an inaccuracy;
• determining a corrective action based on the multi-part response and the error, wherein the corrective action is descriptive of at least one of a replacement for the error or an action to fix the error;
• generating one or more augmented images based on the correction action and the one or more images, wherein the one or more augmented images comprise one or more user interface elements rendered into the one or more images, wherein the one or more user interface elements comprise text superimposed over at least a portion of the one or more images and one or more indicators, wherein the text of the one or more user interface elements comprise informational data descriptive of the corrective action, and wherein the one or more indicators indicate a position-of-interest in the one or more images associated with the error;
• providing the one or more augmented images for display based on the corrective action;
• obtaining additional image data, wherein the additional image data is descriptive of one or more additional images, wherein the one or more additional images are descriptive of the one or more pages with user-generated text;
• processing the additional image data and the multi-part response to determine a particular portion of the user-generated text deviates from the multi-part response;
• and
• generating one or more second augmented images that indicate the particular portion of the user-generated text that has a determined error.