Dynamic input-sensitive validation of machine learning model outputs and methods and systems of the same

A system comprising:

• at least one hardware processor;
• and
• at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:
• determine a performance metric associated with processing an output generation request comprising a prompt for generation of an output using a first LLM of a plurality of LLMs;
• determine a system state associated with available system resources for processing requests using the first LLM of the plurality of LLMs,
• wherein the system state includes information relating to at least one of computational resource usage, memory usage, hard disk space usage, or cost incurred;
• compare a first estimated performance metric value for the determined performance metric with a threshold metric value associated with the determined performance metric,
• wherein the threshold metric value is based on the system state and indicates an allotment of the available system resources for processing the output generation request;
• in response to determining that the first estimated performance metric value satisfies the threshold metric value:
• provide the prompt to the first LLM to generate a first output by processing the prompt included in the output generation request;
• and
• enable access to the first output;
• based on determining that the first estimated performance metric value does not satisfy the threshold metric value:
• provide the prompt to a second LLM of the plurality of LLMs to generate a second output by processing the prompt included in the output generation request;
• and
• enable access to the second output.

A method comprising:

• determining a performance metric associated with processing an output generation request comprising an input for generation of an output using a first model of a plurality of models;
• determining a system state associated with available system resources for processing requests using the first model of the plurality of models,
• wherein the system state includes information relating to at least one of computational resource usage, memory usage, hard disk space usage, or cost incurred;
• comparing a first estimated performance metric value for the determined performance metric with a threshold metric value associated with the determined performance metric,
• wherein the threshold metric value is based on the system state and indicates an allotment of the available system resources for processing the output generation request;
• in response to determining that the first estimated performance metric value satisfies the threshold metric value:
• providing the input to the first model to generate a first output by processing the input included in the output generation request;
• and
• enabling access to the first output;
• based on determining that the first estimated performance metric value does not satisfy the threshold metric value:
• providing the input to a second model of the plurality of models to generate a second output by processing the input included in the output generation request;
• and
• enabling access to the second output.