End-to-end measurement, grading and evaluation of pretrained artificial intelligence models via a graphical user interface (GUI) systems and methods

A non-transitory, computer-readable storage medium storing instructions for grading a pre-trained Artificial Intelligence (AI) model, wherein the instructions when executed by at least one data processor of a system, cause the system to:

• obtain a set of application domains of the pre-trained AI model in which a pre-trained AI model will be used,
• wherein the pre-trained AI model is configured to generate, in response to a received input, a response;
• using the set of application domains, determine a set of guidelines defining one or more operation boundaries of the pre-trained AI model by mapping each application domain of the set of application domains to one or more guidelines of the set of guidelines;
• accessing a set of assessment domains associated with the one or more guidelines of the set of guidelines,
• wherein each of the set of assessment domains includes a set of benchmarks,
• wherein each benchmark in the set of benchmarks is configured to indicate a degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with corresponding assessment domains;
• for each assessment domains in the set of assessment domains, construct a set of assessments,
• wherein each assessment is configured to test the degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with a corresponding assessment domain;
• evaluate the pre-trained AI model against the set of assessments to determine the degree of satisfaction of the pre-trained AI model with the set of guidelines for the corresponding assessment domain by transmitting a command indicating one or more assessments into an input layer of the AI model;
• using the evaluation, assign a grade, for each assessment domain, to the pre-trained AI model in accordance with the set of benchmarks for the corresponding assessment domain by;
• using the assigned grades, map the assigned grades for each assessment domain to a particular degree of satisfaction corresponding to one or more application domains of the pre-trained AI model;
• and
• generate a representation indicating application-domain-specific grades, wherein the representation includes each application domain of the pre-trained AI model and corresponding application-domain-specific grades.

A system for grading an Artificial Intelligence (AI) model, comprising:

• at least one processor;
• and
• one or more non-transitory computer-readable media storing instructions, which when executed by at least one processor, perform operations comprising:
• obtaining a set of application domains of the AI model in which an AI model will be used,
• wherein the AI model is configured to generate, in response to a received input, a response;
• using the set of application domains, determining a set of guidelines defining one or more operation boundaries of the AI model by mapping each application domain of the set of application domains to one or more guidelines of the set of guidelines;
• accessing a set of assessment domains associated with the one or more guidelines of the set of guidelines,
• wherein each of the set of assessment domains includes a set of benchmarks,
• wherein each benchmark in the set of benchmarks is configured to indicate a degree of satisfaction of the AI model with the one or more guidelines associated with corresponding assessment domains;
• for each assessment domains in the set of assessment domains, constructing a set of assessments,
• wherein each assessment is configured to test the degree of satisfaction of the AI model with the one or more guidelines associated with a corresponding assessment domain;
• evaluating the AI model against the set of assessments to determine the degree of satisfaction of the AI model with the set of guidelines for the corresponding assessment domain;
• using the evaluation, assigning a grade, for each assessment domain, to the AI model in accordance with the set of benchmarks for the corresponding assessment domain by;
• using the assigned grades, mapping the assigned grades for each assessment domain to a particular degree of satisfaction corresponding to one or more application domains of the AI model;
• generating a representation indicating application-domain-specific grades, wherein the representation includes each application domain of the AI model and corresponding application-domain-specific grades;
• and
• using the application-domain-specific grades, automatically generating a set of actions to adjust one or more parameters of the AI model to increase a degree of satisfaction of the AI model with corresponding operative boundaries of one or more application domains.