Automating bias evaluation for machine learning projects

A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:

• obtaining descriptive information for a first machine learning project;
• identifying, based on the descriptive information, a plurality of past machine learning projects which are detected to be similar to the first machine learning project;
• retrieving a plurality of digital documents, wherein each digital document of the plurality of digital documents describes a bias evaluation pipeline that was used to evaluate a different past machine learning project of the plurality of past machine learning projects;
• detecting a common bias evaluation pipeline step among at least a subset of the plurality of digital documents;
• extracting, from the subset of the plurality of digital documents, a snippet of machine-executable code that corresponds to the common bias evaluation pipeline step;
• modifying the snippet of machine-executable code with use case data that is specific to the first machine learning project to generate modified machine-executable code;
• and
• generating a proposed bias evaluation pipeline for evaluating the first machine learning project, wherein the proposed bias evaluation pipeline includes the modified machine-executable code.

A device comprising:

• a processing system including at least one processor;
• and
• a non-transitory computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
• obtaining descriptive information for a first machine learning project;
• identifying, based on the descriptive information, a plurality of past machine learning projects which are detected to be similar to the first machine learning project;
• retrieving a plurality of digital documents, wherein each digital document of the plurality of digital documents describes a bias evaluation pipeline that was used to evaluate a different past machine learning project of the plurality of machine learning projects;
• detecting a common bias evaluation pipeline step among at least a subset of the plurality of digital documents;
• extracting, from the subset of the plurality of digital documents, a snippet of machine-executable code that corresponds to the common bias evaluation pipeline step;
• modifying the snippet of machine-executable code with use case data that is specific to the first machine learning project to generate modified machine-executable code;
• and
• generating a proposed bias evaluation pipeline for evaluating the first machine learning project, wherein the proposed bias evaluation pipeline includes the modified machine-executable code.