Automatically curating existing machine learning projects into a corpus adaptable for use in new machine learning projects

One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause a system to perform operations, the operations comprising:

• gathering a set of machine learning (ML) projects from one or more repositories of ML projects based on a filtering criteria;
• ensuring executability of ML pipelines in the set of ML projects;
• identifying irrelevant portions of the ML pipelines in the set of ML projects;
• generating quality features for the set of ML projects;
• generating diversity features for the set of ML projects;
• selecting a subset of ML projects from the set of ML projects based on the quality features and the diversity features;
• and
• storing the subset of ML projects in a corpus of ML projects that may be adapted for use in new ML projects.

A system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed by the one or more processors, cause the system to perform operations, the operations comprising:
• gathering a set of machine learning (ML) projects from one or more repositories of ML projects based on a filtering criteria;
• ensuring executability of ML pipelines in the set of ML projects;
• identifying irrelevant portions of the ML pipelines in the set of ML projects;
• generating quality features for the set of ML projects;
• generating diversity features for the set of ML projects;
• selecting a subset of ML projects from the set of ML projects based on the quality features and the diversity features;
• and
• storing the subset of ML projects in a corpus of ML projects that may be adapted for use in new ML projects.