Machine learning engineering through hybrid knowledge representation

A system, comprising:

• a hardware processor;
• a memory device operatively coupled with the hardware processor, the memory device storing hybrid knowledge representation structured as nodes representing machine learning workflow components and edges connecting the nodes based on relationships between the nodes;
• the hardware processor configured to at least:
• receive a specification;
• parse the specification and identify from the specification a machine learning task, wherein the specification includes at least characteristics about dataset, characteristics of model, and metric associated with machine learning for performing the machine learning task;
• search the hybrid knowledge representation for a machine learning model that performs the machine learning task;
• responsive to finding the machine learning model in the hybrid knowledge representation return the machine learning model;
• responsive to not finding in the hybrid knowledge representation the machine learning model to perform the machine learning task specified according to the specification including the characteristics about dataset, characteristics of model, and metric, search the hybrid knowledge representation for machine learning model fragments associated with the machine learning task, the hybrid knowledge representation including at least components of a machine learning pipeline including at least datasets that can be split to provide a training set, validating set and testing set, and generate a new machine learning model to perform the machine learning task by combining the machine learning model fragments.

A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to:

• receive, by the device, a specification;
• search, by the device, a hybrid knowledge representation stored on a memory device for a machine learning model corresponding to the specification, the hybrid knowledge representation structured as nodes representing machine learning workflow components and links connecting the nodes based on relationships between the nodes;
• responsive to finding the machine learning component in the hybrid knowledge representation, return, by the device, the machine learning model;
• responsive to not finding in the hybrid knowledge representation the machine learning model to perform a machine learning task specified according to the specification including characteristics about dataset, characteristics of model, and metric, search, by the device, the hybrid knowledge representation for machine learning model fragments associated with building the machine learning model, the hybrid knowledge representation including at least components of a machine learning pipeline including at least datasets that can be split to provide a training set, validating set and testing set, and generate, by the device, a new machine learning model by combining the machine learning model fragments and return the new machine learning model.