Data Matching Using a Pipeline of Models

A non-transitory computer-readable medium having instructions stored thereon that are executable by a computing device to perform operations comprising:

• embedding values of records of a first data source and a second, different data source within a multi-dimensional embedding space;
• calculating, for respective pairs of clusters within the multi-dimensional embedding space, similarity scores;
• identifying, based on the similarity scores, correlations between values of records from the first data source and values of records from the second, different data source;
• generating, based on the identified correlations, a set of matching features;
• inputting the set of matching features to a matching model, wherein the matching model is a machine learning model;
• combining, based on output of the matching model for the set of matching features, matching records from the first data source and the second, different data source, wherein the combining produces an enhanced data source;
• and
• inputting the enhanced data source into a prediction model, wherein the prediction model is a machine learning classifier, wherein the matching model and the prediction model are included in a pipeline of machine learning models.

A method for automatically generating a prediction using a pipeline of machine learning models that includes a matching model and a prediction model, the method comprising:

• executing, by a computer system, the matching model included in the pipeline of machine learning models, wherein the matching model determines whether records from a first data source and records from a second, different data source match;
• generating, by the computer system based on output of the matching model for the first data source and the second, different data source, an enhanced data source;
• executing, by the computer system based on the enhanced data source, the prediction model, wherein the pipeline of machine learning models is generated by:
• embedding values of records of the first data source and the second, different data source within a multi-dimensional embedding space;
• identifying, based on similarities between embedded values of the first data source and the second, different data source within the multi-dimensional embedding space, correlations between values of records from the first data source and values of records from the second, different data source;
• generating, based on the identified correlations, a set of matching features;
• inputting the set of matching features to the matching model, wherein the matching model is a machine learning model;
• combining, based on output of the matching model for the set of matching features, similar records from the first data source and the second, different data source, wherein the combining generates the enhanced data source;
• and
• inputting the enhanced data source into the prediction model.