Automating data preprocessing code generation using large language models

A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for computer-executed entity matching using one or more machine learning (ML) models, the operations comprising:

• receiving a first source data schema (SDS) representing a first format for recording data in a computer-readable document;
• providing a first prompt comprising at least a portion of the first SDS;
• generating description data by processing the first prompt using a first large language model (LLM);
• providing a second prompt comprising of at least a portion of the first SDS, at least a portion of a first target data schema (TDS), and the description data;
• generating first schema conversion code (SCC) by processing the second prompt using a second LLM;
• converting a first document from the first SDS to the first TDS using the first SCC;
• processing the first document in the first TDS using a ML model to provide inference results;
• and
• automatically executing one or more tasks responsive to the inference results.

A system, comprising:

• a computing device;
• and
• a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for computer-executed entity matching using one or more machine learning (ML) models, the operations comprising:
• receiving a first source data schema (SDS) representing a first format for recording data in a computer-readable document;
• providing a first prompt comprising at least a portion of the first SDS;
• generating description data by processing the first prompt using a first large language model (LLM);
• providing a second prompt comprising of at least a portion of the first SDS, at least a portion of a first target data schema (TDS), and the description data;
• generating first schema conversion code (SCC) by processing the second prompt using a second LLM;
• converting a first document from the first SDS to the first TDS using the first SCC;
• processing the first document in the first TDS using a ML model to provide inference results;
• and
• automatically executing one or more tasks responsive to the inference results.