Artificial intelligence based generation of data connectors

A non-transitory computer-readable medium comprising memory with instructions encoded thereon for generation of a data connector, the instructions comprising instructions to:

• access an index based on representations of documents storing information describing APIs (Application Programming Interfaces) for accessing data source systems;
• determine a subset of documents referenced by the index having at least a threshold similarity to required connector information;
• generate a prompt for a large language model (LLM) based on information extracted from the index, the prompt requesting information associated with building a new data connector for accessing data from the data source systems;
• provide the generated prompt to the large language model along with instructions to query the subset of documents with the generated prompt;
• receive a respective interim response from each respective document of the subset;
• receive, from the large language model, a response describing information associated with the new data connector, the response comprising each interim response;
• determine a measure of ambiguity for the response;
• responsive to the measure of ambiguity indicating more than a threshold ambiguity in the response based on two or more interim responses comprising conflicting information, generate an additional prompt for the large language model the additional prompt automatically requesting more specific information compared to the information received in the response, the more specific information requested to resolve the conflicting information, wherein as further conflicting information is received, further prompts requesting further resolution are requested until an ultimate response without an ambiguity is received;
• generate the data connector based on a connector knowledge representation corresponding to the ultimate response;
• and
• automatically deploy the data connector for accessing data from the data source systems.

A system for generation of a data connector, the system comprising:

• memory with instructions encoded thereon;
• and
• one or more processors that, when executing the instructions, are caused to perform operations comprising:
• accessing an index based on representations of documents storing information describing APIs (Application Programming Interfaces) for accessing data source systems;
• determining a subset of documents referenced by the index having at least a threshold similarity to required connector information;
• generating a prompt for a large language model (LLM) based on information extracted from the index, the prompt requesting information associated with building a new data connector for accessing data from the data source systems;
• providing the generated prompt to the large language model along with instructions to query the subset of documents with the generated prompt;
• receiving a respective interim response from each respective document of the subset;
• receiving, from the large language model, a response describing information associated with the new data connector, the response comprising each interim response;
• determining a measure of ambiguity for the response;
• responsive to the measure of ambiguity indicating more than a threshold ambiguity in the response based on two or more interim responses comprising conflicting information, generating an additional prompt for the large language model the additional prompt automatically requesting more specific information compared to the information received in the response, the more specific information requested to resolve the conflicting information, wherein as further conflicting information is received, further prompts requesting further resolution are requested until an ultimate response without an ambiguity is received;
• generating the data connector based on a connector knowledge representation corresponding to the ultimate response;
• and
• automatically deploying the data connector for accessing data from the data source systems.