Agentic data fabric with data profiler agents

A system comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the system to perform:
• deploying, by agentic an artificial intelligence (AI) orchestration system of a multi-tenant platform, a plurality of AI agents, wherein at least one of the plurality of AI agents includes an orchestrator agent, wherein at least one other AI agent of the plurality of AI agents includes a data profiler agent, wherein the orchestrator agent is configured to instruct the plurality of AI agents, wherein each of the plurality of AI agents execute a respective machine learning model, and wherein the respective machine learning model executed by the AI orchestrator agent and the respective machine learning model executed by the data profiler agent each comprise a respective large language model (LLM);
• receiving, by the orchestrator agent, a natural language query through a conversational graphical user interface (GUI);
• instructing, by the orchestrator agent based on the natural language query, the data profiler agent to profile an external dataset prior to ingestion of the external dataset by the multi-tenant platform;
• profiling, by the data profiler agent in response to the instruction from the orchestrator agent, the external dataset intended for integration into the multi-tenant platform, the profiling the external dataset comprising:
• determining, by the data profiler agent, data quality metrics for the dataset, the data quality metrics including:
• (i) for each of one or more attributes of the dataset, a completeness metric determined as a function of a count of records having a non-null attribute value for the attribute relative to a total count of records in the dataset,
• (ii) for each of the one or more attributes of the data set, a consistency metric determined as a function of conformance of attribute values for the attribute to at least one validation rule, and
• (iii) for at least one identifier attribute of the one or more attributes of the data, a uniqueness metric determined as a function of duplicate identifier attribute values among the records;
• identifying, by the data profiler agent based on the data quality metrics and one or more data quality metric threshold values, a plurality of data quality issues in the dataset that are predicted to adversely impact entity resolution operations of the multi-tenant platform;
• identifying, by the data profiler agent based on the metrics and the one or more metric threshold values, at least a portion of the dataset that is predicted not to adversely impact entity resolution operations of the multi-tenant platform;
• generating, by the data profiler agent in response to identifying the data quality issues, a plurality of recommended data improvement actions, wherein the plurality of improvement actions includes corrections or enrichments for the dataset;
• generating, by the data profiler agent, data profiling results for the external dataset, the data profiling results comprising at least (i) the data quality metrics, (ii) an identification of the one or more data quality issues, and (iii) the one or more recommended data improvement actions;
• and
• storing the data profiling results for the external dataset;
• ingesting the at least a portion of the dataset that is predicted not to adversely impact the entity resolution operations of the multi-tenant platform into the multi-tenant platform;
• executing a first data improvement action of the plurality of data improvement actions on the portion of the dataset associated with the first data improvement action, thereby generating a first improved portion of the dataset;
• and
• ingesting the first improved portion of the dataset.