Systems and methods for improved file processing

A non-transitory computer-readable medium storing instructions thereon that, when executed by one or more processors, cause the one or more processors to:

• extract, using an extraction function, (i) textual data from a file and (ii) metadata from the file in a hierarchical format configured to preserve hierarchical data structures of the file including at least a first one-to-many relationship indicated in the textual data;
• convert, by an embedding function, the extracted textual data into at least one embedding corresponding to the file;
• decompose, using a smart chunking function based on the extracted metadata, the extracted textual data into a set of text chunks at least partially by performing (i) a semantic search and (ii) a keyword search on the extracted textual data;
• cluster the file into a first cluster of a set of clusters by:
• executing a semantic clustering function to cluster the at least one embedding into a semantic cluster of a set of semantic clusters,
• executing a structural clustering process to cluster the set of text chunks into a structural cluster of a set of structural clusters, and
• clustering the file into the first cluster based on a similarity between the semantic cluster and the structural cluster;
• generate, by a language model and based on the first cluster, an output corresponding to the file;
• and
• cause the output to be displayed for viewing by a user.

A computer-implemented method for improved file processing, the computer-implemented method comprising:

• extracting, by one or more processors and using an extraction function, (i) textual data from a file and (ii) metadata from the file in a hierarchical format configured to preserve hierarchical data structures of the file including at least a first one-to-many relationship indicated in the textual data;
• converting, by the one or more processors using an embedding function, the extracted textual data into at least one embedding corresponding to the file;
• decomposing, by the one or more processors using a smart chunking function based on the extracted metadata, the extracted textual data into a set of text chunks at least partially by performing (i) a semantic search and (ii) a keyword search on the extracted textual data;
• clustering, by the one or more processors, the file into a first cluster of a set of clusters by:
• executing a semantic clustering function to cluster the at least one embedding into a semantic cluster of a set of semantic clusters,
• executing a structural clustering process to cluster the set of text chunks into a structural cluster of a set of structural clusters, and
• clustering the file into the first cluster based on a similarity between the semantic cluster and the structural cluster;
• generating, by the one or more processors using a language model and based on the first cluster, an output corresponding to the file;
• and
• causing, by the one or more processors, the output to be displayed for viewing by a user.