Computer implemented systems and methods for processing semi-structured documents

A computer system comprising:

• one or more processors;
• and
• a non-transitory computer readable medium having stored thereon one or more programs, which when executed by the one or more processors, causes the one or more processors to:
• receive a plurality of semi-structured documents from a data store, wherein the plurality of semi-structured documents comprise a plurality of fields having associated values;
• analyze the plurality of semi-structured documents to determine particular fields in each of the plurality of semi-structured documents, wherein a first field comprises a first document collection and a second field comprises a second document collection;
• receiving further information from a user designating the first document collection and the second document collection as a linked collection;
• determine a frequency that each particular field appears in the plurality of semi-structured documents;
• store in a memory, the particular fields and associated frequencies in a tree data structure, wherein the storing comprises storing an identifier of the linked collections with the first document collection and with the second document collection;
• display a graphical visualization reflecting a hierarchy of the plurality of fields, the graphical visualization comprising a leaf node having an atomic value, the graphical visualization further indicating the linked collections by repeating the first document collection and the second document collection in the tree data structure;
• receive an interaction with the leaf node of the graphical visualization, wherein the interaction comprises a filter applied to the first document collection;
• in response to the interaction with the leaf node, generate a flattened subset of the particular fields and associated values, wherein the flattened subset reflects removal of an element from the first document collection, with removal of the element from the second document collection being based upon a consistency with the linked collection;
• display the flattened subset in a table;
• determine a similarity between nodes in the graphical visualization;
• collapse the nodes into a single graphical element in the visual representation when the similarity is greater than a threshold;
• and
• collapse the tree data structure in memory when the similarity is greater than the threshold.

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions for:

• receiving a plurality of semi-structured documents from a data store, wherein the plurality of semi-structured documents comprise a plurality of fields having associated values;
• analyzing the plurality of semi-structured documents to determine particular fields in each of the plurality of semi-structured documents, wherein a first field comprises a first document collection and a second field comprises a second document collection;
• receiving further information from a user designating the first document collection and the second document collection as a linked collection;
• determining a frequency that each particular field appears in the plurality of semi-structured documents;
• storing in a memory the particular fields and associated frequencies in a tree data structure, wherein the storing comprises storing an identifier of the linked collections with the first document collection and with the second document collection;
• displaying a graphical visualization reflecting a hierarchy of the plurality of fields, the graphical visualization comprising a leaf node having an atomic value, the graphical visualization further indicating the linked collections by repeating the first document collection and the second document collection in the tree data structure;
• receiving an interaction with the leaf node of the graphical visualization, wherein the interaction comprises a filter applied to the first document collection;
• in response to the interaction with the leaf node, generating a flattened subset of the particular fields and associated values, wherein the flattened subset reflects removal of an element from the first document collection, with removal of the element from the second document collection being based upon a consistency with the linked collection;
• displaying the flattened subset in a table;
• determining a similarity between nodes in the graphical visualization;
• collapsing the nodes into a single graphical element in the visual representation when the similarity is greater than a threshold;
• and
• collapsing the tree data structure in the memory when the similarity is greater than the threshold.