Universal container for structured and unstructured data from disparate sources
AssigneeBioInventors & Entrepreneurs Network, LLC
InventorsMichael Seul, Reinhard Klemm
Life of this patent
Estimating the term…
Abstract
The present invention discloses a universal data container, in the form of a generic tabular representation, and methods of navigating and transforming said generic tabular representation by invoking standard SQL-Expressions, implementing the equivalent of XPath expressions, obviating the need for SQL extensions or format-specific query languages, the container accommodating disparate sources regardless of type or format, placing structured and unstructured sources on an equal footing.
Claims23
A method performed by way of a program executed on at least one processor, comprising:
- storing content from data sources each having an arbitrarily deeply nested source data structure, and representing the data sources in a tabular representation that encodes the arbitrarily deeply nested source data structure, wherein the data sources which must be accepted for the storing and representing are:
- tables, API payloads, web pages, databases, documents, and files, in any of the file formats XML, xHTML, HTML, SVG, JSON, FHIR, YAML, RDF/JSON-LD, CSV and TXT, and wherein the tabular representation has at least a first PathCode, a Key and a Value column, and has a total number of columns independent of said source data structure, and wherein the storing and the representing includes:
- mapping the arbitrarily deeply nested source data structure to the tabular representation by:
- recursively traversing an in-memory representation of the said arbitrarily deeply nested source data structure and concurrently constructing at least a first type of PathCode, wherein each at least first PathCode comprises a unique encoding of an absolute path to a designated node among a collection of nodes in said arbitrarily deeply nested source data structure, thereby encoding the arbitrarily deeply nested source data structure;
- extracting information associated with said designated node;
- and
- recording said at least first PathCode and the information in said at least first PathCode, Key and Value columns of the tabular representation;
- and
- navigating and transforming the tabular representation by non-recursive SQL expressions, thereby providing a generic tabular representation for arbitrarily deeply nested data sources of disparate type or format, capable of being queried by non-recursive SQL-Expressions.
A method using a program executed on at least one processor, comprising:
- storing content from data sources each having an arbitrarily deeply nested source data structure, and representing the data sources in a tabular representation that encodes the arbitrarily deeply nested source data structure, wherein the data sources which must be accepted for the storing and representing are:
- tables, API payloads, web pages, databases, documents, and files, in any of the file formats XML, xHTML, HTML, SVG, JSON, FHIR, YAML, RDF/JSON-LD, CSV and TXT, and wherein the storing and the representing includes:
- mapping the arbitrarily deeply nested source data structure to a tabular representation having at least a first PathCode, a Key and a Value field column, and having a total number of columns independent of the source data structure, by:
- recursively traversing an in-memory representation of the said data structure, and concurrently therewith constructing at least the first PathCode comprising a unique encoding of absolute paths to nodes in said arbitrarily deeply nested source data structure, thereby preserving the hierarchical organization of the deeply nested source data structure;
- and
- recording the first PathCode in the first PathCode column, and recording information associated with said nodes in the Key and Value columns of the tabular representation;
- querying said tabular representation by invoking t-Expressions for navigating and transforming the tabular representation or content selected therefrom;
- and
- thereby obtaining query results from an arbitrarily deeply nested source structure without invoking recursive SQL expressions.
A method using a program executed on at least one processor, comprising:
- storing content from data sources each having an arbitrarily deeply nested source data structure in a tabular representation that encodes the arbitrarily deeply nested source data structure, wherein the data sources which must be accepted for the storing are:
- tables, API payloads, web pages, databases, documents, and files, in any of the file formats XML, xHTML, HTML, SVG, JSON, FHIR, YAML, RDF/JSON-LD, CSV and TXT, and wherein the storing comprises mapping the arbitrarily deeply nested source data structure to a tabular representation having at least a first PathCode, a Key and a Value field column, and having a total number of columns independent of the source data structure, by:
- recursively traversing an in-memory representation of the said data structure, and concurrently therewith constructing at least the first PathCode comprising a unique encoding of absolute paths to nodes in said arbitrarily deeply nested source data structure, thereby preserving the hierarchical organization of the deeply nested source data structure;
- and
- recording the first PathCode in the first PathCode column, and information associated with said nodes in the Key and Value columns of the tabular representation;
- navigating and transforming the tabular representation or content selected therefrom by invoking t-Expressions to query said tabular representation;
- and
- thereby obtaining query results from an arbitrarily deeply nested source structure without invoking recursive SQL expressions.
Dependencies are inferred from the claim text.