# US 12,111,797 B1 · Schema inference system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,797 B1 - Document ID: US12111797B1 - Status: Granted - Kind: B1 - Application number: 18/371,931 - Filed: 2023-09-22 - Granted: 2024-10-08 - Earliest claimed: 2023-09-22 - Assignees: Storytellers.ai LLC - Applicants: Storytellers.ai LLC - Inventors: Kirsten Rae Lum; Wing Yew Lum; Christopher John Gutierrez - Web page: https://patentgrep.com/patent/US12111797B1 ## CPC - G06F 16/211: {Schema design and management} - G06F 16/221: {Column-oriented storage; Management thereof} - G06F 16/24544: {Join order optimisation} - G06F 16/2456: {Join operations} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Concrete data types for raw data organized in one or more columns of one or more tables may be determined. Functional data types for the one or more columns may be determined based on the raw data and the concrete data types such that a portion of the one or more columns may be associated with an identifier data type. Relationships between the one or more tables may be determined based on the portion of the one or more columns associated with the identifier data type. A schema representing the raw data may be generated based on the one or more relationships and the one or more tables. ## Claims ### Claim 1 A method for managing data in a network using one or more processors to execute instructions that are configured to cause actions, comprising: employing raw data to determine one or more tables with one or more columns, wherein the raw data includes metadata and is organized in the one or more columns of one or more tables; employing the raw data to determine one or more concrete data types that correspond to the raw data organized in the one or more columns of the one or more tables; determining one or more functional data types for the one or more columns of the one or more tables based on the raw data and correspondence with the one or more concrete data types, wherein a portion of the one or more columns are associated with an identifier data type; determining one or more existing relationships associated with the one or more tables based on one or more of a metric or a statistical feature of the metadata for the raw data; determining one or more inferred relationships associated with the one or more tables based on the portion of the one or more columns associated with the identifier data type and one or more common values associated with the one or more functional data types for the one or more columns; employing one or more large language models to predict one or more inferred relationships associated with the one or more tables, wherein the one or more large language models are trained by one or more semantic relationship evaluators to generate the one or more predicted inferred relationships between the one or more tables; executing one or more join expressions on two or more predicted inferred relationships for validation, wherein each predicted inferred relationship associated with one or more join execution errors is invalidated; and generating a schema representing the raw data based on the one or more existing relationships, the one or more inferred relationships, each predicted inferred relationship that is validated, and the one or more tables. ### Claim 2 (depends on 1) The method of claim 1, further comprising: providing one or more queries that include one or more join expressions that are based on the one or more relationships; and validating the schema based on an execution of the one or more queries, wherein the validation is based on an absence of errors associated with the execution of the one or more queries. ### Claim 3 (depends on 1) The method of claim 1, further comprising: employing one or more data sources to provide the raw data, wherein the raw data includes one or more of a comma separated value file, a spreadsheet, an extensible markup language file, a database export file, an application export file, or a network connection to data source. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the one or more concrete data types, further comprises: determining one or more evaluators that declare one or more operations to infer the one or more concrete data types; and inferring the one or more concrete data types based on the one or more operations, wherein the one or more concrete data types include one or more of an integer type, a floating point type, a character type, or a string type. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the one or more functional data types, further comprises: determining one or more evaluators that declare one or more operations to infer the one or more functional data types; and inferring the one or more functional data types based on the one or more operations, wherein the one or more functional data types include one or more of the identifier data type, a category data type, a text data type, or a numeric data type. ### Claim 6 (depends on 1) The method of claim 1, further comprising: determining one or more metrics associated with the one or more columns based on the raw data, wherein the one or more metrics include one or more of a row count, a median value, a mean value, a cardinality, or a distribution of values; and providing a profile for each column, wherein the one or more metrics for each column are included in the profile; and employing the profile for each column to further determine the one or more concrete data types or the one or more functional data types. ### Claim 7 (depends on 1) The method of claim 1, further comprising: determining a portion of the one or more relationships based on an evaluation of header information associated with the one or more columns, wherein the portion of the one or more relationships are associated with a portion of the one or more columns that are associated with related header information. ### Claim 8 (depends on 1) The method of claim 1, further comprising: determining a portion of the one or more relationships based on an evaluation of one or more statistical features of a portion of the raw data associated with the portion of the one or more relationships. ### Claim 9 (depends on 1) The method of claim 1, further comprising: determining a portion of the one or more relationships based on an evaluation of one or more semantic characteristics of the one or more columns, wherein the evaluation of one or more semantic characteristics is based on a response from a large language model that is trained by a prompt that includes information associated with the one or more columns. ### Claim 10 A network computer for managing data, comprising: a memory that stores at least instructions; and one or more processors that execute instructions that are configured to cause actions, including: employing raw data to determine one or more tables with one or more columns, wherein the raw data includes metadata and is organized in the one or more columns of one or more tables; employing the raw data to determine one or more concrete data types that correspond to the raw data organized in the one or more columns of the one or more tables; determining one or more functional data types for the one or more columns of the one or more tables based on the raw data and correspondence with the one or more concrete data types, wherein a portion of the one or more columns are associated with an identifier data type; determining one or more existing relationships associated with the one or more tables based on one or more of a metric or a statistical feature of the metadata for the raw data; determining one or more inferred relationships associated with the one or more tables based on the portion of the one or more columns associated with the identifier data type and one or more common values associated with the one or more functional data types for the one or more columns; employing one or more large language models to predict one or more inferred relationships associated with the one or more tables, wherein the one or more large language models are trained by one or more semantic relationship evaluators to generate the one or more predicted inferred relationships between the one or more tables; executing one or more join expressions on two or more predicted inferred relationships for validation, wherein each predicted inferred relationship associated with one or more join execution errors is invalidated; and generating a schema representing the raw data based on the one or more existing relationships, the one or more inferred relationships, each predicted inferred relationship that is validated, and the one or more tables. ### Claim 11 (depends on 10) The network computer of claim 10, further comprising: providing one or more queries that include one or more join expressions that are based on the one or more relationships; and validating the schema based on an execution of the one or more queries, wherein the validation is based on an absence of errors associated with the execution of the one or more queries. ### Claim 12 (depends on 10) The network computer of claim 10, further comprising: employing one or more data sources to provide the raw data, wherein the raw data includes one or more of a comma separated value file, a spreadsheet, an extensible markup language file, a database export file, an application export file, or a network connection to data source. ### Claim 13 (depends on 10) The network computer of claim 10, wherein determining the one or more concrete data types, further comprises: determining one or more evaluators that declare one or more operations to infer the one or more concrete data types; and inferring the one or more concrete data types based on the one or more operations, wherein the one or more concrete data types include one or more of an integer type, a floating point type, a character type, or a string type. ### Claim 14 (depends on 10) The network computer of claim 10, wherein determining the one or more functional data types, further comprises: determining one or more evaluators that declare one or more operations to infer the one or more functional data types; and inferring the one or more functional data types based on the one or more operations, wherein the one or more functional data types include one or more of the identifier data type, a category data type, a text data type, or a numeric data type. ### Claim 15 (depends on 10) The network computer of claim 10, further comprising: determining one or more metrics associated with the one or more columns based on the raw data, wherein the one or more metrics include one or more of a row count, a median value, a mean value, a cardinality, or a distribution of values; and providing a profile for each column, wherein the one or more metrics for each column are included in the profile; and employing the profile for each column to further determine the one or more concrete data types or the one or more functional data types. ### Claim 16 (depends on 10) The network computer of claim 10, further comprising: determining a portion of the one or more relationships based on an evaluation of header information associated with the one or more columns, wherein the portion of the one or more relationships are associated with a portion of the one or more columns that are associated with related header information. ### Claim 17 (depends on 10) The network computer of claim 10, further comprising: determining a portion of the one or more relationships based on an evaluation of one or more statistical features of a portion of the raw data associated with the portion of the one or more relationships. ### Claim 18 (depends on 10) The network computer of claim 10, further comprising: determining a portion of the one or more relationships based on an evaluation of one or more semantic characteristics of the one or more columns, wherein the evaluation of one or more semantic characteristics is based on a response from a large language model that is trained by a prompt that includes information associated with the one or more columns. ### Claim 19 A processor readable non-transitory storage media that includes instructions configured for managing data over a network, wherein execution of the instructions by one or more processors on one or more network computers performs actions, comprising: employing raw data to determine one or more tables with one or more columns, wherein the raw data includes metadata and is organized in the one or more columns of one or more tables; employing the raw data to determine one or more concrete data types that correspond to the raw data organized in the one or more columns of the one or more tables; determining one or more functional data types for the one or more columns of the one or more tables based on the raw data and correspondence with the one or more concrete data types, wherein a portion of the one or more columns are associated with an identifier data type; determining one or more existing relationships associated with the one or more tables based on one or more of a metric or a statistical feature of the metadata for the raw data; determining one or more inferred relationships associated with the one or more tables based on the portion of the one or more columns associated with the identifier data type and one or more common values associated with the one or more functional data types for the one or more columns; employing one or more large language models to predict one or more inferred relationships associated with the one or more tables, wherein the one or more large language models are trained by one or more semantic relationship evaluators to generate the one or more predicted inferred relationships between the one or more tables; executing one or more join expressions on two or more predicted inferred relationships for validation, wherein each predicted inferred relationship associated with one or more join execution errors is invalidated; and generating a schema representing the raw data based on the one or more existing relationships, the one or more inferred relationships, each predicted inferred relationship that is validated, and the one or more tables. ### Claim 20 (depends on 19) The media of claim 19, further comprising: providing one or more queries that include one or more join expressions that are based on the one or more relationships; and validating the schema based on an execution of the one or more queries, wherein the validation is based on an absence of errors associated with the execution of the one or more queries. ### Claim 21 (depends on 19) The media of claim 19, further comprising: employing one or more data sources to provide the raw data, wherein the raw data includes one or more of a comma separated value file, a spreadsheet, an extensible markup language file, a database export file, an application export file, or a network connection to data source. ### Claim 22 (depends on 19) The media of claim 19, wherein determining the one or more concrete data types, further comprises: determining one or more evaluators that declare one or more operations to infer the one or more concrete data types; and inferring the one or more concrete data types based on the one or more operations, wherein the one or more concrete data types include one or more of an integer type, a floating point type, a character type, or a string type. ### Claim 23 (depends on 19) The media of claim 19, wherein determining the one or more functional data types, further comprises: determining one or more evaluators that declare one or more operations to infer the one or more functional data types; and inferring the one or more functional data types based on the one or more operations, wherein the one or more functional data types include one or more of the identifier data type, a category data type, a text data type, or a numeric data type. ### Claim 24 (depends on 19) The media of claim 19, further comprising: determining one or more metrics associated with the one or more columns based on the raw data, wherein the one or more metrics include one or more of a row count, a median value, a mean value, a cardinality, or a distribution of values; and providing a profile for each column, wherein the one or more metrics for each column are included in the profile; and employing the profile for each column to further determine the one or more concrete data types or the one or more functional data types. ### Claim 25 (depends on 19) The media of claim 19, further comprising: determining a portion of the one or more relationships based on an evaluation of header information associated with the one or more columns, wherein the portion of the one or more relationships are associated with a portion of the one or more columns that are associated with related header information. ### Claim 26 (depends on 19) The media of claim 19, further comprising: determining a portion of the one or more relationships based on an evaluation of one or more statistical features of a portion of the raw data associated with the portion of the one or more relationships. ### Claim 27 (depends on 19) The media of claim 19, further comprising: determining a portion of the one or more relationships based on an evaluation of one or more semantic characteristics of the one or more columns, wherein the evaluation of one or more semantic characteristics is based on a response from a large language model that is trained by a prompt that includes information associated with the one or more columns.