# US 9,524,473 B2 · System and method for auto-query generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,524,473 B2 - Document ID: US9524473B2 - Status: Granted - Kind: B2 - Application number: 14/202,780 - Filed: 2014-03-10 - Published: 2014-09-11 - Granted: 2016-12-20 - Earliest claimed: 2012-08-31 - Assignees: Nutonian, Inc. - Applicants: Nutonian, Inc. - Inventors: Michael Schmidt - Web page: https://patentgrep.com/patent/US9524473B2 ## CPC - G06N 20/00: Machine learning - G06F 16/2428: {Query predicate definition using graphical user interfaces, including menus and forms (G06F16/2423 takes precedence)} - G06F 16/2453: Query optimisation - G06F 40/18: of spreadsheets (form-filling G06F40/174) - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Various systems and methods provide an intuitive user interface that enables automatic specification of queries and constraints for analysis by ML component. Various implementations provide methodologies for automatically formulating machine learning (“ML”) and optimization queries. The automatic generation of ML and/or optimization queries can be configured to use examples to facilitate formulation of ML and optimization queries. One example method includes accepting input data specifying variables and data values associated with the variables. Within the input data any unspecified data records are identified, and a relationship between the variables specified in the input data and a variable associated with the at least one unspecified data record is automatically determined. The relationship can be automatically determined based on training data contained within the input data. Once a relationship is established a ML problem can be automatically generated. ## Claims ### Claim 1 A computer implemented method, comprising: accessing, by a computer system, input data comprising: training data comprising a first plurality of data records, each of the first plurality of data records having a numeric data value for each of a plurality of variables; and a second plurality of data records, each of the second plurality of data records having a numeric data value for at least one of the plurality of variables; identifying, by the computer system, at least a first not fully specified data record within the second plurality of data records that does not include at least a first numeric data value for at least a first variable of the plurality of variables; determining, automatically by the computer system and based on the training data, a relationship between the first variable and at least one other variable in the plurality of variables, wherein determining the relationship comprises: generating, using a machine learning technique and the first plurality of records in the training data, a mathematical model representing the relationship between the first variable and the at least one other variable; and obtaining, using the mathematical model representing the relationship between the first variable and the at least one other variable, at least the first numeric data value for the first variable in the first not fully specified data record. ### Claim 2 A system comprising: at least one processor configured to perform: accessing input data comprising: training data comprising a first plurality of data records, each of the first plurality of data records having a numeric data value for each of a plurality of variables; and a second plurality of data records, each of the second plurality of data records having a numeric data value for at least one of the plurality of variables; identifying at least a first not fully specified data record within the second plurality of data records that does not include at least a first numeric data value for at least a first variable of the plurality of variables; determining, automatically and based on the training data, a relationship between the first variable and at least one other variable in the plurality of variables, wherein determining the relationship comprises: generating, using a machine learning technique and the first plurality of records in the training data, a mathematical model representing the relationship between the first variable and the at least one other variable; and obtaining, using the mathematical model representing the relationship between the first variable and the at least one other variable, at least the first numeric data value for the first variable in the first not fully specified data record. ### Claim 3 (depends on 1) The method according to claim 1, wherein generating the mathematical model comprises identifying a function for calculating a value of the first variable given at least one value for the at least one other variable. ### Claim 4 (depends on 1) The method according to claim 1, further comprising displaying, by the computer system, the relationship between the first variable and the at least one other variable. ### Claim 5 (depends on 1) The method according to claim 1, wherein accessing the input data includes receiving, within a spreadsheet user interface, a selection of the first plurality of data records and the second plurality of data records. ### Claim 6 (depends on 1) The method according to claim 1, further comprising determining a confidence value associated with the first numeric data value. ### Claim 7 (depends on 1) The method according to claim 1, wherein generating the mathematical model representing the relationship between the first variable and the at least one other variable is performed by using symbolic regression. ### Claim 8 (depends on 1) The method according to claim 1, wherein accessing the input data is performed using a spreadsheet user interface, and wherein generating the mathematical model includes: generating one or more candidate query patterns that relate a blank cell in the spreadsheet user interface to its surrounding non-blank cells in the spreadsheet user interface; for each pattern in the one or more candidate query patterns: collecting data from the spreadsheet user interface that matches the pattern, generating, using the machine learning technique and the collected data, a respective candidate mathematical model representing a relationship between a variable associated with the blank cell and one or more variables associated with the non-blank cells; determining a confidence associated with the generated candidate mathematical model; and selecting the generated candidate mathematical model having a highest confidence level as the mathematical model. ### Claim 9 (depends on 1) The method according to claim 1, wherein the first not fully specified data record does not include a second numeric data value for a second variable of the plurality of variables, the method further comprising: determining the second numeric data value based, at least in part, on the first numeric data value. ### Claim 10 (depends on 1) The method according to claim 1, wherein generating the mathematical model representing the relationship between the first variable and the at least one other variable comprises generating a mathematical model that may be used for estimating a value of the first variable given at least one respective value for the at least one other variable. ### Claim 11 (depends on 1) The method according to claim 1, wherein the first not fully specified data record does not include a second numeric data value for a second variable of the plurality of variables and does include a third numeric data value for a third variable of the plurality of variables, and wherein the obtaining comprises obtaining, using the mathematical model and at least one optimization technique, the first numeric data value and the second numeric data value based on the third numeric data value. ### Claim 12 (depends on 2) The system according to claim 2, wherein generating the mathematical model comprises identifying a function for calculating a value of the first variable given at least one value for the at least one other variable. ### Claim 13 (depends on 2) The system according to claim 2, further comprising a display, wherein the at least one processor is configured to cause the display to show the relationship between the first variable and the at least one other variable. ### Claim 14 (depends on 2) The system according to claim 2, wherein accessing the input data includes receiving, within a spreadsheet user interface, a selection of the first plurality of data records and the second plurality of data records. ### Claim 15 (depends on 2) The system according to claim 2, wherein the at least one processor is further configured to determine a confidence value associated with the first numeric data value. ### Claim 16 (depends on 2) The system according to claim 2, wherein generating the mathematical model representing the relationship between the first variable and the at least one other variable is performed by using symbolic regression. ### Claim 17 (depends on 2) The system according to claim 2, wherein accessing the input data is performed using a spreadsheet user interface, and wherein generating the mathematical model includes: generating one or more candidate query patterns that relate a blank cell in the spreadsheet user interface to its surrounding non-blank cells in the spreadsheet user interface; for each pattern in the one or more candidate query patterns: collecting data from the spreadsheet user interface that matches the pattern, generating, using the machine learning technique and the collected data, a respective candidate mathematical model representing a relationship between a variable associated with the blank cell and one or more variables associated with the non-blank cells; determining a confidence associated with the generated candidate mathematical model; and selecting the generated candidate mathematical model having a highest confidence level as the mathematical model. ### Claim 18 (depends on 2) The system according to claim 2, wherein the first not fully specified data record does not include a second numeric data value for a second variable of the plurality of variables, and wherein the at least one processor is further configured to perform determining the second numeric data value based, at least in part, on the first numeric data value. ### Claim 19 (depends on 2) The system according to claim 2, wherein generating the mathematical model representing the relationship between the first variable and the at least one other variable comprises generating a mathematical model that may be used for estimating a value of the first variable given at least one respective value for the at least one other variable. ### Claim 20 (depends on 2) The system according to claim 2, wherein the first not fully specified data record does not include a second numeric data value for a second variable of the plurality of variables and does include a third numeric data value for a third variable of the plurality of variables, and wherein the obtaining comprises obtaining, using the mathematical model and at least one optimization technique, the first numeric data value and the second numeric data value based on the third numeric data value. ### Claim 21 (depends on 5) The method according to claim 5, wherein the identifying, determining, and obtaining occur in response to receiving the selection. ### Claim 22 (depends on 6) The method according to claim 6, further comprising encoding, visually, the confidence value associated with the first numeric data value. ### Claim 23 (depends on 8) The method according to claim 8, wherein generating the one or more candidate query patterns includes generating one or more patterns including both absolute cell positions and cell positions that are relative to the first not fully specified data record in the spreadsheet user interface. ### Claim 24 (depends on 10) The method according to claim 10, comprising using the mathematical model to estimate values of the first variable in the second plurality data records based on at least one value of the at least one other variable in the second plurality of data records. ### Claim 25 (depends on 11) The method according to claim 11, wherein the optimization technique comprises a gradient search technique. ### Claim 26 (depends on 11) The method according to claim 11, wherein obtaining the first and second numeric data values is performed at least in part by using the mathematical model to compute an estimate of the third numeric data value based, at least in part, on the first and second numeric data values and comparing the estimate of the third numeric data value to the third numeric data value. ### Claim 27 (depends on 11) The method according to claim 11, wherein the first not fully specified data record includes a fourth numeric data value for a fourth variable of the plurality of variables, and wherein obtaining the first and second numeric data values comprises using the fourth numeric data value as a search constraint in the optimization technique. ### Claim 28 (depends on 14) The system according to claim 14, wherein the identifying, determining, and obtaining occur in response to receiving the selection. ### Claim 29 (depends on 15) The system according to claim 15, wherein the at least one processor is further configured to encode, visually, the confidence value associated with the first numeric data value. ### Claim 30 (depends on 17) The system according to claim 17, wherein generating the one or more candidate query patterns includes generating one or more patterns including both absolute cell positions and cell positions that are relative to the first not fully specified at least one unspecified data record in the spreadsheet user interface. ### Claim 31 (depends on 19) The system according to claim 19, wherein the at least one processor is configured to perform: using the mathematical model to estimate values of the first variable in the second plurality data records based on at least one value of the at least one other variable in the second plurality of data records. ### Claim 32 (depends on 20) The system according to claim 20, wherein the optimization technique comprises a gradient search technique. ### Claim 33 (depends on 20) The system according to claim 20, wherein obtaining the first and second numeric data values is performed at least in part by using the mathematical model to compute an estimate of the third numeric data value based, at least in part, on the first and second numeric data values and comparing the estimate of the third numeric data value to the third numeric data value. ### Claim 34 (depends on 20) The system according to claim 20, wherein the first not fully specified data record includes a fourth numeric data value for a fourth variable of the plurality of variables, and wherein obtaining the first and second numeric data values comprises using the fourth numeric data value as a search constraint in the optimization technique. ### Claim 35 (depends on 22) The method according to claim 22, wherein encoding visually the confidence value associated with the first numeric data value includes displaying the first numeric data value with an intensity that depends on the confidence value. ### Claim 36 (depends on 29) The system according to claim 29, wherein encoding visually the confidence value associated with the first numeric data value includes displaying the first numeric data value with an intensity that depends on the confidence value.