US 2026/0310654 A1Published application

Automated feature generation for predictive analytics with generative artificial intelligence

AssigneeBeforepay IP Pty Ltd

InventorsJames Spencer TWISS, Kasey KAPLAN

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system for automated generation of features for a machine learning model obtains a machine-readable schema of a training dataset and generates candidate feature concepts by providing at least one first prompt, including at least one seed value not derived from the training dataset, to a first large language model operated at a first randomness setting. The system generates normalized feature specifications using a second large language model operated at a second randomness setting lower than the first randomness setting, and generates feature computation code using a third large language model fine-tuned on feature computation code. Prior to storage or execution, the system performs schema-aware validation by parsing the feature computation code into an abstract syntax tree, extracting identifier references, enforcing an identifier allowlist, verifying type rules, and rejecting prohibited constructs. Validated code is executed in a sandboxed execution environment enforcing runtime restrictions.

Claims19

Dependent claims

A system for automated generation of features for a machine learning model, comprising:

one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to:
(a) obtain a machine-readable schema of a training dataset, the schema identifying column names and corresponding data types;
(b) generate a plurality of candidate feature concepts by providing at least one first prompt to a first large language model operated at a first randomness setting, the at least one first prompt including at least one seed value that is not derived from the training dataset;
(c) generate a plurality of normalized feature specifications by providing at least one second prompt, derived from the plurality of candidate feature concepts, to a second large language model operated at a second randomness setting lower than the first randomness setting, each normalized feature specification including (i) a feature name, (ii) one or more referenced columns selected from the schema, (iii) an operation definition defining one or more computations to be performed using the referenced columns, and (iv) an output data type;
(d) for each normalized feature specification, generate feature computation code by providing at least one third prompt, including the schema and the normalized feature specification, to a third large language model that is fine-tuned on feature computation code, the feature computation code defining a feature computation function configured to compute a feature value for an input record;
and
(e) prior to storing or executing the feature computation code, perform schema-aware validation of the feature computation code comprising:
(i) parsing the feature computation code into an abstract syntax tree;
(ii) extracting, from the abstract syntax tree, identifier references comprising referenced column names, function calls, and attribute accesses;
(iii) enforcing an identifier allowlist that includes at least (A) the one or more referenced columns identified in the normalized feature specification, (B) local variables defined within the feature computation code, and (C) functions in a predefined allowlist of permitted built-in functions;
(iv) verifying that each column usage is consistent with the corresponding data type in the schema by enforcing one or more type rules;
and
(v) rejecting the feature computation code in response to detecting, in the abstract syntax tree, at least one prohibited construct comprising an import statement, a dynamic code execution instruction, a file system access instruction, or an external network access instruction.

10 more elements

A system for automated generation of features for a machine learning model, comprising:

a network interface;
one or more processors;
a memory storing instructions;
a feature store;
and
a sandboxed execution environment,
wherein the one or more processors are configured to execute the instructions to cause the system to:
(a) obtain a machine-readable schema of a training dataset, the schema identifying at least column names and corresponding data types;
(b) obtain first large language model output generated at a first randomness setting by transmitting, via the network interface, at least one first prompt that includes at least one seed value that is not derived from the training dataset and by receiving, via the network interface, as the first large language model output, a plurality of candidate feature concepts expressed as natural language descriptions;
(c) obtain second large language model output generated at a second randomness setting lower than the first randomness setting by transmitting, via the network interface, at least one second prompt derived from the plurality of candidate feature concepts and by receiving, via the network interface, as the second large language model output, a plurality of normalized feature specifications;
(d) for each normalized feature specification, obtain third large language model output generated by a third large language model that is fine-tuned on feature computation code by transmitting, via the network interface, at least one third prompt that includes the schema and the normalized feature specification and by receiving, via the network interface, as the third large language model output, feature computation code for the normalized feature specification, the feature computation code defining a feature computation function having a prescribed function signature;
(e) validate the feature computation code prior to execution by applying at least one schema-consistency check comprising:
(i) parsing the feature computation code into an abstract syntax tree;
(ii) extracting identifier references from the abstract syntax tree;
(iii) enforcing an identifier allowlist derived from the normalized feature specification by permitting references to only (A) the one or more referenced columns identified in the normalized feature specification, (B) local variables defined within the feature computation code, and (C) functions in a predefined allowlist of permitted built-in functions;
(iv) verifying that each column usage is consistent with the corresponding data type in the schema by enforcing one or more type rules;
and
(v) rejecting the feature computation code in response to detecting, in the abstract syntax tree, at least one prohibited construct comprising an import statement, a dynamic code execution instruction, a file system access instruction, or an external network access instruction;
(f) after validating the feature computation code, execute the feature computation function in the sandboxed execution environment, the sandboxed execution environment enforcing at least a time limit and a memory limit and being configured to disable at least (i) import resolution at runtime and (ii) file system and external network access at runtime;
and
(g) store, in the feature store, an association between the feature computation code and at least one evaluation metric for prioritizing candidate features, the at least one evaluation metric including at least one of an execution status, a validation status, or a predictive value score computed on a labeled sample dataset.

18 more elements

A method for automated generation of features for a machine learning model, comprising:

(a) obtaining, by a computer system, a machine-readable schema of a training dataset, the schema identifying at least column names and corresponding data types;
(b) obtaining, by the computer system, first large language model output generated at a first randomness setting by transmitting at least one first prompt that includes at least one seed value not derived from the training dataset and receiving, as the first large language model output, a plurality of candidate feature concepts expressed as natural language descriptions;
(c) obtaining, by the computer system, second large language model output generated at a second randomness setting lower than the first randomness setting by transmitting at least one second prompt derived from the plurality of candidate feature concepts and receiving, as the second large language model output, a plurality of normalized feature specifications;
(d) for each normalized feature specification, obtaining, by the computer system, third large language model output generated by a third large language model that is fine-tuned on feature computation code by transmitting at least one third prompt that includes the schema and the normalized feature specification and receiving, as the third large language model output, feature computation code for the normalized feature specification, the feature computation code defining a feature computation function having a prescribed function signature for computing a feature value from an input record;
and
(e) validating, by the computer system, the feature computation code prior to storage or execution by applying at least one schema-consistency check comprising:
(i) parsing the feature computation code into an abstract syntax tree;
(ii) extracting identifier references from the abstract syntax tree;
(iii) enforcing an identifier allowlist derived from the normalized feature specification by permitting references to only (A) the one or more referenced columns identified in the normalized feature specification, (B) local variables defined within the feature computation code, and (C) functions in a predefined allowlist of permitted built-in functions;
(iv) verifying that each column usage is consistent with the corresponding data type in the schema by enforcing one or more type rules;
and
(v) rejecting the feature computation code in response to detecting, in the abstract syntax tree, at least one prohibited construct comprising an import statement, a dynamic code execution instruction, a file system access instruction, or an external network access instruction;
and
(f) after validating the feature computation code, executing, by the computer system, the feature computation function in a sandboxed execution environment that enforces at least a time limit and a memory limit and that is configured to disable at least (i) import resolution at runtime and (ii) file system and external network access at runtime.

11 more elements

Dependencies are inferred from the claim text.