Back to results
US 12,001,467 B1Granted
Feature engineering based on semantic types
AssigneeAmerican Express Travel Related Services Company, Inc.
InventorsMan Chon U, Beau Britain, Thai Chiun Huang, Yan Yang, Linou Zhu
Life of this patent
Estimating the term…
Abstract
A computer-based system may engineer features based on semantic types. The computer-based system may implement deep learning algorithms and derive a domain-specific feature engineering strategy from semantic type predictions and data profiling. The computer-based system may utilize embedded domain (e.g., financial industry, etc.) knowledge to generate curated features from raw data (e.g., transactional datasets, relational datasets, etc.).
Claims17
A computer-implemented method for determining features from raw data, the method comprising:
- receiving a plurality of data structures of raw data, wherein each data structure of the plurality of data structures comprises a respective plurality of data elements;
- determining a data profile for the raw data based on an amount of data elements of the respective plurality of data elements for a first data structure of a portion of the plurality of data structures satisfying a statistical threshold for indicating the data profile;
- determining, based on respective semantic rules that describe how to infer respective semantic types from data elements of the respective plurality of data elements for the portion of the plurality of data structures, a respective semantic type for each data structure of the portion of the plurality of data structures;
- determining that the semantic types for the portion of the plurality of data structures correspond to the data profile;
- validating, based on correspondence between the respective semantic types for the portion of the plurality of data structures and the data profile, the respective semantic types for the portion of the plurality of data structures;
- determining, based on respective semantic rules that describe how to infer respective semantic types from data elements of the respective plurality of data elements for each remaining data structure of the plurality of data structures, a respective semantic type for each remaining data structure of the plurality of data structures;
- and
- selecting, based on the respective semantic type for each data structure of the plurality of data structures, a respective feature engineering instruction for each data structure of the plurality of data structures that describes how to calculate an input feature for a machine learning model based on the respective plurality of data elements.
5 more elements
A non-transitory computer-readable device having instructions stored thereon for determining features from raw data, that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
- receiving a plurality of data structures of raw data, wherein each data structure of the plurality of data structures comprises a respective plurality of data elements;
- determining a data profile for the raw data based on an amount of data elements of the respective plurality of data elements for a first data structure of a portion of the plurality of data structures satisfying a statistical threshold for indicating the data profile;
- determining, based on respective semantic rules that describe how to infer respective semantic types from data elements of the respective plurality of data elements for the portion of the plurality of data structures, a respective semantic type for each data structure of the portion of the plurality of data structures;
- determining that the semantic types for the portion of the plurality of data structures correspond to the data profile;
- validating, based on correspondence between the respective semantic types for the portion of the plurality of data structures and the data profile, the respective semantic types for the portion of the plurality of data structures;
- determining, based on respective semantic rules that describe how to infer respective semantic types from data elements of the respective plurality of data elements for each remaining data structure of the plurality of data structures, a respective semantic type for each remaining data structure of the plurality of data structures;
- and
- selecting, based on the respective semantic type for each data structure of the plurality of data structures, a respective feature engineering instruction for each data structure of the plurality of data structures that describes how to calculate an input feature for a machine learning model based on the respective plurality of data elements.
5 more elements
A system for determining features from raw data comprising:
- a memory;
- and
- at least one processor coupled to the memory and configured to:
- receive a plurality of data structures of raw data, wherein each data structure of the plurality of data structures comprises a respective plurality of data elements;
- determine a data profile for the raw data based on an amount of data elements of the respective plurality of data elements for a first data structure of a portion of the plurality of data structures satisfying a statistical threshold for indicating the data profile;
- determine, based on respective semantic rules that describe how to infer respective semantic types from data elements of the respective plurality of data elements for the portion of the plurality of data structures, a respective semantic type for each data structure of the portion of the plurality of data structures;
- determine that the semantic types for the portion of the plurality of data structures correspond to the data profile;
- validate, based on correspondence between the respective semantic types for the portion of the plurality of data structures and the data profile, the respective semantic types for the portion of the plurality of data structures;
- determine, based on respective semantic rules that describe how to infer respective semantic types from data elements of the respective plurality of data elements for each remaining data structure of the plurality of data structures, a respective semantic type for each remaining data structure of the plurality of data structures;
- and
- select, based on the respective semantic type for each data structure of the plurality of data structures, a respective feature engineering instruction for each data structure of the plurality of data structures that describes how to calculate an input feature for a machine learning model based on the respective plurality of data elements.
8 more elements
Dependencies are inferred from the claim text.