Method and apparatus for progressively selecting features from a large feature space in statistical modeling

A system for selecting an optimum feature subset from an initial feature set, the system comprising one or more processor elements comprising:

• first hardware circuit means for splitting the initial feature space into a plurality of first feature sets using a dimension-based split strategy that splits the initial feature space into a plurality of disjoint feature sets, each feature set based on a feature dimension for the spoken input, wherein the feature dimensions comprise word, word position, speech tagger, and prosody characteristics, and further wherein the number of features in each feature set is defined in proportion to a relative importance of the respective dimension in a natural language processing application;
• feature selection hardware means for selecting a subset of features from each of the plurality of first feature sets to produce a plurality of second feature sets, the feature selection means having a threshold size indicating a maximum number of features that can be accommodated;
• second hardware circuit means for performing a merge-split operation on numbers of the second feature sets to produce a plurality of third feature sets, wherein the merge-split operation includes merging the numbers of the second feature sets together to form a combined set, verifying that the size of combined set is less than or equal to the threshold size, and splitting the combined set into the plurality of third feature sets if the combined set size exceeds the threshold size;
• and
• third hardware circuit means for selecting a subset of features from each of the plurality of third feature sets to produce a plurality of fourth feature sets and continuing a merging and selecting operation on successive feature sets until a final feature set is selected.

A non-transitory machine-readable medium including instructions which when executed in a processing system select an optimum feature subset from an initial feature set comprising:

• splitting the initial feature space into a plurality of first feature sets using a dimension-based split strategy that splits the initial feature space into a plurality of disjoint feature sets, each feature set based on a feature dimension for the spoken input, wherein the feature dimensions comprise word, word position, speech tagger, and prosody characteristics, and further wherein the number of features in each feature set is defined in proportion to a relative importance of the respective dimension in a natural language processing application;
• performing a feature selection process to select a subset of features from each of the plurality of first feature sets to produce a plurality of second feature sets, the feature selection process having a threshold size indicating a maximum number of features that can be accommodated;
• performing a merge-split operation using the dimension-based split strategy on numbers of the second feature sets to produce a plurality of third feature sets, wherein the merge-split operation includes merging the numbers of the second feature sets together to form a combined set, verifying that the size of combined set is less than or equal to the threshold size, and splitting the combined set into the plurality of third feature sets if the combined set size exceeds the threshold size;
• selecting, through the feature selection process, a subset of features from each of the plurality of third feature sets to produce a plurality of fourth feature sets;
• and
• continuing the merge-split and selecting steps on successive feature sets until a final feature set is selected.