Two step algorithm for non-exact matching of large datasets
AssigneeBottomline Technologies, Inc.
InventorsMark G. Kane, Richard J. Diekema, Jr., Kaiyu Pan
Life of this patent
Estimating the term…
Abstract
A two-step algorithm for conducting near real-time fuzzy searches of a target on one or more large data sets is described. This algorithm includes the simplification of the data by removing grammatical constructs to bring the target search term (and the stored database) to their base elements and then perform a Levenstein comparison to create a subset of the data set that may be a match. Then performing a scoring algorithm while comparing the target to the subset of the data set to identify any matches.
Claims20
A non-transitory machine-readable media programmed to:
- search for a subset of records in a list that most closely matches a target, wherein the search comprises software programmed to:
- read a record from the list;
- perform a search Levenstein distance calculation on the target and the record;
- and
- store the record in a subset list if the search Levenstein distance calculation is greater than a search threshold;
- score the records in the subset list, wherein the score comprises software programmed to:
- loop through fields of the records in the subset list;
- calculate a score Levenstein distance between the field in the record in the subset list and the field in the target;
- check that the score Levenstein distance is greater than a field-level threshold, and to go to a next record in the loop if the score Levenstein distance is greater than the field-level threshold;
- sum a product of the score Levenstein distance multiplied by a weight factor for the field;
- and
- divide the product by a sum of the weight factor for the fields to derive a weighted mean score for the record;
- indicate that the target is located in the list if at least one weighted mean score is calculated.
A method comprising:
- searching for a subset of records in a list that most closely matches a target, wherein the searching comprises:
- performing a search Levenstein distance calculation on the target and a record from the list;
- and
- storing the record in a subset list if the search Levenstein distance calculation is greater than a search threshold;
- scoring the records in the subset list, wherein the scoring comprises:
- looping through fields of the records in the subset list;
- calculating a score Levenstein distance between the field in the record in the subset list and the field in the target;
- checking that the score Levenstein distance is greater than a field-level threshold, and going to a next record in the loop if the score Levenstein distance is greater than the field-level threshold;
- summing a product of the score Levenstein distance multiplied by a weight factor for the field;
- and
- dividing the product by a sum of the weight factor for the fields to derive a weighted mean score for the record;
- indicating that the target is located in the list if at least one weighted mean score is calculated.
Dependencies are inferred from the claim text.