Back to results
US 11,687,491 B2Granted
Generating weights for finding duplicate records
AssigneeInternational Business Machines Corporation, Armonk, NY
InventorsNamit Kabra, Manish A. Bhide
Life of this patent
Estimating the term…
Abstract
Data-deduplicating includes comparing a first record of a data-store with a second record of the data-store but instead of using a static weight for a field, the present data-deduplicating dynamically assigns a first weight for the first score to generate a first weighted score, wherein the first weight is based on one or both of the first value or the second value; and assigns a second weight for the second score to generate a second weighted score. A composite score is calculated based on the first weighted score and the second weighted score; and it is determined whether or not the first record and the second record are duplicate records, based on the composite score.
Claims20
A method comprising:
- comparing, by a computer, a first record of a data-store with a second record of the data-store, wherein comparing comprises:
- comparing a first value of a first field of the first record with a second value of a second field of the second record and determining a first score indicative of whether or not the first value matches the second value;
- comparing a third value of a third field of the first record with a fourth value of a fourth field of the second record and determining a second score indicative of whether or not the third value matches the fourth value;
- dynamically assigning, by the computer, a first weight for the first score to generate a first weighted score, wherein the first weight is based on one or both of the first value or the second value;
- assigning, by the computer, a second weight for the second score to generate a second weighted score;
- calculating by the computer, a composite score based on the first weighted score and the second weighted score;
- determining, by the computer, whether or not the first record and the second record are duplicate records, based on the composite score;
- and
- eliminating from the data-store, based upon a determination that the first record and the second record are duplications records, one of the first record and the second record.
6 more elements
A system, comprising:
- a processor programmed to initiate executable operations comprising:
- comparing a first record of a data-store with a second record of the data-store, wherein comparing comprises:
- comparing a first value of a first field of the first record with a second value of a second field of the second record and determining a first score indicative of whether or not the first value matches the second value;
- comparing a third value of a third field of the first record with a fourth value of a fourth field of the second record and determining a second score indicative of whether or not the third value matches the fourth value;
- dynamically assigning a first weight for the first score to generate a first weighted score, wherein the first weight is based on one or both of the first value or the second value;
- assigning a second weight for the second score to generate a second weighted score;
- calculating a composite score based on the first weighted score and the second weighted score;
- determining whether or not the first record and the second record are duplicate records, based on the composite score;
- and
- eliminating from the data-store, based upon a determination that the first record and the second record are duplications records, one of the first record and the second record.
7 more elements
A computer program product, comprising:
- a computer readable storage medium having program code stored thereon,
- the program code executable by a data processing system to initiate operations including:
- comparing, by the data processing system, a first record of a data-store with a second record of the data-store, wherein comparing comprises:
- comparing a first value of a first field of the first record with a second value of a second field of the second record and determining a first score indicative of whether or not the first value matches the second value;
- comparing a third value of a third field of the first record with a fourth value of a fourth field of the second record and determining a second score indicative of whether or not the third value matches the fourth value;
- dynamically assigning, by the data processing system, a first weight for the first score to generate a first weighted score, wherein the first weight is based on one or both of the first value or the second value;
- assigning, by the data processing system, a second weight for the second score to generate a second weighted score;
- calculating by the data processing system, a composite score based on the first weighted score and the second weighted score;
- determining, by the data processing system, whether or not the first record and the second record are duplicate records, based on the composite score;
- and
- eliminating from the data-store, based upon a determination that the first record and the second record are duplications records, one of the first record and the second record.
8 more elements
Dependencies are inferred from the claim text.