Back to results
US 2006/0053129 A1Published application
Robust detector of fuzzy duplicates
AssigneeMicrosoft Corporation
InventorsRajeev Motwani, Surajit Chaudhuri, Venkatesh Ganti
Life of this patent
Estimating the term…
Abstract
At least one implementation, described herein, detects fuzzy duplicates and eliminates such duplicates. Fuzzy duplicates are multiple, seemingly distinct tuples (i.e., records) in a database that represent the same real-world entity or phenomenon.
Claims21
One or more processor-readable program media having processor-executable instructions that, when executed by a processor, perform acts comprising:
- obtaining a dataset comprising multiple tuples from a database;
- for each of the multiple tuples of the dataset, computing one or more nearest neighbor tuples in the dataset;
- defining multiple disjoint partitions of multiple tuples, wherein tuples in each partition comprise fuzzy duplicates of one another, such that each fuzzy duplicate tuple in a partition represents a common real world entity or phenomenon.
A fuzzy duplicate elimination system comprising:
- a dataset obtaining means for obtaining a dataset comprising multiple tuples from a database;
- a computing means for computing one or more nearest neighbor tuples in the dataset for each of the multiple tuples of the dataset;
- a partitioning means for defining multiple disjoint partitions of multiple tuples, wherein seemingly distinct tuples in each partition comprise fuzzy duplicates of one another, such that each fuzzy duplicate tuple in a partition represents a common real world entity or phenomenon;
- a duplicate-elimination means for eliminating duplicates of multiple fuzzy duplicate tuples within the multiple partitions so that each of such partitions is left with one, unduplicated tuple.
1 more element
A method for eliminating fuzzy duplicate tuples in a dataset, the method comprising:
- for each of multiple tuples in a dataset, computing one or more nearest neighbor tuples;
- delimitating delimiting multiple disjoint partitions of multiple tuples, wherein seemingly distinct tuples in each partition comprise fuzzy duplicates of one another, such that each fuzzy duplicate tuple in a partition represents a common real world entity or phenomenon;
- eliminating duplicates within the multiple partitions of multiple fuzzy duplicate tuples so that each of such partitions is left with one, unduplicated tuple.
Dependencies are inferred from the claim text.