Back to results
US 11,308,133 B2Granted
Entity matching using visual information
AssigneeInternational Business Machines Corporation
InventorsSunhwan Lee, Pawan Chowdhary, Guangjie Ren
Life of this patent
Estimating the term…
Abstract
One embodiment provides a method comprising receiving a first set of information corresponding to a first entity and receiving a second set of information corresponding to a second entity. Each set of information received corresponding to each entity comprises visual information relating to the entity. The method further comprises encoding each set of information, and classifying the first entity and the second entity as either matching entities or non-matching entities based on the encoding.
Claims18
A method comprising:
- receiving a first set of information corresponding to a first entity, wherein the first set of information comprises first text information relating to the first entity and further comprises one or more first images of the first entity;
- receiving a second set of information corresponding to a second entity, wherein the second set of information comprises second text information relating to the second entity and further comprises one or more second images of the second entity;
- performing visual encoding by:
- generating a first set of visual embeddings by embedding the first images of the first entity using a convolutional neural network (CNN);
- generating a second set of visual embeddings by embedding the second images of the second entity using the CNN;
- and
- encoding the first set of visual embeddings and the second set of visual embeddings to a first set of statistics indicative of a similarity between the first images of the first entity and the second images of the second entity by applying a pairwise comparison algorithm to the first set of visual embeddings and the second set of visual embeddings;
- performing text encoding by:
- encoding the first text information relating to the first entity to a first set of features indicative of occurrence of terms within the first text information;
- and
- encoding the second text information relating to the second entity to a second set of features indicative of occurrence of terms within the second text information;
- and
- classifying the first entity and the second entity as either matching entities or non-matching entities based on a concatenation of the first set of statistics, the first set of features, and the second set of features.
10 more elements
A system comprising:
- at least one processor;
- and
- a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including:
- receiving a first set of information corresponding to a first entity, wherein the first set of information comprises first text information relating to the first entity and further comprises one or more first images of the first entity;
- receiving a second set of information corresponding to a second entity, wherein the second set of information comprises second text information relating to the second entity and further comprises one or more second images of the second entity;
- performing visual encoding by:
- generating a first set of visual embeddings by embedding the first images of the first entity using a convolutional neural network (CNN);
- generating a second set of visual embeddings by embedding the second images of the second entity using the CNN;
- and
- encoding the first set of visual embeddings and the second set of visual embeddings to a first set of statistics indicative of a similarity between the first images of the first entity and the second images of the second entity by applying a pairwise comparison algorithm to the first set of visual embeddings and the second set of visual embeddings;
- performing text encoding by:
- encoding the first text information relating to the first entity to a first set of features indicative of occurrence of terms within the first text information;
- and
- encoding the second text information relating to the second entity to a second set of features indicative of occurrence of terms within the second text information;
- and
- classifying the first entity and the second entity as either matching entities or non-matching entities based on a concatenation of the first set of statistics, the first set of features, and the second set of features.
13 more elements
A computer program product comprising a computer-readable hardware storage medium having program code embodied therewith, the program code being executable by a computer to implement a method comprising:
- receiving a first set of information corresponding to a first entity, wherein the first set of information comprises first text information relating to the first entity and further comprises one or more first images of the first entity;
- receiving a second set of information corresponding to a second entity, wherein the second set of information comprises second text information relating to the second entity and further comprises one or more second images of the second entity;
- performing visual encoding by:
- generating a first set of visual embeddings by embedding the first images of the first entity using a convolutional neural network (CNN);
- generating a second set of visual embeddings by embedding the second images of the second entity using the CNN;
- and
- encoding the first set of visual embeddings and the second set of visual embeddings to a first set of statistics indicative of a similarity between the first images of the first entity and the second images of the second entity by applying a pairwise comparison algorithm to the first set of visual embeddings and the second set of visual embeddings;
- performing text encoding by:
- encoding the first text information relating to the first entity to a first set of features indicative of occurrence of terms within the first text information;
- and
- encoding the second text information relating to the second entity to a second set of features indicative of occurrence of terms within the second text information;
- and
- classifying the first entity and the second entity as either matching entities or non-matching entities based on a concatenation of the first set of statistics, the first set of features, and the second set of features.
10 more elements
Dependencies are inferred from the claim text.