Document retrieval using intra-image relationships
AssigneeGeorgetown University
InventorsCristopher Flagg, Ophir Frieder
Life of this patent
Estimating the term…
Abstract
Technologies are described for retrieving documents using image representations in the documents and is based on intra-image features. The identification of elements within an image representation can allow for deeper understanding of the image representation and for better relating image representations based on their intra-image features. The intra-image features present in image representations can be used in searches. Search results can further be reranked to improve search results. For example, reranking can allow search results to conform to intra-image dominant image features.
Claims20
A method, performed by one or more computing devices, the method comprising:
- obtaining an image representation comprising two or more segmentations of an image;
- generating one or more latent space representations for each of a first segmentation and a second segmentation of the two or more segmentations;
- generating first one or more feature vectors for the first segmentation and second one or more feature vectors the second segmentation based on the one or more latent space representations;
- determining that the first segmentation is a dominant segmentation in the image representation and the second segmentation is a non-dominant segmentation in the image representation;
- assigning a greater weight to the first one or more feature vectors and a lower weight to the second one or more feature vectors to generate a weighted set of feature vectors;
- comparing the weighted set of feature vectors to one or more other feature vectors, the one or more other feature vectors generated from one or more latent space representations based on one or more other image representations;
- and
- retrieving, based on the comparing of the weighted set of feature vectors to one or more other feature vectors, at least one image representation from the one or more other image representations that has a greater similarity to the dominant segmentation than the non-dominant segmentation.
A method, performed by one or more computing devices, the method comprising:
- obtaining an image representation comprising two or more segmentations of an image, the two or more segmentations comprising a first segmentation and a second segmentation;
- determining that the first segmentation is a dominant segmentation in the image representation and the second segmentation is a non-dominant segmentation in the image representation;
- based at least on the first segmentation being the dominant segmentation, generating one or more latent space representations for the first segmentation;
- generating one or more feature vectors for the first segmentation based at least on the one or more latent space representations;
- comparing the one or more feature vectors to one or more other feature vectors, the one or more other feature vectors generated from one or more latent space representations based on one or more other image representations;
- and
- retrieving, based on the one or more feature vectors to the one or more other feature vectors, at least one image representation from the one or more other image representations that have similarity to the dominant segmentation.
One or more computing devices comprising:
- one or more processors;
- and
- memory having a plurality of computer-executable instructions stored thereon;
- wherein the computer-executable instructions are configured to, when executed by the one or more processors, cause the one or more computing devices to perform a plurality of operations, the operations comprising:
- obtaining an image representation comprising two or more segmentations of an image;
- generating one or more latent space representations for each of a first segmentation and a second segmentation of the two or more segmentations;
- generating first one or more feature vectors for the first segmentation and second one or more feature vectors the second segmentation based on the one or more latent space representations;
- determining that the first segmentation is a dominant segmentation in the image representation and the second segmentation is a non-dominant segmentation in the image representation;
- assigning a greater weight to the first one or more feature vectors and a lower weight to the second one or more feature vectors to generate a weighted set of feature vectors;
- comparing the weighted set of feature vectors to one or more other feature vectors, the one or more other feature vectors generated from one or more latent space representations based on one or more other image representations;
- and
- retrieving, based on the comparing of the weighted set of feature vectors to one or more other feature vectors, at least one image representation from the one or more other image representations that has a greater similarity to the dominant segmentation than the non-dominant segmentation.
Dependencies are inferred from the claim text.