# US 2017/0255693 A1 · Providing images for search queries

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0255693 A1
- Document ID: US20170255693A1
- Status: Published application
- Kind: A1
- Application number: 15/061,629
- Filed: 2016-03-04
- Published: 2017-09-07
- Earliest claimed: 2016-03-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Nikola Trifunovic
- Web page: https://patentgrep.com/patent/US20170255693A1

## CPC

- G06F 16/334: Query execution (filtering based on additional data G06F16/335)
- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/58: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06F 16/907: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/909: using geographical or spatial information, e.g. location (spatiotemporally dependent retrieval from the web G06F16/9537)
- G06F 16/951: Indexing; Web crawling techniques
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques and technologies for providing images for search queries are described. In at least some embodiments, a system includes a scraping query component, a search component, and a search results analysis component. The scraping query component provides a scrape query based on textual information associated with an entity of interest. The search component conducts an electronic search via one or more networks to obtain search results based at least partially on the scrape query, the search results including at least a search result image and image metadata associated with the search result image. The search results analysis component determines a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest, and determines whether to store, provide, or discard the search result image based at least partially on the determined similarity.

## Claims

### Claim 1

A system for providing images for search queries, comprising:
a scraping query component configured to provide a scrape query based on textual information associated with an entity of interest;
a search component configured to conduct an electronic search via one or more networks to obtain search results based at least partially on the scrape query, the search results including at least a search result image and image metadata associated with the search result image; and
a search results analysis component configured to:
determine a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest; and
determine whether to at least one of store, provide, or discard the search result image based at least partially on the determined similarity.

### Claim 2 (depends on 1)

The system of claim 1, wherein the scraping query component configured to provide a scrape query based on textual information associated with an entity of interest comprises:
a scraping query component configured to:
obtain textual information associated with an entity of interest from an entity index, and
generate one or more scrape queries using one or more scrape query templates and the textual information associated with the entity of interest.

### Claim 3 (depends on 1)

The system of claim 1, wherein the search results analysis component configured to determine a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest comprises:
a search results analysis component configured to apply a machine learning model to determine a similarity based on a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest.

### Claim 4 (depends on 3)

The system of claim 3, wherein the machine learning model is configured to determine the similarity by applying at least one of a fuzzy string distance function, a cosine string distance function, a Levenshtein string distance function, or a containment binary feature distance function.

### Claim 5 (depends on 3)

The system of claim 3, wherein the machine learning model is configured to determine the similarity by applying a first string distance function when comparing a first attribute of the at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest comprises, and by applying a second string distance function when comparing a second attribute of the at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest.

### Claim 6 (depends on 1)

The system of claim 1, wherein the search results analysis component configured to determine a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest comprises:
a search results analysis component configured to determine a similarity score based on a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest.

### Claim 7 (depends on 1)

The system of claim 1, wherein the search results analysis component configured to determine a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest comprises:
a search results analysis component configured to:
tokenize the at least part of the image metadata associated with the search result image into one or more first tokens;
tokenize the at least part of the textual information associated with the entity of interest into one or more second tokens;
compare at least some of the one or more first tokens with at least some of the one or more second tokens;
determine a connection similarity score associated with each comparison of the one or more first tokens with the one or more second tokens; and
determine an overall similarity score based at least partially on the one or more connection similarity scores.

### Claim 8 (depends on 7)

The system of claim 7, wherein the search results analysis component configured to compare at least some of the one or more first tokens with at least some of the one or more second tokens comprises:
a search results analysis component configured to compare at least some of the one or more first tokens with at least some of the one or more second tokens based on a specified edit distance function relationship between the at least some of the one or more first tokens and the at least some of the one or more second tokens.

### Claim 9 (depends on 7)

The system of claim 7, wherein the search results analysis component configured to determine an overall similarity score based at least partially on the one or more connection similarity scores comprises:
a search results analysis component configured to determine an overall similarity score by applying a token-weighted cosine similarity function to the one or more connection similarity scores.

### Claim 10 (depends on 7)

The system of claim 7, wherein the search results analysis component is further configured to:
disregard one or more connection similarity scores that do not meet or exceed a specified threshold.

### Claim 11 (depends on 7)

The system of claim 7, wherein the search results analysis component is further configured to:
assign first token weights to the one or more first tokens and second token weights to the one or more second tokens; and
wherein the overall similarity score is determined based at least partially on the one or more connection similarity scores and at least partially on the one or more first token weights and the one or more second token weights.

### Claim 12 (depends on 7)

The system of claim 7, wherein the search results analysis component configured to determine an overall similarity score based at least partially on the one or more connection similarity scores and at least partially on the token weights comprises:
a search results analysis component configured to determine an overall similarity score by applying a token-weighted cosine similarity function to the one or more connection similarity scores and the one or more first token weights and the one or more second token weights.

### Claim 13

A method for providing images for search queries, comprising:
conducting an image search via one or more networks to obtain image search results, the image search results including at least a search result image and image metadata associated with the search result image;
determining a similarity between the search result image and an entity of interest, including at least:
comparing one or more attributes of at least part of the image metadata with textual information associated with the entity of interest using a string distance function to determine an overall similarity score; and
determining whether to at least one of store, provide, or discard the search result image based at least partially on the overall similarity score.

### Claim 14 (depends on 13)

The method of claim 13, wherein comparing one or more attributes of at least part of the image metadata with textual information associated with the entity of interest using a string distance function to determine an overall similarity score comprises:
applying a first string distance function to compare a first attribute of at least part of the image metadata with textual information associated with the entity of interest; and
applying a second string distance function to compare a second attribute of at least part of the image metadata with textual information associated with the entity of interest.

### Claim 15 (depends on 13)

The method of claim 13, wherein comparing one or more attributes of at least part of the image metadata with textual information associated with the entity of interest using a string distance function to determine an overall similarity score comprises:
tokenizing at least part of the image metadata associated with the search result image into one or more first tokens;
tokenizing at least part of the textual information associated with the entity of interest into one or more second tokens;
assigning one or more first token weights to the one or more first tokens;
assigning one or more second token weights to the one or more second tokens;
comparing at least some of the one or more first tokens with at least some of the one or more second tokens;
determining a connection similarity score associated with each comparison of the one or more first tokens with the one or more second tokens; and
determining the overall similarity score based at least partially on the one or more connection similarity scores, the one or more first token weights, and the one or more second token weights.

### Claim 16 (depends on 15)

The method of claim 15, wherein comparing one or more attributes of at least part of the image metadata with textual information associated with the entity of interest using a string distance function to determine an overall similarity score further comprises:
disregarding one or more connection similarity scores that do not meet or exceed a specified threshold.

### Claim 17 (depends on 15)

The method of claim 15, wherein comparing at least some of the one or more first tokens with at least some of the one or more second tokens comprises:
determining whether to compare at least some of the one or more first tokens with at least some of the one or more second tokens based on a specified edit distance function.

### Claim 18

A system for providing images for search queries, comprising:
a query component configured to receive a search query, the search query including textual information associated with an entity of interest;
an external image retrieval component configured to conduct a search for one or more images associated with the search query via one or more networks;
a receiving component configured to receive a search result from the external image retrieval component, the search result including at least a search result image and image metadata associated with the search result image; and
a search result analysis component configured to:
determine a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest; and
determine whether to at least one of store, provide, or discard the search result image based at least partially on the determined similarity.

### Claim 19 (depends on 18)

The system of claim 18, wherein the search result analysis component configured to determine a similarity between at least part of the image metadata associated with the search result image and at least part of the textual information associated with the entity of interest comprises:
a search result analysis component configured to compare one or more attributes of at least part of the image metadata with one or more attributes of at least part of the textual information associated with the entity of interest using a string distance function to determine an overall similarity score.

### Claim 20 (depends on 19)

The system of claim 19, wherein the search result analysis component configured to compare one or more attributes of at least part of the image metadata with one or more attributes of at least part of the textual information associated with the entity of interest using a string distance function to determine an overall similarity score comprises:
a search result analysis component configured to:
separate at least part of the image metadata associated with the search result image into one or more first tokens;
separate at least part of the textual information associated with the entity of interest into one or more second tokens;
apply one or more first token weights to the one or more first tokens;
apply one or more second token weights to the one or more second tokens;
compare at least some of the one or more first tokens with at least some of the one or more second tokens;
compute a connection similarity score associated with each comparison of the one or more first tokens with the one or more second tokens; and
compute the overall similarity score based at least partially on the one or more connection similarity scores, the one or more first token weights, and the one or more second token weights.
