# US 2020/0394447 A1 · Search system, search method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394447 A1
- Document ID: US20200394447A1
- Status: Published application
- Kind: A1
- Application number: 16/971,292
- Filed: 2018-06-20
- Published: 2020-12-17
- Earliest claimed: 2018-06-20
- Assignees: Rakuten, Inc.
- Applicants: Rakuten, Inc.
- Inventors: Yeongnam CHAE
- Web page: https://patentgrep.com/patent/US20200394447A1

## CPC

- G06F 16/90335: {Query processing}
- G06F 16/53: Querying
- G06F 16/55: Clustering; Classification
- G06F 18/211: Selection of the most significant subset of features
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/2413: based on distances to training or reference patterns
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

To improve accuracy of search, a learner (L) of a search system calculates a feature quantity of information that is input and outputs a first analysis result of the information in a first viewpoint and a second analysis result of the information in a second viewpoint based on the feature quantity. Storing means stores a feature quantity of information to be searched, which has been input in the learner (L), in a database. Input means inputs input information in the learner (L). Search means searches for information to be searched that is similar to the input information in the feature quantity based on the database.

## Claims

### Claim 1

A search system comprising:
a learner that calculates a feature quantity of information that is input and outputs a first analysis result of the information in a first viewpoint and a second analysis result of the information in a second viewpoint based on the feature quantity; and
at least one processor configured to:
store a feature quantity of information to be searched, which has been input in the learner, in a database,
input information in the learner; and
search for information to be searched that is similar to the input information in the feature quantity based on the database.

### Claim 2 (depends on 1)

The search system according to claim 1, wherein
the learner includes an intermediate layer, a first output layer, and a second output layer,
the intermediate layer is disposed before the learner branches into the first output layer and the second output layer and calculates the feature quantity,
the first output layer outputs the first analysis result based on the feature quantity calculated by the intermediate layer before branching, and
the second output layer outputs the second analysis result based on the feature quantity calculated by the intermediate layer before branching.

### Claim 3 (depends on 1)

The search system according to claim 1, wherein
the at least one processor searches for information to be searched that is similar to the input information in the first analysis result, and, among the retrieved information to be searched, searches for information to be searched that is similar to the input information in the feature quantity.

### Claim 4 (depends on 3)

The search system according to claim 3, wherein
the at least one processor searches for information to be searched that is similar to the input information in the second analysis result among the information to be searched that is similar to the input information in the first analysis result, and, among the retrieved information to be searched, searches for information to be searched that is similar to the input information in the feature quantity.

### Claim 5 (depends on 1)

The search system according to claim 1, wherein
the first analysis result is a classification result of the information that is input,
the at least one processor stores the feature quantity of the information to be searched in a database corresponding to a classification of the information to be searched, each database being prepared for a corresponding classification, and
the at least one processor searches for information to be searched that is similar to the input information in the feature quantity based on the database corresponding to the classification result of the input information among the databases prepared for respective classifications.

### Claim 6 (depends on 1)

The search system according to claim 1, wherein
the learner calculates a feature vector as the feature quantity, and
the at least one processor performs the search based on a distance between a feature vector of information to be searched stored in the database and a feature vector of the input information.

### Claim 7 (depends on 5)

The search system according to claim 5, wherein
the at least one processor stores the information to be searched, which is output from the learner, in a database corresponding to a classification result of the information to be searched.

### Claim 8 (depends on 7)

The search system according to claim 7, wherein
the learner outputs a probability of each classification as the classification result, and
the at least one processor stores the information to be searched, which is output from the learner, in a database of a classification having a probability of the information to be searched, the probability being equal to or more than a threshold value.

### Claim 9 (depends on 5)

The search system according to claim 5, wherein
the learner outputs a probability of each classification as the classification result, and
the at least one processor performs search based on a database of a classification having a probability of the input information, which is output from the learner, the probability being equal to or more than the threshold value.

### Claim 10 (depends on 5)

The search system according to claim 5, wherein
in a case where there are a plurality of databases that correspond to the classification result of the input information, based on each of the plurality of databases, the at least one processor searches for candidates of information to be searched that is similar to the input information in the feature quantity, and narrows down the candidates.

### Claim 11 (depends on 1)

The search system according to claim 1, at least one processor:
obtains a similarity based on the feature quantity of the input information and a feature quantity of information to be searched; and
displays the similarity in association with the information to be searched.

### Claim 12 (depends on 1)

The search system according to claim 1, wherein
the learner calculates a feature quantity of an image that is input and outputs a first analysis result in which a feature of the image is analyzed in the first viewpoint and a second analysis result in which a feature of the image is analyzed in the second viewpoint,
the information to be searched is an image to be searched,
the input information is an input image, and
the at least one processor searches for an image to be searched that is similar to the input image in the feature quantity.

### Claim 13 (depends on 12)

The search system according to claim 12, wherein
the learner calculates a feature quantity of an area indicating an object included in the input image and outputs the first analysis result and the second analysis result of the area, and
in a case where a plurality of areas overlapping with one another are included in the input image, based on a feature quantity of an area having a highest probability of at least one of the first analysis result or the second analysis result, the learner outputs the first analysis result and the second analysis result of the area.

### Claim 14 (depends on 12)

The search system according to claim 12, wherein
the at least one processor:
stores, in a database, the feature quantity of the area indicating the object included in the image to be searched, and
in a case where a plurality of areas overlapping with one another are included in the image to be searched, stores a feature quantity of an area having a highest probability of at least one of the first analysis result or the second analysis result.

### Claim 15 (depends on 12)

The search system according to claim 12, wherein
the learner outputs a classification result of an object included in an image that is input as the first analysis result, and outputs position information about a position of the object as the second analysis result, and
the at least one processor displays the position information of the image to be searched in association with the image to be searched.

### Claim 16 (depends on 12)

The search system according to claim 12, wherein
the learner outputs a classification result of an object included in an image that is input as the first analysis result, and outputs position information about a position of the object as the second analysis result, and
the at least one processor displays the position information of the input image in association with the input image.

### Claim 17 (depends on 12)

The search system according to claim 12, wherein
in a case where an image that is input includes a plurality of objects, the learner calculates a feature quantity of each object and outputs the first analysis result and the second analysis result,
the input image and the image to be searched each include a plurality of objects, and
the at least one processor searches for an image to be searched that is similar to the input image in the feature quantity of some of objects of the image to be searched.

### Claim 18

A search method comprising:
storing a feature quantity of information to be searched, which has been input in a learner, in a database, the learner calculating a feature quantity of information that is input and outputting a first analysis result of the information in a first viewpoint and a second analysis result of the information in a second viewpoint based on the feature quantity;
inputting input information in the learner; and
searching for information to be searched that is similar to the input information in the feature quantity based on the database.

### Claim 19

A non-transitory computer-readable information storage medium for storing a program that causes a computer to:
store a feature quantity of information to be searched, which has been input in a learner, in a database, the learner calculating a feature quantity of information that is input and outputting a first analysis result of the information in a first viewpoint and a second analysis result of the information in a second viewpoint based on the feature quantity;
input information in the learner; and
search for information to be searched that is similar to the input information in the feature quantity based on the database.
