# US 11,308,151 B2 · Search system, search method, and program recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,151 B2 - Document ID: US11308151B2 - Status: Granted - Kind: B2 - Application number: 16/789,521 - Filed: 2020-02-13 - Published: 2020-08-27 - Granted: 2022-04-19 - Earliest claimed: 2019-02-22 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Shigeo Yamazaki - Web page: https://patentgrep.com/patent/US11308151B2 ## CPC - G06F 16/535: Filtering based on additional data, e.g. user or group profiles - G06F 16/538: Presentation of query results - G06F 16/583: using metadata automatically derived from the content - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/172: {Classification, e.g. identification} - G06V 40/50: Maintenance of biometric data or enrolment thereof - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A search system includes a database, a face recognition unit, a similarity degree processing unit, and a search result output unit. The database stores image data. The face recognition unit detects a face from the image data stored in the database. When a target image indicating a search target person and an auxiliary image group including auxiliary image that assists a search for the search target person are designated, the face recognition unit evaluates whether a face detected from the image data is included in the target image and the auxiliary image group. The similarity degree processing unit determines whether the search target person is captured in the image data by using the evaluated result. The search result output unit acquires, from the database, the image data determined that the search target person is captured based on the determined result, and outputs a search result including the acquired image data. ## Claims ### Claim 1 A search system, comprising: a database in which at least one piece of image data being a selection target and metadata of the image data being the selection target are stored; at least one memory storing instructions; and at least one processor connected to the at least one memory and configured to execute the instructions to: detect a face from the image data being the selection target stored in the database; evaluate, when a target image indicating a search target person and an auxiliary image group including at least one auxiliary image that assists a search for the search target person are designated, whether a face detected from the image data being the selection target is included in the target image and the auxiliary image group; determine whether the search target person is captured in the image data, by using an evaluation result; acquire, from the database, the image data determined that the search target person is captured, based on a determination result; calculate a similarity degree between a face detected from the image data and a face of a person included in the target image and the auxiliary image group; acquire metadata of the image data being the selection target from the database; determine whether the search target person is captured in the image data, based on the acquired metadata of the image data being the selection target and the similarity degree calculated for the target image and the auxiliary image group; and output a search result including the acquired image data. ### Claim 2 (depends on 1) The search system according to claim 1, wherein an image indicating an associated person of the search target person is provided as the auxiliary image. ### Claim 3 (depends on 1) The search system according to claim 1, wherein the at least one processor is configured to execute the instructions to determine whether the search target person is captured in the image data, based on the similarity degree calculated for the target image and the auxiliary image group. ### Claim 4 (depends on 1) The search system according to claim 1, wherein the at least one processor is configured to execute the instructions to provide, for the image data, a similarity degree analysis indicating a certainty degree that the search target person is captured, based on the metadata of the image data being the selection target and the similarity degree calculated for the target image and the auxiliary image group. ### Claim 5 (depends on 4) The search system according to claim 4, wherein the search system is connected to an output device including a monitor, the at least one processor is configured to execute the instructions to output the search result to the output device, and the output device causes the monitor to display a first user interface for selecting the image data included in the search result. ### Claim 6 (depends on 1) The search system according to claim 1, wherein the at least one processor is configured to execute the instructions to: acquire at least one piece of the image data stored in the database; and generate a candidate image of the target image and the auxiliary image by using an image including the detected face. ### Claim 7 (depends on 6) The search system according to claim 6, wherein the search system is connected to an input device including a monitor, the at least one processor is configured to execute the instructions to output the candidate image to the input device, and the input device causes the monitor to display a second user interface for selecting the target image and the auxiliary image from among a plurality of candidate images including the candidate image, designates each candidate image selected via the second user interface as any of the target image and the auxiliary image, and outputs information related to the designated target image and the designated auxiliary image to the search system. ### Claim 8 A search method performed by a computer, the search method comprising: detecting a face from at least one piece of image data being a selection target stored in a database; evaluating, when a target image indicating a search target person and an auxiliary image group including at least one auxiliary image that assists a search for the search target person are designated, whether a face detected from the image data being the selection target is included in the target image and the auxiliary image group; determining whether the search target person is captured in the image data, by using an evaluation result; and acquiring, from the database, the image data determined that the search target person is captured, based on a determination result; calculating a similarity degree between a face detected from the image data and a face of a person included in the target image and the auxiliary image group; acquiring metadata of the image data being the selection target from the database; determining whether the search target person is captured in the image data, based on the acquired metadata of the image data being the selection target and the similarity degree calculated for the target image and the auxiliary image group; and outputting a search result including the acquired image data. ### Claim 9 (depends on 8) The search method according to claim 8, wherein an image indicating an associated person of the search target person is provided as the auxiliary image. ### Claim 10 (depends on 8) The search method according to claim 8, further comprising determining whether the search target person is captured in the image data, based on the similarity degree calculated for the target image and the auxiliary image group. ### Claim 11 (depends on 8) The search method according to claim 8, further comprising providing, for the image data, a similarity degree analysis indicating a certainty degree that the search target person is captured, based on the metadata of the image data being the selection target and the similarity degree calculated for the target image and the auxiliary image group. ### Claim 12 (depends on 8) The search method according to claim 8, further comprising outputting the search result to an output device including a monitor, and causing the monitor to display a first user interface for selecting the image data included in the search result. ### Claim 13 A non-transitory program recording medium that records a program causing a computer to perform: processing of detecting a face from at least one piece of image data being a selection target stored in a database; processing of evaluating, when a target image indicating a search target person and an auxiliary image group including at least one auxiliary image that assists a search for the search target person are designated, whether a face detected from the image data being the selection target is included in the target image and the auxiliary image group; processing of determining whether the search target person is captured in the image data, by using an evaluation result; and processing of acquiring, from the database, the image data determined that the search target person is captured, based on a determination result; processing of calculating a similarity degree between a face detected from the image data and a face of a person included in the target image and the auxiliary image group; processing of acquiring metadata of the image data being the selection target from the database; processing of determining whether the search target person is captured in the image data, based on the acquired metadata of the image data being the selection target and the similarity degree calculated for the target image and the auxiliary image group; and processing of outputting a search result including the acquired image data. ### Claim 14 (depends on 13) The non-transitory program recording medium according to claim 13, wherein an image indicating an associated person of the search target person is provided as the auxiliary image. ### Claim 15 (depends on 13) The non-transitory program recording medium according to claim 13, wherein the program causes the computer to further perform: processing of determining whether the search target person is captured in the image data, based on the similarity degree calculated for the target image and the auxiliary image group. ### Claim 16 (depends on 13) The non-transitory program recording medium according to claim 13, wherein the program causes the computer to further perform: processing of providing, for the image data, a similarity degree analysis indicating a certainty degree that the search target person is captured, based on the metadata of the image data being the selection target and the similarity degree calculated for the target image and the auxiliary image group. ### Claim 17 (depends on 13) The non-transitory program recording medium according to claim 13, wherein the program causes the computer to further perform: processing of outputting the search result to an output device including a monitor, and processing of causing the monitor to display a first user interface for selecting the image data included in the search result.