# US 2014/0193077 A1 · Image retrieval apparatus, image retrieval method, query image providing apparatus, query image providing method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0193077 A1 - Document ID: US20140193077A1 - Status: Published application - Kind: A1 - Application number: 14/148,300 - Filed: 2014-01-06 - Published: 2014-07-10 - Earliest claimed: 2013-01-08 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Hirotaka Shiiyama; Masahiro Matsushita - Web page: https://patentgrep.com/patent/US20140193077A1 ## CPC - G06F 16/583: using metadata automatically derived from the content - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An image retrieval apparatus generates an image feature list having a high and/or a low discrimination capability against other images based on the number of stored image features. Based on the image feature list, the image retrieval apparatus selects feature points extracted from a query image. ## Claims ### Claim 1 An image retrieval apparatus comprising: a storage unit configured to store image features extracted from an input image and the image in association with each other; a generation unit configured to generate a frequently appearing image feature list collecting image features including similar image features stored in the storage unit which are greater than a predetermined number; a selection unit configured to select image features to be used as image features of a inputted query image, from image features extracted from the query image, based on the image features included in the frequently appearing image feature list; and a comparison unit configured to compare image features selected to be used as the image features of the query image with image features of registered images stored in the storage unit. ### Claim 2 (depends on 1) The image retrieval apparatus according to claim 1, wherein a size or the number of image features of the query image to be used in the comparison unit is determined in advance. ### Claim 3 (depends on 1) The image retrieval apparatus according to claim 1, wherein the generation unit is configured to use a fixed value as the predetermined number. ### Claim 4 (depends on 1) The image retrieval apparatus according to claim 1, wherein the generation unit is configured to use a variable value as the predetermined number, the variable value being determined according to the number of image features stored in the storage unit or the number of images stored in the storage unit in association with the image features. ### Claim 5 (depends on 1) The image retrieval apparatus according to claim 1, wherein the generation unit is configured to determine as the predetermined number, a variable value or a fixed value, according to the total number of images stored in the storage unit in association with the image features. ### Claim 6 (depends on 1) The image retrieval apparatus according to claim 1, wherein the image features includes local feature points obtained by applying a Harris operator and local feature amounts corresponding to the local feature points. ### Claim 7 (depends on 1) The image retrieval apparatus according to claim 1, wherein the selection unit is configured to select the image features by excluding all or part of image features matching the image features included in the frequently appearing image feature list, from the image features of the query image. ### Claim 8 (depends on 1) The image retrieval apparatus according to claim 1, wherein the selection unit is configured to select the image feature by being identifiable all or part of image features matching the image features included in the frequently appearing image feature list in the image features of the query image; and the comparison unit is configured to compare using the image features except for the image features been identifiable. ### Claim 9 (depends on 1) The image retrieval apparatus according to claim 1, wherein the selection unit is configured to select the image features of the query image based on a size or the number of image features of the query image determined in advance to be used in the comparison unit. ### Claim 10 An image retrieval apparatus comprising: a storage unit configured to store image features corresponding to an image; a generation unit configured to generate a frequently appearing image feature list collecting image features in which the number of image features stored in the storage unit or the number of images stored in the storage unit in association with the image features is greater than a predetermined number; a transmission unit configured to transmit the frequently appearing image feature list to a request source in response to a request; a reception unit configured to receive image features selected by the request source as image features to be used as image features of a query image, based on the image features included in the frequently appearing image feature list; and a comparison unit configured to compare the received image features of the query image with the image features stored in the storage unit. ### Claim 11 A query image providing apparatus comprising: an input unit configured to input a query image; an extraction unit configured to extract image features of the query image; a reception unit configured to receive a frequently appearing image feature list from an image retrieval apparatus; a selection unit configured to select image features to be used as image features of the query image and image features not to be used, from among the image features of the query image, based on image features included in the frequently appearing image feature list; and a transmission unit configured to transmit image features selected to be used as the image features of the query image, to the image retrieval apparatus. ### Claim 12 An image retrieval apparatus comprising: a storage unit configured to store image features extracted from an input image, attribute information about photographing obtained from the input image, and the image in association with each other; a generation unit configured to generate with respect to each attribute information, an image feature list in which the number of image features co-occurring in the images with respect to each attribute information is less than a predetermined number; a selection unit configured to select image features to be used as image features of a inputted query image and image features not to be used, from among the image features extracted with respect to the query image input by the input unit, based on image features included in the image feature list corresponding to attribute information about photographing of the query image obtained by an acquisition unit; and a comparison unit configured to perform a similarity comparison between image features of the query image selected to be used as the image features of the query image and image features of registered images stored in the storage unit. ### Claim 13 (depends on 12) The image retrieval apparatus according to claim 12, wherein a size or the number of image features of the query image to be used in the comparison unit is determined in advance. ### Claim 14 (depends on 12) The image retrieval apparatus according to claim 12, wherein the attribute information about photographing includes at least one of a location, a photographing azimuth direction, time, a weather condition, a photographing mode, and an object attribute. ### Claim 15 (depends on 12) The image retrieval apparatus according to claim 12, wherein the generation unit is configured to describe, in the image feature list, image features stored in the storage unit and having a co-occurrence value smaller than a threshold with respect to registered images matching at least one piece of the attribute information, or a predetermined number of image features in ascending order of the co-occurrence value. ### Claim 16 (depends on 12) The image retrieval apparatus according to claim 12, wherein the generation unit is configured to describe, in the image feature list, image features in which a co-occurrence value of the images stored in association with image features with respect to registered images matching at least one piece of the attribute information is smaller than a threshold, or the number of image features is determined in an ascending order of the co-occurrence value. ### Claim 17 (depends on 12) The image retrieval apparatus according to claim 12, wherein the generation unit is configured to describe a user-selected image features in the image feature list, based on a co-occurrence value of images stored in association with the image features with respect to registered images matching at least one piece of the attribute information. ### Claim 18 (depends on 12) The image retrieval apparatus according to claim 12, wherein the generation unit is configured to generate with respect to each of the compass direction in a photographing location of the images, an image feature list in which the number of image features co-occurring in the images with respect to each of the compass direction is less than a predetermined number. ### Claim 19 (depends on 12) The image retrieval apparatus according to claim 12, wherein the selection unit is configured to select the image features by extracting image features matching the image features included in the image feature list from the image features of the query image. ### Claim 20 (depends on 12) The image retrieval apparatus according to claim 12, wherein the selection unit is configured to select the image features by deleting image features not existing in the image feature list from the image features of the query image. ### Claim 21 (depends on 1) The image retrieval apparatus according to claim 1, wherein the selection unit is configured to select the image features by identifying image features matching the image feature list in the image features of the query image. ### Claim 22 An image retrieval apparatus comprising: a storage unit configured to store image features corresponding to an image and attribute information about photographing; a generation unit configured to generate with respect to each of attribute information, the image feature list in which the number of image features co-occurring in the image with respect to each of the attribute information is less than a predetermined number; a transmission unit configured to transmit the image feature list corresponding to attribute information about photographing of a query image to a request source in response to a request; a reception unit configured to receive image features selected by the request source as an image feature to be used as an image feature of the query image, based on image features included in the image feature list; and a comparison unit configured to compare the received image features of the query image with the image features stored in the storage unit. ### Claim 23 A query image providing apparatus comprising: an input unit configured to input a query image; an acquisition unit configured to obtain attribute information about photographing of the query image; an extraction unit configured to extract image features of the query image; a reception unit configured to receive an image feature list from an image retrieval apparatus; a selection unit configured to select an image feature to be used as an image feature of the query image and an image feature which use is restricted, from the image features of the query image, based on image features included in the image feature list corresponding to the attribute information; and a transmission apparatus configured to transmit image features selected to be used as the image feature of the query image to the image retrieval apparatus. ### Claim 24 An image retrieval method comprising: storing image features extracted from an input image and the image into a storage unit in association with each other; generating a frequently appearing image feature list summarizing image features including similar image features stored in the storage unit which are greater than a predetermined number; selecting an image feature to be used as an image feature of a query image input by an input unit, from image features extracted from the query image by an extraction unit, based on the image features included in the frequently appearing image feature list; and comparing image features selected to be used as the image feature of the query image with image features of a registration image stored in the storage unit. ### Claim 25 An image retrieval method comprising: storing image features corresponding to an image into a storage unit; generating a frequently appearing image feature list summarizing image features in which the number of registration of image features stored in the storage unit or the number of images stored in association with the image features in the storage unit is greater than a predetermined number; transmitting the frequently appearing image feature list to a request source in response to a request; receiving image features selected by the request source as an image feature to be used as an image feature of a query image, based on the image features included in the frequently appearing image feature list; and comparing the received image features of the query image with the image features stored in the storage unit. ### Claim 26 A query image providing method comprising: inputting a query image; extracting image features of the query image; receiving a frequently appearing image feature list from an image retrieval apparatus; selecting an image feature to be used as an image feature of the query image and an image feature which use is restricted, from the image features of the query image, based on image features included in the frequently appearing image feature list; and transmitting image features selected to be used as the image feature of the query image, to the image retrieval apparatus. ### Claim 27 An image retrieval method comprising: storing image features extracted from an input image, attribute information about photographing obtained from the input image, and the image in a storage unit in association with each other; generating with respect to each of the attribute information, an image feature list in which the number of image features co-occurring in the image with respect to each of the attribute information is less than a predetermined number; selecting an image feature to be used as an image feature of a query image and an image feature which use is restricted, from among image features extracted from the query image by an extraction unit with respect to the query image input by an input unit, based on image features included in the image feature list corresponding to attribute information about photographing of the query image obtained by an acquisition unit; and performing a similarly comparison between image features of the query image selected to be used as the image feature of the query image and image features of a registration image stored in the storage unit. ### Claim 28 An image retrieval method comprising: storing image features corresponding to an image and attribute information about photographing into a storage unit; generating with respect to each of the attribute information, the image feature list in which the number of image features co-occurring in the image with respect to each of the attribute information is less than a predetermined number; transmitting the image feature list corresponding to attribute information about photographing of a query image to a request source in response to a request; receiving image features selected by the request source as an image feature to be used as an image feature of the query image, based on image features included in the image feature list; and comparing the received image features of the query image with the image features stored in the storage unit. ### Claim 29 A query image providing method comprising: inputting a query image; obtaining attribute information about photographing of the query image; extracting image features of the query image; receiving an image feature list from an image retrieval apparatus; selecting an image feature to be used as an image feature of the query image and an image feature which use is restricted, from the image features of the query image, based on image features included in the image feature list corresponding to the attribute information; and transmitting image features selected to be used as the image feature of the query image to the image retrieval apparatus. ### Claim 30 (depends on 24) A medium storing a program for causing a computer to perform the image retrieval method according to claim 24. ### Claim 31 (depends on 25) A medium storing a program for causing a computer to perform the image retrieval method according to claim 25. ### Claim 32 (depends on 26) A medium storing a program for causing a computer to perform the query image providing method according to claim 26. ### Claim 33 (depends on 27) A medium storing a program for causing a computer to perform the image retrieval method according to claim 27. ### Claim 34 (depends on 28) A medium storing a program for causing a computer to perform the image retrieval method according to claim 28. ### Claim 35 (depends on 29) A medium storing a program for causing a computer to perform the query image providing method according to claim 29.