# US 2014/0195560 A1 · Two way local feature matching to improve visual search accuracy > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195560 A1 - Document ID: US20140195560A1 - Status: Published application - Kind: A1 - Application number: 14/060,314 - Filed: 2013-10-22 - Published: 2014-07-10 - Earliest claimed: 2013-01-09 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., LTD - Inventors: Xin Xin; Zhu Li; Abhishek Nagar; Gaurav Srivastava; Zhan Ma; Kong Posh Bhat; Felix Carlos Fernandes - Web page: https://patentgrep.com/patent/US20140195560A1 ## CPC - G06F 16/583: using metadata automatically derived from the content - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract To improve precision of visual search processing, SIFT points within a query image are forward matched to features in each of a plurality of repository images and SIFT points within each repository image are backward matched to features within the query image. Forward-only, backward-only and forward-and-backward matches may be weighted differently in determining an image match. Two way matching may be triggered by query image bit rate in excess of a threshold or by a sum of weighted distances between matching points exceeding a threshold. Significant performance gains in eliminating false positive matches are achieved. ## Claims ### Claim 1 A method, comprising: receiving, at a visual search server having access to one or more repository images, information relating to distinctive features within a query image for a visual search request; in the visual search server, forward matching the distinctive features within the query image to distinctive features within each of the one or more repository images, when selected criteria are met, backward matching distinctive features within the respective repository image to the distinctive features within the query image, and determining whether each repository image correlates to the query image based upon results of matching distinctive features. ### Claim 2 (depends on 1) The method according to claim 1, wherein the backward matching is selectively performed when a sum of weights based upon distances in the forward matched distinctive features between the query image and the respective repository image exceed a threshold. ### Claim 3 (depends on 1) The method according to claim 1, wherein backward matching the distinctive features within the respective repository image to the distinctive features within the query image is performed for all of the repository images. ### Claim 4 (depends on 1) The method according to claim 1, wherein determining whether each repository image correlates to the query image based upon results of matching distinctive features involves considering only forward matching and backward matching distinctive features. ### Claim 5 (depends on 1) The method according to claim 1, wherein distinctive feature matches between the query image and one of the repository images used to determine an image match are weighted based upon whether the match is forward-matching only, backward-matching only, or both forward-matching and backward-matching. ### Claim 6 (depends on 1) The method according to claim 1, wherein the distinctive features within the query image and within the repository images are each Scale Invariant Feature Transform (SIFT) points. ### Claim 7 (depends on 1) The method according to claim 1, wherein only the forward matching is performed for query images corresponding to a bit rate less than a predetermined threshold, and the forward matching and the backward matching are performed for query images corresponding to a bit rate higher than the predetermined threshold. ### Claim 8 A method, comprising: receiving, at a visual search server having access to one or more repository images, information relating to distinctive features within a query image for a visual search request; in the visual search server, forward matching the distinctive features within the query image to distinctive features within each of the one or more repository images, when at least one of (a) a sum of weights based upon distances in the forward matched distinctive features between the query image and the respective repository image, and (b) a bit rate for the query image is higher than the predetermined threshold, backward matching distinctive features within the respective repository image to the distinctive features within the query image, and determining whether each repository image correlates to the query image based upon the forward matching and the backward matching. ### Claim 9 (depends on 8) The method according to claim 8, wherein distinctive feature matches between the query image and one of the repository images used to determine an image match are weighted based upon whether the match is forward-matching only, backward-matching only, or both forward-matching and backward-matching. ### Claim 10 (depends on 8) The method according to claim 8, wherein the distinctive features within the query image and within the repository images are each Scale Invariant Feature Transform (SIFT) points. ### Claim 11 (depends on 10) The method according to claim 10, wherein the SIFT points are described in the visual search request by local descriptors. ### Claim 12 A visual search server system, comprising: a network connection configured to provide access to one or more repository images and configured to receive information relating to distinctive features within a query image for a visual search request; a processing system configured to forward match the distinctive features within the query image to distinctive features within each of the one or more repository images, when selected criteria are met, backward match distinctive features within the respective repository image to the distinctive features within the query image, and determine whether each repository image correlates to the query image based upon results of matching distinctive features. ### Claim 13 (depends on 12) The visual search server system according to claim 12, wherein the backward matching is selectively performed when a sum of weights based upon distances in the forward matched distinctive features between the query image and the respective repository image exceed a threshold. ### Claim 14 (depends on 12) The visual search server system according to claim 12, wherein backward matching the distinctive features within the respective repository image to the distinctive features within the query image is performed for all of the repository images. ### Claim 15 (depends on 12) The visual search server system according to claim 12, wherein correlation of each repository image to the query image is determined by considering only forward matching and backward matching distinctive features. ### Claim 16 (depends on 12) The visual search server system according to claim 12, wherein distinctive feature matches between the query image and one of the repository images used to determine an image match are weighted based upon whether the match is forward-matching only, backward-matching only, or both forward-matching and backward-matching. ### Claim 17 (depends on 12) The visual search server system according to claim 12, wherein the distinctive features within the query image and within the repository images are each Scale Invariant Feature Transform (SIFT) points. ### Claim 18 (depends on 12) The visual search server system according to claim 12, wherein only the forward matching is performed for query images corresponding to a bit rate less than a predetermined threshold, and the forward matching and the backward matching are performed for query images corresponding to a bit rate higher than the predetermined threshold. ### Claim 19 A visual search server, comprising: a network connection configured to provide access to one or more repository images and to receive information relating to distinctive features within a query image for a visual search request; a processing system configured to forward match the distinctive features within the query image to distinctive features within each of the one or more repository images, when at least one of (a) a sum of weights based upon distances in the forward matched distinctive features between the query image and the respective repository image, and (b) a bit rate for the query image is higher than the predetermined threshold, backward match distinctive features within the respective repository image to the distinctive features within the query image, and determine whether each repository image correlates to the query image based upon the forward matching and the backward matching. ### Claim 20 (depends on 19) The visual search server system according to claim 19, wherein distinctive feature matches between the query image and one of the repository images used to determine an image match are weighted based upon whether the match is forward-matching only, backward-matching only, or both forward-matching and backward-matching. ### Claim 21 (depends on 19) The visual search server system according to claim 19, wherein the distinctive features within the query image and within the repository images are each Scale Invariant Feature Transform (SIFT) points. ### Claim 22 (depends on 21) The visual search server system according to claim 21, wherein the SIFT points are described in the visual search request by local descriptors.