Generating a combination of a visual query and matching canonical document

A computer-implemented method of processing a visual query, performed by a server system having one or more processors and memory storing one or more programs for execution by the one or more processors, the method comprising:

• receiving a visual query from a client system distinct from the server system;
• performing optical character recognition (OCR) on the visual query to produce text recognition data representing textual characters, including a plurality of textual characters in a contiguous region of the visual query;
• scoring each textual character in the plurality of textual characters;
• identifying, in accordance with the scoring, one or more high quality textual strings, each comprising a plurality of high quality textual characters from among the plurality of textual characters in the contiguous region of the visual query;
• retrieving a canonical document having the one or more high quality textual strings, the retrieving comprising:
• calculating a quality score corresponding to at least one respective high quality textual string of the one or more high quality textual strings;
• retrieving an image version of the canonical document if the quality score is below a predetermined value;
• and
• retrieving a machine readable text version of the canonical document if the quality score is at or above a predetermined value;
• generating a combination of the visual query and at least a portion of the canonical document;
• and
• sending the combination to the client system.

A server system, for processing a visual query, comprising:

• one or more central processing units for executing programs;
• memory storing one or more programs be executed by the one or more central processing units;
• the one or more programs comprising instructions for:
• receiving a visual query from a client system;
• performing optical character recognition (OCR) on the visual query to produce text recognition data representing textual characters, including a plurality of textual characters in a contiguous region of the visual query;
• scoring each textual character in the plurality of textual characters, wherein the scoring of a respective textual character is based on both an OCR quality score of the respective textual character alone and an OCR quality score of one or more neighboring textual characters;
• identifying, in accordance with the scoring, one or more high quality textual strings, each comprising a plurality of high quality textual characters from among the plurality of textual characters in the contiguous region of the visual query;
• retrieving a canonical document having the one or more high quality textual strings;
• generating a combination of the visual query and at least a portion of the canonical document;
• and
• sending the combination to the client system.

A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:

• receiving a visual query from a client system;
• performing optical character recognition (OCR) on the visual query to produce text recognition data representing textual characters, including a plurality of textual characters in a contiguous region of the visual query;
• scoring each textual character in the plurality of textual characters, wherein the scoring of a respective textual character is based on both an OCR quality score of the respective textual character alone and an OCR quality score of one or more neighboring textual characters;
• identifying, in accordance with the scoring, one or more high quality textual strings, each comprising a plurality of high quality textual characters from among the plurality of textual characters in the contiguous region of the visual query;
• retrieving a canonical document having the one or more high quality textual strings;
• generating a combination of the visual query and at least a portion of the canonical document;
• and
• sending the combination to the client system.