# US 2003/0229628 A1 · Method and apparatus for processing user input selecting images from a web page in a data processing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0229628 A1
- Document ID: US20030229628A1
- Status: Published application
- Kind: A1
- Application number: 10/165,875
- Filed: 2002-06-10
- Published: 2003-12-11
- Earliest claimed: 2002-06-10
- Assignees: International Business Machines Corporation
- Inventors: Dwip N. Banerjee; Marilyn Smith Dawkins; Rabindranath Dutta
- Web page: https://patentgrep.com/patent/US20030229628A1

## CPC

- G06F 16/38: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/532: Query formulation, e.g. graphical querying
- Y10S 707/915: Image
- Y10S 707/917: Text
- Y10S 707/99933: Query processing, i.e. searching
- Y10S 707/99943: Generating database or data structure, e.g. via user interface
- Y10S 707/99945: Object-oriented database structure processing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A method, apparatus, and computer instructions for processing a user input. A user input selecting an image on a page is detected. The image selected by the user input is sent to a server to perform a search in response to detecting the user input. The server performs object recognition analysis on the image to generate a text description of the image. Alternatively, the server may perform an image search for resources with similar images or properties. This text description, which may include one or more words, is used as the basis of the search. A result of the search is received in response to sending the image to the server to form a received result. The received result is displayed. Further, other contextual information associated with the image may be used in addition to or in place of the image in generating the search.

## Claims

### Claim 1

A method in a data processing system for processing a user input comprising:
detecting a user input selecting an image on a page;
sending the image to a server to perform a search in response to detecting the user input;
receiving a result of the search in response to sending the image to the server to form a received result; and
displaying the received result.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
determining whether the image has tag information; and
responsive to an ALT tag being present, sending the tag information to the server to perform the search.

### Claim 3 (depends on 2)

The method of claim 2, wherein the tag information is sent instead of the image.

### Claim 4 (depends on 3)

The method of claim 3, wherein the tag information includes at least one of ALT tag information and meta tag information.

### Claim 5 (depends on 1)

The method of claim 1, wherein the result is a Web page.

### Claim 6 (depends on 5)

The method of claim 5, wherein the Web page contains links to Web pages having a subject matter related to the image.

### Claim 7 (depends on 1)

The method of claim 1, wherein the step of sending the image to the server includes sending other information associated with the image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the other information is contextual information about the image.

### Claim 9 (depends on 8)

The method of claim 8, wherein the contextual information includes at least one of ALT tag information and meta tag information.

### Claim 10 (depends on 1)

The method of claim 1, wherein the search includes at least one of matching the image with images contained in Web pages and performing object recognition to generate text used to search for Web pages.

### Claim 11

A method in a data processing system for processing a user input comprising:
detecting a user input selecting an image on a page;
identifying contextual information associated with the image;
sending the contextual information to a server to perform a search;
receiving a result of the search in response to sending the contextual information to the server to form a received result; and
displaying the received result.

### Claim 12 (depends on 11)

The method of claim 11, wherein the contextual information is found in association with at least one of an ALT tag and a meta tag for the image.

### Claim 13 (depends on 12)

The method of claim 12, wherein the contextual information is dynamically associated with the image.

### Claim 14

A method in a data processing system for processing a search request, the method comprising:
receiving an image from a requester, wherein the image forms the search request;
performing object recognition on the image to generate a text identification of the image;
initiating a search using the text identification of the image; and
responsive to receiving a result, sending the result to the requestor.

### Claim 15

A data processing system for processing a user input, the data processing system comprising:
a bus system;
a communications unit connected to the bus system;
a memory connected to the bus system, wherein the memory includes a set of instructions; and
a processing unit connected to the bus system, wherein the processing unit executes the set of instructions to detect a user input selecting an image on a page; send the image to a server to perform a search in response to detecting the user input; receive a result of the search in response to sending the image to the server to form a received result; and display the received result.

### Claim 16

A data processing system for processing a user input, the data processing system comprising:
a bus system;
a communications unit connected to the bus system;
a memory connected to the bus system, wherein the memory includes a set of instructions; and
a processing unit connected to the bus system, wherein the processing unit executes the set of instructions to detect a user input selecting an image on a page; identify contextual information associated with the image; send the contextual information to a server to perform a search; receive a result of the search in response to sending the contextual information to the server to form a received result; and display the received result.

### Claim 17

A data processing system for processing a search request, the data processing system comprising:
a bus system;
a communications unit connected to the bus system;
a memory connected to the bus system, wherein the memory includes a set of instructions; and
a processing unit connected to the bus system, wherein the processing unit executes the set of instructions to receive an image from a requester in which the image forms the search request; perform object recognition on the image to generate a text identification of the image; initiate a search using the text identification of the image; and send the result to the requester in response to receiving a result.

### Claim 18

A data processing system for processing a user input comprising:
detecting means for detecting a user input selecting an image on a page;
sending means for sending the image to a server to perform a search in response to detecting the user input;
receiving means for receiving a result of the search in response to sending the image to the server to form a received result; and
displaying means for displaying the received result.

### Claim 19 (depends on 18)

The data processing system of claim 18, wherein the sending means is a first sending means and further comprising:
determining means for determining whether the image has tag information; and
second sending means, responsive to an ALT tag being present, for sending the tag information to the server to perform the search.

### Claim 20 (depends on 19)

The data processing system of claim 19, wherein the tag information is sent instead of the image.

### Claim 21 (depends on 20)

The data processing system of claim 20, wherein the tag information includes at least one of ALT tag information and meta tag information.

### Claim 22 (depends on 21)

The data processing system of claim 21, wherein the result is a Web page.

### Claim 23 (depends on 22)

The data processing system of claim 22, wherein the Web page contains links to Web pages having a subject matter related to the image.

### Claim 24 (depends on 18)

The data processing system of claim 18, wherein the means of sending the image to the server includes sending other information associated with the image.

### Claim 25 (depends on 24)

The data processing system of claim 24, wherein the other information is contextual information about the image.

### Claim 26 (depends on 25)

The data processing system of claim 25, wherein the contextual information includes at least one of ALT tag information and meta tag information.

### Claim 27 (depends on 18)

The data processing system of claim 18, wherein the search includes at least one of matching the image with images contained in Web pages and performing object recognition to generate text used to search for Web pages.

### Claim 28

A data processing system for processing a user input comprising:
detecting means for detecting a user input selecting an image on a page;
identifying means for identifying contextual information associated with the image;
sending means for sending the contextual information to a server to perform a search;
receiving means for receiving a result of the search in response to sending the contextual information to the server to form a received result; and
displaying means for displaying the received result.

### Claim 29 (depends on 28)

The data processing system of claim 28, wherein the contextual information is found in association with at least one of an ALT tag and a meta tag for the image.

### Claim 30 (depends on 29)

The data processing system of claim 29, wherein the contextual information is dynamically associated with the image.

### Claim 31

A data processing system for processing a search request, the data processing system comprising:
receiving means for receiving an image from a requester in which the image forms the search request;
performing means for performing object recognition on the image to generate a text identification of the image;
initiating means for initiating a search using the text identification of the image; and
sending means, responsive to receiving a result, for sending the result to the requester.

### Claim 32

A computer program product in a computer readable medium for processing a user input, the computer program product comprising:
first instructions for detecting a user input selecting an image on a page;
second instructions for sending the image to a server to perform a search in response to detecting the user input;
third instructions for receiving a result of the search in response to sending the image to the server to form a received result; and
fourth instructions for displaying the received result.

### Claim 33

A computer program product in a computer readable medium for processing a user input, the computer program product comprising:
first instructions for detecting a user input selecting an image on a page;
second instructions for identifying contextual information associated with the image;
third instructions for sending the contextual information to a server to perform a search;
fourth instructions for receiving a result of the search in response to sending the contextual information to the server to form a received result; and
fifth instructions for displaying the received result.

### Claim 34

A computer program product in a computer readable medium for processing a search request, the computer program product comprising:
first instructions for receiving an image from a requester, wherein the image forms the search request;
second instructions for performing object recognition on the image to generate a text identification of the image;
third instructions for initiating a search using the text identification of the image; and
fourth instructions, responsive to receiving a result, for sending the result to the requester.

### Claim 35

A method for processing a search request, the method comprising:
receiving an image from a requester;
analyzing the image to generate an identification identifying at least one of a good and a service;
identifying a business based on the identification; and
directing the requester to the business.

### Claim 36 (depends on 35)

The method of claim 35, wherein the directing step comprises:
returning a link to a Web page for the business.

### Claim 37 (depends on 35)

The method of claim 35, wherein the directing step comprises:
redirecting the requester to a Website for the business.

### Claim 38

A method in a data processing system for processing a Web page, the method comprising:
responsive to receiving the Web page from a submitter, searching the Web page for an image; responsive to identifying an image contained in the Web page, determining whether a replacement image is available;
responsive to a determination the replacement image is present, replacing the image with the replacement image to form a modified Web page, wherein the replacement image includes a link to a selected Website; and
returning the modified Web page to the submitter.

### Claim 39 (depends on 38)

The method of claim 38 further comprising:
billing a client for generating the modified Web page.
