# US 6,731,826 B1 · Image search apparatus and method, and computer readable memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 6,731,826 B1
- Document ID: US6731826B1
- Status: Granted
- Kind: B1
- Application number: 09/385,430
- Filed: 1999-08-30
- Granted: 2004-05-04
- Earliest claimed: 1998-08-31
- Assignees: Canon Kabushiki Kaisha
- Inventors: Kunihiro Yamamoto; Kentaro Matsumoto; Miyuki Enokida; Kiyoshi Kusama
- Web page: https://patentgrep.com/patent/US6731826B1

## CPC

- G06F 16/532: Query formulation, e.g. graphical querying
- Y10S 707/99933: Query processing, i.e. searching
- Y10S 707/99936: Pattern matching access
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A plurality of image data are stored in a hard disk device in correspondence with their image feature amounts. An image is using an input window. A CPU computes the image feature amount of the input image. The CPU computes image similarity on the basis of the computed image feature amount and the image feature amounts of the stored image data.

## Claims

### Claim 1

An image search apparatus for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:
input means for inputting a stroke or strokes that form an image using an input window displayed on a display unit wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;
image feature amount computing means for computing an image feature amount of an image drawn every time a stroke that forms a partial of the image is input by said input means;
image similarity computing means for computing image similarity every time a stroke that forms the image is input by said input means on the basis of the image feature amount computed by said image feature amount computing means; and
image search means for searching for desired image data from the image database every time a stroke that forms the image is input by said input means on the basis of the image similarity computed by said image similarity computing means.

### Claim 2 (depends on 1)

The apparatus according to claim 1, further comprising image display means for displaying a list of image data as search results on the basis of the image similarity computed by said image similarity computing means.

### Claim 3 (depends on 2)

The apparatus according to claim 2, wherein said image display means displays reduced-scale images corresponding to the image data as search results.

### Claim 4 (depends on 3)

The apparatus according to claim 3, wherein said image display means comprises display control means for displaying a detailed image which is associated with each of the reduced-scale images, and
said display control means displays the detailed image associated with the selected reduced-scale image, when one or a plurality of the reduced-scaled images are selected.

### Claim 5 (depends on 2)

The apparatus according to claim 2, wherein the input window and a display window displayed by said image display means are simultaneously displayed on a display unit.

### Claim 6 (depends on 2)

The apparatus according to claim 2, wherein the input window and a display window displayed by said image display means are alternately displayed on a display unit in accordance with an instruction.

### Claim 7 (depends on 1)

The apparatus according to claim 1, wherein said input means comprises a pointing device.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein said pointing device comprises a mouse.

### Claim 9 (depends on 7)

The apparatus according to claim 7, wherein said pointing device comprises a touch screen.

### Claim 10 (depends on 7)

The apparatus according to claim 7, wherein said pointing device comprises a pen tablet.

### Claim 11 (depends on 2)

The apparatus according to claim 2, wherein said image display means displays the image data as the search results to line up one-dimensionally.

### Claim 12 (depends on 2)

The apparatus according to claim 2, wherein said image display means displays the image data as the search results to line up two-dimensionally.

### Claim 13 (depends on 2)

The apparatus according to claim 2, wherein said image display means displays the image data as the search results to line up three-dimensionally.

### Claim 14 (depends on 2)

The apparatus according to claim 2, wherein said image display means displays the image data as the search results in descending order of image similarity.

### Claim 15

An image search method for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:
an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;
an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial image of the image is input on the input window;
an image similarity computing step, of computing image similarity every time a stroke that forms the image is input on the basis of the image feature amount computed in said image feature amount computing step; and
an image searching step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step.

### Claim 16 (depends on 15)

The method according to claim 15, further comprising an image display step of displaying a list of image data as search results on the basis of the image similarity computed in said image similarity computing step.

### Claim 17 (depends on 16)

The method according to claim 16, wherein said image display step includes the step of displaying reduced-scale images corresponding to the image data as search results.

### Claim 18 (depends on 17)

The method according to claim 17, wherein said image display step comprises a display control step of displaying a detailed image which is associated with each of the reduced-scale images, and
said display control step includes the step of displaying the detailed image associated with the selected reduced-scale image, when one or a plurality of the reduced-scaled images are selected.

### Claim 19 (depends on 16)

The method according to claim 16, wherein the image input on the input window and the list of image data as the search results displayed in said image display step are simultaneously displayed on a display unit.

### Claim 20 (depends on 16)

The method according to claim 16, wherein the image input on the input window and the list of image data as the search results displayed in said image display step are alternately displayed on a display unit in accordance with an instruction.

### Claim 21 (depends on 15)

The method according to claim 15, wherein the image is input by a pointing device.

### Claim 22 (depends on 21)

The method according to claim 21, wherein the pointing device comprises a mouse.

### Claim 23 (depends on 21)

The method according to claim 21, wherein the pointing device comprises a touch screen.

### Claim 24 (depends on 21)

The method according to claim 21, wherein the pointing device comprises a pen tablet.

### Claim 25 (depends on 16)

The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results to line up one-dimensionally.

### Claim 26 (depends on 16)

The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results to line up two-dimensionally.

### Claim 27 (depends on 16)

The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results to line up three-dimensionally.

### Claim 28 (depends on 16)

The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results in descending order of image similarity.

### Claim 29

A computer readable memory that stores a program code of an image search for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:
a program code of an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;
a program code of an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial image of the image is input on the input window;
a program code of an image similarity computing step, of computing image similarity every time a stroke that forms the image is input on the basis of the image feature amount computed in said image feature amount computing step; and
a program code for an image searching step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step.

### Claim 30

An image search apparatus for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:
input means for inputting a stroke or strokes that form an image using an input window displayed on a display unit wherein the stroke or strokes are handwritten stroke or strokes and can be imputted succesively and directly in the input window;
image feature amount computing means for computing an image feature amount of an image drawn every time a stroke that forms a partial of the image is input by said input means;
image similarity computing means for computing image similarity every time a stroke that forms the image is input by said input means, on the basis of the image feature amount computed by said image feature amount computing means;
image search means for searching for desired image data from the image database every time a stroke that forms the image is input by said input means, on the basis of the image similarity computed by said image similarity computing means;
image display means for displaying a list of image data as search results obtained by said image search means; and
display control means for controlling to display the input window and a display window displayed by said image display means on a single screen of a display unit.

### Claim 31 (depends on 30)

The apparatus according to claim 30, wherein said image feature amount computing means computes the image feature amount of an image drawn every time the image is modified by said input means.

### Claim 32 (depends on 30)

The apparatus according to claim 30, further comprising instruction means for instructing start of a search using the image input by said input means as a query criteria, and
wherein said image feature amount computing means computes the image feature amount of the image input by said input means upon receiving the instruction from said instruction means.

### Claim 33 (depends on 30)

The apparatus according to claim 30, wherein said image feature amount computing means comprises:
management means for managing an input start time of the image by said input means; and
computing means for computing a time elapsed from the start time every time a stroke that forms the image is input by said input means, and
wherein said image feature amount computing means computes the image feature amount of the image input by said input means on the basis of the elapsed time computed by said computing means.

### Claim 34 (depends on 30)

The apparatus according to claim 30, wherein said image feature amount computing means computes the image feature amount of the image input by said input means at a predetermined time interval.

### Claim 35 (depends on 30)

The apparatus according to claim 30, wherein said image feature amount computing means comprises monitoring means for monitoring a load on said image search apparatus, and
wherein said image feature amount computing means computes the image feature amount of the image input by said input means in accordance with a monitoring result of said monitoring means.

### Claim 36 (depends on 30)

The apparatus according to claim 30, wherein said image feature amount computing means computes the image feature amount of the image input by said input means so far in accordance with the number of strokes that form the image input by said input means.

### Claim 37

An image search method for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:
an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;
an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial image of the image is input on the input window;
an image similarity computing step, of computing image similarity every time a stroke that forms the image is input, on the basis of the image feature amount computed in said image feature amount computing step;
an image search step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step;
an image display step, of displaying a list of image data as search results obtained in said image search step; and
an display control step, of controlling to display the input window and a display window displayed in the image display step on a single screen of a display unit.

### Claim 38 (depends on 37)

The method according to claim 37, wherein said image feature amount computing step includes the step of computing the image feature amount of an image drawn every time the image input on the input window is modified.

### Claim 39 (depends on 37)

The method according to claim 37, further comprising an instruction step of instructing start of a search using the image input on the input window as a query criteria,
wherein said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window upon receiving the instruction issued in said instruction step.

### Claim 40 (depends on 37)

The method according to claim 37, wherein, said image feature amount computing step comprises:
a management step of managing an input start time of the image on the input window; and
a computing step of computing a time elapsed from the start time every time a stroke that forms the image is input on the input window, and
said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window on the basis of the elapsed time computed in said computing step.

### Claim 41 (depends on 37)

The method according to claim 37, wherein said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window at a predetermined time interval.

### Claim 42 (depends on 37)

The method according to claim 37, wherein said image feature amount computing step comprises the monitoring step of monitoring a load on an image search apparatus, and
said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window in accordance with a monitoring result in said monitoring step.

### Claim 43 (depends on 37)

The method according to claim 37, wherein said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window so far in accordance with the number of strokes that form the image input on the input window.

### Claim 44

A computer readable memory that stores a program code of an image search for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:
a program code of an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;
a program code of an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial of the image is input on the input window;
a program code of an image similarity computing step, of computing image similarity every time a stroke that forms the image is input, on the basis of the image feature amount computed in said image feature amount computing step;
a program code of an image search step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step;
a program code of an image display step, of displaying a list of image data as search results obtained in said image search step; and
a program code of a display control step, of controlling to display the input window and a display window displayed in said image display step on a single screen of a display unit.
