Method for indexing and retrieving manufacturing-specific digital imagery based on image content

A method for indexing and retrieving manufacturing-specific digital images based on image content comprising the steps of:

• extracting at least one feature vector from a manufacturing-specific digital image stored in an image database, said at least one feature vector corresponding to a particular characteristic of said manufacturing-specific image;
• using an unsupervised clustering method to index said at least one feature vector in a hierarchical search tree;
• and,
• retrieving a manufacturing-specific image corresponding to a feature vector stored in said hierarchical search tree, said manufacturing-specific image having image content comparably related to image content of a query image, wherein the retrieving step comprises the steps of:
• converting a query image into at least one query vector corresponding to said particular characteristic of said manufacturing-specific digital image;
• performing a first-level data reduction of feature vectors stored in said hierarchical data structure, said first-level data reduction based upon said at least one query vector, said first-level data reduction forming a subset of said feature vectors comparable to said query vector;
• accepting relevance feedback comprising a user-chosen selection of manufacturing-specific digital images corresponding to said subset of feature vectors, wherein the accepting step comprises the steps of:
• for each manufacturing-specific digital image in said selection, extracting three independent feature vectors of manufacturing-based digital imagery, said three independent feature vectors corresponding to a digital image modality and overall characteristic, a substrate/background characteristic, and an anomaly/defect characteristic;
• logically combining each said independent feature vector for each manufacturing-specific digital image in said selection, said logical combination forming a prototype vector for each said independent feature vector;
• calculating at least one prototype vector from said selection, said prototype vector corresponding to said particular characteristic of said manufacturing-specific digital image;
• and,
• performing a second-level data reduction of feature vectors stored in said hierarchical data structure, said second-level data reduction based upon said at least one prototype vector, said second-level data reduction forming a subset of said feature vectors comparable to said at least one prototype vector, and further comparable to said at least one query vector.

A computer apparatus programmed with a routine set of instructions stored in a fixed medium, said computer apparatus comprising:

• means for extracting a plurality of feature vectors from each of a plurality of digital images of in-process or completed semiconductors, said plurality of feature vectors corresponding to distinct descriptive characteristics of said semiconductor images;
• means for recording said plurality of feature vectors from said plurality of semiconductor images;
• means for indexing said plurality of feature vectors from said plurality of semiconductor images using an image clustering method to produce a hierarchical search tree, said clustering method mapping said plurality of images into a set of groups based on similar image content, said groups numbering less than a number of said plurality of images, said hierarchical search tree being a searchable library of said semiconductor images, wherein searching is based on said groups to achieve efficient retrieval times from said library;
• means for extracting at least one of said feature vectors from said query image,
• means for retrieving at least one image from said library based on an image similarity criterion to said feature vector of said query image, and means for displaying said at least one image.

A computer apparatus programmed with a routine set of instructions stored in a fixed medium, said computer apparatus comprising:

• means for extracting at least one feature vector from a manufacturing-specific digital image stored in an image database, said at least one feature vector corresponding to a particular characteristic of said manufacturing-specific image;
• means for using an unsupervised clustering method to index said at least one feature vector in a hierarchical search tree;
• means for retrieving a manufacturing-specific image corresponding to a feature vector stored in said hierarchical search tree, said manufacturing-specific image having image content comparably related to image content of a query image, wherein the retrieving means comprises:
• means for converting a query image into at least one query vector corresponding to said particular characteristic of said manufacturing-specific digital image;
• means for performing a first-level data reduction of feature vectors stored in said hierarchical data structure, said first-level data reduction based upon said at least one query vector, said first-level data reduction forming a subset of said feature vectors comparable to said query vector;
• means for accepting relevance feedback comprising a user-chosen selection of manufacturing-specific digital images corresponding to said subset of feature vectors, wherein the accepting means comprises:
• for each manufacturing-specific digital image in said selection, means for extracting three independent feature vectors of manufacturing-based digital imagery, said three independent feature vectors corresponding to a digital image modality and overall characteristic, a substrate/background characteristic, and an anomaly/defect characteristic;
• means for logically combining each said independent feature vector for each manufacturing-specific digital image in said selection, said logical combination forming a prototype vector for each said independent feature vector;
• means for calculating at least one prototype vector from said selection, said prototype vector corresponding to said particular characteristic of said manufacturing-specific digital image;
• and,
• means for performing a second-level data reduction of feature vectors stored in said hierarchical data structure, said second-level data reduction based upon said at least one prototype vector, said second-level data reduction forming a subset of said feature vectors comparable to said at least one prototype vector, and further comparable to said at least one query vector.

A method of content-based image retrieval (CBIR) for manufacturing, comprising the steps of:

• extracting a plurality of descriptive features from each of a plurality of digital images which represent in-process or completed semiconductors, said images obtained from a plurality of different measurement tools at a plurality of different manufacturing steps;
• recording said plurality of descriptive features to form a historical image collection;
• indexing said historical image collection using an image clustering method to produce a hierarchical search tree, said clustering method mapping said plurality of imaging into a set of groups based on similar image content, said groups numbering less than a number of said plurality of images, said hierarchical search tree constituting a searchable library of said semiconductor images, wherein searching is based on said groups to achieve efficient retrieval times from said library;
• extracting said query image from an in-process or completed manufactured article to be characterized, said query image comprising a plurality of said descriptive features,
• retrieving at least one candidate image from said library based on an image similarity criterion to said descriptive features of said query image, and displaying said at least one image.