Automatic method of identifying drop words in a document image without performing character recognition

A method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, each word occurrence of the second multiplicity of word occurrences having a font size, a processor implementing the method by executing instructions stored in electronic form in a memory coupled to the processor, the method comprising the steps of:

• a) identifying a dominant font size within the document image;
• b) identifying sentence boundaries for sentences of the dominant font size within the document image;
• c) analyzing the document image to identify word equivalence classes of the dominant font size, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences;
• d) for each word equivalence class determining the likelihood that word equivalence class is not a drop word based upon a width of each word equivalence class, a number of occurrences of the word equivalence class, and a number of occurrences of the word equivalence class as a first non-short word of a sentence;
• e) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are drop words.

An article of manufacture comprising:

• a) a memory;
• and
• b) instructions stored in the memory for a method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, method comprising the steps of:
• 1) analyzing the document image to identify word equivalence classes, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences;
• 2) for each word equivalence class determining the likelihood that word equivalence class is not a drop word;
• 3) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are drop words.

An article of manufacture comprising:

• a) a memory;
• and
• b) instructions stored in the memory for a method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, each word occurrence of the second multiplicity of word occurrences having a font size, the method comprising the steps of:
• 1) identifying a dominant font size within the document image;
• 2) identifying sentence boundaries for sentences of the dominant font size within the document image;
• 3) analyzing the document image to identify word equivalence classes of the dominant font size, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences;
• 4) for each word equivalence class determining the likelihood that word equivalence class is not a drop word based upon a width of each word equivalence class, a number of occurrences of the word equivalence, and a number of occurrences of the word equivalence class as a first non-short word of a sentence;
• 5) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are