Method and system for extracting characteristic string, method and system for searching for relevant document using the same, storage medium for storing characteristic string extraction program, and storage medium for storing relevant document searching program

A method for extracting words contained in document data specified by a user comprising the steps of:

• extracting a substring from said document data and looking up word boundary probability information at a head or tail of a previously-prepared partial character string to calculate division probabilities at at least two character positions;
• and
• comparing the division probabilities calculated in said extracting step at at least two or more character positions to determine a division point in the word in the specified text,
• wherein said comparing step is replaced by a step of comparing the division probabilities calculated in said extracting step at at least two or more character positions to determine a division probability in a word in the specified text.

A method for extracting words contained in document data specified by a user comprising the steps of:

• extracting a substring from said document data and looking up word boundary probability information at a head or tail of a previously-prepared partial character string to calculate division probabilities at at least two character positions;
• and
• comparing the division probabilities calculated in said extracting step at at least two or more character positions to determine a division point in the word in the specified text,
• wherein said extracting step is replaced by a step of extracting a substring from said specified text and looking up a probability that said substring is adjacent to a boundary of a predetermined character type at a head or tail of a previously-prepared substring to calculate a division probability of the characterposition.

A relevant document searching method comprising the steps of:

• extracting one or more words from a text, which will be referred to as specified text, of a sentence or document, which will be referred to collectively as a document, specified by a user in a text database storing character data as code data therein;
• counting occurrence frequencies of the words extracted in said word extracting step in the specified text;
• acquiring occurrence frequencies of the words extracted in said word extracting step in document texts which will be referred to as registration texts, stored in said text database;
• calculating similarities of the registration texts to the specified text in accordance with a predetermined calculation expression with use of the occurrence frequencies counted in said occurrence frequency counting step as well as the occurrence frequencies acquired in said occurrence frequency acquiring step;
• and
• outputting the similarities of the registration texts to the specified text calculated in said similarity calculating step as a searched result,
• wherein said word extracting step comprises the steps of:
• extracting a substring from said specified text and looking up word boundary probability information at a head or tail of a previously-prepared partial character string and to calculate division probabilities at at least two character positions, and
• comparing the division probabilities calculated in said extracting a substring step at at least two or more character positions to determine a division point in the word in the specified text.

A system for extracting a word from a sentence or document, which will be referred to collectively as a document, specified by a user, comprising:

• means for extracting a substring from a text, which will be referred to as specified text, of the document specified by the user and looking up word boundary possibility information at a head or tail of a previously-prepared substring with respect to the extracted substring to calculate division probabilities at at least two character positions,
• wherein said word boundary possibility information is used to determine a likely position a compound word should be divided;
• and
• means for comparing the division probabilities at at least two character positions calculated in said means for extracting a substring to determine a division point of a word in the specified text.

A system for extracting a word from a sentence or document, which will be referred to collectively as a document, specified by a user, comprising:

• means for extracting a substring from a text, which will be referred to as specified text, of the document specified by the user and looking up word boundary possibility information at a head or tail of a previously-prepared substring with respect to the extracted substring to calculate a division probability at a character position;
• and
• means for comparing the division probabilities at at least two character positions calculated in said means for extracting a substring to determine a division point of a word in the specified text,
• wherein said means for comparing is replaced by means for comparing the division probabilities calculated in said means for extracting a substring at at least two or more character positions to determine a division probability in a word in the specified text.

A system for extracting a word from a sentence or document, which will be referred to collectively as a document, specified by a user, comprising:

• means for extracting a substring from a text, which will be referred to as specified text, of the document specified by the user and looking up word boundary possibility information at a head or tail of a previously-prepared substring with respect to the extracted substring to calculate division probabilities at at least two character positions;
• and
• means for comparing the division probabilities at at least two character positions calculated in said means for extracting a substring to determine a division point of a word in the specified text,
• wherein said means for extracting a substring is replaced by means for extracting a substring from said specified text and looking up a probability that said substring is adjacent to a boundary of a predetermined character type at a head or tail of a previously-prepared substring to calculate a division probability of the character position.

A relevant document searching system comprising:

• means for extracting one or more words from a text, which will be referred to as a specified text, of a sentence or document, which will be referred to collectively as a document, specified by a user in a text database storing character data as code data therein;
• means for counting occurrence frequencies of the words extracted in said word extracting means in the specified text;
• means for acquiring occurrence frequencies of the words extracted in said word extracting means in document texts, which will be referred to as registration texts, stored in said text database;
• means for calculating similarities of the registration texts to the specified text in accordance with a predetermined calculation expression with use of occurrence frequencies counted in said occurrence frequency counting means as well as occurrence frequencies acquired in said occurrence frequency acquiring means;
• and
• means for outputting the similarities of the registration texts to the specified text calculated in said similarity calculating means as a searched result,
• wherein said word extracting means comprises:
• means for extracting a substring from said specified text and looking up word boundary probability information at a head or tail of a previously-prepared partial character string to calculate division probabilities at at least two character positions, and
• means for comparing the division probabilities calculated in said means for extracting a substring at at least two character positions to determine a division point in the word in the specified text.

A storage medium for storing a program, executable by a computer, for extraction of a word from a sentence or document, which will be referred to collectively as a document, specified by a user, said program when executed causes said computer to perform the steps of:

• extracting a substring from a text which will be referred to as specified text, of the document specified by the user and looking up word boundary possibility information at a head or tail of a previously-prepared substring with respect to the extracted substring to calculate division probabilities at at least two character positions,
• wherein said word boundary possibility information is used to determine a likely position a compound word should be divided;
• and
• comparing the division probabilities at at least two character positions calculated in said division probability calculating step to determine a division point of a word in the specified text.

A storage medium for storing a relevant document search program, said program when executed by a computer causes the computer to perform the steps of:

• extracting one or more words from a text, which will be referred to as specified text, of a sentence or document, which will be referred to collectively as a document, specified b y a user in a text database storing character data as code data therein;
• counting occurrence frequencies of the words extracted in said word extracting step in the specified text;
• acquiring occurrence frequencies of the words extracted in said word extracting step in document texts, which will be referred to as registration texts, stored in said text database;
• calculating similarities of the registration texts to the specified text in accordance with a predetermined calculation expression with use of occurrence frequencies counted in said occurrence frequency counting step as well as occurrence frequencies acquired in said occurrence frequency acquiring step;
• and
• outputting the similarities of the registration texts to the specified text calculated in said similarity calculating step as a searched result,
• wherein said word extracting step comprises the steps of:
• extracting a substring from said specified text and looking up word boundary probability information at a head or tail of a previously-prepared partial character string to calculate division probabilities at at least two character positions, and
• comparing the division probabilities calculated in said extracting a substring step at at least two character positions to determine a division point in the word in the specified text.

A method for extracting characteristic string from a document including a text, comprising the steps of:

• extracting a candidate string which is a candidate of a word starting or ending at an inter-word boundary from the text;
• calculating a division probability that the extracted candidate string is divided at a position in the candidate string and repeating the calculation with respect to a plurality of positions;
• comparing the division probabilities with one another and dividing the candidate string into substrings at the position having the high division probability determined by said comparing step;
• and
• extracting at least one of the substrings as the characteristic string.

A method for searching a text database storing documents for a document relevant to a user specified document, comprising the steps of:

• detecting a inter-word boundary from a text in the user specified document and extracting a candidate string which is a candidate of a word starting or ending at the inter-word boundary from the text;
• calculating a division probability that the extracted candidate string is divided at a position in the candidate string and repeating the calculating with respect to a plurality of positions;
• comparing the division probabilities with one another and dividing the candidate string into substrings at the position having the high division probability determined by the comparing step;
• extracting at least one of the substrings as the characteristic string;
• counting occurrence frequency of the extracted characteristic string in the text;
• obtaining occurrence frequency of the extracted characteristic string in each document in the text database;
• calculating a similarity between the text and each document using the characteristic string occurrence frequency in the text and the characteristic string occurrence frequency in each document;
• and
• outputting the similarity as a search result.