Apparatus and method for generating digest according to hierarchical structure of topic

A digest generator apparatus comprising:

• a structure detecting unit detecting a hierarchical structure of topic passages in a document, in which each of the topic passages corresponds to a part of the document describing a common topic, the hierarchical structure includes a plurality of levels, the topic passages in each of the levels compose the document, and each topic passage in a higher level includes one or more shorter topic passages in a lower level, wherein said structure detecting unit calculates a lexical cohesion degree in a vicinity area of each position in said document, detects a set of the topic boundaries that separate said document into the topic passages of almost the same size based on the cohesion degree, and by repeating detection of the set of the topic boundaries while reducing the size of the vicinity area, detects the hierarchical structure of the topic passages, wherein the topic passages range from a size of about a fraction of the document to about one paragraph;
• a keyword extracting unit extracting a plurality of keywords using the hierarchical structure;
• a sentence selecting unit selecting a plurality of key sentences from one of the topic passages in the hierarchical structure depending on a use condition of the keywords;
• and
• an outputting unit outputting the key sentences as a digest.

A digest generator apparatus, comprising:

• a keyword extracting unit evaluating whether or not a word is characteristic of a process target topic passage in a document by calculating a likelihood ratio based on a comparison of a use frequency of the word in the process target topic passage with a use frequency of the word in a longer topic passage including the process target topic passage and comparing the likelihood ratio to a predetermined threshold value, and extracting a keyword from the process target topic passage when the likelihood ratio is greater than the predetermined threshold value;
• a generating unit generating a digest according to a use condition of said keyword;
• and
• an outputting unit outputting said digest.

A digest generator apparatus, comprising:

• a unit repeatedly calculating a lexical cohesion degree in a vicinity of each position in a document to define topic passages in the document while varying a size of the vicinity;
• a major part specifying unit specifying a major part of the document to be summarized by removing one or more topic passages having a lower cohesion degree from the document;
• a generating unit generating a digest using said major part;
• and
• an outputting unit outputting said digest.

A computer-readable storage medium for storing a program which enables a computer to perform:

• detecting a set of topic boundaries of a document based on a lexical cohesion degree to detect a hierarchical structure of topic passages in the document, in which each of the topic passages corresponds to a part of the document describing a common topic, the hierarchical structure includes a plurality of levels, the topic passages in each of the levels compose the document, and each of the topic passages in a higher level includes one or more shorter topic passages in a lower level;
• extracting a plurality of keywords using the hierarchical structure;
• selecting a plurality of key sentences from one of the topic passages in the hierarchical structure depending on a use condition of the keywords;
• and
• generating a digest which includes of the key sentences.

A computer-readable storage medium for storing a program which enables a computer to perform:

• calculating a likelihood ratio based on a comparison of a use frequency of a word in a process target topic passage in a document with a use frequency of said word in a longer topic passage including said process target topic passage;
• evaluating whether or not said word is characteristic of said process target topic passage by comparing the likelihood ratio to a predetermined threshold value;
• extracting a keyword from the process target topic passage when the likelihood ratio is greater than the predetermined threshold value ;
• and
• generating a digest according to a use condition of said keyword.

A computer-readable storage medium for storing a program which enables a computer to perform:

• specifying a major part of the document to be summarized by removing
• repeatedly calculating a lexical cohesion degree in a vicinity of each position in a document to define topic passages in the document while varying a size of the vicinity;
• specifying a major part of the document to be summarized by removing one or more topic passages having a lower cohesion from the document;
• and
• generating a digest using said major part.

A method of generating a digest, comprising:

• detecting a set of topic boundaries of a document based on a lexical cohesion degree to detect a hierarchical structure of topic passages in the document, in which each of the topic passages corresponds to a part of the document describing a common topic, the hierarchical structure includes a plurality of levels, the topic passages in each of the levels compose the document, and each of the topic passages in a higher level includes one or more shorter topic passages in a lower level;
• extracting a plurality of keywords using the hierarchical structure;
• selecting a plurality of key sentences from one of the topic passages in the hierarchical structure depending on a use condition of the keywords;
• and
• generating a digest which includes the key sentences.