Method and apparatus for extracting topic sentences of webpages

An apparatus for extracting topic sentences of webpages, comprising one or more processors and memory, wherein the one or more processors are configured to:

• obtain candidate webpage, and pre-build a machine learning model, wherein each candidate webpage contains a plurality of preselected candidate topic sentences, and each candidate topic sentence includes several word segments;
• determine word feature values that indicates importance levels of the word segments in each candidate webpage, and input the word feature values to the machine learning model to obtain an importance value for each word segment;
• determine, for each candidate webpage, a partial order value for each candidate topic sentence according to the importance values of the word segments of the candidate topic sentence;
• and
• select, for each candidate webpage, one of the plurality of candidate topic sentences that is associated with a partial order value larger than a preset threshold value as a target topic sentence of the candidate webpage, wherein:
• the pre-built machine learning model is obtained by the one or more processors configured to:
• obtain page titles of a plurality of webpage samples, wherein each webpage sample corresponds to one or more query terms for retrieving the webpage sample, each query term is associated with a weighted value for indicating a correlation degree between the query term and the retrieved webpage sample;
• calculate a label value for each word segment included in each page title;
• determine a word feature value for each word segment included in each page title, wherein the word feature value indicates an importance degree of the word segment in the corresponding webpage sample;
• and
• use a preset machine training algorithm to train the label values and the word feature values of the word segments in each page title to obtain the machine learning model.

A non-transitory computer readable memory comprising a computer readable program stored thereon, wherein, when being executed, the computer readable program causes a computer to implement a method for extracting topic sentences of webpages, the method comprising:

• obtaining candidate webpage, and a pre-built machine learning model, wherein each candidate webpage contains a plurality of preselected candidate topic sentences, and each candidate topic sentence includes several word segments;
• determining word feature values that indicate importance levels of the word segments in each candidate webpage, wherein the word feature value includes a term frequency, an inverse document frequency, a continuous hit rate, and a proximity of the word segment;
• inputting the word feature values to the machine learning model to obtain an importance value for each word segment;
• for each candidate webpage, determining a partial order value for each candidate topic sentence according to the importance values of the word segments included in the candidate topic sentence;
• and
• for each candidate webpage, selecting one of the plurality of candidate topic sentences that is associated with a partial order value larger than a preset threshold value as a target topic sentence of the candidate webpage, wherein:
• the pre-built machine learning model is obtained by:
• obtaining page titles of a plurality of webpage samples, wherein each webpage sample corresponds to one or more query terms for retrieving the webpage sample, each query term is associated with a weighted value for indicating a correlation degree between the query term and the retrieved webpage sample;
• calculating a label value for each word segment included in each page title;
• determining a word feature value for each word segment included in each page title, wherein the word feature value indicates an importance degree of the word segment in the corresponding webpage sample;
• and
• using a preset machine training algorithm to train the label values and the word feature values of the word segments in each page title to obtain the machine learning model.