# US 2018/0322341 A1 · Method and apparatus for extracting information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322341 A1
- Document ID: US20180322341A1
- Status: Published application
- Kind: A1
- Application number: 15/564,187
- Filed: 2016-06-17
- Published: 2018-11-08
- Earliest claimed: 2015-12-30
- Assignees: Baidu Online Network Technology (Beijing) Co., Ltd.
- Applicants: Baidu Online Network Technology (Beijing) Co., Ltd.
- Inventors: Shouke QIN; You HAN; Zhiyang CHEN; Feichao MA; Peizhi XU
- Web page: https://patentgrep.com/patent/US20180322341A1

## CPC

- G06F 40/143: Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
- G06F 16/353: into predefined classes
- G06F 16/986: {Document structures and storage, e.g. HTML extensions}
- G06F 40/117: Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154)
- G06F 40/154: Tree transformation for tree-structured or markup documents, e.g. XSLT, XSL-FO or stylesheets
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present application discloses a method and apparatus for extracting information. A specific implementation of the method comprises: parsing a pre-acquired web page file into a structure of a tag tree, and recognizing, in nodes of the tag tree, at least one body node at which a web page body in the web page file is located; performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks, and setting a tag attribute for each paragraph block according to an attribute of a tag associated with the each paragraph block; classifying a text content contained in the each paragraph block based on the tag attribute of the each paragraph block; and extracting information comprising a question and an answer from the text content contained in the each paragraph block based on a classification result. This implementation implements the automatic and precise extraction of information.

## Claims

### Claim 1

A method for extracting information, comprising:
parsing a pre-acquired web page file into a structure of a tag tree, and recognizing, in nodes of the tag tree, at least one body node at which a web page body in the web page file is located;
performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks, and setting a tag attribute for each paragraph block according to an attribute of a tag associated with the each paragraph block;
classifying a text content contained in the each paragraph block based on the tag attribute of the each paragraph block; and
extracting information comprising a question and an answer from the text content contained in the each paragraph block based on a classification result.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks comprises:
recognizing, in the content contained in the at least one body node, a title of a web page content of the web page file; and
performing the paragraph division on a content other than the title in the content contained in the at least one body node to generate the paragraph blocks.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the parsing a pre-acquired web page file into a structure of a tag tree, and recognizing, in nodes of the tag tree, at least one body node at which a web page body in the web page file is located comprises:
standardizing the pre-acquired web page file, causing the web page file to conform to a Hyper Text Markup Language specification;
performing document object model tree parsing on the standardized web page file to generate the tag tree; and
accessing the nodes of the tag tree, and determining a body node according to contents contained in the nodes.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the recognizing, in the content contained in the at least one body node, a title of a web page content of the web page file comprises:
selecting at least one candidate title node from the at least one body node based on a location of the content contained in the at least one body node in the web page content; and
calculating an edit distance between a text inside each candidate title node and a text corresponding to a title tag of the web page file and an edit distance between the text inside the each candidate title node and a text corresponding to an anchor tag;
sorting the candidate title nodes according to tag information inside the candidate title nodes and the calculated edit distances; and
determining a body title node in the at least one candidate title node based on a sorting result, and determining a text inside the body title node as the title of the web page content.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks, and setting a tag attribute for each paragraph block according to an attribute of a tag associated with the each paragraph block comprises:
recognizing at least one block element in each body node;
performing the paragraph division on a text contained in the body node by using the subblock element and a new line tag if the block element contains a subblock element; or, performing the paragraph division on a text contained in the body node by using a new line tag, to obtain at least one paragraph block; and
setting a tag attribute for each paragraph block according to an attribute of an associated tag in the block element and the subblock element.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the classifying a text content contained in the each paragraph block based on the tag attribute of the each paragraph block comprises:
grouping paragraph blocks having an identical tag attribute into a given paragraph block set;
recognizing, based on a number of characters contained in each paragraph block in each paragraph block set and a line spacing between the paragraph blocks, a short title structure in each block set; and
classifying a paragraph block having a non-short title structure in the each paragraph block set based on a text contained therein.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the extracting information comprising a question and an answer from the text content contained in the each paragraph block based on a classification result comprises:
performing a logical block division on a web page body content corresponding to the web page file based on the recognized short title structure;
determining, by using a regular expression, whether each short title structure is a question, and if yes, setting the each short title as a candidate question;
extracting a candidate answer corresponding to the candidate question based on a location of the candidate question in the web page content and a logical block division result of the web page content; and
extracting information comprising a question and an answer based on at least one of a number of lines by which the candidate question and the candidate answer are spaced apart from each other, whether numerical symbols in the candidate answer are consecutive, a number of answer entries, and a number of answer entries determined as questions.

### Claim 8

An apparatus for extracting information, comprising:
at least one processor; and
a memory storing instructions, which when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
parsing a pre-acquired web page file into a structure of a tag tree, and recognizing, in nodes of the tag tree, at least one body node at which a web page body in the web page file is located;
performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks, and setting a tag attribute for each paragraph block according to an attribute of a tag associated with the each paragraph block;
classifying a text content contained in the each paragraph block based on the tag attribute of the each paragraph block; and
extracting information comprising a question and an answer from the text content contained in the each paragraph block based on a classification result.

### Claim 9 (depends on 8)

The apparatus according to claim 8, wherein the performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks comprises:
recognizing, in the content contained in the at least one body node, a title of a web page content of the web page file; and
performing the paragraph division on a content other than the title in the content contained in the at least one body node to generate the paragraph blocks.

### Claim 10 (depends on 8)

The apparatus according to claim 8, wherein the parsing a pre-acquired web page file into a structure of a tag tree, and recognizing, in nodes of the tag tree, at least one body node at which a web page body in the web page file is located comprises:
standardizing the pre-acquired web page file, causing the web page file to conform to a Hyper Text Markup Language specification;
performing document object model tree parsing on the standardized web page file to generate the tag tree; and
accessing the nodes of the tag tree, and determining a body node according to contents contained in the nodes.

### Claim 11 (depends on 9)

The apparatus according to claim 9, wherein the recognizing, in the content contained in the at least one body node, a title of a web page content of the web page file comprises:
selecting at least one candidate title node from the at least one body node based on a location of the content contained in the at least one body node in the web page content; and
calculating an edit distance between a text inside each candidate title node and a text corresponding to a title tag of the web page file and an edit distance between the text inside the each candidate title node and a text corresponding to an anchor tag;
sorting the candidate title nodes according to tag information inside the candidate title nodes and the calculated edit distances; and
determining a body title node in the at least one candidate title node based on a sorting result, and determining a text inside the body title node as the title of the web page content.

### Claim 12 (depends on 8)

The apparatus according to claim 8, wherein performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks, and setting a tag attribute for each paragraph block according to an attribute of a tag associated with the each paragraph block comprises:
recognizing at least one block element in each body node;
performing the paragraph division on a text contained in the body node by using the subblock element and a new line tag if the block element comprises a subblock element; or, performing the paragraph division on a text contained in the body node by using a new line tag, to obtain at least one paragraph block; and
setting a tag attribute for each paragraph block according to an attribute of an associated tag in the block element and the subblock element.

### Claim 13 (depends on 8)

The apparatus according to claim 8, wherein classifying a text content contained in the each paragraph block based on the tag attribute of the each paragraph block comprises:
grouping paragraph blocks having an identical tag attribute into a given paragraph block set;
recognizing, based on a number of characters contained in each paragraph block in each paragraph block set and a line spacing between the paragraph blocks, a short title structure in each block set; and
classifying a paragraph block having a non-short title structure in the each paragraph block set based on a text contained therein.

### Claim 14 (depends on 13)

The apparatus according to claim 13, wherein the extracting information comprising a question and an answer from the text content contained in the each paragraph block based on a classification result comprises:
performing a logical block division on a web page body content corresponding to the web page file based on the recognized short title structure;
determining, by using a regular expression, whether each short title structure is a question, and if yes, setting the each short title as a candidate question;
extracting a candidate answer corresponding to the candidate question based on a location of the candidate question in the web page content and a logical block division result of the web page content; and
extracting information comprising a question and an answer based on at least one of a number of lines by which the candidate question and the candidate answer are spaced apart from each other, whether numerical symbols in the candidate answer are consecutive, a number of answer entries, and a number of answer entries determined as questions.

### Claim 15

(canceled)

### Claim 16

A non-transitory computer storage medium, storing computer readable instructions executable by a processor, the computer readable instructions when executed by the processor, causing the processor to execute a method for extracting information, the method comprising:
parsing a pre-acquired web page file into a structure of a tag tree, and recognizing, in nodes of the tag tree, at least one body node at which a web page body in the web page file is located;
performing a paragraph division on a content contained in the at least one body node to generate paragraph blocks, and setting a tag attribute for each paragraph block according to an attribute of a tag associated with the each paragraph block;
classifying a text content contained in the each paragraph block based on the tag attribute of the each paragraph block; and
extracting information comprising a question and an answer from the text content contained in the each paragraph block based on a classification result.