# US 2012/0084636 A1 · Method and system for web information extraction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2012/0084636 A1 - Document ID: US20120084636A1 - Status: Published application - Kind: A1 - Application number: 12/896,942 - Filed: 2010-10-04 - Published: 2012-04-05 - Earliest claimed: 2010-10-04 - Assignees: Yahoo! Inc. - Inventors: Srinivasan Hanumantha Rao SENGAMEDU; Charu Tiwari; Amit Madaan; Rupesh Rasiklal Mehta; S. R. Jeyashankher; Rajeev Rastogi - Web page: https://patentgrep.com/patent/US20120084636A1 ## CPC - G06F 16/81: Indexing, e.g. XML tags; Data structures therefor; Storage structures - G06F 16/951: Indexing; Web crawling techniques - G06F 40/143: Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD] - G06F 40/16: Automatic learning of transformation rules, e.g. from examples - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An example of a method includes determining features of a first type for a web page of a plurality of web pages. The method also includes electronically determining a plurality of rules for an attribute of the first web page, wherein the plurality of rules are determined based on features of the first type. The method also includes electronically identifying a first rule, from the plurality of rules, which satisfies a first predefined criterion. The first predefined criteria include at least one of a first threshold for a precision parameter, a second threshold for a support parameter, a third threshold for a distance parameter and a fourth threshold for a recall parameter. The method further includes storing the first rule to enable extraction of value of the attribute from a second web page. ## Claims ### Claim 1 A method comprising: determining features of a first type for a first web page of a plurality of web pages; electronically determining a plurality of rules for an attribute of the first web page, wherein the plurality of rules are determined based on features of the first type; electronically identifying a first rule, from the plurality of rules, which satisfies a first predefined criteria, the first predefined criteria comprising at least one of a first threshold for a precision parameter, a second threshold for a support parameter, a third threshold for a distance parameter, and a fourth threshold for a recall parameter; and storing the first rule to enable extraction of value of the attribute from a second web page. ### Claim 2 (depends on 1) The method as claimed in claim 1 and further comprising: receiving the second web page; determining the first rule as a match for the second web page; extracting value of the attribute based on the first rule; and storing the value of the attribute to render the value of the attribute when desired. ### Claim 3 (depends on 1) The method as claimed in claim 1 and further comprising: if none of the plurality of rules satisfy the first predefined criteria then electronically identifying a second rule, from the plurality of rules, which satisfies a second predefined criteria, the second predefined criteria comprising a fifth threshold for a precision parameter; determining features of a second type for the first web page of the plurality of web pages; determining an extraction criteria based on the features of the second type; and storing the second rule and the extraction criteria to enable extraction of the value of the attribute from the second web page. ### Claim 4 (depends on 3) The method as claimed in claim 3 and further comprising: receiving the second web page; determining the second rule and the extraction criteria as a match for the second web page; extracting value of the attribute based on the second rule and the extraction criteria; and storing the value of the attribute to render the value of the attribute when desired. ### Claim 5 (depends on 1) The method as claimed in claim 1 and further comprising: determining an offset criteria for the attribute; and storing the offset criteria to enable extraction. ### Claim 6 (depends on 1) The method as claimed in claim 1 and further comprising: determining a regular expression criteria for the attribute; and storing the regular expression criteria. ### Claim 7 (depends on 1) The method as claimed in claim 1, wherein electronically determining the plurality of rules comprises: creating rules based on disjunction of the features of the first type. ### Claim 8 (depends on 1) The method as claimed in claim 1, wherein each rule of the plurality of rules comprises an XPath expression if value of the attribute is comprised in one node; and a plurality of XPath expressions if the value of the attribute is comprised in multiple nodes. ### Claim 9 (depends on 1) The method as claimed in claim 1, wherein electronically identifying the first rule comprises: applying the plurality of rules on the plurality of web pages; and determining values for at least one of the precision parameter, the support parameter, the distance parameter, and the recall parameter for each rule of the plurality of rules; and comparing values for at least one of the precision parameter, the support parameter, the distance parameter, and the recall parameter for each rule of the plurality of rules with respective thresholds. ### Claim 10 An article of manufacture comprising: a machine readable medium; and instructions carried by the machine readable medium and operable to cause a programmable processor to perform: determining features of a first type for a first web page of a plurality of web pages; determining a plurality of rules for an attribute of the first web page, wherein the plurality of rules are determined based on features of the first type; identifying a first rule, from the plurality of rules, which satisfies a first predefined criteria, the first predefined criteria comprising at least one of a first threshold for a precision parameter, a second threshold for a support parameter, a third threshold for a distance parameter, and a fourth threshold for a recall parameter; and storing the first rule to enable extraction of value of the attribute from a second web page. ### Claim 11 (depends on 10) The article of manufacture as claimed in claim 10 and further comprising instructions operable to cause the programmable processor to perform: receiving the second web page; determining the first rule as a match for the second web page; extracting value of the attribute based on the first rule; and storing the value of the attribute to render the value of the attribute when desired. ### Claim 12 (depends on 10) The article of manufacture as claimed in claim 10 and further comprising: if none of the plurality of rules satisfy the first predefined criteria then identifying a second rule, from the plurality of rules, which satisfies a second predefined criteria, the second predefined criteria comprising a fifth threshold for a precision parameter; determining features of a second type for the first web page of the plurality of web pages; determining an extraction criteria based on the features of the second type; and storing the second rule and the extraction criteria to enable extraction of the value of the attribute from the second web page. ### Claim 13 (depends on 12) The article of manufacture as claimed in claim 12 and further comprising instructions operable to cause the programmable processor to perform: receiving the second web page; determining the second rule and the extraction criteria as a match for the second web page; extracting value of the attribute based on the second rule and the extraction criteria; and storing the value of the attribute to render the value of the attribute when desired. ### Claim 14 (depends on 10) The article of manufacture as claimed in claim 10 and further comprising instructions operable to cause the programmable processor to perform: determining an offset criteria for the attribute; and storing the offset criteria to enable extraction. ### Claim 15 (depends on 10) The article of manufacture as claimed in claim 10 and further comprising instructions operable to cause the programmable processor to perform: determining a regular expression criteria for the attribute; and storing the regular expression criteria. ### Claim 16 (depends on 10) The article of manufacture as claimed in claim 10, wherein determining the plurality of rules comprises: creating rules based on disjunction of the features of the first type. ### Claim 17 (depends on 10) The article of manufacture as claimed in claim 10, wherein each rule of the plurality of rules comprises an XPath expression if value of the attribute is comprised in one node; and a plurality of XPath expressions if the value of the attribute is comprised in multiple nodes. ### Claim 18 (depends on 10) The article of manufacture as claimed in claim 10, wherein identifying the first rule comprises: applying the plurality of rules on the plurality of web pages; and determining values for at least one of the precision parameter, the support parameter, the distance parameter, and the recall parameter for each rule of the plurality of rules; and comparing values for at least one of the precision parameter, the support parameter, the distance parameter, and the recall parameter for each rule of the plurality of rules with respective thresholds. ### Claim 19 A system comprising: a communication interface in electronic communication with one or more web servers comprising multiple web pages; a memory that stores instructions; and a processor responsive to the instructions to determine features of a first type for a first web page of a plurality of web pages; determine a plurality of rules for an attribute of the first web page, wherein the plurality of rules are determined based on features of the first type; identify a first rule, from the plurality of rules, which satisfies a first predefined criteria, the first predefined criteria comprising at least one of a first threshold for a precision parameter, a second threshold for a support parameter, a third threshold for a distance parameter, and a fourth threshold for a recall parameter; and store the first rule to enable extraction of value of the attribute from a second web page. ### Claim 20 (depends on 19) The system as claimed in claim 19, wherein the processor is further responsive to the instructions to: receive the second web page; determine the first rule as a match for the second web page; extract value of the attribute based on the first rule; and store the value of the attribute to render the value of the attribute when desired. ### Claim 21 (depends on 19) The system as claimed in claim 19, wherein the processor is further responsive to the instructions, if none of the plurality of rules satisfy the first predefined criteria, to: identify a second rule, from the plurality of rules, which satisfies a second predefined criteria, the second predefined criteria comprising a fifth threshold for a precision parameter; determine features of a second type for the first web page of the plurality of web pages; determine an extraction criteria based on the features of the second type; and store the second rule and the extraction criteria to enable extraction of the value of the attribute from the second web page. ### Claim 22 (depends on 21) The system as claimed in claim 21, wherein the processor is further responsive to the instructions to: receive the second web page; determine the second rule and the extraction criteria as a match for the second web page; extract value of the attribute based on the second rule and the extraction criteria; and store the value of the attribute to render the value of the attribute when desired.