Method and system to identify records that relate to a pre-defined context in a data set

A system for identifying one or more records from a data set based on pre-defined context, the data set being a web page captured from the Internet, the web page being represented as a hierarchical tree structure representing one or more nodes, wherein the pre-defined context defines an objective based on which the one or more records are identified, the pre-defined context being associated with its corresponding domain model, the system comprising:

• a processor;
• a. a node identifier, the node identifier identifying one or more nodes of interest from the one or more nodes of the hierarchical tree structure, the one or more nodes of interest being identified based on the pre-defined context;
• b. a sub-tree extractor, the sub-tree extractor extracting one or more sub-trees based on the identified one or more nodes of interest, wherein each of the one or more extracted sub-trees being a hierarchical path showing the relationship of the one or more nodes of interest with its ancestor nodes in the hierarchical tree structure;
• c. a record recognizer, the record recognizer selecting one or more records corresponding to each of the one or more extracted sub-trees based on the pre-defined context, wherein each record comprises at least one node of interest and the corresponding parent node, the parent node of the at least one node of interest being referred to as a record-child, wherein selecting the one or more records comprises:
• i. counting the number of times each of the nodes of interest appears in the one or more extracted sub-trees by scanning an upward path from the node of interest to the root node corresponding to each of the one or more extracted sub-trees and incrementing a count of the each of the nodes of the sub-tree based on the number of times each of the nodes of interest appear in the one or more extracted sub-trees;
• ii. identifying a record-parent by scanning an upward path from a node of interest towards the root node of the hierarchical tree structure, corresponding to the each of the one or more extracted sub-trees, the parent node of the record child being referred to as the record-parent, the record-parent being the lowest node in each of the corresponding one or more extracted sub-trees, wherein the identified record-parent has a count greater than one;
• iii. identifying record-children for each identified record-parent, the record- children being defined as a child node of the record-parent in the each of the one or more extracted sub-trees;
• iv. selecting the identified record-children and all descendent nodes of the corresponding identified record-children, wherein each of the identified record children along with the descendant nodes is referred to as a record;
• and
• d. a data store, the data store storing the selected one or more records in a database, wherein a search engine is built on the database to facilitate a user to search the database over the Internet based on a user query.

A computer program product for use with a computer, the computer program product comprising a computer usable storage medium having a computer program code embodied therein for identifying one or more records from a data set based on a pre-defined context, the data set being a web page captured from the Internet, the web page being represented as a hierarchical tree structure representing one or more nodes, wherein the pre-defined context defines an objective based on which the one or more records are identified, the pre-defined context being associated with its corresponding domain model, the computer program code performing the steps of:

• a. automatically selecting the pre-defined context for the web page;
• b. identifying one or more nodes of interest from the one or more nodes of the hierarchical tree structure, the one or more nodes of interest being identified based on the pre-defined context;
• c. extracting one or more sub-trees based on the identified one or more nodes of interest, wherein each of the one or more extracted sub-trees being a hierarchical path showing the logical relationship of the one or more nodes of interest with its ancestor nodes in the hierarchical tree structure;
• d. selecting one or more records corresponding to each of the one or more extracted sub-trees based on the pre-defined context, wherein each record comprises at least one node of interest and the corresponding parent node, the parent node of the at least one node of interest being referred to as a record-child, wherein selecting the one or more records comprises:
• i. counting the number of times each of the nodes of interest appears in the one or more extracted sub-trees by scanning an upward path from the node of interest to the root node corresponding to each of the one or more extracted sub-trees and incrementing a count of the each of the nodes of the sub-tree based on the number of times each of the nodes of interest appear in the one or more extracted sub-trees;
• ii. identifying a record-parent by scanning an upward path from a node of interest towards the root node of the hierarchical tree structure, corresponding to the each of the one or more extracted sub-trees, the parent node of the record child being referred to as the record-parent, the record-parent being the lowest node in each of the corresponding one or more extracted sub-trees, wherein the identified record-parent has a count greater than one;
• iii. identifying record-children for each identified record-parent, the record-children being defined as a child node of the record-parent in the each of the one or more extracted sub-trees;
• iv. selecting the identified record-children and all descendent nodes of the corresponding identified record-children, wherein each of the identified record children along with the descendant nodes is referred to as a record;
• and
• e. storing each of the selected one or more records in a database, wherein a search engine is built on the database to facilitate a user to search the database over the Internet based on a user query.