Method and system for information extraction

A system comprising a processor for extracting information related to a predefined context from a group of data sets, the predefined context having a set of attributes that define the context, the system comprising:

• a) a content filter for filtering the group of data sets by removing irrelevant information to obtain one or more filtered data sets, the irrelevant information being information that is not related to the set of attributes corresponding to the pre-defined context;
• b) a data set classifier for identifying a relevant data set from a filtered data set of the one or more filtered data sets the identification of the relevant data set being based on the occurrence of the set of attributes corresponding to the pre-defined context the data set classifier comprising:
• a feature identifier for identifying supporting and non-supporting features in the filtered data set, the supporting features being structural or textual information that contain information corresponding to the pre-defined context, the non-supporting features comprising information having structural or textual features not corresponding to the pre-defined context;
• a weight assigner for assigning positive and negative weights to the identified supporting and non-supporting features respectively in the filtered data set, a positive weight being a positive numerical value based on relevance of a supporting feature to the pre-defined context and a negative weight being a negative numerical value based on digression of a non-supporting feature to the pre-defined context;
• a confidence value calculator for calculating the confidence value of the filtered data set, the confidence value being a numerical value used as a measure to identify the relevance of the filtered data set to the pre-defined context;
• and
• a relevance indicator for selecting the relevant data set for further processing by comparing the confidence value of the filtered data set with a threshold confidence value, the selection being done by selecting the data set that has the confidence value greater than a pre-defined threshold confidence value;
• c) an information identifier for identifying pertinent information in the relevant data set, the pertinent information being the information that contains values of the set of attributes corresponding to the pre-defined context;
• and
• d) an entity extractor for extracting the values of the set of attributes from the pertinent information.

A computer program product for use with a computer, the computer program product comprising a computer storage medium having a computer readable program code embodied therein for extracting information related to a pre-defined context from a group of data sets, the pre-defined context comprising a set of attributes that define the context, the data set comprising structural and textual information, the computer program code performing the steps of:

• a) filtering the group of data sets to obtain one or more filtered data sets by removing irrelevant information, the irrelevant information being information that is not related to the set of attributes corresponding to the pre-defined context;
• b) identifying a relevant data set from a filtered data set of the one or more filtered data sets, the identification of the relevant data set being based on the occurrence of the set of attributes corresponding to the pre-defined context comprising:
• identifying supporting and non-supporting features in the filtered data set, the supporting features being structural or textual features that contain information corresponding to the pre-defined context, the non-supporting features being structural or textual features that do not contain information corresponding to the pre-defined context;
• assigning a positive weight to each supporting feature in the filtered data set, the positive weight being a positive numerical value based on the degree of relevance of the supporting feature to the pre-defined context;
• assigning a negative weight to each non-supporting feature in the filtered data set, the negative weight being a negative numerical value based on the degree of digression of the non-supporting feature from the pre-defined context;
• calculating a confidence value for the filtered data set, the confidence value is a function of the positive and negative weights of the filtered data set, the confidence value being used as a measure to determine the relevance of the filtered data set to the pre-defined context;
• and
• selecting the relevant data sets for further processing by comparing the confidence value of the filtered data set with a threshold confidence value, the selection being done by selecting the data set that has the confidence value greater than a pre-defined threshold confidence value;
• c) identifying pertinent information from the relevant data set, the pertinent information being the information that contains values of the set of attributes corresponding to the pre-defined context;
• d) extracting the values of the set of attributes from the pertinent information;
• and
• e) arranging the extracted values in the form of a pre-defined data structure which logically links the set of attributes to each other, in accordance with their inter-relationships as per the pre-defined context.