Method and system for token based classification for reducing overlap in field extraction during parsing of a text
AssigneeInnoplexus AG
InventorsSudhanshu Kumar, Shubham Patel
Life of this patent
Estimating the term…
Abstract
A system and a method for token-based classification for reducing overlap in field extraction during parsing of a text is disclosed. The method includes extracting text from a resource. The method further includes splitting one or more sentences into a predetermined number of plurality of tokens. The method furthermore includes generating a plurality of lists using a machine learning model for identifying one or more fields in the text. The plurality of lists comprises at least a list of tokens, a list of tags and a list of confidence scores of tokens. The method furthermore includes post processing the plurality of lists for extracting one or more fields for parsing the text.
Claims8
A processor-implemented method of token-based classification for reducing overlap in field extraction during parsing of a text, the method comprising:
- extracting the text from a resource;
- splitting one or more sentences in the text into a predetermined number of plurality of tokens;
- generating a plurality of lists using a machine learning model, for identifying one or more fields in the text, wherein the plurality of lists comprises at least a list of tokens, a list of tags and a list of confidence score of tokens;
- and
- post-processing the plurality of lists for extracting one or more fields for parsing the text.
A processor-implemented method of training a machine learning model for token-based classification for reducing overlap in field extraction during parsing of text, the method comprising:
- extracting the text from a resource;
- generating a training set for the artificial intelligence model based on the extracted text and importing the training set into the artificial intelligence model;
- and
- training and evaluating the artificial intelligence model using the training set for generating a plurality of lists for identifying one or more fields in the text, wherein the plurality of lists comprises at least a list of tokens, a list of tags and a list of confidence score of tokens.
A system for token-based classification for reducing overlap in field extraction during parsing of a text, the system comprising a processor configured to execute non-transitory machine-readable instructions that when executed perform:
- extracting the text from a resource;
- splitting one or more sentences in the text into a predetermined number of plurality of tokens;
- generating a plurality of lists using a machine learning model, for identifying one or more fields in the text, wherein the plurality of lists comprises at least a list of tokens, a list of tags and a list of confidence score of tokens;
- and
- post processing the plurality of lists for extracting one or more fields for parsing the text.
Dependencies are inferred from the claim text.