System and method for transforming unstructured text into structured form
AssigneeAccenture Global Solutions Limited
InventorsJayati Deshmukh, Annervaz K. M., Shubhashis Sengupta
Life of this patent
Estimating the term…
Abstract
A system and method for transforming unstructured text into structured form is disclosed. The system and method include converting an input word sequence (e.g., sentence) into tagged output which can be then easily be converted into a structured format. The system may include a bidirectional recurrent neural network that can generate multiple labels of individual words or phrases. In some embodiments, a customized learning loss equation involving set similarity is used to generate the multiple labels.
Claims20
A method of transforming unstructured text into structured form, comprising:
- obtaining a word sequence, including at least a first word and a second word;
- obtaining a first word embedding and a first part-of-speech (“POS”) tag embedding both corresponding to the first word;
- obtaining a second word embedding and a second POS tag embedding both corresponding to the second word;
- concatenating the first word embedding with the first POS word embedding into a first input and the second word embedding with the second POS word embedding into a second input;
- using self-attention to process the first input and the second input through a bidirectional recurrent neural network (“RNN”) to generate a first output corresponding to the first input and a second output corresponding to the second input, wherein the first output includes at least two labels corresponding to the first word;
- using a loss to perform back-propagation to adjust weights of the bidirectional RNN, wherein the loss is based on both the vector of true labels and the vector of independent probabilities of predicted labels;
- and
- wherein the loss is HLdiff, and wherein
- HLdiff=average(yt*(1−yp)+(1−yt)*yp),
- where yt is the vector of true labels and yp is the vector of independent probabilities of predicted labels.
A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to transform unstructured text into structured form by:
- obtaining a word sequence, including at least a first word and a second word;
- obtaining a first word embedding and a first part-of-speech (“POS”) tag embedding both corresponding to the first word;
- obtaining a second word embedding and a second POS tag embedding both corresponding to the second word;
- concatenating the first word embedding with the first POS word embedding into a first input and the second word embedding with the second POS word embedding into a second input;
- using self-attention to process the first input and the second input through a bidirectional recurrent neural network (“RNN”) to generate a first output corresponding to the first input and a second output corresponding to the second input, wherein the first output includes at least two labels corresponding to the first word;
- using a loss to perform back-propagation to adjust weights of the bidirectional RNN, wherein the loss is based on both the vector of true labels and the vector of independent probabilities of predicted labels;
- and
- wherein the loss is HLdiff, and wherein
- HLdiff=average(yt*(1−yp)+(1−yt)*yp),
- where yt is the vector of true labels and yp is the vector of independent probabilities of predicted labels.
A system for transforming unstructured text into structured form, comprising:
- one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to:
- obtain a word sequence, including at least a first word and a second word;
- obtain a first word embedding and a first part-of-speech (“POS”) tag embedding both corresponding to the first word;
- obtain a second word embedding and a second POS tag embedding both corresponding to the second word;
- concatenate the first word embedding with the first POS word embedding into a first input and the second word embedding with the second POS word embedding into a second input;
- use self-attention to process the first input and the second input through a bidirectional recurrent neural network (“RNN”) to generate a first output corresponding to the first input and a second output corresponding to the second input, wherein the first output includes at least two labels corresponding to the first word;
- using a loss to perform back-propagation to adjust weights of the bidirectional RNN, wherein the loss is based on both the vector of true labels and the vector of independent probabilities of predicted labels;
- and
- wherein the loss is HLdiff, and wherein
- HLdiff=average(yt*(1−yp)+(1−yt)*yp),
- where yt is the vector of true labels and yp is the vector of independent probabilities of predicted labels.
Dependencies are inferred from the claim text.