Systems and methods for end-to-end deep reinforcement learning based coreference resolution

A computer-implement method for coreference resolution using a coreference resolution model comprising:

• receiving a document comprising a set of words;
• generating, using a character and word embeddings encoder, a plurality of embeddings with each embedding as a concatenation of fixed word embeddings and convolutional neural network (CNN) character embeddings;
• computing and concatenating, using a bidirectional Long short-term memory (LSTM) layer, contextualized representation of each word in the document from two directions;
• performing iterative operations comprising:
• generating, with head-finding attention, span representation from the concatenated contextualized representations for a current mention;
• obtaining a mention score using a mention feed-forward neural network with a self-attention mechanism based on the generated span representation;
• obtaining an antecedent score using an antecedent feed-forward neural network with the self-attention mechanism based on the generated span representation;
• and
• obtaining a coreference score for the current mention based on at least the generated mention score and the generated antecedent score;
• computing, using a masked softmax layer, probability distribution over a set of actions in which the set of actions comprise linking the current identified mention with a prior identified mention or not linking the current identified mention to any prior identified mention for the current mention based at least on the coreference score;
• selecting an action from the set of actions using the probability distribution of actions;
• and
• based upon the selected action, updating a coreference graph for the document, in which the coreference graph comprises mentions as nodes and links representing coreference connections between mentions.

A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes the steps for training a coreference resolution model to be performed comprising:

• [a] inputting a document comprising a set of text into a policy network to identify mentions in the document;
• [b] given a current identified mention in the document, using the policy network to obtain a probability distribution of a set of actions in which the set of actions comprise linking the current identified mention with a prior identified mention or not linking the current identified mention to any prior identified mention;
• [c] selecting an action from the set of actions using the probability distribution of actions;
• [d] based upon the selected action, updating a coreference graph for the document, in which the coreference graph comprises mentions as nodes and links representing coreference connections between mentions;
• [e] responsive to the document having another mention, selecting it as the current identified mention and returning to step [b];
• [f] responsive to the document not having another mention, outputting the coreference graph for the document;
• [g] using the outputted coreference graph and a ground truth coreference information for the document, computing a reward based upon one or more metrics;
• [h] using a trajectory of actions selected and the reward to compute a gradient;
• and
• [i] updating the policy network using the gradient.