Enabling rhetorical analysis via the use of communicative discourse trees

A computer-implemented method for training a classification model to predict a complementarity of a pair of two sentences by analyzing communicative discourse trees, the method comprising:

• accessing a set of training data comprising a set of training pairs, wherein each training pair comprises a question communicative discourse tree that represents a question and an answer communicative discourse tree that represents an answer and an expected level of complementarity, wherein each communicative discourse tree comprises a discourse tree with rhetorical relationships between a plurality of fragments of text, wherein each fragment that has a verb is matched to a verb signature by:
• accessing a plurality of verb signatures, wherein each verb signature comprises the verb of a fragment and a sequence of thematic roles, wherein each thematic role describes a respective relationship between the verb and related words;
• determining, for each verb signature of the plurality of verb signatures, a plurality of thematic roles of the respective verb signature, wherein each of the thematic roles matches a role of a word in the fragment;
• selecting a particular verb signature from the plurality of verb signatures based on the particular verb signature comprising a highest number of matches of roles;
• and
• associating the particular verb signature with the fragment;
• training a classification model by iteratively:
• providing one of the training pairs to the classification model,
• receiving, from the classification model, a determined level of complementarity;
• calculating a loss function by calculating a difference between the determined level of complementarity and the respective expected level of complementarity;
• and
• adjusting internal parameters of the classification model to minimize the loss function;
• using the trained classification model to identify a correspondence between a first sentence and a second sentence;
• and
• providing either the first sentence or the second sentence to an external device.

A system comprising:

• a computer-readable medium storing non-transitory computer-executable program instructions;
• and
• a processing device communicatively coupled to the computer-readable medium for executing the non-transitory computer-executable program instructions, wherein executing the non-transitory computer-executable program instructions configures the processing device to perform operations comprising:
• determining, for a question sentence, a question communicative discourse tree comprising a question root node, wherein a communicative discourse tree is a discourse tree that includes communicative actions;
• determining, for an answer sentence, an answer communicative discourse tree, wherein the answer communicative discourse tree comprises an answer root node, wherein each communicative discourse tree comprises a discourse tree with rhetorical relationships between a plurality of fragments of text, wherein each fragment that has a verb is matched to a verb signature by:
• accessing a plurality of verb signatures, wherein each verb signature comprises the verb of a fragment and a sequence of thematic roles, wherein each thematic role describes a respective relationship between the verb and related words;
• determining, for each verb signature of the plurality of verb signatures, a plurality of thematic roles of the respective verb signature, wherein each of the thematic roles matches a role of a word in the fragment;
• selecting a particular verb signature from the plurality of verb signatures based on the particular verb signature comprising a highest number of matches of roles;
• and
• associating the particular verb signature with the fragment;
• merging the communicative discourse trees by identifying that the question root node and the answer root node are identical;
• computing a level of complementarity between the question communicative discourse tree and the answer communicative discourse tree by applying a predictive model to the merged discourse tree;
• and
• responsive to determining that the level of complementarity is above a threshold, identifying the question sentence and the answer sentence as complementary.