Grammaticality classification for natural language generation in assistant systems

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• receive, from a client system associated with a user, a user input;
• generate, by a natural-language generation module, a plurality of candidate responses in response to the user input;
• determine, by a filtering module, for each candidate response of the plurality of candidate responses, a quality-indication for the candidate response based on one or more classification models and one or more classification rules, wherein the one or more classification models and the one or more classification rules each correspond to a respective acceptance-criteria, and wherein the one or more classification rules are based on one or more slots and one or more intents associated with the user input;
• select one or more candidate responses from the plurality of candidate responses based on a plurality of quality-indications associated with the plurality of candidate responses, respectively;
• rank the one or more selected candidate responses based on one or more ranking-criteria;
• and
• send, to the client system associated with the user responsive to receiving the user input, instructions for presenting a top-ranked candidate response to the user.

A system comprising:

• one or more processors;
• and a non-transitory memory coupled to the one or more processors comprising instructions executable by the one or more processors, the one or more processors operable when executing the instructions to:
• receive, from a client system associated with a user, a user input;
• generate, by a natural-language generation module, a plurality of candidate responses in response to the user input;
• determine, by a filtering module, for each candidate response of the plurality of candidate responses, a quality-indication for the candidate response based on one or more classification models and one or more classification rules, wherein the one or more classification models and the one or more classification rules each correspond to a respective acceptance-criteria, and wherein the one or more classification rules are based on one or more slots and one or more intents associated with the user input;
• select one or more candidate responses from the plurality of candidate responses based on a plurality of quality-indications associated with the plurality of candidate responses, respectively;
• rank the one or more selected candidate responses based on one or more ranking-criteria;
• and
• send, to the client system associated with the user responsive to receiving the user input, instructions for presenting a top-ranked candidate response to the user.