Detecting out-of-domain, out-of-scope, and confusion-span (OOCS) input for a natural language to logical form model

A system comprising:

• one or more data processors;
• and
• one or more non-transitory computer readable media storing instructions which, when executed by the one or more data processors, cause the system to perform operations comprising:
• generating first training data, including:
• accessing a set of training samples comprising utterances that correspond to valid queries on a database,
• accessing a set of conversational prefixes and suffixes, the conversational prefixes and suffixes comprising sets of one or more words used in conversation, and
• for each training sample of the set of training samples, generating one or more augmented training samples by appending one or more respective conversational prefixes and/or suffixes of the set of conversational prefixes and suffixes to the training sample of the set of training samples;
• training a first machine learning model based on:
• the first training data, comprising utterances related to database queries that include conversational phrases labeled as translatable by a second machine learning model, the second machine learning model configured to generate a query for the database based on natural language utterances, and
• second training data, comprising out of domain, out of scope, and mis-specified example utterances and labeled as not translatable by the second machine learning model;
• generating an input string by concatenating a natural language utterance with a database schema representation for the database;
• providing the input string to the first machine learning model;
• based on the input string, generating, by the first machine learning model, a score indicating whether the natural language utterance is translatable to a database query for the database and should be routed to the second machine learning model;
• comparing the score to a threshold value;
• and
• responsive to determining that the score exceeds the threshold value, providing the natural language utterance or the input string to the second machine learning model.

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform operations comprising:

• generating first training data, including:
• accessing a set of training samples comprising utterances that correspond to valid queries on a database,
• accessing a set of conversational prefixes and suffixes, the conversational prefixes and suffixes comprising sets of one or more words used in conversation, and
• for each training sample of the set of training samples, generating one or more augmented training samples by appending one or more respective conversational prefixes and/or suffixes of the set of conversational prefixes and suffixes to the training sample of the set of training samples;
• training a first machine learning model based on:
• the first training data, comprising utterances related to database queries that include conversational phrases labeled as translatable by a second machine learning model, the second machine learning model configured to generate a query for the database based on natural language utterances, and
• second training data, comprising out of domain, out of scope, and mis-specified example utterances and labeled as not translatable by the second machine learning model;
• generating an input string by concatenating a natural language utterance with a database schema representation for the database;
• providing the input string to the first machine learning model;
• based on the input string, generating, by the first machine learning model, a score indicating whether the natural language utterance is translatable to a database query for the database and should be routed to the second machine learning model;
• comparing the score to a threshold value;
• and
• responsive to determining that the score exceeds the threshold value, providing the natural language utterance or the input string to the second machine learning model.