Methods and apparatus for natural language understanding in conversational systems using machine learning processes

A method comprising:

• receiving input data comprising a plurality of characters;
• training a Bidirectional Encoder Representation from Transformers (BERT) model based on catalog data identifying one or more attributes of a plurality of items and chat session data;
• applying the BERT model to the input data to generate first output embeddings;
• applying a dependency embedding generation model to the input data to generate second output embeddings, wherein the dependency embedding generation model comprises a syntactic dependency embedding model configured to generate syntactic dependencies based on the input data, a transformer encoder comprising a single layer transformer block configured to generate encoded data from the syntactic dependencies, and a linear network configured to generate the second output embeddings based on the encoded data, and wherein the dependency embedding generation model is applied in parallel with the BERT model;
• generating concatenated embeddings by stepwise concatenation of the first output embeddings and the second output embeddings;
• determining, based on applying a classifier model to the concatenated embeddings, output values;
• and
• storing the output values in a database.

A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:

• receiving input data comprising a plurality of characters;
• training a Bidirectional Encoder Representation from Transformers (BERT) model based on catalog data identifying one or more attributes of a plurality of items and chat session data;
• applying the BERT model to the input data to generate first output embeddings;
• applying a dependency embedding generation model to the input data to generate second output embeddings, wherein the dependency embedding generation model comprises a syntactic dependency embedding model configured to generate syntactic dependencies based on the input data, a transformer encoder comprising a single layer transformer block configured to generate encoded data from the syntactic dependencies, and a linear network configured to generate the second output embeddings based on the encoded data, and wherein the dependency embedding generation model is applied in parallel with the BERT model;
• generate concatenated embeddings by stepwise concatenation of the first output embeddings and the second output embeddings;
• determining, based on applying a classifier model to the concatenated embeddings, output values;
• and
• storing the output values in a database.