Natural language processing techniques using multi-context self-attention machine learning frameworks

A system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to:

• generate, using a multi-context convolutional self-attention machine learning framework, a cross-context token representation based at least in part on an input text token of an input text sequence, wherein:
• the multi-context convolutional self-attention machine learning framework comprises a shared token embedding machine learning model, a plurality of context-specific self-attention machine learning models, and a cross-context representation inference machine learning model,
• the shared token embedding machine learning model is configured to generate an initial token embedding for the input text token,
• a context-specific self-attention machine learning model of the plurality of context-specific self-attention machine learning models is (a) associated with a distinct context window size of a plurality of distinct context window sizes, and (b) configured to generate a context-specific token representation for the input text token based at least in part on the initial token embedding, and
• the cross-context representation inference machine learning model is configured to generate the cross-context token representation based at least in part on the context-specific token representation;
• generate, using a natural language processing machine learning model, a natural language processing output for the input text sequence based at least in part on the cross-context token representation;
• and
• initiate the performance of one or more prediction-based actions based at least in part on the natural language processing output.

One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to:

• generate, using a multi-context convolutional self-attention machine learning framework, a cross-context token representation based at least in part on an input text token of an input text sequence, wherein:
• the multi-context convolutional self-attention machine learning framework comprises a shared token embedding machine learning model, a plurality of context-specific self-attention machine learning models, and a cross-context representation inference machine learning model,
• the shared token embedding machine learning model is configured to generate an initial token embedding for the input text token,
• a context-specific self-attention machine learning model of the plurality of context-specific self-attention machine learning models is (a) associated with a distinct context window size of a plurality of distinct context window sizes, and (b) configured to generate a context-specific token representation for the input text token based at least in part on the initial token embedding, and
• the cross-context representation inference machine learning model is configured to generate the cross-context token representation based at least in part on the context-specific token representation;
• generate, using a natural language processing machine learning model, a natural language processing output for the input text sequence based at least in part on the cross-context token representation;
• and
• initiate the performance of one or more prediction-based actions based at least in part on the natural language processing output.