US 12,744,033 B1Granted

Contextual information for spoken language processing

AssigneeAmazon Technologies, Inc.

InventorsGrant Strimel, Feng-Ju Chang, Srikrishna Kompella, Saumya Yashmohini Sahai, Mingzhi Yu, Kanthashree Mysore Sathyendra, Jing Liu, Ross William McGowan

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Techniques for generating a multi-modal, context aware embedded representation of an entity and contextual data associated with the entity are described. The system may receive a natural language entity for which an embedded representation is to be generated. The system uses the natural language entity to determine/generate various context data associated with the entity (e.g., slot type, usage example, phonemes, audio data, image/video data, etc.). The system may tokenize and encode the natural language entity and the context data. Thereafter, the encoded entity and the context data may be inputted to a language model, which generates a single embedded representation corresponding to the entity and the context data. Thereafter, the embedded representation may be used by a system component for performing processing with respect to a user input.

Claims20

Dependent claims

A computer-implemented method, comprising:

receiving first natural language data corresponding to an entity;
determining first context data associated with the entity, the first context data corresponding to second natural language data representing a first example usage of the entity within at least one phrase;
processing, using a first machine learning (ML) model, the first natural language data and the first context data to generate first embedding data corresponding to the entity and the first context data, wherein the first embedding data represents a first contextual relationship between the entity and the first context data;
storing the first embedding data in a data store;
receiving first input data corresponding to a first user input provided to an input component, wherein the first user input is provided to the input component after the first embedding data has been stored in the data store;
following receipt of the first input data, retrieving the first embedding data from the data store;
determining, using the first input data and the first embedding data retrieved from the data store, first data;
processing the first data to generate first output data responsive to the first user input;
and
causing presentation of the first output data.

7 more elements

A computing system comprising:

at least one processor;
and
at least one memory including instructions that, when executed by the at least one processor, cause the computing system to:
receive first natural language data corresponding to an entity;
determine first context data associated with the entity, the first context data corresponding to second natural language data representing a first example usage of the entity within at least one phrase;
process, using a first machine learning (ML) model, the first natural language data and the first context data to generate first embedding data corresponding to the entity and the first context data, wherein the first embedding data represents a first contextual relationship between the entity and the first context data;
store the first embedding data in a data store:
receive first input data corresponding to a first user input provided to an input component, wherein the first user input is provided to the input component after the first embedding data has been stored in the data store;
following receipt of the first input data, retrieving the first embedding data from the data store:
determine, using the first input data and the first embedding data retrieved from the data store, first data;
process the first data to generate first output data responsive to the first user input;
and
cause presentation of the first output data.

10 more elements

Dependencies are inferred from the claim text.