Integration of semantic context information

A computer system comprising:

• one or more computers;
• an interface to the one or more computers that is programmed to receive a request to predict a next word in a dialog being uttered by a speaker;
• a neural network module that is programmed to access a neural network comprising i) an input layer that includes a first portion representing a local context for the dialog and a second portion representing a semantic context for the dialog, ii) one or more hidden layers that are at least partially interconnected with the input layer by first connections, and iii) an output layer that represents a vocabulary of candidate words and that is at least partially interconnected with at least one of the one or more hidden layers by second connections;
• a local context module that is programmed to identify the local context for the dialog of the speaker;
• a semantic context generator that is programmed to select, using a semantic model, at least one vector that represent the semantic context for the dialog, the at least one vector including values for a plurality of dimensions;
• and
• a probability generator that is programmed to:
• apply input to the input layer of the neural network, the input comprising i) the local context of the dialog and ii) the values for the plurality of dimensions of the at least one vector that represents the semantic context of the dialog,
• generate probability values for at least a portion of the candidate words in the vocabulary of the output layer based on propagation of the input through the neural network using, at least, the first connections and the second connections between layers of the neural network, and
• provide, based on the probability values, information that identifies one or more of the candidate words.

A computer program product embodied in a computer readable storage device storing instructions that, when executed, cause one or more computing devices to perform operations comprising:

• receiving a request to predict a next word in a dialog being uttered by a speaker;
• accessing a neural network comprising i) an input layer that includes a first portion representing a local context for the dialog and a second portion representing a semantic context for the dialog, ii) one or more hidden layers that are at least partially interconnected with the input layer by first connections, and iii) an output layer that represents a vocabulary of candidate words and that is at least partially interconnected with at least one of the one or more hidden layers by second connections;
• identifying the local context for the dialog of the speaker;
• selecting, using a semantic model, at least one vector that represent the semantic context for the dialog, the at least one vector including values for a plurality of dimensions;
• applying input to the input layer of the neural network, the input comprising i) the local context of the dialog and ii) the values for the plurality of dimensions of the at least one vector that represents the semantic context of the dialog;
• generating probability values for at least a portion of the candidate words in the vocabulary of the output layer based on propagation of the input through the neural network using, at least, the first connections and the second connections between layers of the neural network;
• and
• providing, based on the probability values, information that identifies one or more of the candidate words.