Robust explainable artificial intelligence
AssigneeArticul8 AI, Inc.
InventorArun Karthi Subramaniyan
Life of this patent
Estimating the term…
Abstract
Provided is process, including: a parsing a sequence of tokens from natural-language text; computing, with a transformer encoder of a neural network, a feature vector of the sequence of tokens; inputting the feature vector into a probabilistic head of the neural network comprising a Gaussian-process layer and determining, with the probabilistic head, both a latent mean and a latent variance for each of a plurality of candidate output classes; computing, with the probabilistic head, predictive class probabilities from the latent means and latent variances; selecting, with the neural network, one of the candidate output classes based on the predictive class probabilities; and determining an uncertainty of the selection based on the latent means and the latent variances.
Claims20
A tangible, non-transitory, machine-readable medium storing instructions that, when executed, effectuate operations comprising:
- parsing, with a computer system, a sequence of tokens from natural-language text, wherein the parsing comprises subword tokenization used with an attention mask and positional encodings;
- computing, with the computer system, with a transformer encoder of a neural network, a feature vector of the sequence of tokens, wherein the transformer encoder comprises stacked layers that implement multi-head self-attention over query, key, and value projections with residual connections and layer normalization, and the feature vector is obtained by pooling hidden states of an encoder layer;
- inputting, with the computer system, the feature vector into a probabilistic head of the neural network comprising a Gaussian-process layer and determining, with the probabilistic head, both a latent mean and a latent variance for each of a plurality of candidate output classes, wherein for each candidate output class the Gaussian-process layer corresponds to a latent function that yields a posterior mean and a posterior variance;
- computing, with the computer system, predictive class probabilities from the latent means and latent variances;
- selecting, with the computer system, one of the candidate output classes based on the predictive class probabilities;
- determining, with the computer system, an uncertainty of the selection based on the latent variances;
- and
- storing, with the computer system, the selection and the uncertainty in memory.
A method, comprising:
- parsing, with a computer system, a sequence of tokens from natural-language text, wherein the parsing comprises subword tokenization used with an attention mask and positional encodings;
- computing, with the computer system, with a transformer encoder of a neural network, a feature vector of the sequence of tokens, wherein the transformer encoder comprises stacked layers that implement multi-head self-attention over query, key, and value projections with residual connections and layer normalization, and the feature vector is obtained by pooling hidden states of an encoder layer;
- inputting, with the computer system, the feature vector into a probabilistic head of the neural network comprising a Gaussian-process layer and determining, with the probabilistic head, both a latent mean and a latent variance for each of a plurality of candidate output classes, wherein for each candidate output class the Gaussian-process layer corresponds to a latent function that yields a posterior mean and a posterior variance;
- computing, with the computer system, predictive class probabilities from the latent means and latent variances;
- selecting, with the computer system, one of the candidate output classes based on the predictive class probabilities;
- determining, with the computer system, an uncertainty of the selection based on the latent variances;
- and
- storing, with the computer system, the selection and the uncertainty in memory.
Dependencies are inferred from the claim text.