Back to results
US 11,769,055 B2Granted
Systems and methods for a supra-graph based attention model for multi-layered embeddings and deep learning applications
AssigneesArizona Board of Regents on behalf of Arizona State University, Lawrence Livermore National Security, LLC
InventorsUday Shanthamallu, Jayaraman Thiagarajan, Andreas Spanias, Huan Song
Life of this patent
Estimating the term…
Abstract
Various embodiments of systems and methods for attention models with random features for multi-layered graph embeddings are disclosed.
Claims20
A method, comprising:
- providing a multi-layered graph having a plurality of layers, each of the plurality of layers including a plurality of nodes;
- obtaining a supra graph by introducing one or more virtual edges between every node in a layer and its counterparts in other layers;
- generating a set of features for each of the nodes in the supra graph;
- inferring dependencies between nodes in the supra graph by processing each set of features using an attention model, wherein the attention model comprises a plurality of attention layers and wherein each of the plurality of attention layers produces a set of layer-specific node features;
- and
- characterizing each of the plurality of nodes by aggregating layer-specific node features associated with one of the plurality of nodes across each of the plurality of attention layers.
3 more elements
A method comprising:
- providing a multi-layered graph having a plurality of layers, each of the plurality of layers including a plurality of nodes;
- obtaining a supra graph by introducing virtual edges between every node in a layer and its counterparts in other layers;
- learning a plurality of latent feature vectors for each of the plurality of nodes using an attention model, wherein the attention model comprises a plurality of layers, and wherein each of the plurality of layers comprises a plurality of attention heads and a fusion head;
- combining each of the plurality of latent feature vectors for a node determined by each of the plurality of attention heads to produce a layer-specific feature vector for the node using the fusion head, wherein the layer-specific feature vector is associated with one of a plurality of attention layers;
- and
- characterizing each of the plurality of nodes by aggregating latent feature vectors associated with one of the plurality of nodes across each of the plurality of attention layers.
3 more elements
Dependencies are inferred from the claim text.