Embedding-based generative model for protein design

A computer program product for designing biological sequences using inverse folding of three-dimensional structures, the computer program product comprising a non-transitory storage medium readable by a processing circuit and storing computer readable instructions run by the processing circuit for performing a method comprising:

• using a first transformer model encoder as part of a sequence-to-sequence model to generate latent sequence representations of corresponding input one-dimensional biological sequences of nucleotides or amino acids represented as text which correspond with respective three-dimensional biological structures, a generated latent sequence representation of the generated latent sequence representations comprising an encoded vector representation of a one-dimensional biological sequence of the one-dimensional biological sequences in a first latent space;
• using a second transformer model encoder as part of a fold-to-sequence model to generate respective latent fold representations of the respective three-dimensional biological structures corresponding to the respective input one-dimensional biological sequences, a generated latent fold representation of the generated latent fold representations comprising an encoded vector representation of a three-dimensional biological structure of the three-dimensional biological structures in a second latent space, wherein the three-dimensional biological structure comprises a density of structural elements represented in 3D voxels of a 3D voxel grid space, each 3D voxel comprising a feature representation comprising a probabilistic approximation determined from contributions of residues within neighboring voxels;
• training a transformer model decoder to learn a joint latent space between said generated latent sequence representations of input one-dimensional biological sequences and said generated respective latent fold representations of the respective three-dimensional biological structures corresponding to the input one-dimensional biological sequences;
• receiving, at said second transformer model encoder running at the processing circuit, an encoded representation of a target three-dimensional biological structure;
• generating, using said second transformer model encoder, an encoded vector representation of said received target three-dimensional biological structure in the second latent space;
• and
• inputting the encoded vector representation of the target three-dimensional biological structure to said trained transformer model decoder running on said processing circuit, said trained transformer model outputting, in response, an alternative one-dimensional sequence of nucleotides or amino acids which will result in a similar three-dimensional biological structure having the same folds as said target three-dimensional biological structure.

A computer-implemented system to design biological sequences using inverse folding of three-dimensional structures, the system comprising:

• a memory storage device;
• and
• a hardware processor coupled to said memory storage device and configured to perform a method to:
• use a first transformer model encoder as part of a sequence-to-sequence model to generate latent sequence representations of corresponding input one-dimensional biological sequences of nucleotides or amino acids represented as text which correspond with respective three-dimensional biological structures, a generated latent sequence representation of the generated latent sequence representations comprising an encoded vector representation of a one-dimensional biological sequence of the one-dimensional biological sequences in a first latent space;
• use a second transformer model encoder as part of a fold-to-sequence model to generate respective latent fold representations of the respective three-dimensional biological structures corresponding to the respective input one-dimensional biological sequences, a generated latent fold representation of the generated latent sequence representations comprising an encoded vector representation of a three-dimensional biological structure of the three-dimensional biological structures in a second latent space, wherein the three-dimensional biological structure comprises a density of structural elements represented in 3D voxels of a 3D voxel grid space, each 3D voxel comprising a feature representation comprising a probabilistic approximation determined from contributions of residues within neighboring voxels;
• train a transformer model decoder to learn a joint latent space between said generated latent sequence representations of input one-dimensional biological sequences and said generated respective latent fold representations of the respective three-dimensional biological structures corresponding to the input one-dimensional biological sequences;
• receive, at said second transformer model encoder, an encoded representation of a target three-dimensional biological structure;
• generate, using said second transformer model encoder, an encoded vector representation of said received target three-dimensional biological structure in the second latent space;
• and
• input the encoded vector representation of the target three-dimensional biological structure to said trained transformer model decoder, said trained transformer model outputting, in response, an alternative one-dimensional sequence of nucleotides or amino acids which will result in a similar three-dimensional biological structure having the same folds as said target three-dimensional biological structure.