Protein structure prediction

A system for performing a software-implemented application for antibody structure prediction, wherein a predicted structure of a given antibody is defined by values of a plurality of structure parameters, the system comprising:

• one or more processors;
• and
• one or more computer-readable memories coupled to the one or more processors and having instructions stored thereon that are executable by the one or more processors to perform a method comprising:
• receiving a target antibody sequence of a target antibody that includes a sequence of amino acids;
• inputting the target antibody sequence into an antibody language model (ALM), wherein the ALM is a protein language model trained from antibody sequences, and the ALM comprises a plurality of self-attention layers;
• obtaining, using the ALM without performing multiple sequence alignment (MSA), a residue encoding and an attention weight encoding, wherein:
• the residue encoding comprises a respective first embedding corresponding to each of the amino acids in the target antibody sequence output by the ALM;
• and
• the attention weight encoding comprises a respective second embedding corresponding to a pair of amino acids in the target antibody sequence calculated from attention weights of the self-attention layers of the ALM;
• transforming the residue encoding and the attention weight encoding into a single representation and a pair representation;
• inputting the single representation and the pair representation into a structure prediction model, wherein parameters of the structure prediction model are trained based on a loss function reflecting a difference between a predicted structure and an actual structure of an antibody;
• determining the predicted structure of the target antibody using the structure prediction model based on the single representation and the pair representation;
• and
• outputting the predicted structure of the target antibody.

One or more non-transitory, computer-readable media storing one or more instructions executable by a computer system to perform operations comprising:

• receiving a target antibody sequence of a target antibody that includes a sequence of amino acids;
• inputting the target antibody sequence into an antibody language model (ALM), wherein the ALM is a protein language model trained from antibody sequences, and the ALM comprises a plurality of self-attention layers;
• obtaining, using the ALM without performing multiple sequence alignment (MSA), a residue encoding and an attention weight encoding, wherein:
• the residue encoding comprises a respective first embedding corresponding to each of the amino acids in the target antibody sequence output by the ALM;
• and
• the attention weight encoding comprises a respective second embedding corresponding to a pair of amino acids in the target antibody sequence calculated from attention weights of the self-attention layers of the ALM;
• transforming the residue encoding and the attention weight encoding into a single representation and a pair representation;
• inputting the single representation and the pair representation into a structure prediction model, wherein parameters of the structure prediction model are trained based on a loss function reflecting a difference between a predicted structure and an actual structure of an antibody;
• determining the predicted structure of the target antibody using the structure prediction model based on the single representation and the pair representation;
• and
• outputting the predicted structure of the target antibody.