Systems and methods for language modeling of protein engineering
AssigneeSalesforce, Inc.
InventorsAli Madani, Bryan McCann, Nikhil Naik
Life of this patent
Estimating the term…
Abstract
The present disclosure provides systems and methods for controllable protein generation. According to some embodiments, the systems and methods leverage neural network models and techniques that have been developed for other fields, in particular, natural language processing (NLP). In some embodiments, the systems and methods use or employ models implemented with transformer architectures developed for language modeling and apply the same to generative modeling for protein engineering.
Claims20
A method for protein generation via a neural network based language model based on target protein properties, the method comprising:
- receiving, via a communication interface, a data sequence of amino acids and a set of target protein properties;
- forming an input sequence of tokens by prepending the set of target protein properties to the data sequence of amino acids;
- generating, via the neural network based language model implemented on one or more hardware processors, which is trained on a protein sequence dataset to generate a predicted amino acid sequence according to desired protein properties, a set of scores indicating conditional distributions of a next output token in response to the input sequence of tokens representing the data sequence of amino acids and the set of target protein properties;
- sequentially determining a constituent amino acid from the data sequence of amino acids based on the set of scores;
- forming an output data sequence of amino acids representing a protein having the target protein properties from the sequentially determined constituent amino acids;
- and
- transmitting, via the communication interface, the output data sequence of amino acids to a protein synthesis facility for synthesizing a new protein having the target protein properties based on the output data sequence of amino acids.
A system for protein generation via a neural network based language model, the system comprising:
- a communication interface configured to receive a data sequence of amino acids and a set of target protein properties;
- a memory configured to store the neural network based language model and a plurality of processor-executable instructions;
- and
- a processor executing the instructions to:
- form an input sequence of tokens by prepending the set of target protein properties to the data sequence of amino acids;
- generate, via the neural network based language model implemented on the processor, which is trained on a protein sequence dataset to generate a predicted amino acid sequence according to desired protein properties, a set of scores indicating conditional distributions of a next output token in response to the input sequence of tokens representing the data sequence of amino acids and the set of target protein properties;
- sequentially determine a constituent amino acid from the data sequence of amino acids based on the set of scores;
- form an output data sequence of amino acids representing a protein having the target protein properties from the sequentially determined constituent amino acids;
- and
- wherein the communication interface is configured to transmit the output data sequence of amino acids to a protein synthesis facility for synthesizing a new protein having the target protein properties based on the output data sequence of amino acids.
A process-readable non-transitory storage medium storing processor-executable instructions for protein generation via a neural network based language model, the processor-executable instructions being executable by a processor to perform:
- receiving, via a communication interface, a data sequence of amino acids and a set of target protein properties;
- forming an input sequence of tokens by prepending the set of target protein properties to the data sequence of amino acids;
- generating, via the neural network based language model implemented on one or more hardware processors, which is trained on a protein sequence dataset to generate a predicted amino acid sequence according to desired protein properties, a set of scores indicating conditional distributions of a next output token in response to the input sequence of tokens representing the data sequence of amino acids and the set of target protein properties;
- sequentially determining a constituent amino acid from the data sequence of amino acids based on the set of scores;
- forming an output data sequence of amino acids representing a protein having the target protein properties from the sequentially determined constituent amino acids;
- and
- transmitting, via the communication interface, the output data sequence of amino acids to a protein synthesis facility for synthesizing a new protein having the target protein properties based on the output data sequence of amino acids.
Dependencies are inferred from the claim text.