Method for Sequence-Based Prediction of Controlled Terms and Generating Protein Sequences from Controlled Terms using Enhanced Large Language Models

Abstract

The present invention relates to a method for enhancing the creativity of a generative pre-trained Large Language Model (LLM) in protein sequence generation and predicting controlled terms from protein sequences. The method includes incorporating 22 novel names representing the 22 amino acids into the vocabulary of the pre-trained LLM, conducting self-supervised learning using protein sequences encoded with the novel names to improve the LLM's comprehension and generation of coherent protein sequences, performing supervised learning using protein sequences to refine the LLM's ability to predict controlled terms based on protein sequences, and performing supervised learning to refine the LLM's ability to generate protein sequences based on controlled terms. The method includes generating the novel names either through a computer program or manually and utilizing datasets of protein sequences and their corresponding controlled terms from a protein database. The self-supervised learning employed is a masked language model (MLM). Additionally, an alternative method is disclosed, which involves identifying a set of 22 amino acid names from the original vocabulary of the pre-trained LLM and proceeding with self-supervised and supervised learning steps using the selected names. The two methods can be used independently or in combination to enhance the creativity of the LLM in achieving protein sequence generation and predicting controlled terms from protein sequences.

Dependencies are inferred from the claim text.