Method and system for creating pruned large language models

A system for creating a pruned large language model (LLM), comprising:

• a processor;
• a memory communicably coupled to the processor, wherein the memory stores processor-executable instructions, which, on execution, cause the processor to:
• receive a weight file of a pretrained LLM, a first metadata, a predefined pruning criterion, and a predefined pruning ratio,
• wherein the first metadata comprises architecture information of the pretrained LLM,
• wherein the pretrained LLM comprises a plurality of layers, and
• wherein the weight file comprises a set of weight matrices each representing learned parameters of a corresponding layer from the plurality of layers;
• identify a set of target layers to be pruned from the plurality of layers based on the set of weight matrices;
• determine at least one weight to be removed from each of the set of target layers based on the predefined pruning criterion and the predefined pruning ratio;
• determine a position index of the at least one weight corresponding to each of the set of target layers based on the set of weight matrices;
• generate a compressed weight file and a second metadata by removing the at least one weight from each of the set of target layers based on the position index;
• and
• create the pruned LLM corresponding to the pretrained LLM using the compressed weight file and the second metadata.

A non-transitory computer-readable medium storing computer-executable instructions for creating a pruned large language model (LLM), the computer-executable instructions configured for:

• receiving a weight file of a pretrained LLM, a first metadata, a predefined pruning criterion, and a predefined pruning ratio;
• wherein the first metadata comprises architecture information of the pretrained LLM, and
• wherein the pretrained LLM comprises a plurality of layers, and
• wherein the weight file comprises a set of weight matrices each representing learned parameters of a corresponding layer from the plurality of layers;
• identifying a set of target layers to be pruned from the plurality of layers based on the set of weight matrices;
• determining at least one weight to be removed from each of the set of target layers based on the predefined pruning criterion and the predefined pruning ratio;
• determining a position index of the at least one weight corresponding to each of the set of target layers based on the set of weight matrices;
• generating a compressed weight file and a second metadata by removing the at least one weight from each of the set of target layers based on the position index;
• and
• creating the pruned LLM corresponding to the pretrained LLM model using the compressed weight file and the second metadata.