Knowledge transfer between language models

A system comprising:

• processing circuitry;
• and
• a memory device including instruction stored thereon that, when executed by a machine, cause the machine to perform operations for knowledge transfer between language models, the operations comprising:
• receiving input data, the input data indicating a user-defined domain;
• extracting, by a large language model and based on the input data and an upper-level ontology, a first domain element that is an example of an entity of the upper-level ontology;
• appending the first domain element to a domain ontology resulting in a knowledge base;
• extracting, by the large language model and based on the input data and the first domain element, a second domain element;
• appending the second domain element to the knowledge base to form an expanded knowledge base;
• selecting a small language model;
• and
• transferring knowledge from the expanded knowledge base to the small language model resulting in a domain-specialized small language model.

A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for knowledge transfer between language model, the operations comprising:

• receiving input data, the input data indicating a user-defined domain;
• extracting, by a large language model and based on the input data and an upper-level ontology, a first domain element that is an example of an entity of the upper-level ontology;
• appending the first domain element to a domain ontology resulting in a knowledge base;
• extracting, by the large language model and based on the input data and the first domain element, a second domain element;
• appending the second domain element to the knowledge base to form an expanded knowledge base;
• selecting a small language model;
• and
• transferring knowledge from the expanded knowledge base to the small language model resulting in a domain-specialized small language model.