Domain-focused natural language question and answer

A method for generating a fine-tuned machine learning model to provide a domain-focused natural language answer to a natural language question, the method comprising:

• creating, from a plurality of files, a plurality of fine-tuning data examples, each respective fine-tuning data example of the plurality of fine-tuning data examples including a question, an answer, and a supporting citation;
• fine-tuning a pre-trained foundational machine learning model using the plurality of fine-tuning data examples to generate a fine-tuned machine learning model;
• inputting a natural language question to the fine-tuned machine learning model;
• and
• retrieving, from the fine-tuned machine learning model, a natural language answer to the natural language question and an indication of one or more files, one or more relevant sections, or both that the natural language answer is based on, wherein the fine-tuned machine learning model is configured to generate the natural language answer and the indication of the one or more files, the one or more relevant sections, or both that the answer is based on by:
• retrieving, using a retrieval model, the one or more files including the one or more sections relevant to the natural language question;
• and
• constructing, using a large language model, the natural language answer to the natural language question based on the one or more relevant sections of the one or more retrieved files,
• the method further comprising augmenting a vocabulary of the pre-trained foundational machine learning model using the plurality of files, wherein augmenting a vocabulary of the pre-trained foundational machine learning model using the plurality of files includes:
• generating, from the plurality of files, a plurality of tokens associated with a minimal likelihood of creating noise in the pre-trained foundational machine learning model, wherein each token represents a domain-specific term included in the plurality of files,
• adding the plurality of tokens to the vocabulary;
• and
• initializing, in the pre-trained foundational machine learning model, a number of vectors equal to the plurality of tokens.

A domain-focused natural language question and answer system, the system comprising:

• one or more electronic processors, the one or more electronic processors configured to:
• create, from a plurality of files, a plurality of fine-tuning data examples, each respective fine-tuning data example of the plurality of fine-tuning data examples including a question, an answer, and a supporting citation;
• fine-tune a pre-trained foundational machine learning model using the plurality of fine-tuning data examples to generate a fine-tuned machine learning model;
• input a natural language question to the fine-tuned machine learning model;
• and
• retrieve, from the fine-tuned machine learning model, an indication of one or more files, one or more sections, or both that are relevant to the natural language question, wherein the fine-tuned machine learning model is configured to generate the indication of the one or more files, the one or more sections, or both that are relevant to the natural language question by:
• retrieving, using a retrieval model, the one or more files including the one or more sections relevant to the natural language question,
• wherein the one or more electronic processors are further configured to augment a vocabulary of the pre-trained foundational machine learning model using the plurality of files by:
• generating, from the plurality of files, a plurality of tokens associated with a minimal likelihood of creating noise in the pre-trained foundational machine learning model, wherein each token represents a domain-specific term included in the plurality of files;
• adding the plurality of tokens to the vocabulary;
• and
• initializing, in the pre-trained foundational machine learning model, a number of vectors equal to the plurality of tokens.