# US 2026/0064575 A1 · Method and system of testing a fine-tuned llm for domain specific code generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064575 A1
- Document ID: US20260064575A1
- Status: Published application
- Kind: A1
- Application number: 19/084,851
- Filed: 2025-03-20
- Published: 2026-03-05
- Earliest claimed: 2024-09-02
- Assignees: HCL Technologies Limited
- Applicants: HCL Technologies Limited
- Inventors: Arun Singh; Yogesh Gupta; Harikrishna Warrier
- Web page: https://patentgrep.com/patent/US20260064575A1

## CPC

- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 8/35: model driven
- G06F 11/3684: {for test design, e.g. generating new test cases}
- G06F 11/3692: {for test results analysis}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method and system of testing a fine-tuned LLM for domain specific code generation is disclosed. Further, a processor receives a test dataset corresponding to a domain from a code repository. Further, the processor determines an LLM generated problem statement corresponding to the test code using the fine-tuned LLM. The fine-tuned LLM is fine-tuned based on a training dataset. Further, the fine-tuned LLM is prompted based on the LLM generated problem statement to determine an LLM generated code for a corresponding test function. The accuracy level of the fine-tuned LLM is determined based on a percentage match between the LLM generated code with the test code for each of the set of test functions.

## Claims

### Claim 1

A method of testing a fine-tuned large language model (LLM), comprising:
receiving, by a processor, a test dataset corresponding to a domain from a code repository,
wherein the test dataset comprises a set of test functions and a test code corresponding to each test function of the set of test functions;
for each of the set of test functions:
determining, by the processor, an LLM generated problem statement based on the corresponding test code using the fine-tuned LLM,
wherein the fine-tuned LLM is fine-tuned based on a training dataset corresponding to the domain; and
prompting, by the processor, the fine-tuned LLM based on the LLM generated problem statement to determine an LLM generated code for a corresponding test function; and
determining, by the processor, an accuracy level of the fine-tuned LLM based on a percentage match between the LLM generated code with the test code for each of the set of test functions.

### Claim 2 (depends on 1)

The method of claim 1, further comprises:
determining, by the processor, a test assert corresponding to the LLM generated code for each of the set of test functions.

### Claim 3 (depends on 2)

The method of claim 2, wherein the training dataset corresponding to the domain comprises a set of predefined functions, a predefined code and a prompt corresponding to each predefined function from the set of predefined functions, and a test case corresponding to each of the predefined code for each of the set of predefined functions.

### Claim 4 (depends on 1)

The method of claim 1, wherein the test dataset is extracted based on a python script from the code repository.

### Claim 5 (depends on 1)

The method of claim 1, wherein the fine-tuned LLM is fine-tuned based on the training dataset using in-context learning techniques.

### Claim 6 (depends on 1)

The method of claim 1, comprising:
updating, by the processor, the training dataset with the LLM generated code that is about same as the test code for a corresponding predefined function from the set of predefined functions.

### Claim 7

A system for testing a fine-tuned large language model (LLM), comprising:
a processor; and
a memory communicably coupled to the processor, wherein the memory stores processor-executable instructions, which when executed by the processor, cause the processor to:
receive a test dataset corresponding to a domain from a code repository,
wherein the test dataset comprises a set of test function and a test code corresponding to each test function of the set of test functions;
for each of the set of test functions:
determine an LLM generated problem statement based on the corresponding test code using the fine-tuned LLM,
wherein the fine-tuned LLM is fine-tuned based on a training dataset corresponding to the domain; and
prompt the fine-tuned LLM based on the LLM generated problem statement to determine an LLM generated code for a corresponding test function; and
determine an accuracy level of the fine-tuned LLM based on a percentage match between the LLM generated code with the test code for each of the set of test functions.

### Claim 8 (depends on 7)

The system of claim 7, wherein the processor-executable instructions cause the processor to:
determine a test assert corresponding to the LLM generated code for each of the set of test functions.

### Claim 9 (depends on 8)

The system of claim 8, wherein the training dataset corresponding to the domain comprises a set of predefined functions, a predefined code, and a prompt corresponding to each predefined function from the set of predefined functions, and a test case corresponding to each of the predefined code for each of the set of predefined functions.

### Claim 10 (depends on 7)

The system of claim 7, wherein the test dataset is extracted based on a Python script from the code repository.

### Claim 11 (depends on 7)

The system of claim 7, wherein the fine-tuned LLM is fine-tuned based on the training dataset using in-context learning techniques.

### Claim 12 (depends on 7)

The system of claim 7, wherein the processor is further configured to update the training dataset with the LLM generated code that is about same as the test code for a corresponding predefined function from the set of predefined functions.

### Claim 13

A non-transitory computer-readable medium storing computer-executable instructions for testing a fine-tuned large language model (LLM), the stored instructions, when executed by a processor, cause the processor to perform operations comprising:
receiving a test dataset corresponding to a domain from a code repository,
wherein the test dataset comprises a set of test functions and a test code corresponding to each test function of the set of test functions;
for each of the set of test functions:
determining an LLM generated problem statement, based on the corresponding test code using the fine-tuned LLM,
wherein the fine-tuned LLM is fine-tuned based on a training dataset corresponding to the domain; and
prompting the fine-tuned LLM, based on the LLM generated problem statement to determine an LLM generated code for a corresponding test function; and
determining an accuracy level of the fine-tuned LLM, based on a percentage match between the LLM generated code with the test code for each of the set of test functions.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the stored instructions, when executed by the processor, cause the processor to perform operations comprises:
determining a test assert corresponding to the LLM generated code for each of the set of test functions.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the training dataset corresponding to the domain comprises a set of predefined functions, a predefined code and a prompt corresponding to each predefined function from the set of predefined functions, and a test case corresponding to each of the predefined code for each of the set of predefined functions.

### Claim 16 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the test dataset is extracted based on a python script from the code repository.

### Claim 17 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the fine-tuned LLM is fine-tuned based on the training dataset using in-context learning techniques.

### Claim 18 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the stored instructions, when executed by the processor, cause the processor to perform operations comprising:
updating the training dataset with the LLM generated code that is about same as the test code for a corresponding predefined function from the set of predefined functions.
