# US 2026/0154514 A1 · Language model performance on low resource languages

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0154514 A1
- Document ID: US20260154514A1
- Status: Published application
- Kind: A1
- Application number: 19/346,369
- Filed: 2025-09-30
- Published: 2026-06-04
- Earliest claimed: 2024-10-19
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Raviraj JOSHI; Kanishk SINGLA; Anusha KAMATH; Raunak KALANI; Utkarsh VAIDYA; Sanjay Singh CHAUHAN; Niranjan WARTIKAR; Eileen Margaret Peters LONG
- Web page: https://patentgrep.com/patent/US20260154514A1

## CPC

- G06F 40/44: Statistical methods, e.g. probability models
- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06F 40/216: using statistical methods
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/30: Semantic analysis
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06F 40/42: Data-driven translation
- G06F 40/45: Example-based machine translation; Alignment
- G06F 40/47: Machine-assisted translation, e.g. using translation memory
- G06F 40/51: Translation evaluation
- G06F 40/55: Rule-based translation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In various examples, techniques are described for adapting a multilingual Large Language Model (LLM) into a bilingual Small Language Model (SLM) that exhibits model capacity to understand, process, and generate content in both English and a Low-Resource Language (LRL). The techniques include compressing the LLM to generate a multilingual SLM and performing continued pre-training on the multilingual SLM to generate the bilingual SLM. The techniques also include performing one or more alignment techniques on the bilingual SLM to adapt the SLM's outputs to human values and expectations regarding, e.g., profanity, privacy, politeness, bias, and/or conversational style. The techniques may generate various training corpora, each including one or more of natural English content, natural LRL content, synthetic LRL content generated via translation from English sources, and transliterated synthetic LRL content based on transliterations of natural and/or synthetic LRL content.

## Claims

### Claim 1

One or more processors comprising processing circuitry to:
translate one or more data sources included in a set of natural English data sources into a Low-Resource Language (LRL) to generate a set of synthetic LRL data sources;
transliterate at least a subset of the set of synthetic LRL data sources and one or more data sources included in a set of natural LRL data sources into a Roman script to generate a set of transliterated LRL data sources; and
perform continued pre-training on a trained multilingual Small Language Model (SLM) using data sources included in one or more of the set of natural English data sources, the set of natural LRL data sources, the set of synthetic LRL data sources, or the set of transliterated LRL data sources to generate a bilingual Small Language Model (SLM).

### Claim 2 (depends on 1)

The one or more processors of claim 1, wherein the one or more processors are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more multimodal language models;
a system implementing one or more large language models (LLMs);
a system implementing one or more small language models (SLMs);
a system implementing one or more vision language models (VLMs);
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system using or deploying one or more inference microservices;
a system incorporating one or more machine learning models deployed in
a service or microservice along with an OS-level virtualization package;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.

### Claim 3 (depends on 1)

The one or more processors of claim 1, wherein the one or more processors further comprise processing circuitry to apply one or more compression techniques to a trained multilingual Large Language Model (LLM) to generate the trained multilingual SLM.

### Claim 4 (depends on 3)

The one or more processors of claim 3, wherein the one or more processors further comprise processing circuitry to re-train the trained multilingual SLM based at least on a portion of training data upon which the trained multilingual LLM was previously trained.

### Claim 5 (depends on 1)

The one or more processors of claim 1, wherein the one or more processors further comprise processing circuitry to perform one or more alignment techniques on the bilingual SLM to generate an aligned SLM, wherein the alignment techniques include one or more of a Supervised Fine-Tuning (SFT) technique or a Direct Preference Optimization (DPO) technique.

### Claim 6 (depends on 1)

The one or more processors of claim 1, wherein the one or more processors further comprise processing circuitry to remove one or more synthetic LRL data sources from the set of synthetic LRL data sources, based at least on perplexity scores associated with one or more synthetic LRL data sources included in the set of synthetic LRL data sources.

### Claim 7 (depends on 1)

The one or more processors of claim 1, wherein the one or more processors further comprise processing circuitry to remove a synthetic LRL data source from the set of synthetic LRL data sources, based at least on a comparison of a back-translation of the synthetic LRL data source into English and a natural English data source associated with the synthetic LRL data source.

### Claim 8 (depends on 1)

The one or more processors of claim 1, wherein the bilingual SLM is operable to understand and process a query that includes both English and LRL textual content.

### Claim 9 (depends on 1)

The one or more processors of claim 1, wherein the one or more processors further comprise processing circuitry to identify a pair of documents exhibiting at least a threshold level of similarity, and remove one document of the pair of documents.

### Claim 10 (depends on 1)

The one or more processors of claim 1, wherein the continued pre-training further comprises assigning a first weight to each natural LRL data source included in the set of natural LRL data sources and assigning a second weight to each synthetic LRL data source included in the set of synthetic LRL data sources, wherein the first weight is numerically greater than the second weight.

### Claim 11

A system comprising one or more processors to:
translate one or more data sources included in a set of natural English data sources into a Low-Resource Language (LRL) to generate a set of synthetic LRL data sources;
transliterate at least a subset of the set of synthetic LRL data sources and one or more data sources included in a set of natural LRL data sources to generate a set of transliterated LRL data sources; and
perform continued pre-training on a trained multilingual Small Language Model (SLM) using data sources included in one or more of the set of natural English data sources, the set of natural LRL data sources, the set of synthetic LRL data sources, or the set of transliterated LRL data sources to generate a bilingual Small Language Model (SLM).

### Claim 12 (depends on 11)

The system of claim 11, wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more multimodal language models;
a system implementing one or more large language models (LLMs);
a system implementing one or more small language models (SLMs);
a system implementing one or more vision language models (VLMs);
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system using or deploying one or more inference microservices;
a system incorporating one or more machine learning models deployed in a service or microservice along with an OS-level virtualization package;
a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

### Claim 13 (depends on 11)

The system of claim 11, wherein the one or more processors are further comprised to apply one or more compression techniques to a trained multilingual Large Language Model (LLM) to generate the trained multilingual SLM.

### Claim 14 (depends on 13)

The system of claim 13, wherein the one or more processors are further comprised to re-train the multilingual SLM based at least on a portion of training data upon which the multilingual LLM was previously trained.

### Claim 15 (depends on 11)

The system of claim 11, wherein the one or more processors are further comprised to perform one or more alignment techniques on the bilingual SLM to generate an aligned SLM, wherein the alignment techniques include one or more of a Supervised Fine-Tuning (SFT) technique or a Direct Preference Optimization (DPO) technique.

### Claim 16 (depends on 11)

The system of claim 11, wherein the one or more processors are further comprised to remove one or more synthetic LRL data sources from the set of synthetic LRL data sources, based at least on perplexity scores associated with one or more of the synthetic LRL data sources included in the set of synthetic LRL data sources.

### Claim 17 (depends on 11)

The system of claim 11, wherein the one or more processors are further comprised to remove a synthetic LRL data source from the set of synthetic LRL data sources, based at least on a comparison of a back-translation of the synthetic LRL data source into English and a natural English data source associated with the synthetic LRL data source.

### Claim 18 (depends on 11)

The system of claim 11, wherein the continued pre-training further comprises assigning a first weight to each data source included in the set of natural LRL data sources and assigning a second weight to each data source included in the set of synthetic LRL data sources, wherein the first weight is numerically greater than the second weight.

### Claim 19

A method comprising:
translating, using a trained multilingual small language model, an input from a first language into a second, non-English language, wherein the trained multilingual small language model is generated at least by: (i) compressing a multilingual large language model to generate an initial multilingual small language model; and (ii) training the initial multilingual small language model using a training dataset to generate the trained multilingual small language model, the training dataset including at least natural Low-Resource Language (LRL) data and synthetic LRL data generated at least by translating High-Resource Language (HRL) data to the LRL.

### Claim 20 (depends on 19)

The method of claim 19, wherein the method is performed by at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more multimodal language models;
a system implementing one or more large language models (LLMs);
a system implementing one or more small language models (SLMs);
a system implementing one or more vision language models (VLMs);
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system using or deploying one or more inference microservices;
a system incorporating one or more machine learning models deployed in a service or microservice along with an OS-level virtualization package;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
