# US 2026/0093898 A1 · Generating corrected sentence-case text

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093898 A1
- Document ID: US20260093898A1
- Status: Published application
- Kind: A1
- Application number: 18/901,596
- Filed: 2024-09-30
- Published: 2026-04-02
- Earliest claimed: 2024-09-30
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Silu Wang; Tong Yao; Zigeng Wang; Wei Shen
- Web page: https://patentgrep.com/patent/US20260093898A1

## CPC

- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/232: Orthographic correction, e.g. spell checking or vowelisation
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/295: Named entity recognition
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Examples relate to a system including a processor that can perform certain operations. The operations can include obtaining input text. The operations also can include generating a set of vectors from an ensemble of machine-learning models based on the input text. The ensemble of machine-learning models can include a pre-trained language model configured to determine capitalization for mixed cases and acronyms, a pre-trained named entity recognition (NER) model configured to determine capitalization for general proper nouns, and a question-answer NER (QA-NER) model configured to determine capitalization for brand names. The QA-NER model can include a transformer language model and a linear layer. The operations additionally can include generating corrected sentence-case text by modifying capitalization of the input text based on the set of vectors and outputting the corrected sentence-case text on a draft advertisement user interface. Other embodiments are described.

## Claims

### Claim 1

A system comprising:
a processor; and
a non-transitory computer-readable medium storing computing instructions that, when executed on the processor, cause the processor to perform operations comprising:
obtaining input text;
generating a set of vectors from an ensemble of machine-learning models based on the input text, wherein the ensemble of machine-learning models comprise a pre-trained language model configured to determine capitalization for mixed cases and acronyms, a pre-trained named entity recognition (NER) model configured to determine capitalization for general proper nouns, and a question-answer NER (QA-NER) model configured to determine capitalization for brand names, wherein the QA-NER model comprises a transformer language model and a linear layer, wherein the linear layer is configured to reduce a vector output from the transformer language model to a two-dimensional vector comprising a start position and an end position of a brand in the input text; and
generating corrected sentence-case text by modifying capitalization of the input text based on the set of vectors.

### Claim 2 (depends on 1)

The system of claim 1, wherein the operations further comprise:
causing the corrected sentence-case text to be outputted on a draft advertisement user interface.

### Claim 3 (depends on 1)

The system of claim 1, wherein generating the corrected sentence-case text further comprises:
performing a majority voting based on the set of vectors, wherein each respective vector of the set of vectors indicates whether to capitalize each respective character of the input text.

### Claim 4 (depends on 3)

The system of claim 3, wherein the majority voting is performed on (i) an original casing vector, (ii) a true-case head vector that is output from the pre-trained language model, and (iii) a logical disjunction of a proper noun head vector that is output from the pre-trained NER model and a brand name head vector that is output from the QA-NER model.

### Claim 5 (depends on 1)

The system of claim 1, wherein the transformer language model and the linear layer are trained in epochs to optimize cross entropy loss.

### Claim 6 (depends on 1)

The system of claim 1, wherein the start position and the end position that are output from the linear layer are converted to probabilities through a softmax function in training the QA-NER model.

### Claim 7 (depends on 1)

The system of claim 1, wherein the QA-NER model takes as input a concatenation of the input text and a facet type and outputs an answer from the input text for the facet type.

### Claim 8 (depends on 1)

The system of claim 1, wherein the operations further comprise, before generating the set of vectors:
preprocessing the input text to remove special characters and extra spaces.

### Claim 9

A computer-implemented method comprising:
obtaining input text;
preprocessing the input text to remove special characters and extra spaces;
generating a set of vectors from an ensemble of machine-learning models based on the input text, wherein the ensemble of machine-learning models comprise a pre-trained language model configured to determine capitalization for mixed cases and acronyms, a pre-trained named entity recognition (NER) model configured to determine capitalization for general proper nouns, and a question-answer NER (QA-NER) model configured to determine capitalization for brand names, wherein the QA-NER model comprises a transformer language model and a linear layer, wherein the linear layer is configured to reduce a vector output from the transformer language model to a two-dimensional vector comprising a start position and an end position of a brand in the input text; and
generating corrected sentence-case text by modifying capitalization of the input text based on the set of vectors.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9 further comprising:
causing the corrected sentence-case text to be outputted on a draft advertisement user interface.

### Claim 11 (depends on 9)

The computer-implemented method of claim 9, wherein generating the corrected sentence-case text further comprises:
performing a majority voting based on the set of vectors, wherein each respective vector of the set of vectors indicates whether to capitalize each respective character of the input text.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein the majority voting is performed on (i) an original casing vector, (ii) a true-case head vector that is output from the pre-trained language model, and (iii) a logical disjunction of a proper noun head vector that is output from the pre-trained NER model and a brand name head vector that is output from the QA-NER model.

### Claim 13 (depends on 9)

The computer-implemented method of claim 9, wherein the transformer language model and the linear layer are trained in epochs to optimize cross entropy loss.

### Claim 14 (depends on 9)

The computer-implemented method of claim 9, wherein the start position and the end position that are output from the linear layer are converted to probabilities through a softmax function in training the QA-NER model.

### Claim 15 (depends on 9)

The computer-implemented method of claim 9, wherein the QA-NER model takes as input a concatenation of the input text and a facet type and outputs an answer from the input text for the facet type.

### Claim 16

A non-transitory computer-readable medium storing computing instructions that, when executed on a processor, cause the processor to perform operations comprising:
obtaining input text;
generating a set of vectors from an ensemble of machine-learning models based on the input text, wherein the ensemble of machine-learning models comprise a pre-trained language model configured to determine capitalization for mixed cases and acronyms, a pre-trained named entity recognition (NER) model configured to determine capitalization for general proper nouns, and a question-answer NER (QA-NER) model configured to determine capitalization for brand names, wherein the QA-NER model comprises a transformer language model and a linear layer, wherein the linear layer is configured to reduce a vector output from the transformer language model to a two-dimensional vector comprising a start position and an end position of a brand in the input text;
generating corrected sentence-case text by modifying capitalization of the input text based on the set of vectors; and
causing the corrected sentence-case text to be outputted on a draft advertisement user interface.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein generating the corrected sentence-case text further comprises:
performing a majority voting based on the set of vectors, wherein each respective vector of the set of vectors indicates whether to capitalize each respective character of the input text.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the majority voting is performed on (i) an original casing vector, (ii) a true-case head vector that is output from the pre-trained language model, and (iii) a logical disjunction of a proper noun head vector that is output from the pre-trained NER model and a brand name head vector that is output from the QA-NER model.

### Claim 19 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein:
the transformer language model and the linear layer are trained in epochs to optimize cross entropy loss; and
the start position and the end position that are output from the linear layer are converted to probabilities through a softmax function in training the QA-NER model.

### Claim 20 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein:
the QA-NER model takes as input a concatenation of the input text and a facet type and outputs an answer from the input text for the facet type; and
the operations further comprise, before generating the set of vectors:
preprocessing the input text to remove special characters and extra spaces.
