# US 2025/0165752 A1 · Systems and methods for processing data for large language models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165752 A1
- Document ID: US20250165752A1
- Status: Published application
- Kind: A1
- Application number: 18/517,600
- Filed: 2023-11-22
- Published: 2025-05-22
- Earliest claimed: 2023-11-22
- Assignees: Infobip Ltd.
- Applicants: Infobip Ltd.
- Inventors: Ivica LOVRIC
- Web page: https://patentgrep.com/patent/US20250165752A1

## CPC

- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method includes: receiving a query; determining a capability associated with the query using at least one of a capability machine learning model or a segmentation algorithm; determining, using a routing system, a large language model provider, among a plurality of large language model providers, that best matches the capability associated with the query; providing the query to the large language model provider; and receiving a response from the large language model provider.

## Claims

### Claim 1

A method comprising:
receiving a query;
determining a capability associated with the query using at least one of a capability machine learning model or a segmentation algorithm;
determining, using a routing system, a large language model provider, among a plurality of large language model providers, that best matches the capability associated with the query;
providing the query to the large language model provider; and
receiving a response from the large language model provider.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
checking a request cache; and
determining that the query does not match a cached query.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
generating a modified query for the large language model provider.

### Claim 4 (depends on 3)

The method of claim 3, wherein the generating the modified query includes:
compressing the received query.

### Claim 5 (depends on 4)

The method of claim 4, wherein the compressing the received query includes:
removing one or more tokens from the query.

### Claim 6 (depends on 4)

The method of claim 4, wherein the compressing the received query includes:
providing a difference between the received query and the compressed query.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
providing the received response from the large language model provider.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
receiving an updated capability from a large language model provider, among the plurality of large language model providers; and
updating the routing system based on the updated capability.

### Claim 9 (depends on 1)

The method of claim 1, wherein the determining the large language model provider further includes:
determining the large language model provider based on one or more of least cost, fallback, quality, or accuracy.

### Claim 10 (depends on 1)

The method of claim 1, wherein the determining the large language model provider further includes determining the large language model provider based on a requested parameter in the query.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
generating an intent based on the query,
wherein the determining the large language model provider further includes determining the large language model provider based on the generated intent.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
performing a health check of one or more of the plurality of large language model providers.

### Claim 13 (depends on 12)

The method of claim 12, wherein the determining the large language model provider includes determining the large language model provider based on the health check.

### Claim 14

A method comprising:
receiving a query;
determining that the query matches a cached query;
retrieving a cached response from a large language model provider for the cached query; and
providing the cached response.

### Claim 15 (depends on 14)

The method of claim 14, wherein the determining that the query matches a cached query includes:
determining whether a similarity of the query to the cached query is above a similarity threshold.

### Claim 16 (depends on 14)

The method of claim 14, wherein the cached response includes a response generated by:
receiving the cached query;
determining a capability associated with the cached query using at least one of a capability machine learning model or a segmentation algorithm;
determining the large language model provider, among a plurality of large language model providers, that best matches the capability associated with the cached query;
providing the cached query to the large language model provider; and
receiving the cached response from the large language model provider.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
performing a health check of one or more of the plurality of large language model providers,
wherein the determining the large language model provider includes determining the large language model provider based on the health check.

### Claim 18

A system comprising one or more processors configured to execute a method including:
receiving a query;
determining a capability associated with the query using at least one of a capability machine learning model or a segmentation algorithm;
determining, using a routing system, a large language model provider, among a plurality of large language model providers, that best matches the capability associated with the query;
providing the query to the large language model provider; and
receiving a response from the large language model provider.

### Claim 19 (depends on 18)

The system of claim 18, the method further including:
removing one or more tokens from the query.

### Claim 20 (depends on 18)

The system of claim 18, the method further including:
performing a health check of one or more of the plurality of large language model providers, wherein the determining the large language model provider includes determining the large language model provider based on the health check.
