# US 2026/0065094 A1 · Local small foundation system for inferencing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065094 A1
- Document ID: US20260065094A1
- Status: Published application
- Kind: A1
- Application number: 18/963,962
- Filed: 2024-11-29
- Published: 2026-03-05
- Earliest claimed: 2024-09-05
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: William Jeffery White; Xin Ma; George Currie
- Web page: https://patentgrep.com/patent/US20260065094A1

## CPC

- G06F 16/285: {Clustering or classification}
- G06N 3/045: Combinations of networks
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 5/04: Inference or reasoning models
- G06N 5/043: Distributed expert systems; Blackboards
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The disclosure is directed to a local small foundation system which leverages a generative Artificial Intelligence (AI) model to produce classification results. The local small foundation system includes less than 10 billion parameters and is located on an edge site. The local small foundation system is customized towards the edge site. The local small foundation system produces classification results based on a query by making an inference with the generative AI model. An output guardrail module determines if the classification results can be resolved to an existing class of a plurality of existing classes. The output guardrail module also provides instructions on how to implement classification results at the endpoint management system. The output guardrail module also determines if a centralized large foundation system needs to be consulted and what to send to the centralized large foundation system.

## Claims

### Claim 1

A method of operating a local small foundation system, comprising:
receiving a first query at an endpoint management system manager of the local small foundation system, wherein:
first endpoint classification results are formed by an endpoint management system using first input data,
the endpoint management system determines that the first endpoint classification results cannot be resolved to an existing class of a plurality of existing classes,
the endpoint management system transmits the first query based on the first endpoint classification results and the first input data, and
the endpoint management system and the local small foundation system are parts of an artificial intelligence (AI) hybrid distributed inference system;
obtaining retrieval augmented generation (RAG) data from an enhanced distributed RAG system at a RAG input manager of the local small foundation system;
searching the RAG data to produce relevant data to the first query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a prompt from the documents, the objects, and the first query to use on a local small foundation model;
performing an inference on the local small foundation model to generate first local classification results using the prompt as an input;
making, by an output guardrail of the local small foundation system, a first determination that the first local classification results are resolved to an existing class of a plurality of existing classes;
in response to the first determination:
transmitting the first endpoint classification results and the first local classification results to a knowledge acquirer; and
transmitting the first local classification results to the endpoint management system wherein a first operation is performed at the endpoint management system in response to the first local classification results.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a second query at the endpoint management system manager of the local small foundation system, wherein:
second endpoint classification results are formed by the endpoint management system using second input data,
the endpoint management system determines that the second endpoint classification results cannot be resolved to an existing class of the plurality of existing classes, and
the endpoint management system transmits the second query based on the second endpoint classification results and the second input data;
obtaining second RAG data from an enhanced distributed RAG system at the RAG input manager of the local small foundation system;
searching the second RAG data to produce relevant data to the second query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a second prompt from the documents, the objects, and the second query to use on the local small foundation model;
performing a second inference on the local small foundation model to generate second local classification results using the second prompt as an input;
after generating the first local classification results, making, by the output guardrail, a second determination that the first local classification results are resolved to an existing class of a plurality of existing classes and that a centralized action is required in response to the first local classification results;
in response to the second determination:
transmitting the second local classification results from the output guardrail to a centralized large foundation system;
performing the centralized action in response to the second local classification results;
after performing the centralized action:
transmitting the second endpoint classification results, the second local classification results, and action data associated with the centralized action to the knowledge acquirer; and
transmitting the second local classification results to the endpoint management system wherein a second operation is performed at the endpoint management system in response to the second local classification results.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving a second query at the endpoint management system manager of the local small foundation system, wherein:
second endpoint classification results are formed by the endpoint management system using second input data,
the endpoint management system determines that the second endpoint classification results cannot be resolved to an existing class of the plurality of existing classes, and
the endpoint management system transmits the second query based on the second endpoint classification results and the second input data;
obtaining second RAG data from an enhanced distributed RAG system at the RAG input manager of the local small foundation system;
searching the second RAG data to produce relevant data to the second query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a second prompt from the documents, the objects, and the second query to use on the local small foundation model;
performing a second inference on the local small foundation model to generate second local classification results using the second prompt as an input;
after generating the second local classification results, making, by the output guardrail, a second determination that the second local classification results are not resolved to an existing class of the plurality of existing classes;
in response to the second determination:
transmitting the second query to a centralized large foundation system;
performing a third inference using global RAG data based on the second query to form first central classification results;
after forming the first central classification results:
performing a centralized action in response to the first central classification results;
transmitting the second endpoint classification results, the second local classification results, the first central classification results, and data associated with the centralized action to the knowledge acquirer; and
transmitting the first central classification results to the endpoint management system wherein a second operation is performed at the endpoint management system in response to the first central classification results.

### Claim 4 (depends on 1)

The method of claim 1, wherein the local small foundation system comprises:
the endpoint management system manager configured to receive the first query from the endpoint management system;
the RAG input manager configured to obtain the RAG data from the enhanced distributed RAG system;
a search manager configured to produce the relevant data from the RAG data;
a document and object retrieval manager configured to receive the documents and objects from the enhanced distributed RAG system related to the relevant data;
a prompt generation manager connected to the endpoint management system manager and the RAG input manager and configured to develop a prompt using the documents and objects that relate to the first query; and
a local small foundation model connected to the prompt generation manager, the local small foundation model comprising generative AI and configured to make the inference with the first local classification results as an output with the prompt as an input.

### Claim 5 (depends on 4)

The method of claim 4, wherein the local small foundation system further comprises:
an output guardrail module connected to the local small foundation model, wherein:
the output guardrail module determines using programed rules if the first local classification results can be resolved to an existing class of the plurality of existing classes,
the output guardrail module determines if data needs to be sent to a centralized large foundation system, and
the output guardrail is configured to transmit data to the centralized large foundation system, the endpoint management system, and the knowledge acquirer.

### Claim 6 (depends on 3)

The method of claim 3, wherein the centralized large foundation system, comprises:
a guardrail module configured to receive the second query and transmit the first central classification results to the local small foundation system;
a global federated RAG module configured to produce Global RAG data;
a centralized large foundation model connected to the guardrail module and the global federated RAG module, the centralized large foundation model comprising generative AI and configured to make the second inference with the first central classification results as an output and the Global RAG data related to the second query as an input; and
a state tracking module connected to the guardrail module and the centralized large foundation model, the state tracking module configured to perform the centralized action.

### Claim 7 (depends on 6)

The method of claim 6, wherein a tokenization and compression manager is connected to the guardrail module of the centralized large foundation system and a output guardrail module of the local small foundation system, the tokenization and compression manager configured to send lower memory index of data to be sent between the centralized large foundation system and the local small foundation system when the data to be indexed is present on both the local small foundation system and the centralized large foundation system.

### Claim 8 (depends on 1)

The method of claim 1, wherein the AI hybrid distributed inference system comprises:
a centralized system comprising a centralized large foundation system; and
an edge site of a plurality of edge sites, comprising:
the local small foundation system, and
the endpoint management system, wherein the plurality of the edge sites are connected to the centralized system and the local small foundation system is connected to the centralized large foundation system.

### Claim 9 (depends on 1)

The method of claim 1, wherein the local small foundation system has less than 10 billion parameters.

### Claim 10

A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method of operating a local small foundation system, comprising:
receiving a first query at an endpoint management system manager of the local small foundation system, wherein:
first endpoint classification results are formed by an endpoint management system using first input data,
the endpoint management system determines that the first endpoint classification results cannot be resolved to an existing class of a plurality of existing classes,
the endpoint management system transmits the first query based on the first endpoint classification results and the first input data, and
the endpoint management system and the local small foundation system are parts of an artificial intelligence (AI) hybrid distributed inference system;
obtaining retrieval augmented generation (RAG) data from an enhanced distributed RAG system at a RAG input manager of the local small foundation system;
searching the RAG data to produce relevant data to the first query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a prompt from the documents, the objects, and the first query to use on a local small foundation model;
performing an inference on the local small foundation model to generate first local classification results using the prompt as an input;
making, by an output guardrail of the local small foundation system, a first determination that the first local classification results are resolved to an existing class of a plurality of existing classes;
in response to the first determination:
transmitting the first endpoint classification results and the first local classification results to a knowledge acquirer; and
transmitting the first local classification results to the endpoint management system wherein a first operation is performed at the endpoint management system in response to the first local classification results.

### Claim 11 (depends on 10)

The non-transitory computer readable medium of claim 10, further comprising:
receiving a second query at the endpoint management system manager of the local small foundation system, wherein:
second endpoint classification results are formed by the endpoint management system using second input data,
the endpoint management system determines that the second endpoint classification results cannot be resolved to an existing class of the plurality of existing classes, and
the endpoint management system transmits the second query based on the second endpoint classification results and the second input data;
obtaining second RAG data from an enhanced distributed RAG system at the RAG input manager of the local small foundation system;
searching the second RAG data to produce relevant data to the second query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a second prompt from the documents, the objects, and the second query to use on the local small foundation model;
performing a second inference on the local small foundation model to generate second local classification results using the second prompt as an input;
after generating the first local classification results, making, by the output guardrail, a second determination that the first local classification results are resolved to an existing class of a plurality of existing classes and that a centralized action is required in response to the first local classification results;
in response to the second determination:
transmitting the second local classification results from the output guardrail to a centralized large foundation system;
performing the centralized action in response to the second local classification results;
after performing the centralized action:
transmitting the second endpoint classification results, the second local classification results, and action data associated with the centralized action to the knowledge acquirer; and
transmitting the second local classification results to the endpoint management system wherein a second operation is performed at the endpoint management system in response to the second local classification results.

### Claim 12 (depends on 10)

The non-transitory computer readable medium of claim 10, further comprising:
receiving a second query at the endpoint management system manager of the local small foundation system, wherein:
second endpoint classification results are formed by the endpoint management system using second input data,
the endpoint management system determines that the second endpoint classification results cannot be resolved to an existing class of the plurality of existing classes, and
the endpoint management system transmits the second query based on the second endpoint classification results and the second input data;
obtaining second RAG data from an enhanced distributed RAG system at the RAG input manager of the local small foundation system;
searching the second RAG data to produce relevant data to the second query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a second prompt from the documents, the objects, and the second query to use on the local small foundation model;
performing a second inference on the local small foundation model to generate second local classification results using the second prompt as an input;
after generating the second local classification results, making, by the output guardrail, a second determination that the second local classification results are not resolved to an existing class of the plurality of existing classes;
in response to the second determination:
transmitting the second query to a centralized large foundation system;
performing a third inference using global RAG data based on the second query to form first central classification results;
after forming the first central classification results:
performing a centralized action in response to the first central classification results;
transmitting the second endpoint classification results, the second local classification results, the first central classification results, and data associated with the centralized action to the knowledge acquirer; and
transmitting the first central classification results to the endpoint management system wherein a second operation is performed at the endpoint management system in response to the first central classification results.

### Claim 13 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the local small foundation system comprises:
the endpoint management system manager configured to receive the first query from the endpoint management system;
the RAG input manager configured to obtain the RAG data from the enhanced distributed RAG system;
a search manager configured to produce the relevant data from the RAG data;
a document and object retrieval manager configured to receive the documents and objects from the enhanced distributed RAG system related to the relevant data;
a prompt generation manager connected to the endpoint management system manager and the RAG input manager and configured to develop a prompt using the documents and objects that relate to the first query; and
a local small foundation model connected to the prompt generation manager, the local small foundation model comprising generative AI and configured to make the inference with the first local classification results as an output with the prompt as an input.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the local small foundation system further comprises:
an output guardrail module connected to the local small foundation model, wherein:
the output guardrail module determines using programed rules if the first local classification results can be resolved to an existing class of the plurality of existing classes,
the output guardrail module determines if data needs to be sent to a centralized large foundation system, and
the output guardrail is configured to transmit data to the centralized large foundation system, the endpoint management system, and the knowledge acquirer.

### Claim 15 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the centralized large foundation system, comprises:
a guardrail module configured to receive the second query and transmit the first central classification results to the local small foundation system;
a global federated RAG module configured to produce Global RAG data;
a centralized large foundation model connected to the guardrail module and the global federated RAG module, the centralized large foundation model comprising generative AI and configured to make the second inference with the first central classification results as an output and the Global RAG data related to the second query as an input; and
a state tracking module connected to the guardrail module and the centralized large foundation model, the state tracking module configured to perform the centralized action.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein a tokenization and compression manager is connected to the guardrail module of the centralized large foundation system and a output guardrail module of the local small foundation system, the tokenization and compression manager configured to send lower memory index of data to be sent between the centralized large foundation system and the local small foundation system when the data to be indexed is present on both the local small foundation system and the centralized large foundation system.

### Claim 17 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the AI hybrid distributed inference system comprises:
a centralized system comprising a centralized large foundation system; and
an edge site of a plurality of edge sites, comprising:
the local small foundation system, and
the endpoint management system, wherein the plurality of the edge sites are connected to the centralized system and the local small foundation system is connected to the centralized large foundation system.

### Claim 18 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the local small foundation system has less than 10 billion parameters.

### Claim 19

A local small foundation system, comprising:
a processor;
storage comprising instructions, which when executed by the processor perform a method, the method comprising:
receiving a first query at an endpoint management system manager of the local small foundation system, wherein:
first endpoint classification results are formed by an endpoint management system using first input data,
the endpoint management system determines that the first endpoint classification results cannot be resolved to an existing class of a plurality of existing classes,
the endpoint management system transmits the first query based on the first endpoint classification results and the first input data, and
the endpoint management system and the local small foundation system are parts of an artificial intelligence (AI) hybrid distributed inference system;
obtaining retrieval augmented generation (RAG) data from an enhanced distributed RAG system at a RAG input manager of the local small foundation system;
searching the RAG data to produce relevant data to the first query and ranking the relevant data;
retrieve documents and objects from the enhanced distributed RAG system corresponding to the relevant data;
generating a prompt from the documents, the objects, and the first query to use on a local small foundation model;
performing an inference on the local small foundation model to generate first local classification results using the prompt as an input;
making, by an output guardrail of the local small foundation system, a first determination that the first local classification results are resolved to an existing class of a plurality of existing classes;
in response to the first determination:
transmitting the first endpoint classification results and the first local classification results to a knowledge acquirer; and
transmitting the first local classification results to the endpoint management system wherein a first operation is performed at the endpoint management system in response to the first local classification results.

### Claim 20 (depends on 19)

The system of claim 19, wherein the local small foundation system has less than 10 billion parameters.
