Confidentiality-preserving splitting of a large language model

An apparatus, comprising:

• one or more network interfaces to communicate within a local network;
• a processor coupled to the one or more network interfaces and configured to execute one or more processes;
• and
• a memory configured to store a process that is executable by the processor, the process when executed configured to:
• receive, via a user interface, a prompt for input to a large language model that is external to the local network;
• send the prompt to the large language model, wherein the large language model sends an intermediate embedding as a response to the prompt for input to one or more model layers split from the large language model that is hosted in the local network;
• receive an answer to the prompt from the one or more model layers hosted in the local network;
• and
• provide the answer to the user interface for presentation to a user.

A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:

• receiving, at a device in a local network and via a user interface, a prompt for input to a large language model that is external to the local network;
• sending, by the device, the prompt to the large language model, wherein the large language model sends an intermediate embedding as a response to the prompt for input to one or more model layers split from the large language model that is hosted in the local network;
• receiving, at the device, an answer to the prompt from the one or more model layers hosted in the local network;
• and
• providing, by the device, the answer to the user interface for presentation to a user.