# US 2025/0148260 A1 · Data multiplexing for neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148260 A1
- Document ID: US20250148260A1
- Status: Published application
- Kind: A1
- Application number: 18/838,096
- Filed: 2023-02-14
- Published: 2025-05-08
- Earliest claimed: 2022-02-14
- Assignees: The Trustees of Princeton University
- Applicants: The Trustees of Princeton University
- Inventors: Karthik NARASIMHAN; Vishvak MURAHARI; Carlos JIMENEZ; Runzhe YANG; Ameet DESHPANDE; Yushan SU; Kai LI
- Web page: https://patentgrep.com/patent/US20250148260A1

## CPC

- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed is a technique for improving the throughput of a neural network, using multiplexing and demultiplexing of information. Specifically, the multiplexing may include receiving a plurality of inputs, generating transformed inputs by performing, via a multiplexing layer, a transformation to each input of the plurality of inputs, and combining the transformed inputs into a single compact representation of the plurality of inputs. The demultiplexing may include receiving an output from a neural network, generating a plurality of values by converting, via a demultiplexing layer, the output back into independent representations, and producing predictions for each input based on the plurality of values. Further improvements may be seen when pretraining of the neural network and/or high-throughput transformers are incorporated.

## Claims

### Claim 1

A method for improving the throughput of a neural network, comprising:
receiving a plurality of inputs;
generating transformed inputs by performing, via a multiplexing layer, a transformation to each input of the plurality of inputs; and
combining the transformed inputs into a single compact representation of the plurality of inputs.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the transformation is a fixed linear transformation.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising transmitting the single compact representation of the plurality of inputs to a neural network.

### Claim 4 (depends on 3)

The method according to claim 3, further comprising:
receiving an output from a neural network;
generating a plurality of values by converting, via a demultiplexing layer, the output back into independent representations; and
producing predictions for each input based on the plurality of values.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the demultiplexing layer utilizes a multihead neural network.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the multihead neural network is a multilayer perceptron.

### Claim 7 (depends on 4)

The method according to claim 4, wherein the demultiplexing layer uses a set of input-specific keys or indices.

### Claim 8 (depends on 7)

The method according to claim 7, wherein using a set of input-specific keys or indices comprises index embedding.

### Claim 9 (depends on 4)

The method according to claim 4, further comprising a training phase, the training phase including a warmup step comprising retrieving correct tokens and order for each position and sequence of the plurality of inputs.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the training phase further comprises pretraining the neural network after the warmup step.

### Claim 11 (depends on 10)

The method according to claim 10, wherein pretraining includes using a masked language modeling objective.

### Claim 12 (depends on 10)

The method according to claim 10, wherein the training phase further comprises finetuning the neural network after pretraining.

### Claim 13 (depends on 12)

The method according to claim 12, wherein finetuning includes training on a specific downstream task.

### Claim 14 (depends on 1)

The method according to claim 1, further comprising compressing at least one transformer between the multiplexing layer and a demultiplexing layer via pruning and/or distillation.

### Claim 15 (depends on 14)

The method according to claim 14, further comprising predicting, using a task accuracy model and a throughput model, parameters that improve throughput and meet a given accuracy budget.

### Claim 16

A non-transitory computer-readable storage medium containing instructions that, when executed, cause a processor to perform operations that include:
receiving a plurality of inputs;
generating transformed inputs by performing, via a multiplexing layer, a transformation to each input of the plurality of inputs; and
combining the transformed inputs into a single compact representation of the plurality of inputs.

### Claim at position 17

17-30. (canceled)

### Claim 31

A system, comprising:
a processor; and
a non-transitory computer-readable medium operably coupled to the processor, the non-transitory computer-readable medium containing instructions that, when executed by the processor, causes the processor to perform operations that include:
receiving a plurality of inputs;
generating transformed inputs by performing, via a multiplexing layer, a transformation to each input of the plurality of inputs; and
combining the transformed inputs into a single compact representation of the plurality of inputs.

### Claim at position 19

32-45. (canceled)

### Claim 46

A neural network apparatus, comprising:
a processor operably coupled to memory, the processor configured to generate a neural network with a plurality of layers including:
a multiplexing layer configured to perform a transformation to each received input before combining them into a single compact representation;
one or more layers defining a base neural network, the base neural network configured to receive output from the multiplexing layer; and
a demultiplexing layer configured to convert output of the base neural network back into independent representations.

### Claim at position 21

47-53. (canceled)
