# US 2026/0073225 A1 · Efficient low-rank backpropagation for vision transformer adaptation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073225 A1
- Document ID: US20260073225A1
- Status: Published application
- Kind: A1
- Application number: 19/290,686
- Filed: 2025-08-05
- Published: 2026-03-12
- Earliest claimed: 2024-08-06
- Assignees: Board of Regents, The University of Texas System
- Applicants: Board of Regents, The University of Texas System
- Inventors: Radu Marculescu; Yuedong Yang; Guihong Li; Hung-Yueh Chiang; Diana Marculescu
- Web page: https://patentgrep.com/patent/US20260073225A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system for data processing, comprising a transformer system operating on a processor and configured to receive data sets and to process the data sets to generate a transformer data structure. A low rank space projection system operating on the processor and coupled to the transformer system, the low rank space processing system configured to convert input data into low rank space input data. A matrix multiplication system operating on the processor and coupled to the low rank space projection system, the matrix multiplication system configured to receive the low rank space input data to perform a matrix multiplication process on the low rank space input data to generate low rank space output data.

## Claims

### Claim 1

A system for data processing, comprising:
a transformer system operating on a processor and configured to receive data sets and to process the data sets to generate a transformer data structure;
a low rank space projection system operating on the processor and coupled to the transformer system, the low rank space processing system configured to convert input data into low rank space input data; and
a matrix multiplication system operating on the processor and coupled to the low rank space projection system, the matrix multiplication system configured to receive the low rank space input data to perform a matrix multiplication process on the low rank space input data to generate low rank space output data.

### Claim 2 (depends on 1)

The system of claim 1 further comprising a reverse projection system operating on the processor and configured to receive the low rank space output data and to perform a reverse projection process on the low rank space output data to generate reverse projected output data.

### Claim 3 (depends on 2)

The system of claim 2, wherein the transformer system is configured to receive the reverse projected output data and to process the data sets to generate a transformer output data.

### Claim 4 (depends on 1)

The system of claim 1 wherein the a low rank space projection system comprises a low rank space variable projection system operating on the processor and coupled to the transformer system, the low rank space variable projection system configured to convert input variable data into low rank space input variable data.

### Claim 5 (depends on 1)

The system of claim 1 wherein the a low rank space projection system comprises a low rank space weight projection system operating on the processor and coupled to the transformer system, the low rank space weight projection system configured to convert input weight data into low rank space input weight data.

### Claim 6 (depends on 1)

The system of claim 1 further comprising a reverse projection variable system operating on the processor and configured to receive the low rank space output data and to perform a reverse projection variable process on the low rank space output data to generate reverse projected variable output data.

### Claim 7 (depends on 1)

The system of claim 1 further comprising a reverse projection weight system operating on the processor and configured to receive the low rank space output data and to perform a reverse projection weight process on the low rank space output data to generate reverse projected weight output data.

### Claim 8

A method for data processing, comprising:
receiving data sets at a transformer system operating on a processor and processing the data sets to generate a transformer data structure;
converting input data into low rank space input data using a low rank space projection system operating on the processor and coupled to the transformer system; and
receiving the low rank space input data at a matrix multiplication system operating on the processor and coupled to low rank space projection system performing a matrix the multiplication process on the low rank space input data to generate low rank space output data.

### Claim 9 (depends on 8)

The method of claim 8 further comprising receiving the low rank space output data at a reverse projection system operating on the processor and performing a reverse projection process on the low rank space output data to generate reverse projected output data.

### Claim 10 (depends on 9)

The method of claim 9, further comprising receiving the reverse projected output data with the transformer system and processing the data sets using the reverse projected output data to generate a transformer output data.

### Claim 11 (depends on 8)

The method of claim 8 further comprising converting input variable data into low rank space input variable data using a low rank space variable projection system operating on the processor and coupled to the transformer system.

### Claim 12 (depends on 8)

The method of claim 8 further comprising converting input weight data into low rank space input weight data using a low rank space weight projection system operating on the processor and coupled to the transformer system.

### Claim 13 (depends on 8)

The method of claim 8 further comprising receiving the low rank space output data at a reverse projection variable system operating on the processor and performing a reverse projection variable process on the low rank space output data to generate reverse projected variable output data.

### Claim 14 (depends on 8)

The method of claim 8 further comprising receiving the low rank space output data at a reverse projection weight system operating on the processor and performing a reverse projection weight process on the low rank space output data to generate reverse projected weight output data.

### Claim 15

A system for data processing, comprising:
a transformer system operating on a processor and configured to receive data sets and to process the data sets to generate a transformer data structure;
a low rank space projection system operating on the processor and coupled to the transformer system, the low rank space processing system configured to convert input data into low rank space input data;
a matrix multiplication system operating on the processor and coupled to the low rank space projection system, the matrix multiplication system configured to receive the low rank space input data to perform a matrix multiplication process on the low rank space input data to generate low rank space output data; and
a reverse projection weight system operating on the processor and configured to receive the low rank space output data and to perform a reverse projection weight process on the low rank space output data to generate reverse projected weight output data.

### Claim 16 (depends on 15)

The system of claim 15 further comprising a reverse projection system operating on the processor and configured to receive the low rank space output data and to perform a reverse projection process on the low rank space output data to generate reverse projected output data.

### Claim 17 (depends on 16)

The system of claim 16, wherein the transformer system is configured to receive the reverse projected output data and to process the data sets to generate a transformer output data.

### Claim 18 (depends on 15)

The system of claim 15 wherein the a low rank space projection system comprises a low rank space variable projection system operating on the processor and coupled to the transformer system, the low rank space variable projection system configured to convert input variable data into low rank space input variable data.

### Claim 19 (depends on 15)

The system of claim 15 wherein the a low rank space projection system comprises a low rank space weight projection system operating on the processor and coupled to the transformer system, the low rank space weight projection system configured to convert input weight data into low rank space input weight data.

### Claim 20 (depends on 15)

The system of claim 15 further comprising a reverse projection variable system operating on the processor and configured to receive the low rank space output data and to perform a reverse projection variable process on the low rank space output data to generate reverse projected variable output data.
