# US 2026/0301395 A1 · Vision foundation model and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0301395 A1
- Document ID: US20260301395A1
- Status: Published application
- Kind: A1
- Application number: 19/095,939
- Filed: 2025-03-31
- Published: 2026-10-01
- Earliest claimed: 2025-03-31
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Lingjuan LYU; Weiming ZHUANG; Zhizhong LI; Chen CHEN; Sina SAJADMANESH; Jingtao LI; Jiabo HUANG; Vivek SHARMA; Peter H. STONE; Michael S. SPRANGER
- Web page: https://patentgrep.com/patent/US20260301395A1

## CPC

- G06V 10/82: using neural networks
- G06V 10/96: Management of image or video recognition tasks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Circuitry comprising a vision foundation model comprises a backbone, comprising a frozen pre-trained vision transformer ‘ViT’, and a trainable adapter configured to interact with tokens of the ViT for successive blocks of the ViT, by use of an injector and extractor; the trainable adapter is configured to generate an output for the backbone for passing as input to a decoder, and the trainable adapter is trained in conjunction with training a plurality of different decoders comprising a first set of decoders, respectively coupled to the backbone for respective tasks, to produce the completed vision foundation model based on multi-task learning.

## Claims

### Claim 1

Circuitry comprising a vision foundation model, comprising:
a backbone, comprising:
a frozen pre-trained vision transformer ‘ViT’; and
a trainable adapter configured to interact with tokens of the ViT for successive interaction blocks of the ViT, by use of an injector and extractor;
the trainable adapter configured to generate an output for the backbone for passing as input to a decoder;
and wherein
the trainable adapter is trained in conjunction with training a plurality of different decoders comprising a first set of decoders, respectively coupled to the backbone for respective tasks, to produce the completed vision foundation model based on multi-task learning.

### Claim 2 (depends on 1)

The circuitry of claim 1, in which
the trainable adapter comprises a minority of the weights or parameters of the backbone.

### Claim 3 (depends on 2)

The vision foundation model of claim 2, in which
the trainable adapter comprises fewer than one or more selected from the list consisting of:
i. 40%;
ii. 30%;
iii. 20%;
iv. 15%;
v. 10%; and
vi. 5%
of the weights or parameters of the backbone.

### Claim 4 (depends on 1)

The circuitry of claim 1, in which:
the injector of the adapter scales the respective token norms of a given ViT block to a common scale prior to processing by the adapter, and reverses the scaling when returning values to the ViT.

### Claim 5 (depends on 1)

The circuitry of claim 1, in which:
the extractor of the adapter scales the respective token norms of a given ViT block to a common scale prior to any further processing.

### Claim 6 (depends on 1)

The circuitry of claim 1, in which:
the adapter uses group normalization instead of batch normalization.

### Claim 7 (depends on 6)

The circuitry of claim 6, in which:
the adapter uses group normalization instead of batch normalization for one or more selected from the list consisting of:
i. a spatial prior module of the adapter; and
ii. a projection module of the adapter.

### Claim 8 (depends on 1)

The circuitry of claim 1, comprising:
the backbone comprising the frozen pre-trained ViT and the trained adapter, wherein the trained adapter is also frozen; and
one of the trained decoders of the first set of decoders.

### Claim 9 (depends on 8)

The circuitry of claim 8, wherein:
the trained decoder is fine-tuned to a specific task whilst the ViT and adapter remain frozen.

### Claim 10 (depends on 1)

The circuitry of claim 1, comprising:
the backbone comprising the frozen pre-trained ViT and the trained adapter, wherein the trained adapter is also frozen; and
a new decoder for a specific task; wherein
the new decoder is trained for the specific task whilst the ViT and adapter remain frozen.

### Claim 11 (depends on 1)

Circuitry comprising a vision application, comprising:
the circuitry of claim 1, wherein the ViT and trained adapter are frozen; and
at least a first trained decoder.

### Claim 12 (depends on 11)

The circuitry of claim 11, in which
the vision application is configured to compute a fingerprint of at least one component of the vision foundation model;
the application is configured to pass the fingerprint to an authentication system; and
the application is configured to use the vision foundation model if it is confirmed by the authentication system.

### Claim 13 (depends on 11)

A device operable to perform a vision task, comprising:
an input operable to receive image data;
a processor configured to execute the vision application of claim 11; and
an output operable to provide data from the vision application for subsequent use.

### Claim 14

A method of generating a vision foundation model, comprising the steps of:
combining as a backbone a frozen pre-trained vision transformer ‘ViT’ and a trainable adapter configured to interact with tokens of the ViT for successive interaction blocks of the ViT, by use of an injector and extractor;
training the trainable adapter in conjunction with training a plurality of different decoders comprising a first set of decoders, respectively coupled to the backbone for respective tasks, to produce the completed vision foundation model based on multi-task learning; wherein
the trainable adapter is configured to generate an output for the backbone for passing as input to a decoder.

### Claim 15 (depends on 14)

A method of generating a task-specific vision model, comprising:
fixing the parameters of the backbone of the completed vision foundation model of claim 14, including the trained adapter as well as the pre-trained ViT; and
training a decoder to perform the specific task whilst the parameters of the backbone remain fixed.

### Claim 16 (depends on 1)

A non-transitory, computer readable storage medium containing a computer program comprising computer executable instructions that when executed by a computer system, cause the computer system to perform the method of claim 1.

### Claim 17

A server configured to:
receive a first request for a backbone comprising both of a vision transformer ‘ViT’ and an adapter from a set of differently trained adapters, the request including identification information for first tasks according to which each of the set of adapters have been trained;
provide via interface circuitry, the backbone;
receive a second request for a decoder adapted to perform a second task when appended to the backbone;
provide, via the interface circuitry the decoder adapted to perform a second task.
