# US 2025/0156684 A1 · Parameter-Efficient Adapter for an Artificial Intelligence System > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0156684 A1 - Document ID: US20250156684A1 - Status: Published application - Kind: A1 - Application number: 18/587,052 - Filed: 2024-02-26 - Published: 2025-05-15 - Earliest claimed: 2023-11-10 - Assignees: Mitsubishi Electric Research Laboratories, Inc. - Applicants: Mitsubishi Electric Research Laboratories, Inc. - Inventors: Jing Liu; Toshiaki Koike-Akino; Pu Wang; Matthew Brand; Ye Wang; Kieran Parsons - Web page: https://patentgrep.com/patent/US20250156684A1 ## CPC - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06F 40/30: Semantic analysis - G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/096: Transfer learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06V 10/82: using neural networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An adapter to a base model of an artificial intelligence (AI) system is disclosed. The adapter includes a connector to connect the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model. The adapter includes a non-linear modifier to modify the data received from the base model non-linearly before returning the modified portion of the data back to the base model, and an AI trainer to tune the non-linear modifier of the adapter by propagating training data through the base model and the adapter and updating weights of the non-linear modifier of the adapter for given weights of the base model to optimize a loss function. Further, weight matrices for the base model and the adapter are jointly constructed by an additional module, which efficiently uses a pool of parameters to allocate to save memory requirement for adaptation of the AI system. ## Claims ### Claim 1 An adapter to a base model of an artificial intelligence (AI) system, the adapter comprising: a connector configured to connect the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model; a non-linear modifier configured to modify the data received from the base model non-linearly before returning the modified portion of the data back to the base model; and an AI trainer configured to tune the non-linear modifier of the adapter by propagating training data through the base model and the adapter and updating weights of the non-linear modifier of the adapter for given weights of the base model to optimize a loss function. ### Claim 2 (depends on 1) The adapter of claim 1, wherein the non-linear modifier includes multiple paths formed by multiple AI architectures of data transformation, each of the paths is either a linear path configured to modify the received data linearly or a non-linear path configured to modify the received data non-linearly, wherein an AI architecture of the linear path modifies the received data linearly using one or multiple weight matrices, wherein an AI architecture of the non-linear path modifies the received data linearly using one or multiple weight matrices and modifies the received data non-linearly using one or multiple non-linear functions, and wherein the non-linear modifier includes at least one non-linear path. ### Claim 3 (depends on 2) The adapter of claim 2, wherein the non-linear modifier includes multiple non-linear paths using different non-linear functions, different arrangements of the same non-linear functions with respect to the weight matrices, or both. ### Claim 4 (depends on 3) The adapter of claim 3, wherein the non-linear modifier includes at least one linear path. ### Claim 5 (depends on 3) The adapter of claim 3, wherein the multiple non-linear paths include the same weight matrices. ### Claim 6 (depends on 3) The adapter of claim 3, wherein the multiple non-linear paths share at least some weights. ### Claim 7 (depends on 3) The adapter of claim 3, wherein weights in the weight matrices of the multiple non-linear paths come from a common pool of parameters, such that to tune the non-linear modifier, the AI trainer updates the common pool of parameters. ### Claim 8 (depends on 2) The adapter of claim 2, wherein the non-linear modifier comprises: a path splitter configured to direct the received data to each of the paths; and a path combiner configured to combine outputs of each of the paths to submit a combined output back to the base model. ### Claim 9 (depends on 8) The adapter of claim 8, wherein the path combiner combines the outputs using an operation including one or a combination of: an identity, a duplication, a permutation, a polynomial basis expansion, a Fourier basis expansion, an addition, a multiplication, a division, a subtraction, a modulo-addition, a modulo-product, a Kronecker product, a Kronecker sum, a Hadamard product, a concatenation, a log-sum-exp, an affine transform, a convolution, randomization, a normalization, a nonlinear activation operation, and variants thereof. ### Claim 10 (depends on 9) The adapter of claim 9, wherein the operation of the path combiner includes a parameter learned during the tuning of the AI trainer. ### Claim 11 (depends on 2) The adapter of claim 2, wherein the AI architecture of the non-linear path includes a bottleneck configuration of multiple layers. ### Claim 12 (depends on 1) The adapter of claim 1, wherein the AI trainer is further configured to approximate the base model and train the adapter and to achieve a common objective. ### Claim 13 (depends on 2) The adapter of claim 2, wherein the AI trainer further comprises a weight constructor comprising a pool of parameters and a set of hyperparameters forming rules of propagation of the parameters from the pool of parameters into the weight matrices of the multiple paths of the non-linear modifier, and wherein the weight constructor is configured to: update the pool of parameters and the set of hyperparameters for given weights of the base model; and propagate the parameters from the pool of parameters to different weight matrices of different paths according to the trained hyperparameters. ### Claim 14 (depends on 1) The adapter of claim 1, wherein the AI trainer updates weights of the adapter for frozen weights of the base model. ### Claim 15 (depends on 1) The adapter of claim 1, wherein weight matrices of the adapter have lower dimensions than weight matrices of the base model. ### Claim 16 (depends on 1) The adapter of claim 1, wherein weight matrices of the adapter are coming from a pool of parameters updated by the AI trainer during the tuning, and wherein a number of parameters in the pool of parameters is more than 1000 times less than a number of parameters of the base model. ### Claim 17 A method for adapting a base model of an artificial intelligence (AI) system using an adapter, the method comprising: connecting, using a connector of the adapter, the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model; modifying, using a non-linear modifier of the adapter, the data received from the base model non-linearly before returning the modified portion of the data back to the base model; and tuning, using an AI trainer of the adapter, the non-linear modifier of the adapter by propagating training data through the base model and the adapter and updating weights of the non-linear modifier of the adapter for given weights of the base model to optimize a loss function. ### Claim 18 (depends on 17) The method of claim 17, wherein the non-linear modifier includes multiple paths formed by multiple AI architectures of data transformation, each of the paths is either a linear path configured to modify the received data linearly or a non-linear path configured to modify the received data non-linearly, wherein an AI architecture of the linear path modifies the received data linearly using one or multiple weight matrices, wherein an AI architecture of the non-linear path modifies the received data non-linearly using one or multiple weight matrices and modifies the received data non-linearly using one or multiple non-linear functions, and wherein the non-linear modifier includes at least one non-linear path. ### Claim 19 (depends on 18) The method of claim 18, wherein the AI trainer further comprises a weight constructor comprising a pool of parameters and a set of hyperparameters forming rules of propagation of the parameters from the pool of parameters into the weight matrices of the multiple paths of the non-linear modifier, and wherein the method further comprises: updating, using the weight constructor, the pool of parameters and the set of hyperparameters for given weights of the base model; and propagating, using the weight constructor, the parameters from the pool of parameters to different weight matrices of different paths according to the trained hyperparameters. ### Claim 20 A non-transitory computer readable storage medium embodied thereon a program executable by a processor for performing a method, the method comprising: connecting an adapter to a base model of an artificial intelligence (AI) system such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model; modifying the data received from the base model non-linearly before returning the modified portion of the data back to the base model; and tuning the non-linear modifier of the adapter by propagating training data through the base model and the adapter and updating weights of the non-linear modifier of the adapter for given weights of the base model to optimize a loss function.