# US 2019/0378015 A1 · Training a machine learning model with limited training data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378015 A1 - Document ID: US20190378015A1 - Status: Published application - Kind: A1 - Application number: 16/438,326 - Filed: 2019-06-11 - Published: 2019-12-12 - Earliest claimed: 2018-06-11 - Assignees: The Regents of the University of California - Applicants: The Regents of the University of California - Inventors: Fang Lin; Mohammad Ghasemzadeh; Bita Darvish Rouhani; Farinaz Koushanfar - Web page: https://patentgrep.com/patent/US20190378015A1 ## CPC - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/09: Supervised learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method may include a transforming a trained machine learning model including by replacing at least one layer of the trained machine learning model with a dictionary matrix and a coefficient matrix. The dictionary matrix and the coefficient matrix may be formed by decomposing a weight matrix associated with the at least one layer of the trained machine learning model. A product of the dictionary matrix and the coefficient matrix may form a reduced-dimension representation of the weight matrix associated with the at least one layer of the trained machine learning model. The transformed machine learning model may be deployed to a client. Related systems and computer program products are also provided. ## Claims ### Claim 1 A system, comprising: at least one processor; and at least one memory including program code which when executed by the at least one processor provides operations comprising: transforming a trained machine learning model including by replacing at least one layer of the trained machine learning model with a dictionary matrix and a coefficient matrix, the dictionary matrix and the coefficient matrix formed by at least decomposing a weight matrix associated with the at least one layer of the trained machine learning model, and a product of the dictionary matrix and the coefficient matrix comprising a reduced-dimension representation of the weight matrix associated with the at least one layer of the trained machine learning model; and deploying, to a client, the transformed machine learning model. ### Claim 2 (depends on 1) The system of claim 1, wherein the at least one layer of the trained machine learning model comprise a fully-connected layer and/or a convolution layer of the trained machine learning model. ### Claim 3 (depends on 1) The system of claim 1, further comprising: training, based at least a first training data, a machine learning model to generate the trained machine learning model, the training of the machine learning model including adjusting one or more weights included in the weight matrix to at least minimize an error in an output of the machine learning model. ### Claim 4 (depends on 3) The system of claim 3, wherein the transformed machine learning model is updated, at the client, based at least on a second training data, and wherein the update to the transformed machine learning model is limited to a first portion of the transformed machine learning model while a second portion of the transformed machine learning model remains unaffected by the update. ### Claim 5 (depends on 4) The system of claim 4, wherein the update to the transformed machine learning model is limited to one or more coefficient matrices associated with one or more layers of the transformed machine learning model. ### Claim 6 (depends on 4) The system of claim 4, wherein the update to the transformed machine learning model is limited to a single dictionary matrix associated with a single layer of the transformed machine learning model. ### Claim 7 (depends on 6) The system of claim 6, wherein the update to the transformed machine learning model is further limited to inserting, into the single dictionary matrix, one or more rows corresponding to the second training data without modifying any existing portions of the single dictionary matrix. ### Claim 8 (depends on 6) The system of claim 6, wherein the single layer of the trained machine learning model comprises a final layer of the transformed machine learning model prior an output of the transformed machine learning model. ### Claim 9 (depends on 1) The system of claim 1, wherein the decomposing of the weight matrix includes increasing a length of the dictionary matrix until a decomposition error associated with the at least one layer of the trained machine learning model is below a threshold value, and wherein the decomposition error comprises a difference between the weight matrix and the reduced-dimension representation of the weight matrix comprising the product of the dictionary matrix and the coefficient matrix. ### Claim 10 (depends on 9) The system of claim 9, wherein a first layer of the trained machine learning model is associated with a first threshold value, and wherein a second layer of the trained machine learning model is associated with a second threshold value. ### Claim 11 (depends on 10) The system of claim 10, wherein the first threshold value and/or the second threshold value are adjusted based at least on one or more resource constraints associated with the client. ### Claim 12 (depends on 10) The system of claim 10, wherein the first threshold value and/or the second threshold value are adjusted based at least on a target accuracy for the transformed machine learning model. ### Claim 13 (depends on 1) The system of claim 1, wherein each column in the weight matrix comprises a linear combination of one or more columns from the dictionary matrix as determined by the coefficient matrix. ### Claim 14 (depends on 1) The system of claim 1, wherein the trained machine learning model comprises a neural network. ### Claim 15 A method, comprising: transforming a trained machine learning model including by replacing at least one layer of the trained machine learning model with a dictionary matrix and a coefficient matrix, the dictionary matrix and the coefficient matrix formed by at least decomposing a weight matrix associated with the at least one layer of the trained machine learning model, and a product of the dictionary matrix and the coefficient matrix comprising a reduced-dimension representation of the weight matrix associated with the at least one layer of the trained machine learning model; and deploying, to a client, the transformed machine learning model. ### Claim 16 (depends on 15) The method of claim 15, further comprising: training, based at least a first training data, a machine learning model to generate the trained machine learning model, the training of the machine learning model including adjusting one or more weights included in the weight matrix to at least minimize an error in an output of the machine learning model. ### Claim 17 (depends on 16) The method of claim 16, wherein the transformed machine learning model is updated, at the client, based at least on a second training data, and wherein the update to the transformed machine learning model is limited to a first portion of the transformed machine learning model while a second portion of the transformed machine learning model remains unaffected by the update. ### Claim 18 (depends on 17) The method of claim 17, wherein the update to the transformed machine learning model is limited to one or more coefficient matrices associated with one or more layers of the transformed machine learning model. ### Claim 19 (depends on 17) The method of claim 17, wherein the update to the transformed machine learning model is limited to a single dictionary matrix associated with a single layer of the transformed machine learning model, and wherein the update to the transformed machine learning model is further limited to inserting, into the single dictionary matrix, one or more rows corresponding to the second training data without modifying any existing portions of the single dictionary matrix. ### Claim 20 A non-transitory computer-readable medium storing instructions, which when executed by at least one data processor, result in operations comprising: transforming a trained machine learning model including by replacing at least one layer of the trained machine learning model with a dictionary matrix and a coefficient matrix, the dictionary matrix and the coefficient matrix formed by at least decomposing a weight matrix associated with the at least one layer of the trained machine learning model, and a product of the dictionary matrix and the coefficient matrix comprising a reduced-dimension representation of the weight matrix associated with the at least one layer of the trained machine learning model; and deploying, to a client, the transformed machine learning model.