# US 2023/0056947 A1 · Learning apparatus, method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0056947 A1 - Document ID: US20230056947A1 - Status: Published application - Kind: A1 - Application number: 17/652,805 - Filed: 2022-02-28 - Published: 2023-02-23 - Earliest claimed: 2021-08-18 - Assignees: Kabushiki Kaisha Toshiba - Applicants: Kabushiki Kaisha Toshiba - Inventors: Shuhei NITTA; Yasutaka Furusho; Albert Rodriguez Mulet; Atsushi Yaguchi; Akiyuki TANIZAWA - Web page: https://patentgrep.com/patent/US20230056947A1 ## CPC - G06N 3/08: Learning methods - G06N 3/096: Transfer learning - G06N 3/045: Combinations of networks - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/09: Supervised learning - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract According to one embodiment, a learning apparatus includes a processing circuit. The processing circuit acquires a first training condition and a first model trained in accordance with the first training condition, sets a second training condition used to reduce a model size of the first model, different from the first training condition, in accordance with the second training condition and based on the first model, trains a second model whose model size is smaller than that of the first model, and in accordance with a third training condition that is not the same as the second training condition and complies with the first training condition, trains a third model based on the second model. ## Claims ### Claim 1 A learning apparatus comprising a processing circuit configured to acquire a first training condition and a first machine learning model trained in accordance with the first training condition, set a second training condition used to reduce a model size of the first machine learning model, different from the first training condition, in accordance with the second training condition and based on the first machine learning model, train a second machine learning model whose model size is smaller than that of the first machine learning model, and in accordance with a third training condition that is not the same as the second training condition and complies with the first training condition, train a third machine learning model based on the second machine learning model. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the processing circuit determines necessity of training of the third machine learning model based on comparison between a first inference accuracy representing accuracy of inference concerning the first machine learning model and a second inference accuracy representing accuracy of inference concerning the second machine learning model, and upon determining that training of the third machine learning model is necessary, trains the third machine learning model. ### Claim 3 (depends on 2) The apparatus according to claim 2, wherein the processing circuit sets a plurality of second training conditions different from each other, trains a plurality of second machine learning models in accordance with the plurality of second training conditions, and determines the necessity of the third machine learning model based on comparison between a best value in second inference accuracies corresponding to the second machine learning models each having a model size not less than a reference value in the plurality of second machine learning models and the reference value based on the first inference accuracy. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein the processing circuit sets the third machine learning model in accordance with the number of nodes, the number of channels, the number of layers, and a kernel size of the second machine learning model and/or linear conversion of an input resolution, or the number of nodes, the number of channels, the number of layers, and the kernel size of the second machine learning model and/or fraction processing of one of a multiple and a multiplier of a predetermined natural number of the input resolution. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein the processing circuit initializes a learning parameter of the third machine learning model in accordance with a predetermined random number, or initializes the learning parameter by copying some trained weight coefficients of the second machine learning model. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein as the second training condition, the processing circuit sets an optimization method to Adam, introduces L2 regularization, and sets an activation function to a saturation nonlinear function, different from the first training condition. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein as the second training condition, the processing circuit adds a BN layer and introduces L1 regularization to the BN layer, different from the first training condition. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the processing circuit displays, on a display device, architectures of the first machine learning model, the second machine learning model, and/or the third machine learning model. ### Claim 9 (depends on 1) The apparatus according to claim 1, wherein the processing circuit displays, on a display device, the model sizes of the first machine learning model, the second machine learning model, and/or the third machine learning model. ### Claim 10 (depends on 1) The apparatus according to claim 1, wherein the processing circuit displays, on a display device, performance of the first machine learning model, the second machine learning model, and/or the third machine learning model. ### Claim 11 (depends on 3) The apparatus according to claim 3, wherein the processing circuit displays, on a display device, a graph that plots a plurality of points representing the inference accuracies and the model sizes of the plurality of second machine learning models. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein the processing circuit displays, on the graph, a point corresponding to the reference value and the best value and/or a region that satisfies the reference value and the best value. ### Claim 13 (depends on 12) The apparatus according to claim 12, wherein of the plurality of points, the processing circuit displays a point that is included in the region and a point that is not included in the region in different colors. ### Claim 14 (depends on 11) The apparatus according to claim 11, wherein the processing circuit plots, on the graph, a point representing the inference accuracy and the model size of the third machine learning model. ### Claim 15 (depends on 14) The apparatus according to claim 14, wherein the processing circuit displays a plurality of points corresponding to the plurality of second machine learning models and a point corresponding to the third machine learning model in different shapes, sizes, and/or colors. ### Claim 16 A learning method comprising: acquiring a first training condition and a first machine learning model trained in accordance with the first training condition; setting a second training condition used to reduce a model size of the first machine learning model, different from the first training condition; in accordance with the second training condition and based on the first machine learning model, training a second machine learning model whose model size is smaller than that of the first machine learning model; and in accordance with a third training condition that is not the same as the second training condition and complies with the first training condition, training a third machine learning model based on the second machine learning model. ### Claim 17 A non-transitory computer readable storage medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform operations comprising: acquiring a first training condition and a first machine learning model trained in accordance with the first training condition; setting a second training condition used to reduce a model size of the first machine learning model, different from the first training condition; in accordance with the second training condition and based on the first machine learning model, training a second machine learning model whose model size is smaller than that of the first machine learning model; and in accordance with a third training condition that is not the same as the second training condition and complies with the first training condition, training a third machine learning model based on the second machine learning model.