# US 11,443,169 B2 · Adaptation of model for recognition processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,169 B2 - Document ID: US11443169B2 - Status: Granted - Kind: B2 - Application number: 15/048,318 - Filed: 2016-02-19 - Published: 2017-08-24 - Granted: 2022-09-13 - Earliest claimed: 2016-02-19 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Gakuto Kurata - Web page: https://patentgrep.com/patent/US11443169B2 ## CPC - G06N 3/047: Probabilistic or stochastic networks - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06N 3/096: Transfer learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A computer implemented method for adapting a model for recognition processing to a target-domain is disclosed. The method includes preparing a first distribution in relation to a part of the model, in which the first distribution is derived from data of a training-domain for the model. The method also includes obtaining a second distribution in relation to the part of the model by using data of the target-domain. The method further includes tuning one or more parameters of the part of the model so that difference between the first and the second distributions becomes small. ## Claims ### Claim 1 A computer implemented method for adapting a model for recognition processing to a target-domain, performed by a computer device, the method comprising: preparing a first distribution in relation to a part of the model, the first distribution being derived from data of a training-domain for the model; obtaining a second distribution in relation to the part of the model by using data of the target-domain; and tuning one or more parameters of the of the model based on the data of the target-domain and one or more output distributions from another part of the model so that a difference between the first and the second distributions becomes small by calculating a change in the one or more parameters with gradient of a loss function that represents the difference between the first and the second distributions and updating the one or more parameters based on the calculated change, the one or more parameters comprising weight matrices for a plurality of localized filters for convolution, wherein each of the target-domain and the training-domain data is split into utterance regions and silence regions and the utterance regions and the silence regions are both separately evaluated and results from each of the regions are combined for use in additional parameter tuning. ### Claim 2 (depends on 1) The method of claim 1, wherein the model includes a neural network having an input layer and a plurality of layers on top of the input layer, the part being one or more lower layers among the plurality of the layers and the input layer, the first and second distributions being output distributions from the part by feeding the data into the input layer from the training-domain and the target-domain, respectively. ### Claim 3 (depends on 2) The method of claim 2, wherein the part of the model is one or more lowest layers among the plurality of the layers and the input layer. ### Claim 4 (depends on 2) The method of claim 2, wherein the part includes a convolutional layer and a subsampling layer on top of the convolutional layer, the first and second distributions being output distributions from the subsampling layer. ### Claim 5 (depends on 1) The method of claim 1, wherein the data of the training-domain and the data of the target-domain both include silence and utterance regions, the tuning including: calculating at least one difference between the first and the second distributions for both the silence and utterance regions. ### Claim 6 (depends on 1) The method of claim 1, wherein the data of the training-domain and the data of the target-domain are both split into a plurality of classes in an unsupervised manner, the plurality of classes including class representing utterance regions and class representing silence regions, both the first and the second distributions including a distribution for each class, the tuning including: calculating at least one difference between the first and the second distributions for each class; and combining the at least one calculated difference over the plurality of the classes. ### Claim 7 (depends on 1) The method of claim 1, wherein the data of the training-domain and the data of the target-domain both include utterance regions split from whole data including the utterance regions and silence regions in an unsupervised manner, the tuning including: calculating at least one difference between the first and the second distributions for the utterance regions. ### Claim 8 (depends on 1) The method of claim 1, wherein the data of the training-domain and the data of the target-domain are both split into a plurality of classes by utilizing supervised information, the plurality of the classes including each class representing a phone, a group of phones, or a group of multi-phones, both the first and the second distributions including a distribution for each class, the tuning including: calculating at least one difference between the first and the second distributions for each class; and combining the at least one calculated difference over the plurality of the classes. ### Claim 9 (depends on 1) The method of claim 1, wherein the obtaining the second distribution and the tuning the one or more parameters are iterated until the difference meets a predetermined condition. ### Claim 10 (depends on 1) The method of claim 1, the difference is calculated by means of square error or cross-entropy and a loss function is set using the means square error or the cross-entropy. ### Claim 11 (depends on 1) The method of claim 1, wherein the method further comprises: performing an additional training to the tuned model by using training data with a label from the target-domain in a supervised manner. ### Claim 12 (depends on 1) The method of claim 1, wherein the adapted model provides an acoustic model for speech recognition processing. ### Claim 13 (depends on 1) The method of claim 1, wherein the preparing, the obtaining and the tuning are performed in a cloud computing environment. ### Claim 14 (depends on 1) The method of claim 1, wherein the preparing, the Obtaining and the tuning are performed by one or more computer devices. ### Claim 15 A computer system for adapting a model for recognition processing to a target-domain, by executing program instructions, the computer system comprising: a memory tangibly storing the program instructions; a processor in communications with the memory, wherein the computer system is configured to: prepare a first distribution in relation to a part of the model, the first distribution being derived from data of a training-domain for the model; obtain a second distribution in relation to the part of the model by using data of the target-domain; and tune one or more parameters of the part of the model based on the data of the target-domain and one or more output distributions from another part of the model so that a difference between the first and the second distributions becomes small by calculating a change in the one or more parameters with gradient of a loss function that represents the difference between the first and the second distributions and updating the one or more parameters based on the calculated change, the one or more parameters comprising weight matrices for a plurality of localized filters for convolution, wherein each of the target-domain and the training-domain data is split into utterance regions and silence regions and the utterance regions and the silence regions are both separately evaluated and results from each of the regions are combined for use in additional parameter tuning. ### Claim 16 (depends on 15) The computer system of claim 15, wherein the model includes a neural network having an input layer and a plurality of layers on top of the input layer, the part being one or more lower layers among the plurality of the layers and the input layer, the first and second distributions being output distributions from the part by feeding the data into the input layer from the training-domain and the target-domain, respectively. ### Claim 17 (depends on 15) The computer system of claim 15, wherein the data of the training-domain and the data of the target-domain both are split into a plurality of classes, both the first and the second distributions including a distribution for each class, the computer system being further configured to: calculate at least one difference between the first and the second distributions for each class; and combine the at least one calculated difference over the plurality of the classes. ### Claim 18 (depends on 15) The computer system of claim 15, wherein the computer system is further configured to: determine whether the difference meets a predetermined condition; and obtain the second distribution and tune the one or more parameters repeatedly in response to determining that the difference does not meet the predetermined condition. ### Claim 19 A computer program product for adapting a model for recognition processing to a target-domain, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising: preparing a first distribution in relation to a part of the model, the first distribution being derived from data of a training-domain for the model; obtaining a second distribution in relation to the part of the model by using data of the target-domain; and tuning one or more parameters of the part of the model based on the data of the target-domain and one or more output distributions from another part of the model so that a difference between the first and the second distributions becomes small by calculating a change in the one or more parameters with gradient of a loss function that represents the difference between the first and the second distributions and updating the one or more parameters based on the calculated change, the one or more parameters comprising weight matrices for a plurality of localized filters for convolution, wherein each of the target-domain and the training-domain data is split into utterance regions and silence regions and the utterance regions and the silence regions are both separately evaluated and results from each of the regions are combined for use in additional parameter tuning. ### Claim 20 (depends on 9) The computer program product of claim 9, wherein the model includes a neural network having an input layer and a plurality of layers on top of the input layer, the part being one or more lower layers among the plurality of the layers and the input layer, the first and second distributions being output distributions from the part by feeding the data into the input layer from the training-domain and the target-domain, respectively. ### Claim 21 (depends on 19) The computer program product of claim 19, wherein the data of the training-domain and the data of the target-domain both are split into a plurality of classes, both the first and the second distributions including a distribution for each class, the tuning including: calculating at least one difference between the first and the second distributions for each class; and combining the at least one calculated difference over the plurality of e classes. ### Claim 22 A computer implemented method for adapting a neural network to a target-domain, performed by a processor, the method comprising: preparing a first output distribution from one or more lower layers of the neural network on a memory operably coupled to the processor, the first output distribution being derived from data of a training-domain for the neural network; calculating a second output distribution from the one or more lower layers of the neural network by feeding data of the target-domain into the neural network; and tuning one or more parameters of the one or more lower layers of the neural network based on the data of the target-domain and one or more output distributions from another part of the model by calculating a change in the one or more parameters so as to minimize a difference between the first and the second output distributions based on the first and the second output distributions stored on the memory by calculating a change in the one or more parameters with gradient of a loss function that represents the difference between the first and the second distributions and updating the one or more parameters based on the calculated change, the one or more parameters comprising weight matrices for a plurality of localized filters for convolution, wherein each of the target-domain and the training-domain data is split into utterance regions and silence regions and the utterance regions and the silence regions are both separately evaluated and results from each of the regions are combined for use in additional parameter tuning. ### Claim 23 (depends on 22) The method of claim 22, wherein the one or more lower layers are lowest layers including a convolutional layer on an input layer of the neural network and a subsampling layer on top of the convolutional layer, the first and second output distributions being output distributions from the subsampling layer by feeding the data into the input layer from the training-domain and the target-domain, respectively. ### Claim 24 A computer system for adapting a model for recognition processing to a target-domain, the computer system comprising: a preparing module configured to prepare a first distribution in relation to a part of the model, the first distribution being derived from data of a training-domain for the model; an obtaining module configured to obtain a second distribution in relation to the part of the model by using data of the target-domain; and a tuning module configured to tune one or more parameters of the part of the model based on the data of the target-domain and one or more output distributions from another part of the model so that a difference between the first and the second distributions becomes small by calculating a change in the one or more parameters with gradient of a loss function that represents the difference between the first and the second distributions and updating the one or more parameters based on the calculated change, the one or more parameters comprising weight matrices for a p orality of localized filters for convolution, wherein each of the target-domain and the training-domain data is split into utterance regions and silence regions and the utterance regions and the silence regions are both separately evaluated and results from each of the regions are combined for use in additional parameter tuning. ### Claim 25 (depends on 24) The computer system of claim 24, wherein the computer system further comprises: a splitting module configured to split the data of the training-domain and the data of the target-domain into a plurality of classes; and/or an additional training module configured to perform an additional training to the tuned model by using training data with a label from the target-domain in a supervised manner.