# US 10,769,550 B2 · Ensemble learning prediction apparatus and method, and non-transitory computer-readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,550 B2 - Document ID: US10769550B2 - Status: Granted - Kind: B2 - Application number: 15/392,377 - Filed: 2016-12-28 - Published: 2018-05-17 - Granted: 2020-09-08 - Earliest claimed: 2016-11-17 - Assignees: Industrial Technology Research Institute - Applicants: Industrial Technology Research Institute - Inventors: Hsin-Lung Hsieh; Chuang-Hua Chueh - Web page: https://patentgrep.com/patent/US10769550B2 ## CPC - G06N 20/00: Machine learning - G06F 16/285: {Clustering or classification} - G06N 20/20: Ensemble learning - G06F 16/35: Clustering; Classification - G06F 18/211: Selection of the most significant subset of features - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The disclosure is directed to an ensemble learning prediction apparatus. The apparatus includes a loss module, a diversity module, a sample weight module, and an integrating weight module. The loss module, the diversity module and the sample weight module calculate a loss, a diversity and a sample weight, respectively. An ensemble weight is learned by an object function built by the loss, diversity and the sample weight. The integrating weight module calculates an adaptive ensemble weight by integrating the ensemble weight and previous ensemble weights at a plurality of previous time points. ## Claims ### Claim 1 An ensemble learning prediction apparatus, comprising: a loss module receiving a sample data and calculating a loss according to a first prediction result of the sample data and an actual result; a diversity module receiving the sample data and calculating a diversity between at least one hypothesis according to a second prediction result of the sample data under the at least one hypothesis; a sample weighting module calculating a correctable value of the sample data according to the first prediction result and the actual result and assigning a sample weight according to the correctable value; and an integrating weighting module, interconnected with the loss module, the diversity module and the sample weighting module, creating an object function according to the loss, the diversity and the sample weight, and training an ensemble weight by use of the object function; wherein an adaptive ensemble weight is calculated by integrating the ensemble weight and the ensemble weight obtained at previous time points, and the correctable value is obtained according to a function having a high point from which the function descends to both sides on which the high point no more appears. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the correctable value is obtained according to a difference between a target class and an erroneous class of the sample data. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein the function is a bilateral decreasing function, a quadratic function having a high point, a polynomial function having a high point, or a combination thereof. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein when at least one prediction of some of the sample data generated under the most of the at least one hypothesis is totally correct or erroneous, the correctable value which is small is provided; for other sample data, the correctable value which is large is provided. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein a product is obtained by multiplying the loss obtained by the loss module by the correctable value obtained by the sample weighting module for each sample of the sample data, a sum is obtained by summing up the products for all samples of the sample data, another product is obtained by multiplying the diversity obtained by the diversity module by a regularization parameter, and the ensemble weight obtained via the object function is obtained by subtracting the another product by the sum. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein the correctable value is calculated according to the sample weight and the first prediction result of the sample data observed at a previous time point and the actual result. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein for the adaptive ensemble weight, a larger weight is assigned to the sample data observed at a time point closer to a current time point and a smaller weight is assigned to the sample data observed at a time point farther away from the current time point. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the adaptive ensemble weight is stored in a data medium. ### Claim 9 (depends on 1) The apparatus according to claim 1, wherein the diversity is calculated from a contingency table. ### Claim 10 (depends on 9) The apparatus according to claim 9, wherein the contingency table has binary classification. ### Claim 11 (depends on 9) The apparatus according to claim 9, wherein the contingency table has multi-class classification. ### Claim 12 (depends on 1) The apparatus according to claim 1, wherein the sample weight of each sample of the sample data is different. ### Claim 13 An ensemble learning method, comprising: calculating a loss according to a first prediction result and an actual result of a sample data; calculating a diversity between at least one hypothesis according to a second prediction result of the sample data under the at least one hypothesis; calculating a correctable value of the sample data according to the first prediction result and the actual result, and assigning a sample weight according to the correctable value; and creating an object function according to the loss, the diversity and the sample weight, and training an ensemble weight by use of the object function; wherein an adaptive ensemble weight is calculated by integrating the ensemble weight and the ensemble weight obtained at previous time points, and the correctable value is obtained according to a function having a high point from which the function descends to both sides on which the high point no more appears. ### Claim 14 (depends on 13) The method according to claim 13, wherein the correctable value is obtained according to a difference between a target class and an erroneous class of the sample data. ### Claim 15 (depends on 13) The apparatus according to claim 13, wherein the function is a bilateral decreasing function, a quadratic function having a high point, a polynomial function having a high point, or a combination thereof. ### Claim 16 (depends on 13) The method according to claim 13, wherein when at least one of some of the sample data generated under most of the at least one the hypothesis is totally correct or erroneous, the correctable value which is small is provided; for other sample data, the correctable value which is large is provided. ### Claim 17 (depends on 13) The method according to claim 13, wherein the diversity is calculated from a contingency table. ### Claim 18 (depends on 17) The method according to claim 17, wherein the contingency table has binary classification. ### Claim 19 (depends on 17) The method according to claim 17, wherein the contingency table has multi-class classification. ### Claim 20 (depends on 17) The method according to claim 17, wherein the sample weight of each sample of the sample data is different. ### Claim 21 (depends on 13) The method according to claim 13, wherein the correctable value is calculated according to the sample weight and the first prediction result of the sample data observed at a previous time point and the actual result. ### Claim 22 (depends on 13) The method according to claim 13, wherein for the adaptive ensemble weight, a larger weight is assigned to the sample data observed at a time point closer to a current time point and a smaller weight is assigned to the sample data observed at a time point farther away from the current time point. ### Claim 23 (depends on 13) The method according to claim 13, wherein the adaptive ensemble weight is stored in a data medium. ### Claim 24 (depends on 13) The method according to claim 13, wherein a product is obtained by multiplying the loss by the correctable value for each sample of the sample data, a sum is obtained by summing up the products for all samples of the sample data, another product is obtained by multiplying the diversity by a regularization parameter, and the ensemble weight obtained via the object function is obtained by subtracting the another product by the sum. ### Claim 25 (depends on 13) A non-transitory computer-readable storage medium used in a computer program product comprising a plurality of instructions allocated to a computing device executing the method as claimed in claim 13.