# US 2023/0069342 A1 · Computer system and method of determining model switch timing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0069342 A1 - Document ID: US20230069342A1 - Status: Published application - Kind: A1 - Application number: 17/675,485 - Filed: 2022-02-18 - Published: 2023-03-02 - Earliest claimed: 2021-08-27 - Assignees: Hitachi, Ltd. - Applicants: Hitachi, Ltd. - Inventors: Keita Mizushina; Satoshi Katsunuma; Keiro Muro - Web page: https://patentgrep.com/patent/US20230069342A1 ## CPC - G06F 11/076: {by exceeding a count or rate limit, e.g. word- or bit count limit} - G06F 11/3419: {by assessing time} - G06F 11/3457: {Performance evaluation by simulation} - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A computer system that detects an abnormality based on time series data, including: an abnormality diagnosis unit that diagnoses an abnormality of the time series data from a machine learning model created based on learning data; a model degradation detection unit that detects degradation in the machine learning model; a learning curve estimation unit that estimates a learning curve and predicts a number of errors per unit time; a model switch cost calculation unit that calculates a number of errors per unit time of a model in operation, a number of errors per unit time of a switch candidate model, a first total cost and a second total cost; and a model switch time prediction unit that compares the first total cost with the second total cost to calculate switch time of a machine learning model. ## Claims ### Claim 1 A computer system that detects an abnormality based on time series data, the system comprising: an abnormality diagnosis unit that diagnoses an abnormality of the time series data from a machine learning model created based on learning data; a model degradation detection unit that detects degradation in the machine learning model; a learning curve estimation unit that estimates a learning curve of the machine learning model and predicts a number of errors per unit time using the learning curve; a model switch cost calculation unit that calculates a number of errors per the unit time of a model in operation that is a machine learning model presently being used, a number of errors per the unit time of a switch candidate model that is a switch candidate of a model in operation, a first total cost when a machine learning model is switched at given first time based on error cost information that defines a cost per error, and a second total cost when a machine learning model is switched at given second time; and a model switch time prediction unit that compares the first total cost with the second total cost to calculate switch time of a machine learning model. ### Claim 2 (depends on 1) The computer system according to claim 1, wherein: the learning curve estimation unit obtains a number of samples of data learnt by the switch candidate model when a learning curve is estimated; and the learning curve estimation unit calculates a learning curve of the switch candidate model based on data indicating a relationship between a number of samples of learnt data in a machine learning model and a number of errors at time of operation used in past. ### Claim 3 (depends on 1) The computer system according to claim 1, wherein: the learning curve estimation unit obtains a ratio of context learned by the switch candidate model when a learning curve is estimated; and the learning curve estimation unit calculates a learning curve of the switch candidate model based on data indicating a relationship between ratio of learned context and number of errors at time of operation in machine learning model used in past. ### Claim 4 (depends on 1) The computer system according to claim 1, wherein the learning curve estimation unit calculates a learning curve of the switch candidate model using a learning curve estimation model created from a learning curve when a model is created in past when a learning curve is estimated. ### Claim 5 (depends on 1) The computer system according to claim 1, wherein the model degradation detection unit detects a change in a distribution of the time series data. ### Claim 6 (depends on 1) The computer system according to claim 1, wherein the model switch cost calculation unit further calculates the first total cost and the second total cost based on a model update cost necessary to switch the machine learning model. ### Claim 7 (depends on 6) The computer system according to claim 6, wherein the model update cost is a fixed value, and assuming that the machine learning model is switched between time t1 and time tx between which is a predetermined duration, the first total cost and the second total cost are calculated. ### Claim 8 (depends on 7) The computer system according to claim 7, wherein: the time t1 is time based on time at which the model degradation detection unit detects degradation in the machine learning model; and the time tx is time specified by a user. ### Claim 9 (depends on 8) The computer system according to claim 8, wherein upon receiving a result of the model switch cost calculation unit, the model switch time prediction unit predicts time T at which the machine learning model is switchable at a minimum total cost. ### Claim 10 A method of determining model switch timing comprising: in a computer system that diagnoses an abnormality of time series data from a machine learning model created based on learning data, when timing of switching of the machine learning model is determined, a model degradation detecting step of detecting degradation in the machine learning model; a learning curve estimation step of estimating a learning curve of the machine learning model to predict a number of errors per unit time using the learning curve; a model switch cost calculating step of calculating a number of errors per the unit time of a model in operation that is a machine learning model presently being used, a number of errors per the unit time of a switch candidate model that is a switch candidate of a model in operation, a first total cost when a machine learning model is switched at given first time based on error cost information that defines a cost per error, and a second total cost when a machine learning model is switched at given second time; and a model switch time predicting step of comparing the first total cost with the second total cost to calculate switch time of a machine learning model. ### Claim 11 (depends on 10) The method of determining model switch timing according to claim 10, wherein in the learning curve estimation step, a learning curve of the switch candidate model is calculated based on at least one of first past data indicating a relationship between a number of samples of learnt data and a number of errors at time of operation in a machine learning model used in past and second past data indicating a relationship between a ratio of learned context and a number of errors at time of operation in a machine learning model used in past. ### Claim 12 (depends on 11) The method of determining model switch timing according to claim 11, wherein in the learning curve estimation step, a learning curve of the switch candidate model is calculated using a learning curve estimation model created from a learning curve when a model is created in past when a learning curve is estimated. ### Claim 13 (depends on 10) The method of determining model switch timing according to claim 10, wherein in the model switch cost calculating step, the first total cost and the second total cost are further calculated based on a model update cost necessary to switch the machine learning model. ### Claim 14 (depends on 13) The method of determining model switch timing according to claim 13, wherein the model update cost is a fixed value, and assuming that the machine learning model is switched between time t1 and time tx between which is a predetermined duration, the first total cost and the second total cost are calculated. ### Claim 15 (depends on 14) The method of determining model switch timing according to claim 14, wherein: the time t1 is time based on time at which degradation in the machine learning model is detected in the model degradation detecting step; and the time tx is time specified by a user.