# US 10,375,098 B2 · Anomaly detection based on relationships between multiple time series > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,098 B2 - Document ID: US10375098B2 - Status: Granted - Kind: B2 - Application number: 15/420,737 - Filed: 2017-01-31 - Published: 2018-08-02 - Granted: 2019-08-06 - Earliest claimed: 2017-01-31 - Applicants: Splunk Inc. - Inventors: Adam Jamison Oliner; Jonathan La; Colleen Kinross; Hongyang Zhang; Jacob Leverich; Shang Cai; Mihai Ganea; Alex Cruise; Toufic Boubez; Manish Sainani - Web page: https://patentgrep.com/patent/US10375098B2 ## CPC - H04L 63/1425: {Traffic logging, e.g. anomaly detection} - B06B 1/0644: {using a single piezoelectric element (B06B1/0688 takes precedence)} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G08B 6/00: Tactile signalling systems, e.g. tactile personal calling systems - H02N 2/04: Constructional details - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - B06B: METHODS OR APPARATUS FOR GENERATING OR TRANSMITTING MECHANICAL VIBRATIONS OF INFRASONIC, SONIC, OR ULTRASONIC FREQUENCY, {e.g.} FOR PERFORMING MECHANICAL WORK IN GENERAL (for particular applications, see the relevant subclasses, e.g. B07B1/40, B23Q17/12, B24B31/06; measurement of mechanical vibrations G01H; in direction finding, locating, distance or velocity measuring G01S; {generating seismic energy G01V1/02}; control of mechanical vibrations in general G05D; sound-producing devices, e.g. bells, sirens, whistles G10K, {e.g. methods or devices for transmitting, conducting, or directing sound in general G10K11/00}; generation of electrical oscillations H03B; electromechanical resonators in general H03H; electromechanical transducers {for communication techniques, e.g. microphones, speakers} H04R) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - H02N: ELECTRIC MACHINES NOT OTHERWISE PROVIDED FOR ## Abstract In some implementations, sequences of time series values determined from machine data are obtained. Each sequence corresponds to a respective time series. A plurality of predictive models is generated for a first time series from the sequences of time series values. Each predictive model is to generate predicted values associated with the first time series using values of a second time series. For each of the plurality of predictive models, an error is determined between the corresponding predicted values and values associated with the first time series. A predictive model is selected for anomaly detection based on the determined error of the predictive model. Transmission is caused of an indication of an anomaly detected using the selected predictive model. ## Claims ### Claim 1 A computer-implemented method comprising: obtaining sequences of time series values determined from raw machine data, each sequence corresponding to a respective time series, wherein the raw machine data is produced by one or more components within an information technology or security environment and reflects activity within the information technology or security environment; generating a plurality of predictive models for a first time series from the sequences of time series values, each predictive model to generate predicted values associated with the first time series using time series values corresponding to a second time series; evaluating one or more characteristics of the plurality of predictive models; and automatically selecting a predictive model from the plurality of predictive models for anomaly detection based on the evaluating of the one or more characteristics. ### Claim 2 (depends on 1) The method of claim 1, further comprising applying the selected predictive model to subsequently received time series values to detect an anomaly. ### Claim 3 (depends on 1) The method of claim 1, further comprising in response to detection of an anomaly using the selected predictive mode, causing transmission of an indication of the anomaly. ### Claim 4 (depends on 1) The method of claim 1, wherein the evaluating comprises, for each of the plurality of predictive models, determining an error between the corresponding predicted values and values associated with the first time series, wherein the selecting of the predictive model is based on the error of the predictive model. ### Claim 5 (depends on 1) The method of claim 1, wherein the sequences of time series values correspond to at least one of performance metrics or security-related metrics. ### Claim 6 (depends on 1) The method of claim 1, wherein a first of the plurality of predictive models corresponds to a different set of time series than a second of the plurality of predictive models. ### Claim 7 (depends on 1) The method of claim 1, wherein a first of the plurality of predictive models is a polynomial model, a second of the predictive models is a neural network, and a third predictive model is a decision tree model. ### Claim 8 (depends on 1) The method of claim 1, wherein the obtaining of the sequences of time series values is from one or more streams of time series data. ### Claim 9 (depends on 1) The method of claim 1, wherein the one or more characteristics correspond to residuals between the predicted values of one or more of the plurality of predictive models and time series values associated with the first time series. ### Claim 10 (depends on 1) The method of claim 1, wherein the one or more characteristics are based on a first explanatory value associated with a model type of a first of the plurality of predictive models and a second explanatory value associated with a different model type of a second of the plurality of predictive models. ### Claim 11 (depends on 1) The method of claim 1 comprising: training a subset of the plurality of predictive models based on the one or more characteristics; and evaluating the subset of the plurality of predictive models based on the training, wherein the predictive model is selected for the anomaly detection from the subset based on the evaluating. ### Claim 12 (depends on 1) The method of claim 1 comprising: training the plurality of predictive models using first portions of the sequences of time series values corresponding to a training period; and evaluating second portions of the sequences of time series values corresponding to a prediction period, wherein the one or more characteristics are based on the evaluated second portions. ### Claim 13 (depends on 1) The method of claim 1, further comprising clustering the sequences of time series values into a plurality of clusters, wherein the first time series corresponds to a representative time series of a first of the plurality of clusters and the second time series corresponds to at least one time series in a second cluster of the plurality of clusters. ### Claim 14 (depends on 1) The method of claim 1, wherein the obtaining of the sequences of time series values is responsive to a user interaction associated with the sequences of the series values. ### Claim 15 (depends on 1) The method of claim 1, wherein the values of the second time series correspond to events, each event comprising a time stamp and a portion of raw data. ### Claim 16 (depends on 1) The method of claim 1, wherein each data point of the second time series is associated with a respective time stamp of a respective event. ### Claim 17 (depends on 1) The method of claim 1, generating the time series values from event data using a late-binding schema. ### Claim 18 (depends on 1) The method of claim 1, wherein the predicted values are associated with later times than the values of the second time series used to generate the predictive model. ### Claim 19 (depends on 1) The method of claim 1, further comprising causing an explanatory message to be presented based on the anomaly detection, the explanatory message indicating a predicted relationship corresponding to the predictive model and an observed relationship associated with an anomaly. ### Claim 20 (depends on 1) The method of claim 1, wherein the selecting is of multiple models from the plurality of predictive models based on the evaluating of the one or more characteristics, and the anomaly is detected using the multiple models. ### Claim 21 One or more non-transitory computer-readable storage media having instructions stored thereon, wherein the instructions, when executed by one or more processors, cause the one or more processors to perform a computer-implemented method comprising: obtaining sequences of time series values determined from raw machine data, each sequence corresponding to a respective time series, wherein the raw machine data is produced by one or more components within an information technology or security environment and reflects activity within the information technology or security environment; generating a plurality of predictive models for a first time series from the sequences of time series values, each predictive model to generate predicted values associated with the first time series using time series values corresponding to a second time series; evaluating one or more characteristics of the plurality of predictive models; and automatically selecting a predictive model from the plurality of predictive models for anomaly detection based on the evaluating of the one or more characteristics. ### Claim 22 (depends on 21) The non-transitory one or more computer-readable storage media of claim 21, wherein the evaluating comprises, for each of the plurality of predictive models, determining an error between the corresponding predicted values and values associated with the first time series, wherein the selecting of the predictive model is based on the error of the predictive model. ### Claim 23 (depends on 21) The non-transitory one or more computer-readable storage media of claim 21, wherein the sequences of time series values correspond to at least one of performance metrics, security-related metrics, industrial metrics, behavioral data, or transactional metrics. ### Claim 24 (depends on 21) The non-transitory one or more computer-readable storage media of claim 21, wherein a first of the plurality of predictive models corresponds to a different set of time series than a second of the plurality of predictive models. ### Claim 25 (depends on 21) The non-transitory one or more computer-readable storage media of claim 21, wherein a first of the plurality of predictive models is a polynomial model, a second of the predictive models is a neural network, and a third predictive model is a decision tree model. ### Claim 26 A computer-implemented system comprising: one or more processors; one or more non-transitory computer-readable storage media having instructions stored thereon, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform a method comprising: obtaining sequences of time series values determined from raw machine data, each sequence corresponding to a respective time series, wherein the raw machine data is produced by one or more components within an information technology or security environment and reflects activity within the information technology or security environment; generating a plurality of predictive models for a first time series from the sequences of time series values, each predictive model to generate predicted values associated with the first time series using time series values corresponding to a second time series; evaluating one or more characteristics of the plurality of predictive models; and automatically selecting a predictive model from the plurality of predictive models for anomaly detection based on the evaluating of the one or more characteristics. ### Claim 27 (depends on 26) The system of claim 26, wherein the evaluating comprises, for each of the plurality of predictive models, determining an error between the corresponding predicted values and values associated with the first time series, wherein the selecting of the predictive model is based on the error of the predictive model. ### Claim 28 (depends on 26) The system of claim 26, wherein the sequences of time series values correspond to at least one of performance metrics or security-related metrics. ### Claim 29 (depends on 26) The system of claim 26, wherein a first of the plurality of predictive models corresponds to a different set of time series than a second of the plurality of predictive models. ### Claim 30 (depends on 26) The system of claim 26, wherein a first of the plurality of predictive models is a polynomial model, a second of the predictive models is a neural network, and a third predictive model is a decision tree model.