A computer implemented method for automating the process of forecasting, said method comprising the following steps:
- collecting data from a plurality of sources including historic time series data, causal information and trend indicators and collecting data corresponding to external factors including macroeconomic factors, market related factors and social media related factors;
- extracting, from the collected data, the predictor data streams and elements to be used for forecasting;
- storing the extracted predictor data streams and elements in a first repository;
- receiving said predictor data streams and elements from said repository and analyzing the received predictor data streams and elements to identify the characteristics of said predictor data streams and elements;
- iteratively determining whether all the predictor data streams and elements required for forecasting are present;
- creating sets of predictor data streams and elements appropriate for forecasting from said predictor data streams and elements by selecting, combining and transforming one or more predictor data streams and elements;
- receiving sets of predictor data streams and elements and grouping the predictor data streams and elements into a plurality of clusters, in accordance with clustering rules such that each cluster contains predictor data streams and elements having a common data pattern;
- extracting said characteristic(s) corresponding to each cluster;
- storing a plurality of forecasting models and the corresponding unique properties and a unique master list of their forecasting parameters, for each of the plurality of forecasting models;
- receiving the properties of each of said forecasting models and the extracted characteristics of predictor data streams and elements present in each of said clusters;
- comparing said characteristics with said properties and identifying a set of forecasting models containing at least one appropriate forecasting model adapted to perform a forecast using said predictor data streams and elements;
- processing the set of forecasting models in accordance with a set of ranking rules and ranking the set of forecasting models, in accordance with its ability to match with the characteristics of the predictor data elements and data streams;
- selecting a ranked forecasting model;
- and
- receiving, at a forecasting engine, the selected forecasting model and the master list of its forecasting parameters;
- applying shrinkage based techniques to eliminate inter-dependent predictor variables, in the event that the selected forecasting model is a regression model, and applying state-space modeling to automate the determination of optimal parameter values, in the event that the selected forecasting model is an exponential smoothing model;
- automatically select an appropriate set of forecasting parameters and corresponding optimal parameter values;
- processing said appropriate set of forecasting parameters and the corresponding optimal parameter values in accordance with the selected forecasting model, to generate a forecast;
- and
- diagnosing the generated forecast and identifying at least one forecast error criteria corresponding to the generated forecast.