Demand sensing and forecasting

A data processing system for demand sensing and forecasting comprises:

• a server configured to receive a plurality of historical data from an entity and storing the data at a data lake;
• a processor for generating a hierarchical dataset from the received historical data wherein a bot creates a data script based on the received data for generating the hierarchical dataset by utilizing a library of functions stored on a functional database, wherein the hierarchical data set is a linear or non-linear time series data set distributed at different levels in a data hierarchy;
• an AI engine coupled to the processor and configured for creating one or more forecasting data models based on the hierarchical data set for predicting demand of one or more objects wherein the AI engine analyzes processing results of the one or more forecasting data models at each level of the hierarchal data set to generate a prediction data model and wherein the AI engine is configured to transform the historical data into a data matrix for training the one or more forecasting models on the historical data;
• a training data set database and a testing data set database wherein the hierarchical dataset is divided into training data set and testing data set at the different levels in the hierarchical data set;
• and
• a forecasting data models database configured for storing the one or more forecasting data models wherein forecasting data for the testing data is generated using the training data set at each level through the one or more forecasting data models;
• and
• a neural network configured to identify one or more data attribute weights assigned to the prediction data model at each level of the hierarchical data set and processing the identified weights to determine an optimum data attribute weight for the prediction data model wherein the neural network includes an auto encoder and decoder configured for data compression wherein a LSTM (long short term memory networks) layer is embedded between the auto encoder and decoder of the neural network for identifying temporal variations in a testing data and determining the data attribute weights considering the variations and wherein a demand of the one or more objects is predicted based on the optimized prediction data model associated with the hierarchal data set, wherein the auto encoder is configured to compress the forecasting data and the decoder is configured to compare the compressed forecasting data with a valid coherent forecast data, wherein the processor is configured to execute supply chain operations including inventory management, wherein executing the supply chain operations including inventory management includes determining a required safety stock of the one or more objects based on analysis of the predicted demand and one or more operational characteristics of the one or more objects, wherein the one or more operational characteristics includes category of object, wherein the category of object includes high consumption objects, and wherein the safety stock for high consumption is determined as:
• S=Zα×√(E(L)σ2D+(E(D))2σ2L)
• where, α is service level and Zα is the inverse distribution function of a standard normal distribution with cumulative probability α of the one or more object;
• E(L) and σL are standard deviation of lead time;
• and
• E(D) and σD are standard deviation of demand in each unit time period.

A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations of demand sensing and forecasting, the operations comprising:

• receiving and storing at a data lake, a plurality of historical data from an entity;
• generating a hierarchical dataset from the received historical data wherein a bot creates a data script based on the received data for generating the hierarchical dataset by utilizing a library of functions stored on a functional database, wherein the hierarchical data set is a linear or non-linear time series data set distributed at different levels in a data hierarchy;
• creating one or more forecasting data models by an AI engine for processing the hierarchical data set to predict data trend wherein the AI engine is configured to transform the historical data into a data matrix for training the one or more forecasting models on the historical data;
• analyzing by the AI engine, processing results of the one or more forecasting data models at each level of the hierarchal data set to generate a prediction data model;
• identifying by a neural network, one or more data attribute weights assigned to the prediction data model at each level of the hierarchical data set and processing the identified weights to determine an optimum data attribute weight for the prediction data model wherein the neural network includes an auto encoder and decoder configured for data compression wherein a LSTM (long short term memory networks) layer is embedded between the auto encoder and decoder of the neural network for identifying temporal variations in a testing data and determining the data attribute weights considering the variations;
• dividing the hierarchical dataset into training data set and testing data set at the different levels in the hierarchical dataset;
• generating forecasting data for the testing data using the training data set at each level through the one or more forecasting data models;
• predicting demand of one or more objects based on the optimized prediction data model associated with the hierarchal data set, wherein the auto encoder is configured to compress the forecasting and the decoder is configured to compare the compressed forecasting data with a valid coherent forecast data;
• and
• executing supply chain operations including inventory management, wherein executing the supply chain operations including inventory management includes determining a required safety stock of the one or more objects based on analysis of the predicted demand and one or more operational characteristics of the one or more objects, wherein the one or more operational characteristics includes category of object, wherein the category of object includes high consumption objects, and wherein the safety stock for high consumption is determined as:
• S=Zα×√(E(L) σ2D+(E(D))2σ2L
• where, α is service level and Zα is the inverse distribution function of a standard normal distribution with cumulative probability α of the one or more object;
• E(L) and σL are standard deviation of lead time;
• and
• E(D) and σD are standard deviation of demand in each unit time period.

A data processing and data analytics method for demand sensing, forecasting and inventory management, the method comprising:

• receiving and storing at a data lake, a plurality of historical data from an entity;
• generating a hierarchical dataset from the received historical data, wherein a bot creates a data script based on the received data for generating the hierarchical dataset by utilizing a library of functions stored on a functional database;
• creating one or more forecasting data models by an AI engine of a support mechanism for processing the hierarchical dataset to predict data trend, wherein the AI engine is configured to transform the historical data into a data matrix for training the one or more forecasting models on the historical data, wherein the support mechanism includes a demand data categorization and classification engine for cleansing and categorizing objects, a data solver and optimizer for processing variables and optimization parameters, a data extraction and mapping module configured for extracting and mapping object data to category by a clustering script generated through the AI engine, and an API (application programming interface) for triggering the one or more forecasting data models through the processor;
• analyzing by the AI engine coupled to a processor, processing results of the one or more forecasting data models at each level of the hierarchal data set based on a hierarchical time series library with a set of exogenous and hyper parameters to generate a prediction data model;
• identifying by a neural network, one or more data attribute weights assigned to the prediction data model at each level of the hierarchical data set and processing the identified weights to determine an optimum data attribute weight for the prediction data model, wherein the neural network includes an auto encoder and decoder configured for data compression, wherein a LSTM layer is embedded between he auto encoder and decoder of the neural network for identifying temporal variations in a testing data and determining the data attribute weights considering the variations;
• predicting demand of one or more objects based on the optimized prediction data model associated with the hierarchal data set, wherein the optimized prediction data model is determined by parameter tuning on different nodes of a big data tool cluster through group mapping tool;
• determining required safety stock of the one or more obj ects based on analysis of the predicted demand and one or more operational characteristics of the one or more objects;
• assigning by an inventory bot, data attribute weights to the one or more operational characteristics at different levels of analytic hierarchy;
• processing by the AI engine, the data attribute weights to categorize and classify the one or more objects based on the demand and based on the classification wherein the processor is configured to analyze demand data patterns of the object through classification of the historical data into a hierarchical taxonomy that provides insights into consumption patterns, wherein the AI engine generates clusters from the historical data and the processor is configured to process the data through an unsupervised and nonparametric clustering approach as number of possible clusters are unknown;
• determining the required safety stock for each category of the one or more objects, wherein a safety stock for medium consumption category object is determined based on mean and variance as:
• if mean×1.1>Variance, then the safety stock is determined by Poisson distribution else
• if mean×1.1<Variance, then the safety stock is determined by bootstrapping methodology;
• and
• a safety stock for high consumption category object is determined as:
• S=Zα×√(E(L)σ2D+(E(D))2σ2L)
• where, α is service level and Zα is the inverse distribution function of a standard normal distribution with cumulative probability α of the one or more object;
• E(L) and σL are standard deviation of lead time;
• and
• E(D) and σD are standard deviation of demand in each unit time period.