Method and apparatus for collecting and storing statistics data from network elements using scalable architecture

A communications network, comprising:

• a scheduler and orchestrator module in a network server coupled to the communications network and configured to generate a list of data collecting tasks for obtaining statistics data from various network nodes, wherein the scheduler and orchestrator module is configured to determine insufficient processing capacity of stateless processes for collecting statistics data from all nodes within predefined time duration;
• a pool of stateless processes residing in one or more network devices coupled to the scheduler and orchestrate module and configured to collect information from the various network nodes in accordance with the list of data collecting tasks, wherein a stateless process is dynamically added to or removed from the pool of stateless processes for increasing or reducing capacity of collecting statistics data in accordance with computing power needed to perform data collecting within a predefined time period;
• and
• a database coupled to the plurality of stateless processes and capable of storing the statistics data collected from one or more network nodes.

A method for collecting statistics over a communications network, comprising:

• partitioning a relational database into a daily level and an interval level;
• detecting a delay associated with availability of statistics data collected from a plurality of network elements (“NEs”) in accordance with a predefined expected period;
• determining insufficient processing capacity of stateless processes to complete collecting statistics data from all nodes within predefined time duration in response to the delay;
• adding one or more stateless processes to a pool of existing stateless processes for scaling up computing power in response to the delay to increase capacity of processing power;
• collecting statistics data from NEs of the communications network in response to a list of tasks;
• wherein collecting statistics data includes fetching statistics data in response to a data collection manager (“DCM”) task generated by a scheduler;
• and
• storing the statistics data in the relational database in accordance with the daily level and the interval level.