Data statistics service

A computer-implemented method, comprising:

• providing range descriptors to processing agents executing in a service provider network, the range descriptors defining sets of data stored by a storage service in the service provider network;
• determining that computing capacity is available from an on-demand computing service operating within a service provider network for generating statistical information for the sets of the data;
• responsive to determining that computing capacity is available from the on-demand computing service for generating the statistical information for the sets of the data, instructing the processing agents to generate, based on availability of the computing capacity, the statistical information and associated confidence values for the sets of the data defined by the provided range descriptors, wherein the associated confidence values define a current degree of completion for the generating by the processing agents;
• providing the statistical information and the associated confidence values for the sets of the data to a plurality of network services executing in the service provider network as a shared source for the statistical information by way of an application programming interface (API), wherein the API is further configured to receive access metadata from one of the plurality of network services, the access metadata describing a frequency or pattern of access to the data observed by the one of the plurality of network services;
• and
• instructing the processing agents to process sets of the data based upon the access metadata.

A non-transitory computer-readable storage medium having instructions stored thereupon which are executable by a processor and which, when executed, cause the processor to:

• provide range descriptors to processing agents executing in a service provider network, the range descriptors defining sets of data stored by a storage service in the service provider network;
• determine that computing capacity is available from computing resources provided by an on-demand computing service operating within a service provider network for generating statistical information for the sets of the data;
• responsive to determining that computing capacity is available from computing resources provided by the on-demand computing service for generating the statistical information for the data, instructing the processing agents to generate, based on availability of the computing capacity, the statistical information and associated confidence values for the sets of the data defined by the provided range descriptors, wherein the associated confidence values define a current degree of completion for the generating by the processing agents;
• provide the statistical information and the associated confidence values for the sets of the data to a plurality of network services executing in the service provider network by way of an application programming interface (API) as a shared source for the statistical information wherein the API is further configured to receive access metadata from one of the plurality of network services, the access metadata describing a frequency or pattern of access to the data observed by the one of the plurality of network services;
• and
• instruct the processing agents to process sets of the data based upon the access metadata.