Cloud access security broker systems and methods via a distributed worker pool

A cloud node in a cloud-based system comprising one or more processors and memory storing instructions that, when executed, cause the one or more processors to:

• communicate with a user associated with a tenant of a plurality of tenants;
• obtain policy and configuration for the user based on the tenant, from a central authority in the cloud-based system;
• provide the one or more cloud services to the user, based on the policy and configuration;
• and
• crawl one or more cloud providers having a plurality of files for the user, based on the policy and configuration.

A non-transitory computer-readable storage medium having computer-readable code stored thereon for programming a processor to perform steps of:

• communicating with a user associated with a tenant of a plurality of tenants;
• obtaining policy and configuration for the user based on the tenant, from a central authority in the cloud-based system;
• providing the one or more cloud services to the user, based on the policy and configuration;
• and
• crawling one or more cloud providers having a plurality of files for the user, based on the policy and configuration.