Scaling By Dynamic Discovery For Data Processing

A method comprising:

• receiving, by a first computing device via a rapid representational state transfer (REST) application programming interface (API), a Get page search command, wherein the Get page search command comprises at least one user specified criterion of content to be searched and a universally unique identifier (UUID) of a user computing device;
• accessing an external storage device to identify a global results collector for assisting the first computing device in processing the Get page command;
• determining whether the global results collector is maintained within the first computing device;
• after determining whether the global results collector is maintained within the first computing device, receiving, by the first computing device from the global results collector, a page and summary of results of content matching the at least one user specified criterion;
• and
• sending, by the first computing device via the rapid REST API, the page and summary of results of content matching the at least one user specified criterion.

A system comprising:

• an external storage device configured to store identification of a plurality of computing devices registered with the external storage device and identification of a global results collector;
• a first computing device, wherein the first computing device comprises:
• one or more first processors;
• and
• first memory storing instructions that, when executed by the one or more first processors, cause the first computing device to perform steps comprising:
• receiving, from a user computing device, a search request, wherein the search request comprises at least one user specified criterion of content to be searched and a universally unique identifier (UUID) of the user computing device;
• receiving, from the external storage device, a list of at least one second computing device registered with the external storage device and available for assisting the first computing device in processing the search request;
• partitioning the content to be searched for the search request across the at least one second computing device;
• generating the global results collector to compile results received from the at least one second computing device;
• registering the global results collector with the external storage device;
• sending, to the global results collector, the UUID and a number of partitions of the partitioned content to be searched;
• sending, to each of the at least one second computing device, a respective one of the number of partitions and reference data of the global results collector;
• determining whether the global results collector has received all expected compiled result lists from each of the at least one second computing device;
• and
• upon determining that the global results collector has received all expected compiled result lists, sending, by the global results collector to the user computing device, a response to the search request, wherein the response comprises results including the expected compiled result lists from each of the at least one second computing device;
• and
• at least one second computing device, wherein each of the second computing device comprises:
• one or more second processors;
• and
• second memory storing instructions that, when executed by the one or more second processors, cause the second computing device to perform steps comprising:
• generating a local collector actor to compile results received from at least one local search actors;
• sending, to the local collector actor, a number of sub-partitions and reference data of the global results collector;
• for each of the at least one local search actors, sending a respective one of the sub-partitions and reference data of the local collector actor to the local search actor;
• for each of the at least one local search actors, processing a search of the respective sub-partition based upon the user specified criterion to obtain a local search actor result list;
• receiving, from each of the at least one local search actors, a respective local search actor result list;
• determining whether the local collector actor has received all expected respective local search actor result lists;
• and
• upon determining that the local collector actor has received all expected respective local search actor result lists, sending, by the local collector actor to the global results collector, a compiled result list of all expected respective local search actor result lists.