# US 10,481,948 B2 · Data transfer in a collaborative file sharing system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,481,948 B2 - Document ID: US10481948B2 - Status: Granted - Kind: B2 - Application number: 14/835,599 - Filed: 2015-08-25 - Published: 2017-03-02 - Granted: 2019-11-19 - Earliest claimed: 2015-08-25 - Assignees: Box, Inc. - Applicants: Box, Inc. - Inventors: Denis Grenader - Web page: https://patentgrep.com/patent/US10481948B2 ## CPC - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) - H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes - H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)} - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - G06F 11/3409: {for performance assessment} - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system and method for facilitating implementing a computing task (such as selectively transforming and/or writing data retrieved from a message stream of a message broker) via concurrent computing in a networked computing environment. The example method includes using a distributed software framework to encapsulate overhead functionality for facilitating extracting data from a data stream output from the message broker and writing associated data to one or more targets in a networked computing environment; and accommodating custom code as a plugin to one or more modules of the software framework. In a more specific embodiment, the one or more modules include one or more committers. The overhead functionality includes functionality for facilitating selective redistribution of work when a process becomes unreliable; tracking of offsets for message batches in the data stream; automatically assigning work to nodes in the networked computing environment. ## Claims ### Claim 1 A method for distributing log information in a cross- enterprise file collaboration system, the system including one or more server computers coupled to one or more client devices, wherein the one or more client devices include a first group of client devices on a first network and a second group of client devices on a second network, the method comprising the following performed by the one or more server computers: receiving log information from a service that is in communication with a web application in response to an operation of a client device; storing log information in a message broker; using a replicator of the one or more server computers to fetch at least a portion of the log information from the message broker, resulting in fetched log information in response thereto, wherein fetching the at least a portion of the log information comprises: using a replicator worker in communication with one or more fetchers to receive a portion of a message stream from the message broker, the one or more fetchers in communication with one or more committers, the replication worker comprising one of a plurality of replication workers for the replicator of the one or more servers, wherein the replication worker retrieves batches of data from the message broker and performs task assignments received from the replicator, using the one or more fetchers to selectively cache date from the message broker and share the cached data among one or more committers, each of the one or more fetchers comprising a child node of the replication worker, wherein each fetcher monitors and handles a predefined partition of a message topic of a plurality of message topics for the log information, and using the one or more committers to write at least a portion of the fetched log information to one or more targets, each of the one or more committers comprising a child node of at least one of the one or more fetchers, wherein each committer replicates data from the message broker to one or more targets for the committer and performs, by execution of a custom code plug-in, one or more overhead functions for the one or more targets of the committer, and wherein the log information written to the one or more targets is also replicated across two or more data centers by the one or more committers. ### Claim 2 (depends on 1) The method of claim 1, further including: using a distributed software framework to encapsulate overhead functionality for facilitating extracting data from a data stream output from the message broker and writing associated data to one or more targets in a networked computing environment, and wherein the overhead functionality includes functionality for providing one or more of (i) process redundancy and redistribution of work when a process becomes unreliable (ii) tracking of offsets for message batches in the data stream, (iii) automatically assigning work to nodes in the networked computing environment. ### Claim 3 (depends on 1) The method of claim 1, further including: employing the one or more committers to implement a software action using the portion of the message stream. ### Claim 4 (depends on 3) The method of claim 3, wherein the action includes writing the portion of the message stream to a target computing resource. ### Claim 5 (depends on 4) The method of claim 4, wherein the target computing resource includes a database or a computing process. ### Claim 6 (depends on 1) The method of claim 1, wherein the message broker includes a message broker for handling real-time data feeds. ### Claim 7 (depends on 3) The method of claim 3, further including: selectively incorporating custom code as a plugin, wherein the custom code includes code to transform data of the message stream in advance of facilitating writing resulting transformed data to the target network resource. ### Claim 8 (depends on 7) The method of claim 7, wherein the portion of the message stream includes a sequence of messages, the sequence characterized by a specified length representing a number of messages in the sequence of messages. ### Claim 9 (depends on 3) The method of claim 3, further including: employing a coordinator service to coordinate communications between operations of a cluster of servers, wherein the coordinator service includes one or more components that are replicated across each of the servers of the cluster of servers. ### Claim 10 (depends on 9) The method of claim 9, further including: employing a toolkit to facilitate orchestrating data flow between software components within an individual server of the cluster of servers. ### Claim 11 (depends on 10) The method of claim 10, further including: incorporating plural replicator workers, at least one replicator worker for two or more servers of the cluster of servers, to facilitate implementing a task indicated via the coordinator service. ### Claim 12 (depends on 11) The method of claim 11, further including: employing a replicator leader to selectively rebalance portions of the task among the replicator workers, wherein the replicator leader includes a message broker monitor that monitors the message stream output by the message broker and selectively informs one or more replicator workers as to changes occurring in the message stream. ### Claim 13 An apparatus for distributing log information in a cross-enterprise file collaboration system, the system including one or more server computers coupled to one or more client devices, wherein the client devices include a first group of client devices on a first network and a second group of client devices on a second network, the apparatus comprising: a digital processor coupled to a display and to a non-transitory processor-readable storage device, wherein the non-transitory processor-readable storage device includes one or more instructions executable by the digital processor to perform the following acts: receiving log information from a service that is in communication with a web application in response to an operation of a client device; storing log information in a message broker; using a replicator of the one or more server computers to fetch at least a portion of the log information from the message broker, resulting in fetched log information in response thereto, wherein fetching the at least a portion of the log information comprises: using a replicator worker in communication with one or more fetchers to receive a portion of a message stream from the message broker, the one or more fetchers in communication with one or more committers, the replication worker comprising one of a plurality of replication workers for the replicator of the one or more servers, wherein the replication worker retrieves batches of data from the message broker and performs task assignments received from the replicator, using the one or more fetchers to selectively cache date from the message broker and share the cached data among one or more committers, each of the one or more fetchers comprising a child node of the replication worker, wherein each fetcher monitors and handles a predefined partition of a message topic of a plurality of message topics for the log information, and using the one or more committers to write at least a portion of the fetched log information to one or more targets, each of the one or more committers comprising a child node of at least one of the one or more fetchers, wherein each committer replicates data from the message broker to one or more targets for the committer and performs, by execution of a custom code plug-in, one or more overhead functions for the one or more targets of the committer, and wherein the log information written to the one or more targets is also replicated across two or more data centers by the one or more committers. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the non-transitory processor-readable storage device further includes one or more instructions executable by the digital processor to perform the following acts: using a distributed software framework to encapsulate overhead functionality for facilitating extracting data from a data stream output from the message broker and writing associated data to one or more targets in a networked computing environment and wherein the overhead functionality includes functionality for providing one or more of (i) process redundancy and redistribution of work when a process becomes unreliable (ii) tracking of offsets for message batches in the data stream, (iii) automatically assigning work to nodes in the networked computing environment. ### Claim 15 (depends on 13) The apparatus of claim 13, wherein the non-transitory processor-readable storage device further includes one or more instructions executable by the digital processor to perform the following acts: employing the one or more committers to implement a software action using the portion of the message stream. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the action includes writing the portion of the message stream to a target computing resource. ### Claim 17 A non-transitory processor-readable storage device including instructions executable by a digital processor, the processor-readable storage device including one or more instructions for: receiving log information from a service that is in communication with a web application in response to an operation of a client device; storing log information in a message broker; using a replicator of the one or more server computers to fetch at least a portion of the log information from the message broker, resulting in fetched log information in response thereto, wherein fetching the at least a portion of the log information comprises: using a replicator worker in communication with one or more fetchers to receive a portion of a message stream from the message broker, the one or more fetchers in communication with one or more committers, the replication worker comprising one of a plurality of replication workers for the replicator of the one or more servers, wherein the replication worker retrieves batches of data from the message broker and performs task assignments received from the replicator, using the one or more fetchers to selectively cache date from the message broker and share the cached data among one or more committers, each of the one or more fetchers comprising a child node of the replication worker, wherein each fetcher monitors and handles a predefined partition of a message topic of a plurality of message topics for the log information, and using the one or more committers to write at least a portion of the fetched log information to one or more targets, each of the one or more committers comprising a child node of at least one of the one or more fetchers, wherein each committer replicates data from the message broker to one or more targets for the committer and performs, by execution of a custom code plug-in, one or more overhead functions for the one or more targets of the committer, and wherein the log information written to the one or more targets is also replicated across two or more data centers by the one or more committers. ### Claim 18 (depends on 17) The non-transitory processor-readable storage device of claim 17, further includes one or more instructions executable by the digital processor for: using a distributed software framework to encapsulate overhead functionality for facilitating extracting data from a data stream output from the message broker and writing associated data to one or more targets in a networked computing environment and wherein the overhead functionality includes functionality for providing one or more of (i) process redundancy and redistribution of work when a process becomes unreliable (ii) tracking of offsets for message batches in the data stream, (iii) automatically assigning work to nodes in the networked computing environment. ### Claim 19 (depends on 17) The non-transitory processor-readable storage device of claim 17, further includes one or more instructions executable by the digital processor for: employing the one or more committers to implement a software action using the portion of the message stream. ### Claim 20 (depends on 19) The non-transitory processor-readable storage device of claim 19, wherein the action includes writing the portion of the message stream to a target computing resource.