# US 9,654,538 B1 · Dynamic partitioning of instances in distributed streaming platform for real-time applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,654,538 B1 - Document ID: US9654538B1 - Status: Granted - Kind: B1 - Application number: 13/928,357 - Filed: 2013-06-26 - Granted: 2017-05-16 - Earliest claimed: 2013-03-11 - Assignees: DataTorrent, Inc. - Applicants: DataTorrent, Inc. - Inventors: Amol J. Kekre; Phu D. Hoang; Chetan Narsude; Thomas Weise - Web page: https://patentgrep.com/patent/US9654538B1 ## CPC - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06F 9/541: {via adapters, e.g. between incompatible applications} - G06F 11/1402: {Saving, restoring, recovering or retrying} - G06F 16/24568: {Data stream processing; Continuous queries} - G06F 16/248: Presentation of query results - G06F 16/252: {between a Database Management System and a front-end application} - G06F 16/258: {Data format conversion from or to a database} - G06F 16/951: Indexing; Web crawling techniques - H04L 65/60: Network streaming of media packets - H04L 65/765: {intermediate} - H04L 67/55: Push-based network services - G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Software for a distributed streaming platform monitors a first operator instance in a container for an application running on a distributed streaming platform over a networked cluster of servers. The first operator instance includes an input port that receives streaming windows from an upstream instance and an output port that emits streaming windows to a downstream instance. And the monitoring includes determining a window identifier for a streaming window which the first operator instance is processing. The software reports results of the monitoring to a master and receives a dynamic adjustment from the master. The dynamic adjustment includes launching a second operator instance of the same operator in the container. The software resumes a flow of streaming windows into the input port and the output port according to first recovery policy for the first operator instance and a second recovery policy for the second operator instance. ## Claims ### Claim 1 A method, comprising: monitoring a first operator instance of a first operator on an upstream node for an application running on a distributed streaming platform over a networked cluster of servers, wherein the first operator instance includes an input port that receives streaming windows from an upstream instance and an output port that emits streaming windows to a second operator instance of a second operator on a downstream node, wherein the monitoring includes determining a window identifier for a streaming window which the first operator instance is processing, and wherein each streaming window is an atomic sequence of tuples associated with a recovery policy; reporting results of the monitoring to a master; receiving a dynamic adjustment from the master, wherein the dynamic adjustment includes launching a plurality of additional operator instances of the first operator and using cascading unifiers to connect them and the first operator instance through a plurality of network interface controllers (NICs) to the second operator instance and wherein the plurality of additional operator instances is three additional operator instances; and resuming a flow of streaming windows into the input port and the output port according to the recovery policy for the first operator instance, wherein the recovery policy is at least once, wherein data loss is not acceptable but extra computations are acceptable and checkpointing of an operator instance does not occur after every streaming window when the recovery policy is at least once, and wherein each of the operations is executed by one or more processors in real time or near real time rather than offline. ### Claim 2 (depends on 1) The method of claim 1, wherein the application includes a graphical user interface (GUI) with a dashboard for decision support. ### Claim 3 (depends on 1) The method of claim 1, wherein the application uses one or more stream modes. ### Claim 4 (depends on 3) The method of claim 3, wherein at least one of the stream modes is in-node. ### Claim 5 (depends on 3) The method of claim 3, wherein at least one of the stream modes is in-rack. ### Claim 6 (depends on 1) The method of claim 1, wherein the results are reported to the master, using a heartbeat protocol. ### Claim 7 (depends on 1) The method of claim 1, wherein the recovery policy is specified on a per-operator basis. ### Claim 8 (depends on 1) The method of claim 1, wherein the recovery policy determines a frequency of checkpointing the operator instance and wherein checkpointing includes serializing the instance to persistent storage. ### Claim 9 (depends on 3) The method of claim 3, wherein at least one of the stream modes is in-line. ### Claim 10 One or more computer-readable media that are non-transitory and that store one or more programs, wherein the one or more programs, when executed, instructs one or more processors to perform the following operations: monitor a first operator instance of a first operator on an upstream node for an application running on a distributed streaming platform over a networked cluster of servers, wherein the first operator instance includes an input port that receives streaming windows from an upstream instance and an output port that emits streaming windows to a second operator instance of a second operator on a downstream node, wherein the monitoring includes determining a window identifier for a streaming window which the first operator instance is processing, and wherein each streaming window is an atomic sequence of tuples associated with a recovery policy; report results of the monitoring to a master; receive a dynamic adjustment from the master, wherein the dynamic adjustment includes launching a plurality of additional operator instances of the first operator and using cascading unifiers to connect them and the first operator instance through a plurality of network interface controllers (NICs) to the second operator instance and wherein the plurality of additional operator instances is three additional operator instances; and resume a flow of streaming windows into the input port and the output port according to the recovery policy for the first operator instance, wherein the recovery policy is at least once, wherein data loss is not acceptable but extra computations are acceptable and checkpointing of an operator instance does not occur after every streaming window when the recovery policy is at least once, and wherein each of the operations is executed in real time or near real time rather than offline. ### Claim 11 (depends on 10) The computer-readable media of claim 10, wherein the application includes a graphical user interface (GUI) with a dashboard for decision support. ### Claim 12 (depends on 10) The computer-readable media of claim 10, wherein the application uses one or more stream modes. ### Claim 13 (depends on 12) The computer-readable media of claim 12, wherein at least one of the stream modes is in-node. ### Claim 14 (depends on 12) The computer-readable media of claim 12, wherein at least one of the stream modes is in-rack. ### Claim 15 (depends on 10) The computer-readable media of claim 10, wherein the results are reported to the master, using a heartbeat protocol. ### Claim 16 (depends on 10) The computer-readable media of claim 10, wherein the recovery policy is specified on a per-operator basis. ### Claim 17 (depends on 10) The computer-readable media of claim 10, wherein the recovery policy determines a frequency of checkpointing the operator instance and wherein checkpointing includes serializing the instance to persistent storage. ### Claim 18 (depends on 12) The computer-readable media of claim 12, wherein at least one of the stream modes is in-line. ### Claim 19 A method, comprising: monitoring a first operator instance of a first operator on an upstream node for an application running on a distributed streaming platform over a networked cluster of servers, wherein the first operator instance includes an input port that receives streaming windows from an upstream instance and an output port that emits streaming windows to a second operator instance of a second operator on a downstream node, wherein the monitoring includes determining a window identifier for a streaming window which the first operator instance is processing, and wherein each streaming window is an atomic sequence of tuples associated with a recovery policy; reporting results of the monitoring to a master; receiving a dynamic adjustment from the master, wherein the dynamic adjustment includes launching a plurality of additional operator instances of the first operator and using cascading unifiers to connect them and the first operator instance through a plurality of network interface controllers (NICs) to the second operator instance and wherein the plurality of additional operator instances is three additional operator instances; and resuming a flow of streaming windows into the input port and the output port according to the recovery policy for the first operator instance, wherein the recovery policy is at least once, wherein data loss is not acceptable but extra computations are acceptable and checkpointing of an operator instance does not occur after every streaming window when the recovery policy is at least once, and wherein each of the operations is executed by one or more processors in real time or near real time rather than offline. ### Claim 20 (depends on 19) The method of claim 19, wherein the application includes a graphical user interface (GUI) with a dashboard for decision support.