# US 10,313,430 B2 · Distributed method and apparatus for processing streaming data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,313,430 B2 - Document ID: US10313430B2 - Status: Granted - Kind: B2 - Application number: 15/281,273 - Filed: 2016-09-30 - Published: 2017-12-28 - Granted: 2019-06-04 - Earliest claimed: 2016-06-23 - Assignees: Baidu Online Network Technology (Beijing) Co., Ltd. - Applicants: Beijing Baidu Netcom Science and Technology Co., Ltd. - Inventors: Yao Xu; Cong Wang; Yuncong Zhang; Jianwei Zhang; Xin Huang - Web page: https://patentgrep.com/patent/US10313430B2 ## CPC - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - H04L 65/765: {intermediate} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A distributed method and apparatus for processing streaming data are disclosed. A specific implementation of the method includes: encapsulating received streaming data as a first resilient distributed dataset; performing a grouping operation on the first resilient distributed dataset based on time windows, the grouping operation comprising: assigning each data element in the first resilient distributed dataset into a group corresponding to a time window to which a recorded timestamp of the data element belongs, and forming second resilient distributed datasets comprising a limited number of data elements and respectively corresponding to the time windows; encapsulating the second resilient distributed datasets as a nested dataset comprising a plurality of the second resilient distributed datasets; passing, using a predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to a batch operator defined in a finite dataset to perform distributed data processing. ## Claims ### Claim 1 A distributed method for processing streaming data, comprising: encapsulating received streaming data as a first resilient distributed dataset; performing a grouping operation on the first resilient distributed dataset based on time windows, the grouping operation comprising: assigning each data element in the first resilient distributed dataset into a group corresponding to a time window to which a recorded timestamp of the data element belongs, and forming second resilient distributed datasets comprising a limited number of data elements and respectively corresponding to the time windows; encapsulating the second resilient distributed datasets as a nested dataset comprising a plurality of the second resilient distributed datasets; passing, using a predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to a batch operator defined in a finite dataset to perform distributed data processing. ### Claim 2 (depends on 1) The method according to claim 1, wherein, the encapsulating the second resilient distributed datasets as the nested dataset comprising the plurality of the second resilient distributed datasets comprises: encapsulating a time point selected from each of the time windows as a key and each of the second resilient distributed datasets as a value as the nested dataset storing the second resilient distributed datasets in a form of a key-value pair, and the passing, using the predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to the hatch operator defined in the finite dataset to perform the distributed data processing comprises: passing each of the second resilient distributed datasets as the value in the nested dataset successively to the batch operator defined in the finite dataset to perform the distributed data processing. ### Claim 3 (depends on 2) The method according to claim 2, wherein the encapsulating the time point selected from each of the time window as the key and using each of the second resilient distributed dataset as the value as the nested dataset storing the second resilient distributed datasets in the form of the key-value pair comprises: setting the key of the key-value pair to a starting time point of the time window. ### Claim 4 (depends on 1) The method according to claim 1, further comprising: obtaining a time step set by an user, and setting a value of a time interval between adjacent time windows to the time step. ### Claim 5 (depends on 4) The method according to claim 4, further comprising: setting a value of a duration of the time window to the time step. ### Claim 6 (depends on 4) The method according to claim 4, further comprising: obtaining a step number set by the user, and setting a value of a duration of the time window to a product of the step number and the time step. ### Claim 7 A distributed apparatus for processing streaming data, comprising: at least one processor; and a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising: encapsulating received streaming data as a first resilient distributed dataset; performing a grouping operation on the first resilient distributed dataset based on time windows, the grouping operation comprising: assigning each data element in the first resilient distributed dataset into a group corresponding to a time window to which a recorded timestamp of the data element belongs, and forming second resilient distributed datasets comprising a limited number of data elements and respectively corresponding to the time windows; encapsulating the second resilient distributed datasets as a nested dataset comprising a plurality of the second resilient distributed datasets; and passing, using a predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to a batch operator defined in a finite dataset to perform distributed data processing. ### Claim 8 (depends on 7) The apparatus according to claim 7, wherein, the encapsulating the second resilient distributed datasets as the nested dataset comprising the plurality of the second resilient distributed datasets comprises: encapsulating a time point selected from each of the time windows as a key and each of the second resilient distributed datasets as a value as the nested dataset storing the second resilient distributed datasets in a form of a key-value pair, and the passing, using the predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to the batch operator defined in the finite dataset to perform the distributed data processing comprises: passing each of the second resilient distributed datasets as the value in the nested dataset successively to the batch operator defined in the finite dataset to perform the distributed data processing. ### Claim 9 (depends on 8) The apparatus according to claim 8, wherein the encapsulating the time point selected from each of the time window as the key and using each of the second resilient distributed dataset as the value as the nested dataset storing the second resilient distributed datasets in the form of the key-value pair comprises: setting the key of the key-value pair to a starting time point of the time window. ### Claim 10 (depends on 7) The apparatus according to claim 7, wherein he operations further comprise: obtaining a time step set by an user, and setting a value of a time interval between adjacent time windows to the time step. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein the operations further comprise: setting a value of a duration of the time window to the time step. ### Claim 12 (depends on 10) The apparatus according to claim 10, wherein the operations further comprise: obtaining a step number set by the user, and setting a value of a duration of the time window to a product of the step number and the time step. ### Claim 13 A non-transitory storage medium storing one or more programs, the one or more programs when executed by an apparatus, causing the apparatus to perform a distributed method for processing streaming data, the method comprising: encapsulating received streaming data as a first resilient distributed dataset; performing a grouping operation on the first resilient distributed dataset based on time windows, the grouping operation comprising: assigning each data element in the first resilient distributed dataset into a group corresponding to a time window to which a recorded timestamp of the data element belongs, and forming second resilient distributed datasets comprising a limited number of data elements and respectively corresponding to the time windows; encapsulating the second resilient distributed datasets as a nested dataset comprising a plurality of the second resilient distributed datasets; passing, using a predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to a batch operator defined in a finite dataset to perform distributed data processing.