Data processing system, computing node, and data processing method

A data processing system, comprising a management node including, a processor and a plurality of computing nodes that form a first class of computing nodes, each of the first class of computing nodes including a processor, wherein

• the processor of the management node is configured to:
• allocate a first processing task to each of at least two computing nodes in the first class of computing nodes, wherein a computing node FCx is the xth computing node in the at least two computing nodes, and x is a positive integer, and wherein the at least two computing nodes in the first class of computing nodes concurrently perform the first processing tasks allocated by the management node;
• the processor of the computing node FCx is configured to:
• obtain, according to the first processing task allocated by the management node, a data block M1x and a data block M2x that are in a to-be-processed dataset, wherein the data block M1x is a matrix comprising m rows and n columns of data, the data block M2x is a matrix comprising n rows and p columns of data, wherein m, n, and p are positive integers, and n is not less than 2;
• and
• perform a combine2 operation and a reduce2 operation on the data block M1x and the data block M2x, to obtain a first intermediate result M′x, wherein the first intermediate result M′x is a matrix comprising m rows and p columns of data;
• and the first intermediate result M′x has an element m′i,j, wherein i and j are variants, the value of i ranges from 1 to m, the value of j ranges from 1 to p, m ′i,j =m ′i,j,n is obtained according to m′i,j,k =reduce2(m′i,j,k−1, combine2(m1[i,k],m2[k,j])), m1[i,k] is an element in the ith row and the kth column of the data block M1x, M2[k,j] is an element in the kth row and the jth column of the data block M2x, k is a variant, and the value of k ranges from 1 to n;
• and the processor of the management node is further configured to:
• obtain a first processing result for the to-be-processed dataset according to first intermediate results obtained by the at least two computing nodes in the first class of computing nodes.

A data processing method for use in a data processing system that comprises a management node including a processor and a plurality of computing nodes that form a first class of computing nodes each of the first class of computing nodes including a processor, the method comprising:

• allocating, by the processor of the management node, a first processing task to each of at least two computing nodes in the first class of computing nodes, wherein a computing node FCx is the xth computing node in the at least two computing nodes, and x is a positive integer, and wherein the at least two computing nodes in the first class of computing nodes concurrently perform the first processing tasks allocated by the management node;
• obtaining, by the processor of the computing node FCx according to the first processing task allocated by the management node, a data block Mx and a data block V1x in a to-be-processed dataset, wherein the data block Mx is a matrix comprising m rows and n columns of data, the data block V1x is a vector comprising n-dimensional data, m and n are positive integers, and n is not less than 2;
• performing, by the processor of the computing node FCx, a combine2 operation and a reduce2 operation on the data block Mx and the data block V1x, to obtain a first intermediate result V′x, wherein the first intermediate result V′x, is a vector comprising m-dimensional data;
• and the first intermediate result V′x, has an element v′i, wherein i is a variant, the value of i ranges from 1 to m, v′i=v′i,n, v′i,n is obtained according to v′i,j=reduce2(v′i,j−1,combine2(mi,j,v j)), mi,j is an element in the data block Mx, vj is an element in the data block V1x, j is a variant, and the value of j ranges from 1 to n;
• and
• obtaining, by the processor of the management node, a first processing result for the to-be-processed dataset according to first intermediate results obtained by the at least two computing nodes in the first class of computing nodes.

A data processing method for use in a data processing system that comprises a management node including a processor and a plurality of computing nodes that form a first class of computing nodes, each of the first class of computing nodes including a processor, the method comprising:

• allocating, by the processor of the management node, a first processing task to each of at least two computing nodes in the first class of computing nodes, wherein a computing node FCx is the xth computing node in the at least two computing nodes, and x is a positive integer, and wherein the at least two computing nodes in the first class of computing nodes concurrently perform the first processing tasks allocated by the management node;
• obtaining, by the processor of the computing node FCX according to the first processing task allocated by the management node, a data block M1x and a data block M2x that are in a to-be-processed dataset, wherein the data block M1x is a matrix comprising m rows and n columns of data, the data block M2x is a matrix comprising n rows and p columns of data, m, n, and p are positive integers, and n is not less than 2;
• performing, by the processor of the computing node FCx, a combine2 operation and a reduce2 operation on the data block M1x and the data block M2x, to obtain a first intermediate result M′x, wherein the first intermediate result M′x is a matrix comprising m rows and p columns of data;
• and the first intermediate result M′x has an element m′ij, wherein i and j are variants, the value of i ranges from 1 to m, the value of j ranges from 1 to p, m′i,j =m′i,j,n,, m′i,j,nis obtained according to m′i,j,k=reduce2(m′i,j,k−1combine2(m1[i,k],m2[k,j])), m1[i,k]is an element in the ith row and the kth column of the data block M1x, m2[k,j]is an element in the kh row and the jth column of the data block M2x, k is a variant, and the value of k ranges from 1 to n;
• and
• obtaining, by the processor of the management node, a first processing result for the to-be-processed dataset according to first intermediate results obtained by the at least two computing nodes in the first class of computing nodes.