Distributed computing on document formats
AssigneeInternational Business Machines Corporation
InventorsGregory S. Felderman, Brian K. Hoyt
Life of this patent
Estimating the term…
Abstract
According to one embodiment of the present invention, a system includes at least one processor. The system partitions a document into a plurality of data blocks, wherein each data block comprises one or more complete logical units of the document. A plurality of sub-documents is produced from the plurality of data blocks. The sub-documents are processed in parallel by a plurality of processing elements. Embodiments of the present invention further include a method and computer program product for processing a document in parallel in substantially the same manner described above.
Claims7
A method comprising:
- partitioning a document into a plurality of data blocks of equal number of bits and storing the plurality of data blocks across a plurality of processing elements of a distributed file system;
- in response to determining that one or more data blocks are not partitioned on a boundary of complete logical units of the document, adjusting the one or more data blocks of the plurality of data blocks to comprise one or more complete logical units of the document, wherein at least one data block of the one or more data blocks is adjusted by removing data from an adjacent data block that is stored on a different processing element from the processing element of the at least one data block, transferring, via a network, the data from the different processing element of the adjacent data block to the processing element of the at least one data block, and inserting the data into the at least one data block to result in the at least one data block including one or more complete logical units;
- producing a plurality of sub-documents from the plurality of data blocks including the one or more adjusted data blocks, wherein each sub-document is a stand-alone document that is formatted in a format of the document;
- and
- processing the plurality of sub-documents in parallel by a plurality of processing elements.
Dependencies are inferred from the claim text.