System and method for auto-scaling optimizing optical character recogin ition (ocr) processing of documents based on workload prediction

A method for improving Optical Character Recognition (OCR) performances of a plurality of uploaded documents, the method comprising:

• determining if the plurality of uploaded documents include large documents by comparing a file size of each of the plurality of uploaded documents with a predetermined file size threshold, wherein the file size is calculated according to a page count and a resolution of each of the plurality of uploaded document, and the predetermined file size threshold is determined by a page count limit and a resolution limit;
• sending small documents among the plurality of uploaded documents, as small document files, to an OCR job queue of at least one OCR node, wherein the file size of each of the small documents is no greater than the predetermine file size threshold;
• dividing each of the large documents to multiple segmented document files, wherein the file size of each of the large documents is greater than the file size threshold, and the file size of each of the multiple segmented document files is no greater than the file size threshold;
• inserting markers to the multiple document files, wherein the markers define locations and orders of individual segmented document file in respective large documents and are used to track the multiple segmented document files that belong to a same large document;
• sending the multiple segmented document files of respective large documents to the job queue of the at least one OCR node;
• determining a total file size of the multiple segmented document files and the small document files received by the OCR job queue of the at least one OCR node;
• and
• if the total file size exceeds a predetermined OCR processing threshold, distributing some of the multiple segmented documents files and the small document files received by the OCR job queue of the at least one OCR node to at least one new OCR node for processing.

A system for performing Optical Character Recognition (OCR) on bulk uploaded document, the system comprising:

• a storage for storing a plurality of uploaded documents;
• a managing device accessible to the plurality of uploaded documents stored in the storage, comprising a processor, wherein the storage further stores medium-readable instructions, which when executed, causes the processor to:
• send small documents among the plurality of uploaded documents, as small document files, to an OCR job queue of at least one OCR node for performing the OCR, wherein each of the small documents has a file size no greater than a predetermined document size threshold;
• divide each of large documents among the plurality of uploaded documents to multiple segmented document files, wherein each of the large documents has a file size greater than the predetermined document size threshold, and wherein the file size of each of the multiple segmented document files is no greater than the predetermined document size threshold;
• send the multiple segmented document files of respective large documents to the OCR job queue of at least one OCR node for performing the OCR;
• and
• distributing some of the documents received in the job queue to at least one new node for processing when either one of the followings exists:
• a total processing time that the at least one OCR node takes to perform the OCR for documents received in the job queue exceeds a predetermined time threshold, and
• a total file size of the multiple segmented document files and the small document files received by the OCR job queue of the at least one OCR node exceeds an predetermined OCR processing threshold,
• wherein the predetermined time threshold is set based on characteristics of the at least one OCR node, and wherein the predetermined OCR processing threshold is set based on maximum file size the at least one OCR node can process within a predetermined period of time.