Systems and methods for handwriting recognition using optical character recognition

A merchant system comprising:

• at least one processor;
• and
• at least one memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to cause the merchant system to perform operations comprising:
• obtaining an image of a receipt, wherein the image comprises a reference region in a pre-set format, a plurality of category identifiers, and at least one set of handwritten characters, wherein the reference region and the plurality of category identifiers are printed, and wherein the plurality of category identifiers is located at pre-set positions relative to the reference region;
• obtaining optical character recognition (OCR) information of the image;
• identifying the reference region from the OCR information based on the pre-set format of the reference region;
• determining a plurality of amounts based on the OCR information and the reference region, wherein at least one of the plurality of amounts is associated with a tip category;
• validating the at least one of the plurality of amounts associated with the tip category based on at least one of a set of validation rules wherein the set of validation rules require (1) a purchase amount, a tip amount, and a total amount that are larger than zero, (2) the purchase amount plus the tip amount equals the total amount, and (3) the tip amount is within a threshold percentage of the purchase amount;
• and
• storing the at least one of the plurality of amounts associated with the tip category into a point of sale (POS) system.

A non-transitory computer storage medium, wherein the non-transitory computer storage medium is coupled to at least one processor of a merchant system and stores programming instructions for execution by the at least one processor to cause the merchant system to perform operations comprising:

• obtaining an image of a receipt, wherein the image comprises a reference region in a pre-set format, a plurality of category identifiers, and at least one set of handwritten characters, wherein the reference region and the plurality of category identifiers are printed, and wherein the plurality of category identifiers is located at pre-set positions relative to the reference region;
• obtaining optical character recognition (OCR) information of the image;
• identifying the reference region from the OCR information based on the pre-set format of the reference region;
• determining a plurality of amounts based on the OCR information and the reference region, wherein at least one of the plurality of amounts is associated with a tip category;
• validating the at least one of the plurality of amounts associated with the tip category based on at least one of a set of validation rules wherein the set of validation rules require (1) a purchase amount, a tip amount, and a total amount that are larger than zero, (2) the purchase amount plus the tip amount equals the total amount, and (3) the tip amount is within a threshold percentage of the purchase amount;
• and
• storing the at least one of the plurality of amounts associated with the tip category into a point of sale (POS) system.