Training-free framework for zero-shot check field detection

A computer-implemented method, comprising:

• providing production input data as input into a field detection model, wherein the field detection model was trained by a method comprising:
• accessing input data, wherein the input data includes image data;
• providing the input data to a first branch comprising a first vision model;
• processing the input data with the first branch, wherein the processing includes using the first vision model to identify a first set of fields in the input data using a first set of bounding boxes;
• recognizing at least one field in the first set of fields as an identified field;
• labeling the recognized at least one field in the first set of fields as a labeled identified field;
• providing the input data to a second branch comprising a second vision model and a third model;
• processing the input data with the second branch, wherein the processing includes:
• using the second vision model to identify a second set of fields in the input data using a second set of bounding boxes;
• passing the input data with the second set of bounding boxes to the third model;
• and
• causing the third model to output a set of recognized fields within the bounding boxes of the second set of bounding boxes;
• recognizing at least one field in the second set of fields as a target field;
• labeling the recognized at least one field in the second set of fields as a labeled target field;
• and
• combining the input data, the labeled identified field, and the labeled target field to form labeled data.

A computer system comprising a non-transitory computer readable medium having stored thereon code of a field detection model trained by operations comprising:

• receiving input data in a first branch comprising a first model, the input data comprising an image of a document, the first model adapted to identify a first set of fields in the input data using a first set of bounding boxes;
• labeling, by the first model, the first set of fields to output a labeled first set of fields;
• identifying an identified field as a desired type of field using a corresponding bounding box from the first set of bounding boxes;
• outputting from the first branch, a localized and labeled identified field;
• passing the input data to a second branch comprising a second model and a third model, the second model adapted to identify a second set of fields in the input data using a visualized second set of bounding boxes;
• passing the input data with the second set of bounding boxes to the third model executing in the second branch, the third model outputting a set of recognizing field within bounding boxes of the second set of bounding boxes;
• identifying at least one recognized field of the second set of fields as a target field;
• labeling the target field to output from the second branch a labeled target field;
• combining to form labeled data in a training data set, the input data, the labeled identified field, and the labeled target field;
• and
• training the model with the training data set.