Training-free framework for zero-shot check field detection

A computer program product comprising one or more non-transitory computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by a set of one or more processors to cause the set of one or more processors to perform operations comprising:

• receiving input data at zero-shot configuration (Zcon) comprising a first Vision-Language Model (VLM), a second VLM, and a Multimodal Large Language Model (MLLM), the input data comprising facsimile representation of a paper document;
• passing the input data to a first branch of the Zcon, the first VLM adapted to operate in the first branch and identify a first set of fields in the input data using a visualized first set of bounding boxes (BB);
• executing in the first branch a first agentic Artificial Intelligence architecture (AAA-1), the executing causing a localization of an identified field as a desired type of filed using a corresponding visualized identified BB from the first set of BBs;
• passing the input data to a second branch of the Zcon, the second VLM adapted to operate in the second branch, identify a second set of fields in the input data, and output a visualized second set of BBs;
• passing the input data with the second set of BBs to the MLLM executing in the second branch, the MLLM outputting a set of recognizing fields within BBs of the second set of BBs;
• and
• localizing, by a second agentic Artificial Intelligence architecture (AAA-2) executing in the second branch, at least one recognized field as a target field.

A computer system comprising a set of one or more processors and one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by the set of one or more processors to cause the processor to perform operations comprising:

• receiving input data at zero-shot configuration (Zcon) comprising a first Vision-Language Model (VLM), a second VLM, and a Multimodal Large Language Model (MLLM), the input data comprising facsimile representation of a paper document;
• passing the input data to a first branch of the Zcon, the first VLM adapted to operate in the first branch and identify a first set of fields in the input data using a visualized first set of bounding boxes (BB);
• executing in the first branch a first agentic Artificial Intelligence architecture (AAA-1), the executing causing a localization of an identified field as a desired type of filed using a corresponding visualized identified BB from the first set of BBs;
• passing the input data to a second branch of the Zcon, the second VLM adapted to operate in the second branch, identify a second set of fields in the input data, and output a visualized second set of BBs;
• passing the input data with the second set of BBs to the MLLM executing in the second branch, the MLLM outputting a set of recognizing fields within BBs of the second set of BBs;
• and
• localizing, by a second agentic Artificial Intelligence architecture (AAA-2) executing in the second branch, at least one recognized field as a target field.