Automatic image annotation

A method of automatically annotating and labeling images including:

• receiving a reference image and at least one query image, wherein the reference image and the at least one query image include similar regions of interest and the reference image further includes at least one annotation delineating the region of interest (ROI) included therein;
• generating an S-matrix for the reference image wherein S-matrix includes singular values for the reference image;
• extracting a low dimensional reference image with the at least one annotation from the reference image using a pre-trained deep learning model;
• obtaining a vector representation of the reference image with the at least one annotation by combining the S-matrix of the reference image with the low dimensional reference image;
• determining a distance metric between the vector representation of the low dimensional reference image with the at least one annotation and a vector representation of a low dimensional query image corresponding to the query image;
• identifying a preliminary region of interest (ROI) within the at least one query image based on the distance metric, wherein the preliminary ROI includes at least a portion of the ROI in the reference image delineated by the at least one annotation;
• generating a preliminary output image, wherein the preliminary output image includes the at least one query image with a preliminary annotation that delineates the preliminary ROI;
• and
• optimizing the preliminary annotation based on a threshold which indicates a difference between a distribution representative of the reference image and a distribution representative of the at least one query image and the optimization causes a resizing of the preliminary ROI.

A non-transitory processor-readable storage medium comprising machine-readable instructions that cause a processor to:

• receive a reference image and at least one query image, wherein the reference image and the at least one query image include similar regions of interest and the reference image further includes at least one annotation highlighting the region of interest (ROI) therein;
• generate an S-matrix for the reference image wherein S-matrix includes singular values for the reference image;
• extract a low dimensional reference image with the at least one annotation from the reference image using a pre-trained deep learning model;
• obtain a vector representation of the reference image using a combination of the low dimensional reference image with the at least one annotation and the S-matrix of the reference image;
• determine a distance metric between the vector representation of the low dimensional reference image with the at least one annotation and a vector representation of a low dimensional query image associated with the query image;
• provide a preliminary output image based on the distance metric wherein the preliminary output image includes the at least one query image with a preliminary annotation that covers at least a portion of the ROI in the query image;
• optimize the preliminary annotation based on a threshold which indicates a difference between a distribution representative of the reference image and a distribution representative of the at least one query image and the optimization causes a resizing of the ROI;
• and
• resize a boundary box that forms the preliminary annotation based on the optimization so that the boundary box encloses the resized ROI.