System and method for extracting object information from digital images to evaluate for realism

A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to extract object information from digital images by:

• receiving, at a realism assessment system, a user prompt involving real-world content;
• submitting, from the realism assessment system and to a web-based search engine, the user prompt;
• receiving, at the realism assessment system, a reference digital image retrieved by the search engine in response to the user prompt;
• submitting, from the realism assessment system and to a first generative AI model, the user prompt;
• receiving, at the realism assessment system, a plurality of synthetic digital images including a first synthetic digital image and a second synthetic digital image, the plurality of synthetic digital images generated by the first generative AI model in response to the user prompt;
• automatically classifying, via a deep learning-based instance segmentation model of the realism assessment system, a first set of pixels in the reference digital image as corresponding to a first object, a second set of pixels in the first synthetic digital image as corresponding to a second object, and a third set of pixels in the second synthetic digital image as corresponding to a third object;
• extracting one or more Hu Moments for each of the first object, the second object, and the third object;
• generating, at the realism assessment system, a first realism score based on a comparison of characteristics of the first object with the second object by comparing the one or more Hu Moments for the first object and the second object;
• generating, at the realism assessment system, a second realism score based on a comparison of characteristics of the first object with the third object by comparing the one or more Hu Moments for the first object and the third object, the second realism score being greater than the first realism score;
• determining, at the realism assessment system, the second synthetic digital image has a greater likelihood of accurately representing the real-world content than the first synthetic digital image based on the second realism score being greater than the first realism score;
• and
• ranking each synthetic digital image of the plurality of synthetic digital images based on their computed realism score and presenting, at a computing device, the synthetic digital images in order of their ranking.

A system for extracting object information from digital images comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to:

• receive, at a realism assessment system, a user prompt involving real-world content;
• submit, from the realism assessment system and to a web-based search engine, the user prompt;
• receive, at the realism assessment system, a reference digital image retrieved by the search engine in response to the user prompt;
• submit, from the realism assessment system and to a first generative AI model, the user prompt;
• receive, at the realism assessment system, a plurality of synthetic digital images including a first synthetic digital image and a second synthetic digital image, the plurality of synthetic digital images generated by the first generative AI model in response to the user prompt;
• automatically classify, via a deep learning-based instance segmentation model of the realism assessment system, a first set of pixels in the reference digital image as corresponding to a first object, a second set of pixels in the first synthetic digital image as corresponding to a second object, and a third set of pixels in the second synthetic digital image as corresponding to a third object;
• extract one or more Hu Moments for each of the first object, the second object, and the third object;
• generate, at the realism assessment system, a first realism score based on a comparison of characteristics of the first object with the second object by comparing the one or more Hu Moments for the first object and the second object;
• generate, at the realism assessment system, a second realism score based on a comparison of characteristics of the first object with the third object by comparing the one or more Hu Moments for the first object and the third object, the second realism score being greater than the first realism score;
• determine, at the realism assessment system, the second synthetic digital image has a greater likelihood of accurately representing the real-world content than the first synthetic digital image based on the second realism score being greater than the first realism score;
• and
• rank each synthetic digital image of the plurality of synthetic digital images based on their computed realism score and present, at a computing device, the synthetic digital images in order of their ranking.