# US 12,118,702 B2 · Artificial intelligence architectures for determining image authenticity > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,702 B2 - Document ID: US12118702B2 - Status: Granted - Kind: B2 - Application number: 18/528,949 - Filed: 2023-12-05 - Published: 2024-04-04 - Granted: 2024-10-15 - Earliest claimed: 2021-06-23 - Assignees: Lendbuzz, Inc. - Applicants: Lendbuzz, Inc. - Inventors: Otkrist Gupta; Dan Raviv; Hailey James - Web page: https://patentgrep.com/patent/US12118702B2 ## CPC - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/24: Classification techniques - G06N 20/00: Machine learning - G06Q 40/03: Credit; Loans; Processing thereof - G06T 7/00: Image analysis - G06V 10/82: using neural networks - G06V 20/95: {Pattern authentication; Markers therefor; Forgery detection} - G06V 30/40: Document-oriented image-based pattern recognition - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/048: Activation functions - G06T 2207/10024: Color image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30176: Document - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present disclosure generally relates to systems that include an artificial intelligence (AI) architecture for determining whether an image is manipulated. The architecture can include a constrained convolutional layer, separable convolutional layers, maximum-pooling layers, a global average-pooling layer, and a fully connected layer. In one specific example, the constrained convolutional layer can detect one or more image-manipulation fingerprints with respect to an image and can generate feature maps corresponding to the image. The global average-pooling layer can generate a vector of feature values by averaging the feature maps. The fully connected layer can then generate, based on the vector of feature values, an indication of whether the image was manipulated or not manipulated. ## Claims ### Claim 1 A system comprising: one or more processors; and a non-transitory computer-readable medium including instructions for a trained machine-learning model that is executable by the one or more processors to detect a manipulation of an image, the trained machine-learning model including: a constrained convolutional layer configured to employ a filter for detecting one or more image-manipulation fingerprints in the image and to generate a feature map associated with the image-manipulation fingerprints; a plurality of separable convolutional layers configured to perform a respective spatial convolution on each color channel of the image based on the feature map to generate a plurality of feature maps corresponding to the image; a global average-pooling layer configured to average the plurality of feature maps to generate a vector of feature values; and one or more additional layers configured to output, based on the vector of feature values, an indication of whether the image was manipulated or not manipulated. ### Claim 2 (depends on 1) The system of claim 1, wherein the trained machine-learning model is configured to: detect whether the image was manipulated via a print-and-scan attack; and based on detecting that the image was manipulated via the print-and-scan attack, generate an output indicating a printer type used in the print-and-scan attack. ### Claim 3 (depends on 1) The system of claim 1, wherein the filter is at least a five-by-five filter, and the plurality of separable convolutional layers include at least 30 separable convolutional layers. ### Claim 4 (depends on 1) The system of claim 1, wherein the trained machine-learning model is a convolutional neural network trained on a set of training images configured to imitate a print-and-scan attack to enable the convolutional neural network to learn the one or more image-manipulation fingerprints corresponding to the print-and-scan attack. ### Claim 5 (depends on 4) The system of claim 4, wherein the set of training images includes a plurality of images generated using a plurality of different printers. ### Claim 6 (depends on 1) The system of claim 1, wherein the non-transitory computer-readable medium further includes instructions that are executable by the one or more processors to use the trained machine-learning model as part of a loan verification process to validate images relating to a loan application. ### Claim 7 (depends on 1) The system of claim 1, wherein the trained machine-learning model further comprises a plurality of max-pooling layers. ### Claim 8 (depends on 1) The system of claim 1, wherein the non-transitory computer-readable medium further includes instructions that are executable by the one or more processors to: provide the image as input to the trained machine-learning model, the trained machine-leaning model being configured to generate the vector of feature values and provide the vector of feature values from the global average-pooling layer to a fully connected layer, the fully connected layer being configured to provide the vector of feature values to a softmax layer, the softmax layer being configured to classify the image as manipulated or not manipulated based on the vector of feature values, wherein the one or more additional layers include the fully connected layer and the softmax layer; and receive the indication as output from the softmax layer of the trained machine-learning model. ### Claim 9 A non-transitory computer-readable medium including instructions for a trained machine-learning model that is executable by one or more processors to detect a manipulation of an image, the trained machine-learning model including: a constrained convolutional layer configured to employ a filter for detecting one or more image-manipulation fingerprints in the image and to generate a feature map associated with the image-manipulation fingerprints; a plurality of separable convolutional layers configured to perform a respective spatial convolution on each color channel of the image based on the feature map to generate a plurality of feature maps corresponding to the image; a global average-pooling layer configured to average the plurality of feature maps to generate a vector of feature values; and one or more additional layers configured to output, based on the vector of feature values, an indication of whether the image was manipulated or not manipulated. ### Claim 10 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the trained machine-learning model is configured to, based on detecting that the image was manipulated via a print-and-scan attack, generate an output indicating a printer type used in the print-and-scan attack. ### Claim 11 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the filter is at least a five-by-five filter, and the plurality of separable convolutional layers include at least 20 separable convolutional layers. ### Claim 12 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the trained machine-learning model is a convolutional neural network trained on a set of training images configured to imitate a print-and-scan attack to enable the convolutional neural network to learn the one or more image-manipulation fingerprints corresponding to the print-and-scan attack. ### Claim 13 (depends on 12) The non-transitory computer-readable medium of claim 12, wherein the set of training images includes a plurality of images generated by using a plurality of different printers. ### Claim 14 (depends on 9) The non-transitory computer-readable medium of claim 9, further including instructions that are executable by the one or more processors to use the trained machine-learning model as part of a loan verification process to validate images relating to a loan application. ### Claim 15 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the trained machine-learning model further comprises a plurality of max-pooling layers. ### Claim 16 (depends on 9) The non-transitory computer-readable medium of claim 9, further including instructions that are executable by the one or more processors to: provide the image as input to the trained machine-learning model, the trained machine-leaning model being configured to generate the vector of feature values and provide the vector of feature values from the global average-pooling layer to a fully connected layer, the fully connected layer being configured to provide the vector of feature values to a softmax layer, the softmax layer being configured to classify the image as manipulated or not manipulated based on the vector of feature values, wherein the one or more additional layers include the fully connected layer and the softmax layer; and receive the indication as output from the softmax layer of the trained machine-learning model. ### Claim 17 A method comprising: receiving, by a processor, an image that is to be analyzed for authenticity; providing, by the processor, the image as input to a trained machine-learning model for detecting a manipulation of the image, wherein the trained machine-learning model includes: a constrained convolutional layer configured to employ a filter for detecting one or more image-manipulation fingerprints in the image and to generate a feature map associated with the image-manipulation fingerprints; a plurality of separable convolutional layers configured to perform a respective spatial convolution on each color channel of the image based on the feature map to generate a plurality of feature maps corresponding to the image; a global average-pooling layer configured to average the plurality of feature maps to generate a vector of feature values; and one or more additional layers configured to output, based on the vector of feature values, an indication of whether the image was manipulated or not manipulated; receiving, by the processor, the indication of whether the image was manipulated or not manipulated from the trained machine-learning model; and outputting, by the processor and based on the indication, a classification of the image as manipulated or not manipulated for display on a display device. ### Claim 18 (depends on 17) The method of claim 17, further comprising: generating the feature map at the constrained convolutional layer by applying the filter to the image; generating the plurality of features maps at the plurality of separable convolutional layers by performing the respective spatial convolution on each color channel of the image; generating the vector of feature values at the global average-pooling layer by averaging the plurality of feature maps; and determining the indication based on the vector of feature values. ### Claim 19 (depends on 17) The method of claim 17, wherein the trained machine-learning model is configured to detect whether images have been manipulated by a print-and-scan attack, wherein the trained machine-learning model is a convolutional neural network trained on a set of training images configured to imitate the print-and-scan attack to enable the convolutional neural network to learn the one or more image-manipulation fingerprints corresponding to the print-and-scan attack, and wherein the set of training images includes a plurality of images generated by using a plurality of different printers. ### Claim 20 (depends on 19) The method of claim 19, wherein the trained machine-learning model is configured to, based on detecting that the image was manipulated via the print-and-scan attack, generate an output indicating a printer type used in the print-and-scan attack.