# US 12,002,054 B1 · Systems and methods for identity document fraud detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,002,054 B1 - Document ID: US12002054B1 - Status: Granted - Kind: B1 - Application number: 18/071,381 - Filed: 2022-11-29 - Published: 2024-05-30 - Granted: 2024-06-04 - Earliest claimed: 2022-11-29 - Assignees: Stripe, Inc. - Applicants: Stripe, Inc. - Inventors: Charlotte Gils; Efstathios Vafeias - Web page: https://patentgrep.com/patent/US12002054B1 ## CPC - G06V 30/1916: {Validation; Performance evaluation} - G06F 40/169: Annotation, e.g. comment data or footnotes - G06N 3/045: Combinations of networks - G06N 20/00: Machine learning - G06Q 20/4016: {involving fraud or risk level assessment in transaction processing} - G06Q 30/0185: {Product, service or business identity fraud} - G06Q 30/0609: {Qualifying participants for shopping transactions (payment transaction verification G06Q20/401)} - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 10/82: using neural networks - G06V 20/95: {Pattern authentication; Markers therefor; Forgery detection} - G06V 30/19013: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching (specially adapted for image segmentation G06T7/10; specially adapted for the analysis of motion G06T7/20; specially adapted for image alignment G06T7/30; specially adapted for the calculation of depth from stereo images G06T7/50; specially adapted for position determination G06T7/70)} - G06V 30/19147: {Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting} - G06V 30/41: Analysis of document content (recognition of printed characters based on code marks G06V30/224) - G06V 30/413: Classification of content, e.g. text, photographs or tables - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract A method and apparatus for fraud detection during transactions using identity graphs are described. The method may include receiving a document image for detecting whether an identity document depicted within the document image is fraudulent. The method may also include extracting data associated with the document image to generate extracted data. The method may also include processing, by a single machine learning model, subsets of the decoded image data used as corresponding inputs to each of a set of machine learning model backbones of the single machine learning model that generate one or more intermediate signals indicative of whether a subset of the extracted image data input into said each machine learning model backbone is associated with a fraudulent identity document. The method may also include processing, by a second machine learning model backbone that generates a final score indicative of whether the document image depicts a fraudulent identity document, at least one or more intermediate signals. ## Claims ### Claim 1 A method for fraudulent identity document detection using document images that depict identity documents, the method comprising: receiving, by a server computer system, a document image for detecting whether an identity document depicted within the document image is fraudulent; extracting, by one or more data extractors and decoders of the server computer system, data associated with the document image to generate extracted data comprising image data extracted from the document image, image file data extracted from an image file for the document image, or a combination thereof; processing, by a single machine learning model of the server computer system, a subset the extracted data used as an input to a set of machine learning model backbones of the single machine learning model, wherein each of the set of machine learning model backbones is associated with a machine learning model head, each machine learning model backbone generating a set of intermediate signals, and each machine learning model head receiving a set of intermediate signals generated by a corresponding machine learning model backbone and generating an intermediate fraud score indicative of whether the extracted data is associated with a fraudulent identity document; generating, by an identity fraud head of the single machine learning model, a final score indicative of whether the document image depicts a fraudulent identity document, the final score generated by the identity fraud head processing the sets of intermediate signals generated by the machine learning model backbones; and when the final score satisfies a fraudulent document detection threshold, determining, by the server computer system, that the document image depicts a fraudulent identity document. ### Claim 2 (depends on 1) The method of claim 1, wherein the set of machine learning model backbones comprises a plurality of different machine learning models that each generate an independent set of intermediate signals indicative of different types of fraud determination of the identity document depicted within the document image. ### Claim 3 (depends on 2) The method of claim 2, wherein the plurality of different machine learning model backbones form intermediate layers of the single machine learning model. ### Claim 4 (depends on 3) The method of claim 3, wherein the plurality of different machine learning models comprise: a convolutional neural network machine learning model backbone forming an intermediate layer of the single machine learning model, and trained as part of training the single machine learning model to generate from at least pixels of the document image a corresponding set of first intermediate signals, wherein the image data extracted from the document image comprises one or more pixel data extracted from the document image; a first transformer machine learning model backbone forming an intermediate layer of the single machine learning model, and trained as part of training the single machine learning model to analyze at least an email address obtained from one or more of user metadata and decoded user data in a context with additional user metadata and decoded user data to generate a second set of intermediate signals; a second transformer machine learning model backbone forming an intermediate layer of the single machine learning model, and trained as part of training the single machine learning model to analyze at least barcode data, machine readable zone data, or a combination thereof obtained from the decoded user data based on an order of individual data within the barcode data, machine readable zone data, or the combination thereof to generate a third set of intermediate signals; an identity feature machine learning model backbone comprising by a machine learning layer forming an intermediate layer of the single machine learning model, and trained as part of training the single machine learning model to determine from identity graph feature inputs a fourth set of intermediate signals; and a categorical feature machine learning model backbone forming an intermediate layer of the single machine learning model, and trained as part of training the single machine learning model to determine whether a set of categorical features are associated with fraud to generate a fifth set of intermediate signals, wherein the categorical features comprises two or more of an internet protocol (IP) address, a geographic location associated with the IP address, a cameral label, an device type, a device operating system, a browser, or a combination thereof. ### Claim 5 (depends on 3) The method of claim 3, wherein each of the plurality of different machine learning model backbones outputs a set of intermediate signals, the method further comprising: receiving, by a combiner layer of the single machine learning model, sets of intermediate signals output by the plurality of different machine learning model backbones; generating, by the combiner layer of the single machine learning model, a combined set of intermediate signals by concatenating the sets of intermediate signals; and providing, by the combiner layer of the single machine learning model, the combined set of intermediate signals as a single input to the identity fraud head for generation of the final score. ### Claim 6 (depends on 1) The method of claim 1, wherein the one or more decoders comprise an optical character recognition decoder, a barcode decoder, a machine readable zone decoder, or a combination thereof. ### Claim 7 (depends on 1) The method of claim 1, wherein the server computer system is comprised in a commerce platform system, the document image is received from a service system of the commerce platform system, and the service system provides the document image to the server computer system with a request to verify the identity document depicted thereon as a condition of the service system providing a service to a user purported to be identified by the identity document. ### Claim 8 (depends on 7) The method of claim 7, further comprising: transmitting a positive fraud determination, by the server computer system to the service system, indicative of the fraudulent identity document determination when the final score satisfies a fraudulent document detection threshold; and transmitting a negative fraud determination, by the server computer system to the service system, indicative of a non-fraudulent identity document determination when the final score does not satisfy the fraudulent document detection threshold. ### Claim 9 (depends on 8) The method of claim 8, wherein each intermediate fraud score is further output to a data store, and the method further comprises: accessing, by the server computer system, one or more intermediate fraud scores in the data store; and generating, by the server computer system, explanation data using the accessed one or more intermediate fraud scores, the explanation data indicative of how one or more machine learning model backbones contributed to the final score indicative of whether the document image depicts a fraudulent identity document. ### Claim 10 (depends on 9) The method of claim 9, further comprising: transmitting, by the server computer system to the service system, the explanation data with the positive fraud determination or the negative fraud determination to explain a determination reached by the single machine learning model with respect to a fraudulent nature of the identity document depicted within the document image. ### Claim 11 (depends on 1) The method of claim 1, wherein the single machine learning model comprises a neural network machine learning model that comprises at least one tree based machine learning model backbone, at least one neural network machine learning model backbone, and at least one transformer machine learning model backbone, and wherein each heads associated with a backbone and the identity fraud head comprise multilayer perceptron heads, and wherein the method further comprises: receiving, by the server computer system, feedback data on the determination as to whether the document image depicts the fraudulent identity document, wherein the feedback data differs from a determined fraudulent nature of the identity document depicted within the document image; annotating the document image with the feedback data; storing the annotated document image in training data store with a collection of machine learning model training data; and periodically retraining the single machine learning model, causing each machine learning model backbone and each head of the single machine learning model to be retrained, using a single sampling of the collection of machine learning model training data from the training data store. ### Claim 12 A non-transitory computer readable storage medium storing instructions, which when executed by a computer processing system, causes the computer processing system to perform operations for fraudulent identity document detection using document images that depict identity documents, the operations comprising: receiving, by a server computer system, a document image for detecting whether an identity document depicted within the document image is fraudulent; extracting, by one or more data extractors and decoders of the server computer system, data associated with the document image to generate extracted data comprising image data extracted from the document image, image file data extracted from an image file for the document image, or a combination thereof; processing, by a single machine learning model of the server computer system, a subset the extracted data used as an input to a set of machine learning model backbones of the single machine learning model, wherein each of the set of machine learning model backbones is associated with a machine learning model head, each machine learning model backbone generating a set of intermediate signals, and each machine learning model head receiving a set of intermediate signals generated by a corresponding machine learning model backbone and generating an intermediate fraud score indicative of whether the extracted data is associated with a fraudulent identity document; generating, by an identity fraud head of the single machine learning model, a final score indicative of whether the document image depicts a fraudulent identity document, the final score generated by the identity fraud head processing the sets of intermediate signals generated by the machine learning model backbones; and when the final score satisfies a fraudulent document detection threshold, determining, by the server computer system, that the document image depicts a fraudulent identity document. ### Claim 13 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the set of machine learning model backbones comprises a plurality of different machine learning models that each generate an independent set of intermediate signals indicative of different types of fraud determination of the identity document depicted within the document image. ### Claim 14 (depends on 13) The non-transitory computer readable storage medium of claim 13, wherein the plurality of different machine learning model backbones form intermediate layers of the single machine learning model, and wherein each of the plurality of different machine learning model backbones outputs a set of intermediate signals, the method further comprising: receiving, by a combiner layer of the single machine learning model, sets of intermediate signals output by the plurality of different machine learning model backbones; generating, by the combiner layer of the single machine learning model, a combined set of intermediate signals by concatenating the sets of intermediate signals; and providing, by the combiner layer of the single machine learning model, the combined set of intermediate signals as a single input to the identity fraud head for generation of the final score. ### Claim 15 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the server computer system is comprised in a commerce platform system, the document image is received from a service system of the commerce platform system, and the service system provides the document image to the server computer system with a request to verify the identity document depicted thereon as a condition of the service system providing a service to a user purported to be identified by the identity document. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, further comprising: transmitting a positive fraud determination, by the server computer system to the service system, indicative of the fraudulent identity document determination when the final score satisfies a fraudulent document detection threshold; and transmitting a negative fraud determination, by the server computer system to the service system, indicative of a non-fraudulent identity document determination when the final score does not satisfy the fraudulent document detection threshold. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein each intermediate fraud score is further output to a data store, and the method further comprises: accessing, by the server computer system, one or more intermediate fraud scores in the data store; and generating, by the server computer system, explanation data using the accessed one or more intermediate fraud scores, the explanation data indicative of how one or more machine learning model backbones contributed to the final score indicative of whether the document image depicts a fraudulent identity document. ### Claim 18 (depends on 17) The non-transitory computer readable storage medium of claim 17, further comprising: transmitting, by the server computer system to the service system, the explanation data with the positive fraud determination or the negative fraud determination to explain a determination reached by the single machine learning model with respect to a fraudulent nature of the identity document depicted within the document image. ### Claim 19 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the single machine learning model comprises a neural network machine learning model that comprises at least one tree based machine learning model backbone, at least one neural network machine learning model backbone, and at least one transformer machine learning model backbone, and wherein each heads associated with a backbone and the identity fraud head comprise multilayer perceptron heads, and wherein the method further comprises: receiving, by the server computer system, feedback data on the determination as to whether the document image depicts the fraudulent identity document, wherein the feedback data differs from a determined fraudulent nature of the identity document depicted within the document image; annotating the document image with the feedback data; storing the annotated document image in training data store with a collection of machine learning model training data; and periodically retraining the single machine learning model, causing each machine learning model backbone and each head of the single machine learning model to be retrained, using a single sampling of the collection of machine learning model training data from the training data store. ### Claim 20 A server computer system for fraudulent identity document detection using document images that depict identity documents, the system comprising: a memory; and one or more processors coupled with the memory configured to perform operations, comprising: receiving a document image for detecting whether an identity document depicted within the document image is fraudulent; extracting, by one or more data extractors and decoders, data associated with the document image to generate extracted data comprising image data extracted from the document image, image file data extracted from an image file for the document image, or a combination thereof; processing, by a single machine learning model, a subset the decoded image data used as an input to a set of machine learning model backbones of the single machine learning model, wherein each of the set of machine learning model backbones is associated with a machine learning model head, each machine learning model backbone generating a set of intermediate signals, and each machine learning model head receiving a set of intermediate signals generated by a corresponding machine learning model backbone and generating an intermediate fraud score indicative of whether the extracted data is associated with a fraudulent identity document; generating, by an identity fraud head of the single machine learning model, a final score indicative of whether the document image depicts a fraudulent identity document, the final score generated by the identity fraud head processing the sets of intermediate signals generated by the machine learning model backbones; and when the final score satisfies a fraudulent document detection threshold, determining that the document image depicts a fraudulent identity document.