Back to results
US 12,762,375 B2Granted
Techniques for matching disparate input data
AssigneeVisa International Service Association
InventorsYichun Shi, Lacey Best-Rowden, Kim Wagner
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed for training a generative adversarial network (GAN) to transform images of one type (e.g., a selfie) to images of a second type (e.g., an ID document image). Once trained, the GAN may be utilized to generate an augmented training set that includes pairs of images (e.g., an image of the first type paired with an image of the second type, an image of the second type generated from an image of the first type paired with an image of the second type). The augmented training data set may be utilized to train a matching model to identify when subsequent input images (e.g., a selfie and an ID image, an ID image generated from a selfie and an actual ID image) match.
Claims15
A computer-implemented method, comprising:
- receiving, by a data processing computer, input data comprising a first input image and a second input image for processing by a machine-learning model, wherein the first input image is a captured user image of a first type that is captured in a real time, the machine-learning model formed by:
- (i) obtaining, by the data processing computer, an initial training set comprising a first set of original first images of the first type and a second set of original second images of a second type,
- (ii) training a neural network using the initial training set by:
- determining a first transformation function for generating images of the second type from the original first images of the first type,
- determining a second transformation function for generating images of the first type from the original second images of the second type,
- verifying the first transformation function by (a) using the first transformation function to transform an original first image of the first type to a transformed image of the second type, (b) providing the transformed image of the second type as an input to the second transformation function to transform the transformed image of the second type to a transformed image of the first type, (c) applying a perceptual loss function on the transformed image of the first type and the original first image of the first type, and (d) determining whether the transformed image of the first type matches the original first image of the first type within a first threshold,
- verifying the second transformation function by (e) using the second transformation function to transform an original second image of the second type to a transformed image of the first type, (f) providing the transformed image of the first type as an input to the first transformation function to transform the transformed image of the first type to a transformed image of the second type, (g) applying a perceptual loss function on the transformed image of the second type and the original second image of the second type, and (h) determining whether the transformed image of the second type matches the original second image of the second type within a second threshold,
- wherein the first transformation function and the second transformation function are verified when the transformed image of the first type matches the original first image of the first type within the first threshold and the transformed image of the second type matches the original second image of the second type within the second threshold, and
- stopping the training the neural network when the first transformation function and the second transformation function are verified,
- wherein the neural network is different from the machine-learning model,
- (iii) in response to the first transformation function and the second transformation function being verified, generating, by the data processing computer using the neural network that is trained, an augmented training data set by
- applying the first transformation function on the original first images from the first set to generate second generated images of the second type,
- applying the second transformation function on the original second images from the second set to generate first generated images of the first type,
- associating the second generated images with matching original first images from the first set, to create first pairs of matching images,
- associating the first generated images with matching original second images from the second set, to create second pairs of matching images, wherein each of the first pairs and each of the second pairs is labeled as match examples,
- generating at least one mismatched pair of images by associating at least one second image of the second type and at least one first image of the first type that are obtained from the initial training set and do not match, wherein the at least one second image of the second type and the at least one first image of the first type are not generated by the neural network, and
- generating the augmented training data set to include the first pairs, the second pairs, and the at least one mismatched pair, and
- (iv) training the machine-learning model to identify whether two input images match or not, using the augmented training data set comprising the first pairs, the second pairs, and the at least one mismatched pair;
- transforming the first image using the first transformation function to a transformed user image of the second type;
- providing, by the data processing computer as an input to the machine-learning model, the transformed user image of the second type and the second input image, to determine whether the first input image matches the second input image;
- determining, by the machine-learning model, whether the first input image matches the second input image based on the transformed user image of the second type and the second input image;
- and
- executing, by the data processing computer, at least one operation in response to receiving output of the machine-learning model indicating the first input image matches the second input image.
20 more elements
A data processing computer, comprising:
- one or more processors;
- and
- one or more memories storing computer-executable instructions, wherein executing the computer-executable instructions by the one or more processors causes the data processing computer to:
- receive input data comprising a first input image and a second input image for processing by a machine-learning model, wherein the first input image is a captured user image of a first type that is captured in a real time, the machine-learning model formed by:
- (i) obtaining an initial training set comprising a first set of original first images of the first type and a second set of original second images of a second type,
- (ii) training a neural network using the initial training set by:
- determining a first transformation function for generating images of the second type from the original first images of the first type,
- determining a second transformation function for generating images of the first type from the original second images of the second type,
- verifying the first transformation function by (a) using the first transformation function to transform an original first image of the first type to a transformed image of the second type, (b) providing the transformed image of the second type as an input to the second transformation function to transform the transformed image of the second type to a transformed image of the first type, (c) applying a perceptual loss function on the transformed image of the first type and the original first image of the first type, and (d) determining whether the transformed image of the first type matches the original first image of the first type within a first threshold,
- verifying the second transformation function by (e) using the second transformation function to transform an original second image of the second type to a transformed image of the first type, (f) providing the transformed image of the first type as an input to the first transformation function to transform the transformed image of the first type to a transformed image of the second type, (g) applying a perceptual loss function on the transformed image of the second type and the original second image of the second type, and (h) determining whether the transformed image of the second type matches the original second image of the second type within a second threshold,
- wherein the first transformation function and the second transformation function are verified when the transformed image of the first type matches the original first image of the first type within the first threshold and the transformed image of the second type matches the original second image of the second type within the second threshold, and
- stopping the training the neural network when the first transformation function and the second transformation function are verified,
- wherein the neural network is different from the machine-learning model,
- (iii) in response to the first transformation function and the second transformation function being verified, generate, using the neural network that is trained, an augmented training data set by:
- applying the first transformation function on the original first images from the first set to generate second generated images of the second type,
- applying the second transformation function on the original second images from the second set to generate first generated images of the first type,
- associating the second generated images with matching original first images from the first set, to create first pairs of matching images,
- associating the first generated images with matching original second images from the second set, to create second pairs of matching images, wherein each of the first pairs and each of the second pairs is labeled as match examples,
- generating at least one mismatched pair of images by associating at least one second image of the second type and at least one first image of the first type that are obtained from the initial training set and do not match, wherein the at least one second image of the second type and the at least one first image of the first type are not generated by the neural network, and
- generating the augmented training data set to include the first pairs, the second pairs, and the at least one mismatched pair, and
- (iv) train the machine-learning model to identify whether two input images match or not, using the augmented training data set comprising the first pairs, the second pairs, and the at least one mismatched pair;
- transform the first image using the first transformation function to a transformed user image of the second type;
- provide, as an input to the machine-learning model, the transformed user image of the second type and the second input image, for determining, by the machine-learning model, whether the first input image matches the second input image based on the transformed user image of the second type and the second input image;
- and
- execute at least one operation in response to receiving output of the machine-learning model indicating the first input image matches the second input image.
22 more elements
Dependencies are inferred from the claim text.