Method, medium, and system for detecting cross-lingual comparable listings for machine translation using image similarity

A computer-implemented method comprising:

• translating a first listing from a first language to a second language, the first listing including at least one image of a first item;
• providing as input to a encoded neural network model at least one image portion and at least one text portion of the translated first listing and at least one image portion and at least one text portion of a second listing in the second language, the second listing including at least one image of a second item;
• receiving from the encoded neural network model a first feature vector for the translated first listing and a second feature vector for the second listing, the first and the second feature vectors both include at least one type of image signature feature and at least one type of listing text-based feature;
• calculating, via at least one processor, a similarity score based on the first feature vector and the second feature vector, and
• based on the similarity score at least meeting a similarity score threshold, generating a pairing of the first listing in the first language with the second listing in the second language for inclusion in a database of training data of a machine translation system, wherein the pairing is based on a degree of likelihood that the first item and the second item are the same.

A non-transitory computer-readable medium storing executable instructions thereon that, when executed by a processor, cause the processor to perform operations including:

• translating a first listing from a first language to a second language, the first listing including at least one image of a first item;
• providing as input to an encoded neural network model at least one image portion and at least one text portion of the translated first listing and at least one image portion and at least one text portion of a second listing in the second language, the second listing including at least one image of a second item;
• receiving from the encoded neural network model a first feature vector for the translated first listing and a second feature vector for the second listing, the first and the second feature vectors both include at least one type of image signature feature and at least one type of listing text-based feature;
• calculating a similarity score based on the first feature vector and the second feature vector, and
• based on the similarity score at least meeting a similarity score threshold, generating a pairing of the first listing in the first language with the second listing in the second language for inclusion in a database training data of a machine translation system, wherein the pairing is based on a degree of likelihood that the first item and the second item are the same.