Difference metric for machine learning-based processing systems

A computer-readable storage medium storing a deep neural network trained to provide a perceptual loss score by:

• initializing a committee of deep neural networks using different sets of labeled training distortion pairs, each deep neural network providing, at an end of initializing, a perceptual loss score for a given distortion pair;
• generating a set of millions of unlabeled distortion pairs by an artifact engine;
• and
• iteratively training the committee of deep neural networks until convergence by:
• identifying a plurality of informative distortion pairs by obtaining a perceptual loss score from each deep neural network in the committee for each unlabeled distortion pair in the set of unlabeled distortion pairs and selecting, as the plurality of informative distortion pairs, unlabeled distortion pairs from the set of unlabeled distortion pairs having highest diversity in the perceptual loss scores,
• obtaining labels for the plurality of informative distortion pairs, and
• retraining the deep neural networks of the committee using the labels for the plurality of informative distortion pairs.

A system comprising:

• at least one processor;
• and
• memory storing instructions that, when executed by the at least one processor, cause the system to perform operations including:
• learning a perceptual loss function by:
• using an artifact engine for generating millions of unlabeled distortion pairs by, for each base file, generating a respective distorted file from the base file,
• from a perceptual loss committee that includes a plurality of deep neural networks, each deep neural network having a respective perceptual loss function, obtaining a respective set of perceptual loss scores for each unlabeled distortion pair of the millions of unlabeled distortion pairs, the respective set including a perceptual loss score from each of the deep neural networks,
• identifying a set of distortion pairs from the millions of unlabeled distortion pairs, each distortion pair in the set of distortion pairs having a respective set of perceptual loss scores that satisfies a diversity metric,
• using a labeling user interface for obtaining labels for unlabeled distortion pairs, obtaining a respective labeled perception score for each distortion pair in the set of unlabeled distortion pairs, and
• retraining the plurality of deep neural networks using the set of distortion pairs and respective labeled perception scores as training examples;
• and
• using the perceptual loss function as an objective function in a machine-learning based digital file processing task.