Audio classifier

A computer-implemented method for providing search results identifying audio resources, the method comprising:

• receiving a query, the query including one or more query terms;
• using the one or more query terms to identify one or more audio resources responsive to the query, wherein each audio resource is associated with one or more audio labels generated by an audio classifier, wherein the audio classifier is trained using a plurality of unlabeled audio segments extracted from a plurality of video frames that are obtained from a plurality of videos and corresponding image labels respectively generated for the plurality of video frames as input training data, and wherein the audio classifier is trained to assign one or more audio labels to unlabeled audio segments input to the audio classifier;
• and
• providing search results identifying the one or more audio resources.

A system comprising:

• one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for generating an audio classifier that assigns audio labels to input audio segments, the operations comprising:
• obtaining a plurality of video frames from a plurality of videos, wherein each of the plurality of video frames is unlabeled;
• generating, for each of the plurality of video frames, one or more image labels of a plurality of image labels based on image recognition and associating the one or more image labels with each of the plurality of video frames;
• obtaining a plurality of unlabeled audio segments corresponding to the plurality of video frames, wherein each unlabeled audio segment has a specified duration relative to the corresponding video frame;
• generating the audio classifier, wherein the audio classifier is trained using the plurality of unlabeled audio segments and the associated image labels generated from the plurality of video frames as input training data, and wherein the audio classifier is trained to assign one or more audio labels to unlabeled audio segments input to the audio classifier;
• receiving a query, the query including one or more query terms;
• using the one or more query terms to identify one or more audio resources responsive to the query using the audio classifier, wherein each audio resource is associated with one or more audio labels generated by the audio classifier based on the query terms;
• and
• providing search results identifying the one or more audio resources.

A system comprising:

• one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for providing search results identifying audio resources, the operations comprising:
• receiving a query, the query including one or more query terms;
• using the one or more query terms to identify one or more audio resources responsive to the query, wherein each audio resource is associated with one or more audio labels generated by an audio classifier, wherein the audio classifier is trained using a plurality of unlabeled audio segments extracted from a plurality of video frames that are obtained from a plurality of videos and corresponding image labels respectively generated for the plurality of video frames as input training data, and wherein the audio classifier is trained to assign one or more audio labels to unlabeled audio segments input to the audio classifier;
• and
• providing search results identifying the one or more audio resources.

A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations for generating an audio classifier that assigns audio labels to input audio segments, the operations comprising:

• obtaining a plurality of video frames from a plurality of videos, wherein each of the plurality of video frames is unlabeled;
• generating, for each of the plurality of video frames, one or more image labels of a plurality of image labels based on image recognition and associating the one or more image labels with each of the plurality of video frames;
• obtaining a plurality of unlabeled audio segments corresponding to the plurality of video frames, wherein each audio segment has a specified duration relative to the corresponding video frame;
• generating the audio classifier, wherein the audio classifier is trained using the plurality of unlabeled audio segments and the associated image labels generated from the plurality of video frames as input training data, and wherein the audio classifier is trained to assign one or more audio labels to unlabeled audio segments input to the audio classifier;
• receiving a query, the query including one or more query terms;
• using the one or more query terms to identify one or more audio resources responsive to the query using the audio classifier, wherein each audio resource is associated with one or more audio labels generated by the audio classifier based on the query terms;
• and
• providing search results identifying the one or more audio resources.

A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations for providing search results identifying audio resources, the operations comprising:

• receiving a query, the query including one or more query terms;
• using the one or more query terms to identify one or more audio resources responsive to the query, wherein each audio resource is associated with one or more audio labels generated by an audio classifier, wherein the audio classifier is trained using a plurality of unlabeled audio segments extracted from a plurality of video frames that are obtained from a plurality of videos and corresponding image labels respectively generated for the plurality of video frames as input training data, and wherein the audio classifier is trained to assign one or more audio labels to unlabeled audio segments input to the audio classifier;
• and
• providing search results identifying the one or more audio resources.