Mapping visual tags to sound tags using text similarity
AssigneeSony Interactive Entertainment Inc.
InventorSudha Krishnamurthy
Life of this patent
Estimating the term…
Abstract
Sound effects (SFX) are registered in a database for efficient search and retrieval. This may be accomplished by classifying SFX and using a machine learning engine to output a first of the classified SFX for a first computer simulation based on learned correlations between video attributes of the first computer simulation and the classified SFX. Subsequently, videos without sound may be processed for object, action, and caption recognition to generate video tags which are semantically matched with SFX tags to associate SFX with the video.
Claims20
An apparatus, comprising:
- at least one processor;
- and
- at least one computer storage that is not a transitory signal and that comprises instructions executable by the at least one processor to:
- classify sound effects (SFX) to render classified SFX;
- and
- semantically match SFX labels of the classified SFX with visual labels derived from video recognition of attributes in at least a first video to incorporate, into the first video, classified SFX associated with the SFX labels.
A method, comprising:
- generating at least one visual tag describing at least one attribute of at least a first video;
- and
- associating with the first video at least one sound effect (SFX) associated with at least one SFX tag at least in part based on a semantic similarity between the visual tag and the SFX tag.
An assembly comprising:
- at least one computer storage that is not a transitory signal and that comprises instructions executable by at least one processor for:
- identifying at least one visual tag describing at least a first video;
- and
- associating with the first video at least one sound effect (SFX) associated with at least one SFX tag at least in part based on a similarity between the visual tag and the SFX tag.
Dependencies are inferred from the claim text.