Labeling Video Files Using Acoustic Vectors
AssigneeFacebook, Inc.
InventorsYing Zhang, Yun Lei
Life of this patent
Estimating the term…
Abstract
In one embodiment, a method includes receiving a video file. The video file includes a corresponding audio stream. The method further includes accessing the audio stream, and generating, based on the audio stream, a representative vector. The vector has a particular number of dimensions. The method further includes accessing a label-embedding space, which has the same particular number of dimensions, and includes a number of regions that each correspond to a respective label. The method further includes determining a region of the label-embedding space that corresponds to the vector, the determined region corresponding to a particular label. The method further includes associating the particular label with the video file.
Claims20
A method comprising:
- by a computing device, receiving a video file, the video file comprising an audio stream;
- by the computing device, accessing the audio stream of the video file;
- by the computing device, generating, based on the audio stream, a vector representing the audio stream, wherein the vector has a particular number of dimensions;
- by the computing device, accessing a label-embedding space, wherein the label-embedding space has:
- the particular number of dimensions;
- and
- a plurality of regions, wherein each region from the plurality of regions corresponds to a label from a plurality of labels;
- by the computing device, determining a region of the label-embedding space that corresponds to the vector representing the audio stream, wherein the region is associated with a particular label;
- and
- by the computing device, associating the particular label with the video file.
One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
- receive a video file, the video file comprising an audio stream;
- access the audio stream of the video file;
- generate, based on the audio stream, a vector representing the audio stream, wherein the vector has a particular number of dimensions;
- access a label-embedding space, wherein the label-embedding space has:
- the particular number of dimensions;
- and
- a plurality of regions, wherein each region from the plurality of regions corresponds to a label from a plurality of labels;
- determine a region of the label-embedding space that corresponds to the vector representing the audio stream, wherein the region is associated with a particular label;
- and
- associate the particular label with the video file.
A system comprising:
- one or more processors;
- and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:
- receive a video file, the video file comprising an audio stream;
- access the audio stream of the video file;
- generate, based on the audio stream, a vector representing the audio stream, wherein the vector has a particular number of dimensions;
- access a label-embedding space, wherein the label-embedding space has:
- the particular number of dimensions;
- and
- a plurality of regions, wherein each region from the plurality of regions corresponds to a label from a plurality of labels;
- determine a region of the label-embedding space that corresponds to the vector representing the audio stream, wherein the region is associated with a particular label;
- and
- associate the particular label with the video file.
Dependencies are inferred from the claim text.