Back to results
US 2021/0279817 A1Published application
Systems and methods for utilizing compressed convolutional neural networks to perform media content processing
AssigneeFacebook, Inc.
InventorsYunchao Gong, Liu Liu, Lubomir Dimitrov Bourdev, Robert D. Fergus, Ming Yang
Life of this patent
Estimating the term…
Abstract
Systems, methods, and non-transitory computer-readable media can receive a compressed convolutional neural network (CNN). A media content item to be processed can be acquired. The compressed CNN to can be utilized to apply a media processing technique to the media content item to produce information about the media content item. It can be determined, based on at least some of the information about the media content item, whether to transmit at least a portion of the media content item to one or more remote servers for additional media processing.
Claims18
A computer-implemented method comprising:
- receiving, by a computing system, a compressed convolutional neural network (CNN) generated based on a compression process performed remotely from the computing system,
- wherein the compression process is configured based on one or more properties associated with the computing system including at least one of an operating system of the computing system or resources of the computing system, and
- wherein the compression process is based on a matrix factorization method that factorizes a parameter in a connected layer into two orthogonal matrices and a diagonal matrix;
- acquiring, by the computing system, a media content item to be processed;
- utilizing, by the computing system, the compressed CNN to apply a media processing technique to the media content item to produce information about the media content item, wherein the information about the media content item includes a score indicating a level of confidence associated with recognizing, by the media processing technique, one or more objects of interest depicted in the media content item;
- determining, by the computing system, based on the score being less than an associated threshold confidence score, to transmit at least a portion of the media content item associated with the one or more objects of interest to one or more remote servers for additional media processing to determine identifying information for the portion;
- cropping out, by the computing system, the at least the portion of the media content item to generate a cropped out portion;
- transmitting, by the computing system, the cropped out portion of the media content item to the one or more remote servers for the additional media processing;
- and
- receiving, by the computing system, the identifying information for the one or more objects of interest.
7 more elements
A system comprising:
- at least one processor;
- and
- a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
- receiving a compressed convolutional neural network (CNN) generated based on a compression process performed remotely from the system,
- wherein the compression process is configured based on one or more properties associated with the system including at least one of an operating system of the system or resources of the system, and
- wherein the compression process is based on a matrix factorization method that factorizes a parameter in a connected layer into two orthogonal matrices and a diagonal matrix;
- acquiring a media content item to be processed;
- utilizing the compressed CNN to apply a media processing technique to the media content item to produce information about the media content item, wherein the information about the media content item includes a score indicating a level of confidence associated with recognizing, by the media processing technique, one or more objects of interest depicted in the media content item;
- determining based on the score being less than an associated threshold confidence score, to transmit at least a portion of the media content item associated with the one or more objects of interest to one or more remote servers for additional media processing to determine identifying information for the portion;
- cropping out the at least the portion of the media content item to generate a cropped out portion;
- transmitting the cropped out portion of the media content item to the one or more remote servers for the additional media processing;
- and
- receiving the identifying information for the one or more objects of interest.
10 more elements
A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
- receiving a compressed convolutional neural network (CNN) generated based on a compression process performed remotely from the computing system,
- wherein the compression process is configured based on one or more properties associated with the computing system including at least one of an operating system of the computing system or resources of the computing system, and
- wherein the compression process is based on a matrix factorization method that factorizes a parameter in a connected layer into two orthogonal matrices and a diagonal matrix;
- acquiring a media content item to be processed;
- utilizing the compressed CNN to apply a media processing technique to the media content item to produce information about the media content item, wherein the information about the media content item includes a score indicating a level of confidence associated with recognizing, by the media processing technique, one or more objects of interest depicted in the media content item;
- determining based on the score being less than an associated threshold confidence score, to transmit at least a portion of the media content item associated with the one or more objects of interest to one or more remote servers for additional media processing to determine identifying information for the portion;
- cropping out the at least the portion of the media content item to generate a cropped out portion;
- transmitting the cropped out portion of the media content item to the one or more remote servers for the additional media processing;
- and
- receiving the identifying information for the one or more objects of interest.
7 more elements
Dependencies are inferred from the claim text.