Neural network for object detection in images
AssigneeSnap Inc.
InventorsWei Han, Jianchao Yang, Ning Zhang, Jia Li
Life of this patent
Estimating the term…
Abstract
Systems, devices, media, and methods are presented for identifying and categorically labeling objects within a set of images. The systems and methods receive an image depicting an object of interest, detect at least a portion of the object of interest within the image using a multilayer object model, determine context information, and identify the object of interest included in two or more bounding boxes.
Claims18
A device implemented method for image recognition;
- the method comprising:
- accessing, using one or more processors of the device coupled to a memory of the device, an image depicting an object of interest;
- generating, by the one or more processors, a set of bounding boxes comprising a first bounding box and a second bounding box within the image, the generating the set of bounding boxes comprising determining an initial resolution for the first bounding box based on training of a multilayer neural network model and at least of a pixel count or a measurement parameter, the measurement parameter based on at least one of color saturation, brightness, grayscale or lightness from the image, the first bounding box being associated with a first layer of the multilayer neural network model, the second bounding box being associated with a second layer of the multilayer neural network model;
- determining whether the initial resolution of the first bounding box exceeds a specified box resolution;
- in response to determining;
- that the initial resolution of the first bounding box exceeds the specified box resolution, resealing the resolution of the first bounding box to match the specified box resolution by identifying a center point of the first bounding box and cropping a portion of the first bounding box outside of the specified box resolution with respect to the center point;
- detecting, by the one or more processors;
- at least a portion of the object of interest within the first bounding box;
- extracting, by the one or more processors, context information from the second bounding box;
- wherein a layer output of the second layer includes the extracted context information;
- and
- identifying, by the one or more processors, the entire object of interest based on the at least detected portion of the object of interest and the context information;
- using the multilayer neural network model, by passing the layer output of the second layer including the extracted context information to the first layer using a deconvolution layer of the multilayer neural network model.
A system comprising:
- one or more processors;
- and
- a processor-readable storage device coupled to the one or more processors, the processor-readable storage device storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
- accessing an image depicting an object of interest;
- generating a set of bounding boxes comprising a first bounding box and a second bounding box within the image, the generating the set of bounding boxes comprising determining an initial resolution for the first bounding box based on training of a multilayer neural network model and at least of a pixel count or a measurement parameter, the measurement parameter based on at least one of color, saturation, brightness, grayscale, or lightness from the image, the first bounding box being associated with a first layer of the multilayer neural network model, the second bounding box being associated with a second layer of the multilayer neural network model;
- determining whether the initial resolution of the first bounding box exceeds a specified box resolution;
- in response to determining that e initial resolution of the first bounding box exceeds the specified box resolution, rescaling the resolution of the first bounding box to match the specified box resolution by identifying a center point of the first bounding box and by cropping a portion of the first bounding box outside of the specified box resolution with respect to the center point;
- detecting at least a portion of the object of interest within the first bounding box;
- extracting context information from the second bounding box, wherein a layer output of the second layer includes the extracted context information;
- and
- identifying the entire object of interest based on the at least detected portion of the object of interest and the context information, using the multilayer neural network model, by passing the layer output of the second layer including the extracted context information to the first layer using a deconvolution layer of the multilayer neural network model.
A processor-readable storage device storing processor-executable instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
- accessing an image depicting an object of interest;
- generating a set of bounding boxes comprising a first bounding box and a second bounding box within the image, the generating the set of bounding boxes comprising determining an initial resolution for the first bounding box based on training of a multilayer neural network model and at least of a pixel count or a measurement parameter, the measurement parameter based on at least one of color, saturation, brightness, grayscale, or lightness from the image, the first bounding box being associated with a first layer of the multilayer neural network model, the second bounding box being associated with a second layer of the multilayer neural network model;
- determining whether the initial resolution of the first bounding box exceeds a specified box resolution;
- in response to determining that the initial resolution of the first bounding box exceeds the specified box resolution, rescaling the resolution of the first bounding box to match the specified box resolution by identifying a center point of the first bounding box and by cropping a portion of the first bounding box outside of the specified box resolution with respect to the center point;
- detecting at least a portion of the object of interest within the first bounding box;
- extracting context information from the second bounding box, wherein a layer output of the second layer includes the extracted context information;
- and
- identifying the entire object of interest based on the at least detected portion of the object of interest and the context information, using the multilayer neural network model, by passing the layer output of the second layer including the extracted context information to the first layer using a deconvolution layer of the multilayer neural network model.
Dependencies are inferred from the claim text.