Back to results
US 2024/0257510 A1Published application
Methods and apparatus localizing object(s) in vision data
AssigneeGoogle Inc.
InventorsWeicheng Kuo, Tsung-Yi Lin, Anelia Angelova, Dahun Kim
Life of this patent
Estimating the term…
Abstract
An object localization network (OLN) can be used to localize object(s) (e.g., known and/or unknown object(s)) in an instance of vision data. Various implementations include detecting the localized object(s) based on the localization. Many implementations include processing the instance of vision data using the OLN to generate a objectness score (e.g., a centerness score) as well as an intersection of union (IoU) score for one or more proposed object locations in the instance of vision data. Object(s) can be localized in the instance of vision data based on the objectness scores and the IoU scores.
Claims15
A method implemented by one or more processors, the method comprising:
- training an Object Localization Network (OLN) to detect one or more objects, in an instance of vision data, based on predicted object localization, wherein training the OLN comprises:
- selecting a training instance including a training instance of vision data and one or more training annotations, where each training annotation in the one or more training annotations identifies a ground truth location of a corresponding object location of an object, in the training instance of vision data;
- processing the training instance of vision data using a feature extraction network portion of the OLN to generate a set of training instance features;
- processing the set of training instance features using a region proposal portion of the OLN to generate a set of proposed object locations in the training instance of vision data;
- processing the set of proposed object locations using a region based portion of the OLN to generate one or more predicted annotations, where each predicted annotation in the one or more predicted annotations identifies a corresponding predicted location of a corresponding object location in the training instance of vision data;
- generating a loss based on comparing the one or more predicted annotations and the one or more training annotations of the training instance of vision data;
- and
- updating one or more portions of the OLN based on the generated loss.
5 more elements
A method implemented by one or more processors, the method comprising:
- identifying an instance of vision data including one or more objects;
- detecting the one or more objects in the instance of vision data, wherein detecting the one or more objects comprises:
- processing the instance of vision data using an Object Localization Network (OLN) to generate one or more predicted annotations, where each predicted annotation in the one or more predicted annotations identifies a corresponding predicted location of a corresponding object, of the one or more objects in the instance of vision data, wherein processing the instance of vision data using the OLN to generate the one or more predicted annotations comprises:
- processing the instance of vision data using a box refinement portion of the OLN to generate a set of proposed object locations in the instance of vision data;
- for each proposed object location in the set of proposed object locations, processing the proposed object location using a centerness portion of the OLN to generate a centerness score corresponding the proposed object location indicating a predicted deviation of the proposed object location from a ground truth representation of the location of the object;
- selecting a subset of proposed object locations from the set of proposed object locations based on the corresponding centerness scores;
- processing the subset of the proposed object locations using an additional box refinement portion of the OLN to generate a set of refined object locations;
- for each refined object location in the set of refined object locations, processing the refined object location using an intersection of union (IoU) portion of the OLN to generate a corresponding IoU score indicating a predicted intersection of union between the refined object location and the ground truth representation of the object;
- ranking the set of refined object locations, where the rank of each refined object location, in the set of refined object locations, is based on the corresponding centerness score and the corresponding IoU score;
- and
- detecting the one or more objects based on the ranking of the set of refined object locations.
8 more elements
Dependencies are inferred from the claim text.