Video object detection with co-occurrence

A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• while training a model to detect objects of any of two or more object types that are depicted in images:
• accessing, for a training image, first data that indicates a detected bounding box for a first object depicted in the training image and a predicted type label for the first object that indicates a predicted type for the first object from the two or more object types;
• accessing, for the training image, ground truth data for one or more ground truth objects that includes second data that indicates a) a labeled bounding box for a ground truth object depicted in the training image and b) a ground truth label that indicates an actual type for the ground truth object from the two or more object types, wherein the ground truth object is a different object than the first object;
• determining, using the first data and the ground truth data, that i) the detected bounding box represents an object that is not a ground truth object represented by the ground truth data or ii) the predicted type label for the first object does not match a ground truth label for the first object identified by the ground truth data;
• in response to determining that i) the detected bounding box represents an object that is not a ground truth object or ii) the predicted type label for the first object does not match a ground truth label for the first object, determining a penalty to adjust the model using a distance between a) a first location of the detected bounding box in the training image and b) a second location of the labeled bounding box in the training image;
• and
• training the model using the penalty.

A computer-implemented method comprising:

• while training a model to detect objects of any of two or more object types that are depicted in images:
• accessing, for a training image, first data that indicates a detected bounding box for a first object depicted in the training image and a predicted type label for the first object that indicates a predicted type for the first object from the two or more object types;
• accessing, for the training image, ground truth data for one or more ground truth objects that includes second data that indicates a) a labeled bounding box for a ground truth object depicted in the training image and b) a ground truth label that indicates an actual type for the ground truth object from the two or more object types, wherein the ground truth object is a different object than the first object;
• determining, using the first data and the ground truth data, that i) the detected bounding box represents an object that is not a ground truth object represented by the ground truth data or ii) the predicted type label for the first object does not match a ground truth label for the first object identified by the ground truth data;
• in response to determining that i) the detected bounding box represents an object that is not a ground truth object or ii) the predicted type label for the first object does not match a ground truth label for the first object, determining a penalty to adjust the model using a distance between a) a first location of the detected bounding box in the training image and b) a second location of the labeled bounding box in the training image;
• and
• training the model using the penalty.