Automated audio caption correction using false alarm and miss detection

A method for generating correct captions of input data, the method comprising:

• receiving a set of audio tags associated with the input data;
• receiving a set of detections, the set of detections generated from a candidate caption associated with the input data;
• determining a set of false negatives in the candidate caption based on a first text-based comparison of the set of audio tags and the set of detections;
• determining a set of false positives in the candidate caption based on a second text-based comparison of the set of audio tags and the set of detections;
• and
• generating, based on an addition of at least one missing term to the candidate caption based on the set of false negatives and removal of at least one false term from the candidate caption based on the set of false positives, a corrected caption relative to the candidate caption.

A non-transitory computer-readable medium having stored thereon instructions which, when executed by one or more processors, cause the one or more processors to be configured to:

• receive a set of audio tags associated with input data;
• receive a set of detections, the set of detections generated from a candidate caption associated with the input data;
• determine a set of false negatives in the candidate caption based on a first text-based comparison of the set of audio tags and the set of detections;
• determine a set of false positives in the candidate caption based on a second text-based comparison of the set of audio tags and the set of detections;
• and
• generate, based on an addition of at least one missing term to the candidate caption based on the set of false negatives and removal of at least one false term from the candidate caption based on the set of false positives, a corrected caption relative to the candidate caption.