AI-Based Transformation of Audio/Video Content

A method for generating a report from a video, the method comprising:

• a) receiving, at a server, the video comprising audio data and multiple frames of image data;
• b) extracting, by the server, individual frames from the video;
• c) performing, by an object detection generative AI component, object detection on the individual frames to identify identified objects within the individual frames;
• d) tracking, by an object tracking generative AI component, the identified objects across the individual frames to maintain object continuity to identify tracked objects;
• e) analyzing, by an audio analysis component, the audio data to create analyzed audio data;
• f) identifying, by a feature point identification component, feature points within the individual frames;
• g) applying, by a geomapping component, geolocation data associated with the video to the tracked objects to generate geolocation-enhanced data;
• h) analyzing, by an LSTM generative AI component, the tracked objects, the analyzed audio data, the identified feature points, and the geolocation-enhanced data to generate a contextual timeline of events;
• i) detecting, by an event detection generative AI component, events within the contextual timeline generated by the LSTM generative AI component;
• j) analyzing, by a scene understanding generative AI component, the detected events to generate a contextual understanding of the scene;
• k) generating, based on the contextual timeline, the detected events, the contextual understanding, and a report template, a formatted report using a report generation generative AI component.

A computing device comprising:

• a) a processor configured to execute computer programming;
• b) a memory containing the computer programming, the computer programming configured to instruct the processor to:
• i) receive a video comprising audio data and multiple frames of image data;
• ii) extract individual frames from the video;
• iii) perform, by an object detection generative AI component, object detection on the individual frames to identify identified objects within the individual frames;
• iv) track, by an object tracking generative AI component, the identified objects across the individual frames to maintain object continuity to identify tracked objects;
• v) analyze, by a Long Short-Term Memory (LSTM) generative AI component, the tracked objects to generate a contextual timeline of events;
• and
• vi) generate, based on the contextual timeline and a report template, a formatted report using a report generation generative AI component.