System and method for identifying a traffic anomaly
OwnerPheng-Ann Heng, Zhenghao Xing +3inventors
InventorsPheng-Ann Heng, Zhenghao Xing, Xiaowei Hu, Hao Chen, Chi-Wing FU
Life of this patent
Estimating the term…
Abstract
A computer-implemented method for identifying a traffic anomaly includes: receiving a video stream of traffic, wherein the video stream includes two input categories wherein the two input categories are audio data and visual data, simultaneously processing audio data and visual data to identify a traffic anomaly, generating an output of the identified traffic anomaly, and; presenting the generated output of the identified traffic anomaly to a user.
Claims18
A computer-implemented method for identifying a traffic anomaly, comprising:
- receiving a video stream of traffic, wherein the video stream comprises two input categories
- wherein the two input categories are audio data and visual data,
- simultaneously processing audio data and visual data to identify a traffic anomaly,
- generating an output of the identified traffic anomaly, and;
- presenting the generated output of the identified traffic anomaly to a user.
A system for identifying a traffic anomaly comprising:
- a computing apparatus comprising a processor and a memory unit, the processor and memory unit being operatively coupled to each other,
- the computing apparatus further comprising a display operatively coupled to the processor, the display adapted to present one or more outputs,
- the computing apparatus further comprising:
- an audio insight frame selector configured to identify one or more anomalous events in an input video stream of traffic by applying spectral flux computation, and select each frame in the video stream that includes the identified one or more anomalous events,
- an audio encoder and a visual encoder, the encoders configured to generate encoded audio data and encoded visual data,
- an audio-visual dynamic connector configured to fuse the encoded visual data and the encoded audio data into a fused audio-visual data (fused AV data),
- a large language model (LLM) that is trained to process the fused AV data and generate one or more natural language outputs,
- wherein the natural language outputs are presented on the display.
Dependencies are inferred from the claim text.