Automated vector database with enhanced semantic augmentation

A system for searching enhanced semantic representations of audio data, the system comprising:

• a vector database;
• a relational database;
• and
• one or more computing devices including at least one hardware processor and memory storing instructions executable by the at least one hardware processor, the one or more computing devices configured to:
• segment a digital audio signal into a plurality of discrete time-based chunks of predefined length,
• extract features from each chunk to generate corresponding audio embeddings defined as vector representations of the digital audio signal,
• store the audio embeddings in the vector database for retrieval and similarity-based search,
• process the audio embeddings using an audio-language model guided by structured prompts of an insights template to generate characteristic metadata associated with corresponding time-based chunks of the digital audio signal, wherein the insights template includes a plurality of predefined metadata fields and the structured prompts instruct the audio-language model to populate the plurality of predefined metadata fields with the characteristic metadata corresponding to each corresponding time-based chunk,
• store the metadata in the relational database, wherein the audio embeddings are indexed by shared identifiers and the metadata is linked to the audio embeddings using the shared identifiers, and
• perform a multimodal similarity-based search to locate chunks of the digital audio signal based on a combination of vector-based similarity scores from the vector database and keyword-based ranking of the metadata in the relational database.

A non-transitory computer-readable medium comprising instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations including to:

• segment a digital audio signal into a plurality of discrete time-based chunks of predefined length;
• extract features from each chunk to generate corresponding audio embeddings defined as vector representations of the digital audio signal;
• store the audio embeddings in a vector database for retrieval and similarity-based search;
• process the audio embeddings using an audio-language model guided by structured prompts of an insights template to generate characteristic metadata associated with corresponding time-based chunks of the digital audio signal, wherein the insights template includes a plurality of predefined metadata fields and the structured prompts instruct the audio-language model to populate the plurality of predefined metadata fields with the characteristic metadata corresponding to each corresponding time-based chunk;
• store the metadata in a relational database, wherein the audio embeddings are indexed by shared identifiers and the metadata is linked to the audio embeddings using the shared identifiers;
• and
• perform a multimodal similarity-based search to locate chunks of the digital audio signal based on a combination of vector-based similarity scores from the vector database and keyword-based ranking of the metadata in the relational database.