Media perspectives in an augmented semantic search system

A method for structured data representation of a set of media perspectives, comprising:

• identifying a set of caption data for a set of media items;
• for each of the set of media items, generating a set of structured data representations of the set of media perspectives, by:
• (i) generating a prompt comprising caption data of the media item and definition of a structured data representation of a media perspective of the set of media perspectives;
• and
• (ii) executing, by a computer processor, a large language model using the prompt to generate the structured data representation of the media perspective for the media item;
• and
• storing the set of structured data representations in a document store, wherein the document store comprises multiple different structured data representations of different media perspectives for each of the set of media items;
• executing an encoder model on the set of structured data representations to generate a set of embeddings;
• and
• storing the set of embeddings in a vector store of a semantic search system for execution of semantic search using a vector similarity operation.

A non-transitory computer-readable storage medium comprising a plurality of instructions for structured data representation of a set of media perspectives, the plurality of instructions configured to execute on at least one computer processor to enable the at least one computer processor to:

• identify a set of caption data for a set of media items;
• for each of the set of media items, generate a set of structured data representations of the set of media perspectives, by:
• (i) generating a prompt comprising caption data of the media item and definition of a structured data representation of a media perspective of the set of media perspectives;
• and
• (ii) executing a large language model using the prompt to generate the structured data representation of the media perspective for the media item;
• and
• store the set of structured data representations in a document store, wherein the document store comprises multiple different structured data representations of different media perspectives for each of the set of media items;
• execute an encoder model on the set of structured data representations to generate a set of embeddings;
• and
• store the set of embeddings in a vector store of a semantic search system for execution of semantic search using a vector similarity operation.