Artificial Intelligence Systems and Methods

A non-transitory, computer-readable storage medium having encoded thereon machine instructions that when executed by a processor, cause the processor to generate retrospectively verified documents, wherein the processor:

• receives an original query on a big data source, the big data source comprising a plurality of structured data objects and unstructured data objects, wherein one or more data objects are represented in a vector database corresponding to the big data source;
• generates a refined query from the original query, wherein the processor:
• applies a system query refiner function and a user query refiner function to the original query, and
• applies a system prompt and a user prompt to the original query;
• generates an embedding of the refined query;
• identifies one or more discrete vectors in the vector database most similar to the embedding;
• applies the refined query and one or more most similar vectors to a large language model;
• applies the large language model to the big data source to generate the document, wherein the refined query comprises a role stated as a persona to be adopted by the large language model, a set of instructions to be followed by the large language model, and a set of conditions and limitations imposed on the large language model, comprising:
• using the one or more most similar vectors as an index to the big data source;
• and generates the verified document using data objects indicated by the index and returns the verified document and an identification for each indicated data object.

A computer-implemented method for generating an hallucination-free, retrospectively verified documents based on searching content of a big data source storing text and image-based data objects, the method, comprising:

• a processor executing a search of the big data source for data objects directed to a specific subject, comprising:
• receiving an original query related to the specific subject;
• generating a refined query from the original query, comprising:
• applying a system query refiner to the original query, and
• applying a user query refiner to the user query;
• generating an embedding of the refined query;
• applying the embedding to a vector database, the vector database the vector database comprising vector representations of data objects in the big data source;
• determining a similarity between the embedding and one or more of the vector representations;
• and
• returning a top number K of the vector representations having a highest similarity to the embedding, the vector representations providing an index to the big data source,
• applying the refined query and one or more most similar vectors to a large language model,
• applying the large language model to the big data source to generate the document, wherein the refined query comprises a role stated as a persona to be adopted by the large language model, a set of instructions to be followed by the large language model, and a set of conditions and limitations imposed on the large language model, comprising using the one or more most similar vectors as an index to the big data source, and
• generating the verified document using data objects indicated by the index and returning the verified document and an identification for each indicated data object.