Methods and systems for automatically evaluating code and software documentation for compliance with relevant standards, requirements, and policies

A computer system configured for evaluating application code or software documentation against relevant standards, requirements, or policies, comprising:

• a memory;
• and
• one or more computing devices coupled to the memory and configured with computer-executable instructions to perform operations comprising:
• receiving from a user interface one or more code or software documents for evaluation and requested analysis or evaluation to be conducted on the one or more code or software documents;
• transforming the received request for analysis or evaluation and the one or more code or software documents into a format suitable for processing by a large language model (LLM);
• prompting an LLM to process the one or more code or software documents and the requested analysis or evaluation to identify concepts or elements that are likely to be a subject of a standard, requirement, or policy;
• reformatting the concepts or elements identified by the LLM into a query suitable for presenting to a compliance criteria vector database;
• submitting the query to the compliance criteria vector database to identify standards, requirements, or policies that are relevant to the requested analysis or evaluation;
• prompting the LLM to determine whether the one or more code or software documents comply with each of the identified relevant standards, requirements, or policies;
• and
• generating or outputting a report comprising determinations, inferences, or outputs of the LLM regarding compliance of the one or more code or software documents with each of the identified relevant standards, requirements, or policies.

A non-transitory computer-readable medium having stored thereon computer-executable instructions configured to cause one or more computing devices of a computing system to perform operations comprising:

• receiving from a user interface one or more code or software documents for evaluation and requested analysis or evaluation to be conducted on the one or more code or software documents;
• transforming the received request for analysis or evaluation and the one or more code or software documents into a format suitable for processing by a large language model (LLM);
• prompting an LLM to process the one or more code or software documents and the requested analysis or evaluation to identify concepts or elements that are likely to be a subject of a standard, requirement, or policy;
• reformatting the concepts or elements identified by the LLM into a query suitable for presenting to a compliance criteria vector database;
• submitting the query to the compliance criteria vector database to identify standards, requirements, or policies that are relevant to the requested analysis or evaluation;
• prompting the LLM to determine whether the one or more code or software documents comply with each of the identified relevant standards, requirements, or policies;
• and
• generating or outputting a report comprising determinations, inferences, or outputs of the LLM regarding compliance of the one or more code or software documents with each of the identified relevant standards, requirements, or policies.