Enhanced generation of formatted and organized guides from unstructured spoken narrative using large language models

A computing device for generating a structured document from a speech audio input comprising an unstructured verbal narrative using a large language model, the computing device comprising:

• one or more processing units;
• and
• a computer-readable storage medium having encoded thereon computer-executable instructions to cause the one or more processing units to:
• receive fine-tuning data having a layout that follows one or more formatting requirements, the fine-tuning data comprising individual sections of content associated with individual categories;
• determine the one or more formatting requirements by analyzing the fine-tuning data, wherein the analysis determines the one or more formatting requirements based on a format or a layout of the content of the fine-tuning data;
• receive the speech audio input at a microphone in communication with the computing system, the speech audio input including the unstructured verbal narrative;
• analyze the speech audio input to determine speech characteristics that include at least one of a volume of the voice, a tone of voice, and one or more inflections of a voice included in the speech audio input and using the speech characteristics to supplement the one or more formatting requirements;
• convert the speech audio input including the unstructured verbal narrative into a text translation of the unstructured verbal narrative;
• determine one or more categories by analyzing the text translation of the unstructured verbal narrative;
• identify individual sections of select content by analyzing the text translation of the unstructured verbal narrative, wherein the individual sections of select content are associated with individual categories of the one or more categories;
• modify the one or more formatting requirements based on the speech characteristics, wherein at least one of the volume of the voice, the tone of the voice, and one or more inflections of a voice modifies as least one of an order of the individual sections or a format of the individual sections;
• and
• provide the one or more formatting requirements identified from the fine-tuning data, the one or more categories, and the individual sections of select content identified from the text translation of the unstructured verbal narrative to the large language model for causing the large language model to generate the structured document, wherein the structured document comprises the individual sections of select content each associated with the one or more categories, wherein the individual sections of select content and the one or more categories are ordered and formatted according to the one or more formatting requirements identified by the analysis of the fine-tuning data and the speech characteristics that include at least one of the volume, the tone of voice, and the one or more inflections of the voice included in the speech audio input.

A method of generating a structured document from a speech audio input comprising an unstructured verbal narrative using a large language model, the method for execution on a computing system, the method comprising:

• receiving fine-tuning data having a layout that follows one or more formatting requirements, the fine-tuning data comprising individual sections of content associated with individual categories;
• determining the one or more formatting requirements by analyzing the fine-tuning data, wherein the analysis determines the one or more formatting requirements based on a format or a layout of the content of the fine-tuning data;
• receiving the speech audio input at a microphone in communication with the computing system, the speech audio input including the unstructured verbal narrative;
• analyzing the speech audio input to determine speech characteristics that include at least one of a volume, a tone of voice, and one or more inflections of a voice included in the speech audio input;
• determining, based on the speech characteristics, a priority level for one or more portions of the unstructured verbal narrative;
• converting the speech audio input including the unstructured verbal narrative into a text translation of the unstructured verbal narrative;
• determining one or more categories by analyzing the text translation of the unstructured verbal narrative;
• identifying individual sections of select content by analyzing the text translation of the unstructured verbal narrative, wherein the individual sections of select content are associated with individual categories of the one or more categories;
• assigning the priority level to one or more of the individual sections of select content or the categories based on the corresponding portions of the speech audio input from which the individual sections of select content were derived;
• and
• providing the one or more formatting requirements identified from the fine-tuning data, the one or more categories, the individual sections of select content identified from the text translation of the unstructured verbal narrative, and the priority level determined from the speech characteristics to the large language model for causing the large language model to generate the structured document,
• wherein the structured document comprises the individual sections of select content each associated with the one or more categories, and
• wherein the large language model orders or emphasizes the individual sections of select content or the categories in the structured document based at least in part on the priority level determined from the speech characteristics in addition to the formatting requirements identified from the analysis of the fine-tuning data.