# US 12,223,605 B2 · Parallel content authoring method and system for procedural guidance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,605 B2 - Document ID: US12223605B2 - Status: Granted - Kind: B2 - Application number: 18/111,440 - Filed: 2023-02-17 - Published: 2023-08-17 - Granted: 2025-02-11 - Earliest claimed: 2022-02-17 - Assignees: The Government of the United States of America, as represented by the Secretary of the Navy - Applicants: The United States of America, AS Represented By the Secretary of the Navy - Inventors: Larry Clay Greunke - Web page: https://patentgrep.com/patent/US12223605B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 16/24573: {using data annotations, e.g. user-defined metadata} - G06T 13/20: Three-dimensional [3D] animation - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip - G09B 19/003: {Repetitive work cycles; Sequence of movements} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2210/52: Parallel processing - G06T 2219/004: Annotating, labelling - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract This disclosure and exemplary embodiments described herein provide a Parallel Content Authoring Method and Tool for Procedural Guidance, and a Remote Expert Method and System Utilizing Quantitative Quality Assurance in Mixed Reality. The implementation described herein is related to the generation of content/instruction set that can be viewed in different modalities, including but not limited to mixed reality, VR, audio text, however it is to be understood that the scope of this disclosure is not limited to such application. ## Claims ### Claim 1 A method for parallel content authoring, the method comprising: creating a 3D representation of a target physical system; receiving a part selection from an editor of the 3D representation, the part selection from one of a plurality of parts included in the target physical system; collecting part actions from the editor, the part actions associated with actions to be performed on the selected part; creating queued annotations for the part actions, wherein the queued annotations are to be displayed in a 3D environment with respect to the 3D representation of the target physical system, and wherein at least one of the queued annotations includes a camera position recording based on a type of the corresponding part action and a location of the target system part; collecting and associating augmented reality data with the queued annotations; publishing a data structure bundle including a data set for generation of the queued annotations, the data set parsable to create mixed reality content; and a mixed reality system creating and presenting to a user content including the queued annotations from the data set, where the user interacts with the target physical system and parts included in the parts selection according to the queued annotations. ### Claim 2 (depends on 1) The method for parallel content authoring according to claim 1, further comprising: collecting a text description for at least one of the queued annotations. ### Claim 3 (depends on 1) The method for parallel content authoring according to claim 1, further comprising: collecting an audio description for at least one of the queued annotations. ### Claim 4 (depends on 1) The method for parallel content authoring according to claim 1, further comprising: receiving and processing validations of the queued annotations from at least one of the editors. ### Claim 5 (depends on 1) The method for parallel content authoring according to claim 1, wherein the queued annotations are stored such that the queued annotations can be translated into at least one medium selected from a group of a 2D medium, a 2.5D medium, and a 3D medium, wherein the queued annotations are presented in the at least one selected medium. ### Claim 6 (depends on 1) The method for parallel content authoring according to claim 1, wherein the queued annotations are stored such that the queued annotations can be translated into at least one format selected from a group of a document format, an audio format, a video format, wherein the queued annotations are presented in the at least one selected format. ### Claim 7 (depends on 1) The method for parallel content authoring according to claim 1, wherein the part selection and the part actions are received from the editors in a mixed reality environment. ### Claim 8 (depends on 1) The method for parallel content authoring according to claim 1, wherein the editors work collaboratively in at least one environment selected from a group of a mixed reality environment and a desktop environment. ### Claim 9 (depends on 1) The method for parallel content authoring according to claim 1, further comprising: collecting completion metrics for at least one of the queued annotations. ### Claim 10 (depends on 1) The method for parallel content authoring according to claim 1, wherein the method for parallel content authoring publishes the data structure bundle including a data set for generation of the queued annotations, and the method for parallel content authoring publishes discrete individual outputs including a text, AR instructions and video. ### Claim 11 A parallel content authoring system comprising: a mixed reality controller performing a method to: define a coordinate system frame of reference for a target physical system, the coordinate system frame of reference including an initial point of the target physical system and at least one directional axis that are specified by a user of the mixed reality controller; define additional points of the target physical system; and define interface elements of the target physical system; and a computer system performing a method to: generate a 3D model of the target physical system based on the coordinate system frame of reference, the additional points, and the interface elements; receive a part selection from an editor of the 3D representation, the part selection from one of a plurality of parts included in the target physical system; collect part actions from the editor, the part actions associated with actions to be performed on the selected part; create queued annotations for the part actions, wherein the queued annotations are to be displayed in a 3D environment with respect to the 3D representation of the target physical system, and wherein at least one of the queued annotations includes a camera position recording based on a type of the corresponding part action and a location of the target system part; collect and associate augmented reality data with the queued annotations; and publish a data structure bundle including a data set for generation of the queued annotations, the data set parsable to create mixed reality content, wherein a mixed reality system creates and presents to a user content including the queued annotations from the data set, where the user interacts with the target physical system and parts included in the parts selection according to the queued annotations. ### Claim 12 (depends on 11) The parallel content authoring system according to claim 11, further comprising: collecting a text description for at least one of the queued annotations. ### Claim 13 (depends on 11) The parallel content authoring system according to claim 11, further comprising: collecting an audio description for at least one of the queued annotations. ### Claim 14 (depends on 11) The parallel content authoring system according to claim 11, further comprising: receiving and processing validations of the queued annotations from at least one of the editors. ### Claim 15 (depends on 11) The parallel content authoring system according to claim 11, wherein the queued annotations are stored such that the queued annotations can be translated into at least one medium selected from a group of a 2D medium, a 2.5D medium, and a 3D medium, wherein the queued annotations are presented in the at least one selected medium. ### Claim 16 (depends on 11) The parallel content authoring system according to claim 11, wherein the queued annotations are stored such that the queued annotations can be translated into at least one format selected from a group of a document format, an audio format, a video format, wherein the queued annotations are presented in the at least one selected format. ### Claim 17 (depends on 11) The parallel content authoring system according to claim 11, wherein the part selection and the part actions are received from the editors in a mixed reality environment. ### Claim 18 (depends on 11) The parallel content authoring system according to claim 11, wherein the editors work collaboratively in at least one environment selected from a group of a mixed reality environment and a desktop environment. ### Claim 19 (depends on 11) The parallel content authoring system according to claim 11, further comprising: collecting completion metrics for at least one of the queued annotations. ### Claim 20 (depends on 11) The parallel content authoring system according to claim 11, wherein the method for parallel content authoring publishes the data structure bundle including a data set for generation of the queued annotations, and the method for parallel content authoring publishes discrete individual outputs including a text, AR instructions and video. ### Claim 21 A non-transitory computer-readable medium comprising executable instructions for causing a computer system to perform a method for parallel content authoring, the instructions when executed causing the computer system to: create a 3D representation of a target physical system; receive a part selection from an editor of the 3D representation, the part selection from one of a plurality of parts included in the target physical system; collect part actions from the editor, the part actions associated with actions to be performed on the selected part; create queued annotations for the part actions, wherein the queued annotations are to be displayed in a 3D environment with respect to the 3D representation of the target physical system, and wherein at least one of the queued annotations includes a camera position recording based on a type of the corresponding part action and a location of the target system part; collect and associating augmented reality data with the queued annotations; publish a data structure bundle including a data set for generation of the queued annotations, the data set parsable to create mixed reality content; and present to a user using an associated mixed reality system content including the queued annotations from the data set, where the user interacts with the target physical system and parts included in the parts selection according to the queued annotations.