# US 2018/0322254 A1 · Multimodal cognitive collaboration and cybernetic knowledge exchange with visual neural networking streaming augmented medical intelligence > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322254 A1 - Document ID: US20180322254A1 - Status: Published application - Kind: A1 - Application number: 15/731,201 - Filed: 2017-05-02 - Published: 2018-11-08 - Earliest claimed: 2017-05-02 - Inventors: James Paul Smurro - Web page: https://patentgrep.com/patent/US20180322254A1 ## CPC - G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring - G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04N 7/15: Conference systems - H04N 7/152: {Multipoint control units therefor} - H04N 21/23605: {Creation or processing of packetized elementary streams [PES]} - H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)} - H04N 21/2389: Multiplex stream processing, e.g. multiplex stream encrypting - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The invention enables multimodal cognitive communications, collaboration, consultation and instruction between and among heterogeneous networked teams of persons, machines, devices, neural networks, robots and algorithms during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. The invention enables both synchronous and asynchronous multiparty collaboration with multichannel, multiplexed streaming imagery data, including interactive curation, multisensory annotation and metadata tagging, as well as multi-formatted encapsulation, saving and sharing of collaborated imagery data as packetized augmented intelligence. The invention acquires both live stream and archived medical modality imagery from network-connected medical devices, cameras, signals and sensors, as well as multiomic data [phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic] maps and clinical data sets from structured reports and clinical documents, including biometric maps and movies, hapmaps, heat maps and data stream visualizations. The invention also acquires both medical and non-medical streaming imagery data from image data repositories, documents and structured reports, workstations and mobile devices, as well as from wearable computing, signals and sensors. The invention enables networked teams to interactively communicate, concurrently collaborate and bi-directionally exchange multichannel multiplexed imagery data streams, singly or together, in real time or asynchronously, generally by curating, annotating and tagging imagery information objects. The invention encapsulates and saves collaborated imagery data, together with multisensory annotations and metadata tags, in standard file formats as packetized augmented intelligence. The invention enables recursive cognitive enrichment of clinical cognitive vismemes, and saves packetized imagery information objects, multisensory annotations and metadata tags in native file formats [PDF, MPEG, JPEG, XML, XMPP, QR,TIFF, RDF, RDF/XML, SVG and DAE] as well as in formats compliant with standards for digital communications in medicine [DICOM]. The invention enables live stream multicasting of multimodal cognitive instruction and collaborative knowledge exchange with multisensory [visual, auditory, haptic] annotation of streaming imagery data, as well as secure, encrypted transmission of streaming augmented intelligence across file sharing data networks for informatics-enabled learning, specialist skills acquisition and accelerated knowledge exchange. ## Claims ### Claim 1 A network system enabling multichannel multiplexed communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during collaboration sessions, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, the network system enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, including recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, allowing each participant cognitive collaborant to capture, retrieve and concurrently view at least one source of streaming medical modality imagery data, and at least one or more sources of heterogeneous streaming imagery data, medical and non-medical streaming imagery data, and combinations thereof including images, video, modality imagery, audio, video and haptic wave forms and files, multiomic-phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents, both live and archived streaming imagery data, enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data in collaboration sessions practiced by and among at least one or more participant cognitive collaborants during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, each participant cognitive collaborant able to view, curate, annotate and tag the heterogeneous streaming imagery data, comprising; a tele-visual imagery informatics management system including, at least one or more tele-visual imagery informatics management system clini-docks, wherein each clini-dock is adapted for independent acquisition and transmission of signals from other sources of streaming imagery data at native, enhanced or reduced resolutions and native enhanced or reduced frame rates, used for the acquisition and transmission of, live or archived streaming imagery data, including images, video, modality imagery, audio, video and haptic wave forms and files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents, analog or digital video signals in standard or non-standard resolutions, medical or non-medical imagery, in compressed or uncompressed imagery formats; at least one or more tele-visual imagery informatics management system clini-pod network servers, wherein each server is a neurosynaptic network node comprising at least one streamer, splitter, router, server and storage device enabling at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, to concurrently view, communicate, collaborate, consult and instruct among participant cognitive collaborants, including curation, annotation and tagging, using at least one or more sources of streaming imagery data acquired and transmitted by tele-visual imagery informatics management system clini-docks, including live streaming imagery data, archived streaming imagery data, appended streaming imagery metadata, including appended semantic metadata and annotations, cognitive collaborant annotations, and archived collaborated imagery files during a synchronous or asynchronous collaboration session; establishing and maintaining channel communications for each and all of the sources of streaming imagery data for at least one or more participant cognitive collaborant during a collaboration session, enabling at least one or more participant cognitive collaborants in at least one or more locations, to concurrently view, communicate, collaborate, consult and instruct among participant cognitive collaborants using at least one or more sources of live streaming imagery data, archived streaming imagery data, appended streaming imagery metadata, cognitive collaborant annotations, and archived collaborated imagery files, including curation, annotation and tagging from each participant cognitive collaborant during a collaboration session, managing and controlling at least one or more associated databases, and privileges for authorization, authentication, identity management, security, access, publication and distribution for viewing, communicating, collaborating, consulting and instructing among participant cognitive collaborants managing and controlling privileges for at least one or more participant cognitive collaborants to view, curate, annotate, tag, encapsulate, save, store, retrieve and distribute live streaming imagery data, archived streaming imagery data, appended streaming imagery metadata, including appended semantic metadata and annotations, cognitive collaborant annotations, and archived collaborated imagery files for each participant cognitive collaborant during collaboration sessions; enabling both synchronous and asynchronous bidirectional communications with combinations of at least one or more local area networks, at least one or more wide area networks, including internet, and at least one or more streaming imagery data repositories during at least one or more collaboration sessions enabling identification, tracking and monitoring of participant cognitive collaborants by assignment of a unique color for annotations of streaming imagery data, archived collaborated imagery files and cognitive collaborant annotations, including telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, outcomes annotations, costs annotations, resource consumption/utilization annotations, haptic annotations and document annotations; and at least one or more tele-visual imagery informatics management system clini-ports allowing at least one or more participant cognitive collaborants, each, capturing live streaming imagery data, capturing associated live streaming imagery metadata, including semantic metadata and annotations, retrieving archived streaming imagery data, retrieving archived associated imagery metadata, including archived semantic metadata and annotations, and transporting live streaming imagery data, transporting associated live streaming imagery metadata, including semantic metadata and annotations, and transporting live streaming imagery data, associated live streaming metadata, including semantic metadata and annotations, archived streaming imagery data, associated archived streaming metadata, including archived semantic metadata and annotations, into collaboration sessions, concurrently viewing, communicating, collaborating, consulting and instructing among participant cognitive collaborants using at least one or more sources of streaming imagery data, curating, annotating and tagging streaming imagery data, including telestrations, drawings, illustrations, alpha-numeric text annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, outcomes annotations, costs annotations, resource consumption/utilization annotations, haptic annotations and document annotations, and encapsulating streaming imagery data and associated streaming imagery metadata, including semantic metadata and annotations, together with cognitive collaborant annotations in native, single file format structures, and saving said streaming imagery data and said associated streaming imagery metadata, including semantic metadata and annotations, together with said cognitive collaborant annotations in at least one or more collaborated imagery files during collaboration sessions, including asynchronous or synchronous collaborations with at least one or more participant cognitive collaborants, communicating, collaborating, consulting and instructing, including viewing, curating, annotating and tagging, using at least one or more sources of streaming imagery data shared among at least one or more participant cognitive collaborants with a multi-channel stream viewer that enables capture, retrieval and concurrent viewing of both live and archived medical imagery streams together with associated metadata, including semantic metadata and annotations, during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting independently adding sources of streaming imagery data, adjust, enhance or reduce resolutions or frame rates of streaming imagery data with a multi-channel communications control interface, and independently view those additional channels of streaming imagery data and independently select which of those channels to bring into a collaboration session conveying instructions with two way communications among participant cognitive collaborants, including source channel selection, for viewing, curating, annotating and tagging imagery data streams with telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, image annotations, wave form annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, outcomes annotations, costs annotations, resource consumption/utilization annotations, haptic annotations and document annotations and not reliant upon any external communications network. ### Claim 2 (depends on 1) The network system of claim 1 for the acquisition and transmission of heterogeneous sources of streaming imagery data, enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during collaboration sessions, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, each participant cognitive collaborant able to view, curate, annotate and tag the heterogeneous streaming imagery data, including medical video, medical modality imagery, medical wave form imagery, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, encapsulating and saving collaborated annotations and tags together with streaming imagery data, relevant imagery metadata, including semantic metadata and annotations, including appended imagery metadata and appended semantic metadata and annotations, from the collaboration session in native, single file format structures, known as collaborated imagery files; storing collaborated imagery files from all participant cognitive collaborants locally in media libraries or image data repositories on their respective computer storage devices, in image data repositories on tele-visual imagery informatics management system servers, in image data repositories on cloud storage devices and locations, in image data repositories on picture archiving and communications systems, in other image data repositories compliant with standards for digital imaging and communications in medicine, or in any other data repository that allows streaming imagery data, annotations and metadata, including semantic metadata and annotations, to be combined in native, single file format structures, including clinical data repositories, personalized knowledge repositories, clinical cognitive vismeme vaults and metadata repositories retrieving collaborated imagery files from all participant cognitive collaborants stored locally in media libraries or image data repositories on their respective computer storage devices, in image data repositories on tele-visual imagery informatics management system servers, in image data repositories on cloud storage devices and locations, in image data repositories on picture archiving and communications systems, in other image data repositories compliant with standards for digital imaging and communications in medicine, or in any other data repository that allows streaming imagery data, annotations and metadata to be combined in native, single file format structures, including clinical data repositories, personalized knowledge repositories, clinical cognitive vismeme vaults and metadata repositories publishing and distributing collaborated imagery files in known native, single file format structures, including those used for digital imaging and communications in medicine comprising both core and non-core data element tags, together with conformance statements that enable prior evaluation and testing of streaming imagery equipment components without an actual physical connection, all of which facilitate network connectivity for imagery equipment components, communication interoperability for imagery data systems, and exchange of collaborated imagery files. ### Claim 3 (depends on 1) The network system of claim 1 for the acquisition and transmission of medical streaming imagery data, including medical images, medical video, medical modality imagery, medical wave form imagery, clinical maps, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents, the network system preserving the clinical integrity of medical streaming imagery data from medical devices, systems and equipment cleared for medical use, including clinical diagnostic purposes, care delivery and patient monitoring, enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during collaboration sessions, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, including viewing, curating, annotating and tagging streaming medical imagery data during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, and including recursive cognitive enrichments thereof, for use with medical devices, equipment, systems, image and data repositories, in native, single file format structures, including those compliant with standards for digital imaging and communications in medicine. ### Claim 4 (depends on 1) The network system of claim 1 for enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during collaboration sessions, among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, together with collaborated imagery files created from cognitive collaborant annotations, session metadata and medical streaming imagery data during collaboration sessions, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, including data cleared for clinical diagnostic purposes, that can be viewed, curated, annotated, tagged, encapsulated and saved together as collaborated medical imagery files and cleared for use with approved medical devices, equipment, systems, image and data repositories, in native, single file format structures, including those compliant with standards for digital imaging and communications in medicine. ### Claim 5 (depends on 1) The network system of claim 1 for archived collaborated imagery files that can be retrieved for use together with streaming imagery data during synchronous or asynchronous collaboration sessions, revised, appended, viewed, curated, annotated, tagged, encapsulated and saved in native, single file format structures, including those compliant with standards for digital imaging and communications in medicine, during collaboration sessions practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, and made available for use together with streaming imagery data during current or subsequent collaboration sessions. ### Claim 6 A method enabling multichannel multiplexed communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, persons, the network system enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, allowing each participant cognitive collaborant to capture, retrieve and concurrently view at least one source of streaming medical modality imagery data, and at least one or more sources of heterogeneous streaming imagery data, medical and non-medical streaming imagery data, and combinations thereof including images, video, modality imagery, audio, video and haptic wave forms and files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents, both live and archived streaming imagery data, enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data in collaboration sessions practiced by and among at least one or more participant cognitive collaborants during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, each participant cognitive collaborant able to view, curate, annotate and tag the heterogeneous streaming imagery data, comprising; a tele-visual imagery informatics management system consisting of the following essential components: at least one or more tele-visual imagery informatics management system clini-docks; wherein each clini-dock is adapted for independent acquisition and transmission of signals from other sources of streaming imagery data at native, enhanced or reduced resolutions and native, enhanced or reduced frame rates, used for the acquisition and transmission of, live or archived streaming imagery data, including images, video, modality imagery, audio, video and haptic wave forms and files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents, analog or digital video signals in standard or non-standard resolutions, medical or non-medical imagery, in compressed or uncompressed imagery formats. at least one or more tele-visual imagery informatics management system clini-pod CNS network servers, wherein each server is a neurosynaptic network node comprising at least one streamer, splitter, router, server and storage device that; enables at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, to concurrently view, communicate, collaborate, consult and instruct among participant cognitive collaborants using at least one or more sources of streaming imagery data acquired and transmitted by tele-visual imagery informatics management system clini-docks, including live streaming imagery data, archived streaming imagery data, appended streaming imagery metadata, including appended semantic metadata and annotations, cognitive collaborant annotations, and archived collaborated imagery files during a synchronous or asynchronous collaboration session; enables concurrent collaboration including viewing, curation, annotation and tagging with each and all of the one or more sources of streaming imagery data acquired and transmitted by tele-visual imagery informatics management system clini-docks, establishes and maintains channel communications for each and all of the one or more sources of streaming imagery data each participant cognitive collaborant wishes to view, monitor and collaborate with, enables at least one or more participant cognitive collaborants to concurrently view, communicate, collaborate, consult and instruct, including curation, annotation and tagging, with live streaming imagery data, archived imagery data, appended imagery metadata, including appended semantic metadata and annotations, collaborated annotations, and archived collaborated imagery files during a synchronous or asynchronous collaboration session, enables at least one or more participant cognitive collaborant in multiple locations, some of whom may be located remotely to the sources of streaming imagery data, to concurrently view, communicate, collaborate, consult and instruct, including curation, annotation and tagging, with live streaming imagery data, archived imagery data, appended imagery metadata, including appended semantic metadata and annotations, collaborated annotations, and archived collaborated imagery files from each participant cognitive collaborant during the collaboration session, dynamically manages and controls with at least one or more associated databases, authorization, authentication, identity management, security, and access, publication and distribution privileges for viewing, communicating, collaborating, consulting and instructing, and cognitive collaborant privileges, including curation, annotation, tagging, encapsulation, saving, storage, retrieval and distribution of live streaming imagery data, archived imagery data, appended imagery metadata, including appended semantic metadata and annotations, collaborated annotations, and archived collaborated imagery files for each participant cognitive collaborant during collaboration sessions; enables both synchronous and asynchronous bidirectional communications with at least one or more local area networks, at least one or more wide area networks (internet) including imagery data repositories and combinations thereof during multiple collaboration sessions enables identification, tracking and monitoring of participant cognitive collaborants by assignment of a unique color for annotations of streaming imagery data, archived collaborated imagery files and cognitive collaborant annotations, that include telestrations, drawings, illustrations, alpha-numeric text annotations, as well as cognitive collaborant annotations combined with alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, outcomes annotations, costs annotations, resource consumption/utilization annotations, haptic annotations and document annotations at least one or more tele-visual imagery informatics management system clini-ports that allows for multiple participant cognitive collaborants, each of whom can capture live streaming imagery data together with associated imagery metadata, including semantic metadata and annotations, and bring into the collaboration session, retrieve archived streaming imagery data together with associated imagery metadata, including semantic metadata and annotations, and bring into the collaboration session, concurrently view, communicate, collaborate, consult and instruct with streaming imagery data, enables curation, annotation and tagging that streaming imagery data with collaborated annotations that include telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, outcomes annotations, costs annotations, resource consumption/utilization annotations, haptic annotations and document annotations, enables encapsulation and saving collaborated streaming imagery data and archived imagery metadata, including archived semantic metadata and annotations, together with appended imagery metadata, including appended semantic metadata and annotations, and collaborated annotations and from each collaboration session, including asynchronous or synchronous collaboration with at least one or more participant cognitive collaborants, in native, single file format structures, known as collaborated imagery files, enables multimodal cognitive communications, collaboration, consultation and instruction, including viewing, curating, annotating and tagging, using at least one or more sources of streaming imagery data shared among at least one or more participant cognitive collaborants with a multi-channel stream viewer that enables capture, retrieval and concurrent viewing of both live and archived medical imagery streams together with associated metadata during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting enables independently adding sources of streaming imagery data, adjust, enhance or reduce resolutions or frame rates of streaming imagery data with a multi-channel communications control interface, and independently view those additional channels of streaming imagery data and independently select which of those channels to bring into a collaboration session enables conveying instructions with two way communications among participant cognitive collaborants, including source channel selection, for viewing, curating, annotating and tagging imagery data streams with telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, outcomes annotations, costs annotations, resource consumption/utilization annotations, haptic annotations and document annotations and not reliant upon any external communications network. ### Claim 7 (depends on 6) The method of claim 6 for the acquisition and transmission of heterogeneous sources of streaming imagery data, enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, during collaboration sessions practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, persons, each participant cognitive collaborant able to view, curate, annotate and tag the heterogeneous streaming imagery data, including medical video, medical modality imagery, medical wave form imagery, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, as well as encapsulate and save collaborated annotations and tags together with the heterogeneous streaming imagery data, including medical video, medical modality imagery, medical wave form imagery, and clinical documents, and save collaborated annotations together with streaming imagery data, relevant imagery metadata, including semantic metadata and annotations, and appended imagery metadata, including appended semantic metadata and annotations, from the collaboration session in native, single file format structures, known as collaborated imagery files; storing collaborated imagery files from all participant cognitive collaborants locally in media libraries or image data repositories on their respective computer storage devices, in image data repositories on tele-visual imagery informatics management system servers, in image data repositories on cloud storage devices and locations, in image data repositories on picture archiving and communications systems, in other image data repositories compliant with standards for digital imaging and communications in medicine, or in any other data repository that allows streaming imagery data, annotations and metadata to be combined in native, single file format structures, including clinical data repositories, personalized knowledge repositories, clinical cognitive vismeme vaults and metadata repositories retrieving collaborated imagery files from all participant cognitive collaborants stored locally in media libraries or image data repositories on their respective computer storage devices, in image data repositories on tele-visual imagery informatics management system servers, in image data repositories on cloud storage devices and locations, in image data repositories on picture archiving and communications systems, in other image data repositories compliant with standards for digital imaging and communications in medicine, or in any other data repository that allows streaming imagery data, annotations and metadata to be combined in native, single file format structures, including clinical data repositories, personalized knowledge repositories, clinical cognitive vismeme vaults and metadata repositories publishing and distributing collaborated imagery files in known native, single file format structures, including those used for digital imaging and communications in medicine comprising both core and non-core data element tags, together with conformance statements that enable prior evaluation and testing of streaming imagery equipment components without an actual physical connection, all of which facilitate network connectivity for imagery equipment components, communication interoperability for imagery data systems, and exchange of collaborated imagery files. ### Claim 8 (depends on 6) The method of claim 6 for the acquisition and transmission of medical streaming imagery data, including medical images, medical video, medical modality imagery, medical wave form imagery, clinical maps, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents, the method preserving the clinical integrity of medical streaming imagery data from medical devices systems and equipment cleared for medical use, including clinical diagnostic purposes, care delivery and patient monitoring, enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation, instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, including viewing, curation, annotation and tagging of streaming medical imagery data during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, including recursive cognitive enrichments thereof, for use with medical devices, equipment, systems, image and data repositories, in native, single file format structures, including those compliant with standards for digital imaging and communications in medicine. ### Claim 9 (depends on 6) The method of claim 6 for enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data in collaboration sessions among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, together with collaborated imagery files created from cognitive collaborant annotations, session metadata and medical streaming imagery data during collaboration sessions, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, including data cleared for clinical diagnostic purposes, that can be viewed, curated, annotated, tagged, encapsulated and saved together as collaborated medical imagery files and cleared for use with approved medical devices, equipment, systems, image and data repositories, including those compliant with standards for digital imaging and communications in medicine. ### Claim 10 (depends on 6) The method of claim 6 for archived collaborated imagery files that can be retrieved for use together with streaming imagery data during synchronous or asynchronous collaboration sessions, revised, appended, viewed, curated, annotated, tagged, encapsulated and saved in native, single file format structures, including those compliant with standards for digital imaging and communications in medicine, during collaboration sessions, practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, and made available for use together with streaming imagery data during current or subsequent collaboration sessions. ### Claim 11 (depends on 6) The method of claim 6 adapted for recursive cognitive enrichment and collaborative mind-machine knowledge exchange between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 12 (depends on 6) The method of claim 6 adapted for informatics-enabled learning specialist skills acquisition and accelerated knowledge exchange with multimodal cognitive instruction by and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 13 (depends on 6) The method of claim 6 adapted for cognitive enterprise imaging with streaming imagery informatics between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 14 (depends on 6) The method of claim 6 adapted for collaborative precision medicine with multiomic data analytics between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 15 (depends on 6) The method of claim 6 adapted for informatics-enabled imagery guided intervention, including robotic-assisted surgery, between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 16 (depends on 6) The method of claim 6 adapted for machine learning with medical imaging, including deep learning, transfer learning reinforcement learning and convolutional neural networks, between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 17 (depends on 6) The method of claim 6 adapted for precision guided biomedical nanorobotics between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 18 (depends on 6) The method of claim 6 adapted for personalized precision targeted theranostic nanomedicine between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 19 (depends on 6) The method of claim 6 adapted for managing clinical knowledge with networked metadata repositories, including semantic metatdata repositories, between and among participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting. ### Claim 20 A method enabling multichannel multiplexed communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data by participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with modular and scalable clusters of gateway streamer servers configured to support multiple network topologies, including peer-to-peer, hub-and-spoke and mesh chord networks, and heterogeneous network combinations thereof, each gateway streamer server enabling neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal cognitive communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, and clinical documents during various stages of medical disease management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting.