Augmenting clinical intelligence with federated learning, imaging analytics and outcomes decision support

A method enabling multichannel multiplexed communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data practiced by and among at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, the network system enabling both synchronous and asynchronous multimodal clinical communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data during various stages of medical disease and injury management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, as well as workflow optimization with operational analytics for outcomes, performance, results, resource utilization, resource consumption and costs, allowing each participant cognitive collaborant to capture, retrieve and concurrently view at least one source of streaming medical modality imagery data, and at least one or more sources of heterogeneous streaming imagery data, medical and non-medical streaming imagery data, and combinations thereof including images, video, modality imagery, audio, video and haptic wave forms and files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, interactive media reports, including interactive multimedia reporting, clinical documents and key performance indicators, including indicators of quality of care, as well as indicators of clinical, operational and financial performance, both live and archived streaming imagery data, enabling both synchronous and asynchronous multimodal clinical communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data in collaboration sessions practiced by and among at least one or more participant cognitive collaborants during various stages of medical disease and injury management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, as well as workflow optimization with operational analytics for outcomes, performance, results, resource utilization, resource consumption and costs, each participant cognitive collaborant able to view, curate, annotate and tag the heterogeneous streaming imagery data, comprising a tele-visual imagery informatics management system consisting of the following essential components:

• at least one or more tele-visual imagery informatics management system clini-docks, wherein each clini-dock is adapted for independent acquisition and transmission of signals from other sources of streaming imagery data at native, enhanced or reduced resolutions and native, enhanced or reduced frame rates, used for the acquisition and transmission of, live or archived streaming imagery data, including images, video, modality imagery, audio, video and haptic wave forms and files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, interactive media reports, including interactive multimedia reporting, clinical documents and key performance indicators, including indicators of quality of care, as well as indicators of clinical, operational and financial performance, analog or digital video signals in standard or non-standard resolutions, medical or non-medical imagery, in compressed or uncompressed imagery to formats;
• at least one or more tele-visual imagery informatics management system clini-pod CNS network servers, wherein each server is a neurosynaptic network node comprising at least one streamer, splitter, router, server and storage device that
• enables at least one or more participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, to concurrently view, communicate, collaborate, consult and instruct among participant cognitive collaborants using at least one or more sources of streaming imagery data acquired and transmitted by tele-visual imagery informatics management system clini-docks, including live streaming imagery data, archived streaming imagery data, appended streaming imagery metadata, including appended semantic metadata and annotations, cognitive collaborant annotations, and archived collaborated imagery files during a synchronous or asynchronous collaboration session,
• enables concurrent collaboration including viewing, curation, annotation and tagging with each and all of the one or more sources of streaming imagery data acquired and transmitted by tele-visual imagery informatics management system clini-docks,
• establishes and maintains channel communications for each and all of the one or more sources of streaming imagery data each participant cognitive collaborant wishes to view, monitor and collaborate with,
• enables at least one or more participant cognitive collaborants to concurrently view, communicate, collaborate, consult and instruct, including curation, annotation and tagging, with live streaming imagery data, archived imagery data, appended imagery metadata, including appended semantic metadata and annotations, collaborated annotations, and archived collaborated imagery files during a synchronous or asynchronous collaboration session,
• enables at least one or more participant cognitive collaborant in multiple locations, some of whom may be located remotely to the sources of streaming imagery data, to concurrently view, communicate, collaborate, consult and instruct, including curation, annotation and tagging, with live streaming imagery data, archived imagery data, appended imagery metadata, including appended semantic metadata and annotations, collaborated annotations, and archived collaborated imagery files from each participant cognitive collaborant during the collaboration session,
• dynamically manages and controls with at least one or more associated databases, authorization, authentication, identity management, security, and access, publication and distribution privileges for viewing, communicating, collaborating, consulting and instructing, and cognitive collaborant privileges, including curation, annotation, tagging, encapsulation, saving, storage, retrieval and distribution of live streaming imagery data, archived imagery data, appended imagery metadata, including appended semantic metadata and annotations, collaborated annotations, and archived collaborated imagery files for each participant cognitive collaborant during collaboration sessions, including managing and controlling security tokens providing access for cognitive collaborants maintained in security metadata repositories, blockchain metadata repositories and blockchain data ledgers,
• enables both synchronous and asynchronous bidirectional communications with at least one or more local area networks, at least one or more wide area networks (internet) including imagery data repositories and combinations thereof during multiple collaboration sessions,
• enables identification, tracking and monitoring of participant cognitive collaborants by assignment of unique colors for annotations of streaming imagery data, archived collaborated imagery files and cognitive collaborant annotations, that include telestrations, drawings, illustrations, alpha-numeric text annotations, as well as cognitive collaborant annotations combined with alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, haptic annotations, document annotations, outcomes annotations, performance annotations, results annotations, resource consumption annotations, resource utilization annotations and costs annotations,
• enables colorized telestration, annotation and masking of colorized attention maps and colorized prediction bases for explainable artificial intelligence by participant cognitive collaborants by assignment of unique colors for annotations of streaming imagery data, archived collaborated imagery files and cognitive collaborant annotations, including telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, haptic annotations, document annotations, outcomes annotations, performance annotations, results annotations, costs annotations, resource consumption annotations and resource utilization annotations;
• and
• at least one or more tele-visual imagery informatics management system clini-ports that allows for multiple participant cognitive collaborants, each of whom can
• capture live streaming imagery data together with associated imagery metadata, including semantic metadata and annotations, and bring into the collaboration session,
• retrieve archived streaming imagery data together with associated imagery metadata, including semantic metadata and annotations, and bring into the collaboration session,
• concurrently view, communicate, collaborate, consult and instruct with streaming imagery data,
• enables curation, annotation and tagging that streaming imagery data with collaborated annotations that include telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, haptic annotations, document annotations, outcomes annotations, performance annotations, results annotations, resource consumption annotations, resource utilization annotations and costs annotations,
• enables encapsulation and saving collaborated streaming imagery data and archived imagery metadata, including archived semantic metadata and annotations, together with appended imagery metadata, including appended semantic metadata and annotations, and collaborated annotations and from each collaboration session, including asynchronous or synchronous collaboration with at least one or more participant cognitive collaborants, in native, single file format structures, known as collaborated imagery files,
• enables multimodal clinical communications, collaboration, consultation and instruction, including viewing, curating, annotating and tagging, using at least one or more sources of streaming imagery data shared among at least one or more participant cognitive collaborants with a multi-channel stream viewer that enables capture, retrieval and concurrent viewing of both live and archived medical imagery streams together with associated metadata during various stages of medical disease and injury management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, as well as workflow optimization with operational analytics for outcomes, performance, results, resource utilization, resource consumption and costs,
• enables independently adding sources of streaming imagery data, adjust, enhance or reduce resolutions or frame rates of streaming imagery data with a multi-channel communications control interface, and independently view those additional channels of streaming imagery data and independently select which of those channels to bring into a collaboration session,
• enables conveying instructions with two way communications among participant cognitive collaborants, including source channel selection, for viewing, curating, annotating and tagging imagery data streams with telestrations, drawings, illustrations, alpha-numeric text annotations, image annotations, wave form annotations, voice annotations, video annotations, augmented reality imagery annotations, 3D/4D imagery annotations, haptic annotations, document annotations, outcomes annotations, performance annotations, results annotations, resource consumption annotations, resource utilization annotations and costs annotations, and not reliant upon any external communications network.

A method enabling multichannel multiplexed communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, with streaming imagery data by participant cognitive collaborants, including persons, machines, devices, neural networks, robots and algorithms, and heterogeneous networked teams composed thereof, with modular and scalable clusters of gateway streamer servers configured to support multiple network topologies, including peer-to-peer, hub-and-spoke, mesh chord and core-spine-leaf networks, as well as in 2-tier, 3-tier, or N-tier application architectures, and heterogeneous network combinations thereof, each gateway streamer server enabling neurosynaptic network connectivity enabling both synchronous and asynchronous multimodal clinical communications, collaboration, consultation and instruction, as well as recursive cognitive enrichment and collaborative knowledge exchange, that includes viewing, curating, annotating and tagging, using at least one or more sources of multichannel, multiplexed heterogeneous streaming imagery data, including both medical and non-medical streaming imagery data, and combinations thereof, and together with images, video, modality imagery, waveforms, audio and haptic files, multiomic—phenotypic, genomic, metabolomic, pathomic, radiomic, radiopathomic and radiogenomic—maps and clinical data sets, biometric maps and movies, hapmaps, heat maps, data stream visualizations, structured reports, interactive media reports, including interactive multimedia reporting, clinical documents and key performance indicators, including indicators of quality of care, as well as indicators of clinical, operational and financial performance, during various stages of medical disease and injury management, including detection, diagnosis, prognosis, treatment, measurement, monitoring and reporting, as well as workflow optimization with operational analytics for outcomes, performance, results, resource utilization, resource consumption and costs.