# US 11,113,530 B2 · System and method for dynamic three dimensional command and control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,530 B2 - Document ID: US11113530B2 - Status: Granted - Kind: B2 - Application number: 16/733,835 - Filed: 2020-01-03 - Published: 2021-07-08 - Granted: 2021-09-07 - Earliest claimed: 2020-01-03 - Assignees: Booz Allen Hamilton Inc. - Applicants: Booz Allen Hamilton Inc. - Inventors: James J. Ter Beest, III; Evan Montgomery-Recht; Mary Palmer; Alexander Tejada; Sanjana Thirumalai; Jeremiah White - Web page: https://patentgrep.com/patent/US11113530B2 ## CPC - H04Q 9/00: Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom - G06Q 10/06: Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06V 20/20: in augmented reality scenes - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - H04L 63/1458: {Denial of Service} - F41H 13/0043: {Directed energy weapons, i.e. devices that direct a beam of high energy content toward a target for incapacitating or destroying the target} - H04Q 2209/20: using a distributed architecture - H04Q 2209/40: using a wireless architecture - H04Q 2209/823: where the data is sent when the measured values exceed a threshold, e.g. sending an alarm - H04Q 2209/826: where the data is sent periodically - H04Q: SELECTING (switches, relays, selectors H01H; wireless communication networks H04W) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - F41H: ARMOUR; ARMOURED TURRETS; ARMOURED OR ARMED VEHICLES; MEANS OF ATTACK OR DEFENCE, e.g. CAMOUFLAGE, IN GENERAL ## Abstract An exemplary system for monitoring an operational area, includes a processor that receive plural data streams. Each data stream can include a different spatial characteristic of the operational area. The processor also generates a three-dimensional (3D) virtual visualization of the operational area based on observational perspectives associated with the data streams and their associated spatial characteristics and dynamically prioritizes operational sub-regions within the operational area based on the spatial characteristics. The processor generates a signal encoded with data for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions. ## Claims ### Claim 1 A system for monitoring an operational area, the system comprising: a sensor arrangement including two or more sensors, wherein: each sensor includes a processor configured to receive plural data streams from the sensor arrangement, each data stream including a different spatial characteristic of the operational area; and each sensor is configured to: generate a three-dimensional (3D) virtual visualization of the operational area based on observational perspectives associated with the data streams and their associated spatial characteristics; dynamically prioritize operational sub-regions within the operational area based on the spatial characteristics; and generate a signal encoded with data for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions. ### Claim 2 (depends on 1) The system according to claim 1, further comprising: an interface that receives the signal from the processor and generates an output verifying the 3D virtual visualization of the operational area. ### Claim 3 (depends on 1) The system according to claim 1, wherein the two or more sensors are configured to move between two or more locations of observational perspective with respect to the operational area. ### Claim 4 (depends on 1) The system according to claim 1, wherein the two or more sensors include any combination of one or more sensor types. ### Claim 5 (depends on 4) The system according to claim 4, wherein each sensor comprises: at least one of an acoustic, radio frequency, electro-optical, thermal, chemical, biological, radiological, nuclear, explosive, temperature, or mechanical sensing capability. ### Claim 6 (depends on 5) The system according to claim 5, further comprising: a communication interface for communicating with a network, and wherein each sensor is configured to: transmit a data stream periodically or non-periodically to the processor over the network. ### Claim 7 (depends on 6) The system according to claim 6, wherein each sensor is configured to: transmit data to the processor when a change in a spatial characteristic of an operational sub-region is detected. ### Claim 8 (depends on 1) The system according to claim 1, comprising: a memory device for storing the plural data streams. ### Claim 9 (depends on 1) The system according to claim 1, wherein each sensor is configured to: generate the 3D virtual visualization of the operational area by dividing the operational area into a plurality of 3D grid spaces. ### Claim 10 (depends on 9) The system according to claim 9, wherein each operational sub-region includes one or more 3D grid spaces. ### Claim 11 (depends on 10) The system according to claim 10, wherein each sensor is configured to: generate the plural data streams by observing the operational area, wherein a 3D geometry of each operational sub-region is determined based on one or more sensing characteristics of the sensor arrangement. ### Claim 12 (depends on 11) The system according to claim 11, wherein the 3D geometry of each operational sub-region is dimensioned according to spatial properties of the operational area. ### Claim 13 (depends on 11) The system according to claim 11, wherein each sensor is configured to: dynamically change the 3D geometry of one or more operational sub-regions based on a detected change in a spatial characteristic of the one or more operational sub-regions, the change in the spatial characteristic is determined from the plural data streams. ### Claim 14 (depends on 10) The system according to claim 10, wherein each sensor is configured to: assign a real-time prioritization value to each operational sub-region based on one or more of: an operational importance to a specified operation, a spatial or temporal relevance to a specified operation activity, the period since last sensor observation, available granularity of the spatial characteristics in the data streams, an observational perspective of an operational sub-region for which the data streams are generated, a sensor type, or velocity of an anomaly within an operational sub-region of interest or a sub-region adjacent to the sub-region. ### Claim 15 (depends on 14) The system according to claim 14, wherein each sensor is configured to assign on a case-by-case basis: a first priority to an operational sub-region determined to be included in a security operation; a null priority to a sub-region determined to be outside of a specified operational area; and a second priority to an operational sub-region determined to be on a fringe of the operational area and/or undetectable by a sensor, the second priority being lower than the first priority. ### Claim 16 (depends on 14) The system according to claim 14, wherein each sensor is configured to assign a priority to one or more operational sub-regions, or one or more sensors with observational perspective of an operational sub-region, based on requirements of a specified operation, wherein at least two or more of the assigned priorities are different. ### Claim 17 (depends on 1) The system according to claim 1, wherein each sensor is configured to process each data stream by detecting one or more of: a position of an anomaly in an operational sub-region or movement of an anomaly in an operational sub-region. ### Claim 18 (depends on 17) The system according to claim 17, wherein each sensor is configured to: extract portions of the spatial characteristics from each data stream and combine the extracted portions into a combined data set, and identify one or more patterns in the combined data set. ### Claim 19 (depends on 1) The system according to claim 1, wherein each sensor is configured to: increase a priority of an operational sub-region when an interval increases between receptions of data streams from a sensor arrangement having observational perspective, if the duration of the interval exceeds a threshold. ### Claim 20 (depends on 1) The system according to claim 1, configured to: assign a priority to an operational sub-region based on the sensor arrangement's observational perspective of the sub-region, and the sensor arrangement's proximity to a detected anomaly. ### Claim 21 (depends on 1) The system according to claim 1, wherein each sensor is configured to: increase a priority of an operational sub-region over time when data streams received from the operational sub-region are acquired by two or more sensor arrangements having a common observational perspective. ### Claim 22 (depends on 1) The system according to claim 1, wherein each sensor has observational perspective of each operational sub-region and the system is configured to: prioritize each sensor based on characteristics including a location of the operational sub-region, the interval between data streams from the operational sub-regions, the location of an anomaly in or near the operational sub-region, or an observational perspective of the operational sub-region. ### Claim 23 (depends on 1) The system according to claim 1, wherein the sensor arrangement comprises: a first sensor arrangement and a second sensor arrangement, and the system is configured to: dynamically adjust the priority of a first operational sub-region, adjust one or more properties of the first sensor arrangement based on the adjusted priority, and acquire data from the second sensor arrangement in a next highest priority operational sub-region. ### Claim 24 (depends on 23) The system according to claim 23, further comprising: a computing system configured to: adjust one or more properties of the first sensor arrangement that has observational perspective of the first operational sub-region, to eliminate gaps in the sensor arrangement's observational perspective of the first operational sub-region, establish granular observation of the first operational sub-region, or perform a triangulation of an anomaly under observation in the first operational sub-region. ### Claim 25 (depends on 23) The system according to claim 23, further comprising: a computing system configured to: identify a first sensor of the first sensor arrangement in the first operational sub-region that is currently engaged in a first observation activity, and identify a second sensor in the first sensor arrangement that is available to engage in a second observation activity in the first operational sub-region. ### Claim 26 (depends on 1) The system according to claim 1, further comprising: a computing system configured to: control one or more sub-systems for performing predefined critical tasks to respond to or mitigate threats such as denial of service attacks or impairing or halting electronic functionality of the system or devices associated with the system. ### Claim 27 (depends on 26) The system according to claim 26, in combination with the one or more subsystems, each of which includes an electronic warfare system, a directed energy system, or a kinetic weapon system. ### Claim 28 A method for monitoring an operational area, the method comprising: in one or more devices connected in a network: receiving one or more data streams from a sensor arrangement that includes two or more sensors, each data stream including a different spatial characteristic of the operational area; and in at least one of the two or more sensors of the sensor arrangement: generating a 3D virtual visualization of the operational area based on an observational perspective associated with each data stream and the different spatial characteristics included in each data stream; dynamically prioritizing operational sub-regions within the operational area based on the received spatial characteristics; and generating a signal for verifying the 3D visual virtualization of the operational area including the prioritized operational sub-regions. ### Claim 29 (depends on 28) The method according to claim 28, comprising: in an interface of the one or more devices or an interface connected to the network: receiving the signal for verifying the 3D virtual visualization of the operational area; and generating an output that verifies the 3D virtual visualization of the operational area. ### Claim 30 (depends on 28) The method according to claim 28, wherein generating the 3D virtual visualization of the operational area comprises: defining one or more 3D grid spaces within the operational area. ### Claim 31 (depends on 28) The method according to claim 28, wherein each operational sub-region includes one or more 3D grid spaces, the method comprising: defining a 3D geometry of each operational sub-region based on the observational perspective of each data stream and the spatial characteristics included in each data stream; and dynamically changing the 3D geometry of one or more operational sub-regions based on a detected change in a spatial characteristic of the one or more operational sub-regions. ### Claim 32 (depends on 31) The method according to claim 31, comprising: assigning a real-time priority to each operational sub-region based on one or more of: operational importance to a specified operation, spatial or temporal relevance to a specified operation activity, the duration of the interval since last sensor observation, available granularity of sensing, the observational perspective of the sensor arrangement within an operational sub-region, the sensor-type, or velocity of an anomaly within an operational sub-region or an adjacent operational sub-region. ### Claim 33 (depends on 32) The method according to claim 32, wherein assigning a real-time priority to each operational sub-region, on a case-by-case basis, comprises: assigning a first priority to an operational sub-region that will be included in security operations; assigning a null priority to an operational sub-region determined to be outside of a specified operational area; or assigning a second priority lower than the first priority to an operational sub-region determined to be on a fringe of the operational area and/or undetectable by a sensor of the two or more sensors. ### Claim 34 (depends on 32) The method according to claim 32, comprising: assigning a priority to one or more operational sub-regions or one or more sensors having observational perspective of an operational sub-region based on characteristics of a specified operational activity, wherein two or more of the assigned priorities are different. ### Claim 35 (depends on 34) The method according to claim 34, comprising: identifying a first sensor in the sensor arrangement having observational perspective of the first operational sub-region that is currently engaged in a first observation activity, and identifying a second sensor in the sensor arrangement that is available to engage in a second observation activity in the first operational sub-region, wherein identifying the second sensor in the first operational sub-region is based on one or more of an availability of resources, a prioritization of operational sub-regions, or a prioritization of anomalies. ### Claim 36 (depends on 34) The method according to claim 34, comprising: dynamically adjusting the priority of the first operational sub-region, and dynamically adjusting one or more properties of a first sensor arrangement in the first operational sub-region based on the adjusted priority and acquiring spatial characteristics of the first operational sub-region from a second data stream received from a second sensor arrangement in a next highest priority operational sub-region. ### Claim 37 (depends on 36) The method according to claim 36, comprising at least one of: adjusting one or more properties of the sensor arrangement having observational perspective on the first operational sub-region to eliminate gaps in observational perspective in coverage of the 3D geometry of the first operational sub-region; establishing granular observation of the first operational sub-region; or performing a triangulation of an anomaly under observation in the first operational sub-region. ### Claim 38 (depends on 28) The method according to claim 28, comprising: analyzing the data streams from a sensor arrangement having observational perspective of a first operational sub-region to detect a position of an anomaly in the first operational sub-region, and/or movement of an anomaly in the first operational sub-region. ### Claim 39 (depends on 38) The method according to claim 38, comprising: extracting portions of the spatial characteristics from the data streams of the first operational sub-region; combining the extracted portions into a combined data set; and identifying one or more patterns in the combined data set. ### Claim 40 A non-transitory computer readable medium encoded with a program for performing a method of monitoring an operational area, which when placed in communicative contact with a processor configures the processor to: generate a three-dimensional (3D) virtual visualization of the operational area based on an observational perspective of plural data streams from a sensor arrangement and different spatial characteristics included in each data stream, wherein the sensor arrangement includes two or more sensors, each sensor configured to: dynamically prioritize operational sub-regions within the operational area based on the spatial characteristics; and generate a signal for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions. ### Claim 41 A system for monitoring an operational area, the system comprising: a sensor arrangement including two or more sensors, each sensor including a processor configured to receive a plurality of data streams from the other sensors in the sensor arrangement, each data stream including a different spatial characteristic of the operational area, each sensor configured to: generate a three-dimensional (3D) virtual visualization of the operational area based on observational perspectives associated with the data streams and their associated spatial characteristics; dynamically prioritize operational sub-regions within the operational area based on the spatial characteristics; increase a priority of one of the operational sub-regions when an interval between reception of the data streams from the sensor arrangement increases beyond a threshold; and generate a signal encoded with data for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions. ### Claim 42 A system for monitoring an operational area, the system comprising: a sensor arrangement including two or more sensors, each sensor including a processor configured to receive a plurality of data streams from the other sensors in the sensor arrangement, each data stream including a different spatial characteristic of the operational area, each sensor configured to: generate a three-dimensional (3D) virtual visualization of the operational area based on observational perspectives associated with the data streams and their associated spatial characteristics; dynamically prioritize operational sub-regions within the operational area based on the spatial characteristics; assign a priority to one of the operational sub-regions based on the observational perspective of the sensors in the sensor arrangement, proximity of the sensors in the sensor arrangement to a detected anomaly; and generate a signal encoded with data for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions. ### Claim 43 A system for monitoring an operational area, the system comprising: a sensor arrangement including two or more sensors, each sensor including a processor configured to receive a plurality of data streams from the other sensors in the sensor arrangement, each data stream including a different spatial characteristic of the operational area, each sensor configured to: generate a three-dimensional (3D) virtual visualization of the operational area based on observational perspectives associated with the data streams and their associated spatial characteristics; dynamically prioritize operational sub-regions within the operational area based on the spatial characteristics; increase a priority of one of the operational sub-regions over time when data streams received from the operational sub-region are acquired by multiple sensor arrangements having a common observational perspective; and generate a signal encoded with data for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions.