# US 2021/0278943 A1 · Optical workspace link

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0278943 A1
- Document ID: US20210278943A1
- Status: Published application
- Kind: A1
- Application number: 17/193,805
- Filed: 2021-03-05
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: Critical Systems, Inc.
- Applicants: Critical Systems, Inc.
- Inventors: Theodore J. Jones; James A. Pasker; Douglas Todd Kaltenecker; Morgan Whitworth; Michael Troy Reese
- Web page: https://patentgrep.com/patent/US20210278943A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0486: Drag-and-drop
- G05B 19/406: characterised by monitoring or safety (G05B19/19 takes precedence)
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06T 11/00: Two-dimensional [2D] image generation
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G07C 3/08: Registering or indicating the production of the machine either with or without registering working or idle time
- G05B 2219/31437: Monitoring, global and local alarms
- G05B 2219/31467: Display of operating conditions of machines, workcells, selected programs
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

Embodiments described herein are directed to methods, systems, apparatuses, and user interfaces for remotely monitoring and controlling objects identified in images. In one scenario, a system is provided that includes an image sensing device configured to capture images, a transceiver, and an interactive interface that allows a user to select objects identified in at least one of the images captured by the image sensing device. Selecting an identified object within the images creates a corresponding node that provides data related to the identified object. The system also includes a data collection hub configured to receive and aggregate data received from the nodes created by the user through the interactive interface.

## Claims

### Claim 1

A system, comprising:
an image sensing device configured to capture images;
a transceiver;
an interactive interface that allows a user to select one or more objects identified in at least one of the images captured by the image sensing device, wherein selecting an identified object within the images creates a corresponding node that provides data related to the identified object; and
a data collection hub configured to receive and aggregate data received from one or more of the nodes created by the user through the interactive interface.

### Claim 2 (depends on 1)

The system of claim 1, wherein the interactive interface allows users to overlay one or more configurable interactive patterns over the identified objects in the images.

### Claim 3 (depends on 2)

The system of claim 2, wherein the configurable interactive patterns are dragged and dropped onto the identified objects, such that the configurable interactive patterns are overlaid on top of the identified objects.

### Claim 4 (depends on 3)

The system of claim 3, wherein the configurable interactive patterns overlaid on top of the identified objects allow users to receive data from the identified objects and transmit data to the identified objects.

### Claim 5 (depends on 4)

The system of claim 4, wherein the data includes current status data for the identified objects.

### Claim 6 (depends on 3)

The system of claim 3, wherein the configurable interactive patterns overlaid on top of the identified objects allow real-time interaction with the identified objects.

### Claim 7 (depends on 3)

The system of claim 3, wherein the identified objects in the images comprise at least one of electronic devices, pieces of machinery, pieces of equipment, people, or sensors.

### Claim 8 (depends on 1)

The system of claim 1, wherein the data received at the data collection hub is presented in a control room monitoring device.

### Claim 9 (depends on 1)

The system of claim 1, wherein the image sensing device is positioned to capture a specific workspace, and wherein the objects identified in the images of the workspace comprise equipment that is to be monitored.

### Claim 10 (depends on 1)

The system of claim 1, wherein the interactive interface includes one or more user interface display elements that display the data related to the identified object.

### Claim 11 (depends on 10)

The system of claim 10, wherein the user interface display elements are displayed on one or more computer systems that are remote from a workspace that is being monitored.

### Claim 12

A computer-implemented method comprising:
capturing one or more images using an image sensing device;
instantiating an interactive interface that allows a user to select one or more objects identified in at least one of the images captured by the image sensing device;
receiving one or more user inputs that select an identified object within the images, wherein the selection creates a corresponding node that provides data related to the identified object; and
instantiating a data collection hub configured to receive and aggregate data received from one or more of the nodes created by the user through the interactive interface.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein the data collection hub is further configured to monitor for changes in state in equipment under surveillance by the image sensing device.

### Claim 14 (depends on 13)

The computer-implemented method of claim 13, further comprising generating one or more alerts or notifications directed to specific individuals upon determining that a specified change in state has occurred.

### Claim 15 (depends on 12)

The computer-implemented method of claim 12, wherein the aggregated data received from the one or more nodes created by the user is further analyzed by one or more machine learning algorithms to identify when the identified object is functioning abnormally.

### Claim 16 (depends on 12)

The computer-implemented method of claim 12, wherein one or more machine learning algorithms are implemented to identify one or more of the objects in the images captured by the image sensing device.

### Claim 17 (depends on 12)

The computer-implemented method of claim 12, wherein the interactive interface provides configurable interactive patterns that are overlaid on top of one or more of the identified objects in the images.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, wherein the configurable interactive patterns allow users to issue commands to the identified objects that are interpreted and carried out by the identified objects.

### Claim 19 (depends on 18)

The computer-implemented method of claim 18, wherein the issued commands specify one or more changes of state that are to be effected on the identified objects.

### Claim 20

A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
capture one or more images using an image sensing device;
instantiate an interactive interface that allows a user to select one or more objects identified in at least one of the images captured by the image sensing device;
receive one or more user inputs that select an identified object within the images, wherein the selection creates a corresponding node that provides data related to the identified object; and
instantiate a data collection hub configured to receive and aggregate data received from one or more of the nodes created by the user through the interactive interface.
