# US 12,189,381 B2 · Extended reality playback of events for manufacturing, installation, operation, maintenance and repair of infrastructure equipment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,189,381 B2
- Document ID: US12189381B2
- Status: Granted
- Kind: B2
- Application number: 17/458,209
- Filed: 2021-08-26
- Published: 2021-12-16
- Granted: 2025-01-07
- Earliest claimed: 2013-03-12
- Assignees: ASCO Power Technologies, L.P.
- Applicants: ASCO Power Technologies, L.P.
- Inventors: Mario Ibrahim; Tewfik Meftah; Olivier Bouffet
- Web page: https://patentgrep.com/patent/US12189381B2

## CPC

- G05B 23/0272: {Presentation of monitored results, e.g. selection of status reports to be displayed; Filtering information to the user}
- G05B 15/02: electric
- G05B 23/0216: {Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration}
- G05B 2219/24055: Trace, store a working, operation history
- G05B 2219/2642: Domotique, domestic, home control, automation, smart house
- Y04S 10/52: Outage or fault management, e.g. fault detection or location
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- Y04S: SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS

## Abstract

System and methods for monitoring and control infrastructure equipment uses extended reality playback to monitor such infrastructure equipment. The system and method advantageously blends videos and images of physical equipment with computer-generated representations of events to enable playback of the events, including any user interactions with the equipment. The extended reality playback presents events captured by real-time event logs for the infrastructure equipment using extended reality media content. The playback is composed of a real-time sequence of media content depicting one or more equipment operations corresponding to the events as a continuous presentation. The real-time sequence can then be displayed in real time and stored for subsequent playback at any time by a user. The real-time sequence can also be compared to a verified “golden” sequence to detect errors in the infrastructure equipment.

## Claims

### Claim 1

An extended reality infrastructure monitoring and control system, comprising:
a media storage processor operable to store extended reality media content for infrastructure equipment; and
a media playback processor coupled to the media storage processor, the media playback processor operable to:
receive real-time event data associated with the infrastructure equipment recorded in a real-time event log of the infrastructure equipment, wherein the real time event data comprises a series of events associated with one or more operations of the infrastructure equipment, the events including one or more of a circuit breaker transitioning from OPEN to CLOSE and CLOSE to OPEN, a switch transitioning from MAIN to BACKUP and BACKUP to MAIN, and a control relay transitioning from ON to OFF and OFF to ON;
identify, from an equipment database, an equipment type of the infrastructure equipment and/or facility where the infrastructure equipment is installed;
access extended reality media content stored by the media storage processor corresponding to the series of events, wherein a media content type of the media content may vary based on the equipment type of the infrastructure equipment;
assemble the extended reality media content into a real-time sequence of extended reality media content corresponding to the series of events associated with the one or more operations of the infrastructure equipment; and
present the real-time sequence at a user device as a continuous presentation of extended reality media content that provides an extended reality playback of the series of events occurring at a visual representation of the infrastructure equipment.

### Claim 2 (depends on 1)

The system of claim 1, wherein the media playback processor is further operable to perform error detection for the real-time sequence of extended reality media content and provide an error indication in the real-time sequence of extended reality media content upon detection of an error.

### Claim 3 (depends on 2)

The system of claim 2, wherein the media playback processor is operable to perform error detection by comparing the real-time sequence of extended reality media content to a corresponding verified sequence of extended reality media content.

### Claim 4 (depends on 1)

The system of claim 1, wherein the media playback processor is further operable to present the real-time sequence at a user device in real time and at any time thereafter as selected by a user.

### Claim 5 (depends on 1)

The system of claim 1, wherein the media playback processor is further operable to present in real time or play back an extended reality service at the user device, the extended reality service including: a guided procedure, X-ray vision, hands-free documentation, real-time equipment statuses, and animated equipment connections.

### Claim 6 (depends on 1)

The system of claim 1, wherein the infrastructure equipment includes electrical power infrastructure equipment.

### Claim 7 (depends on 1)

The system of claim 1, wherein the system is hosted locally relative to the infrastructure equipment, remotely on a private or public cloud, or a combination of locally and remotely.

### Claim 8

A method for event playback in infrastructure equipment, comprising:
receiving event data associated with the infrastructure equipment recorded in a real-time event log of the infrastructure equipment, wherein the event data comprises a series of events associated with one or more operations of the infrastructure equipment, the events including one or more of a circuit breaker transitioning from OPEN to CLOSE and CLOSE to OPEN, a switch transitioning from MAIN to BACKUP and BACKUP to MAIN, and a control relay transitioning from ON to OFF and OFF to ON;
identifying, from an equipment database, an equipment type for the infrastructure equipment and/or a facility where the infrastructure equipment is installed;
assembling media content corresponding to each of the series of events associated with the one or more operations of the infrastructure equipment, wherein a media content type of the media content may vary based on the equipment type of the infrastructure equipment; and
displaying a continuous presentation of the media content at a user device, the continuous presentation providing an extended reality playback of the series of events occurring at a visual representation of the infrastructure equipment.

### Claim 9 (depends on 8)

The method of claim 8, further comprising detecting perform error detection for infrastructure equipment by comparing the extended reality playback of the series of events to a corresponding verified series of events.

### Claim 10 (depends on 9)

The method of claim 9, further comprising providing an error indication in the extended reality playback of the series of events upon detection of an error.

### Claim 11 (depends on 8)

The method of claim 8, wherein the user device includes one of a wearable headset or glasses.

### Claim 12 (depends on 8)

The method of claim 8, wherein the visual representation of the infrastructure equipment includes a real-time video of the infrastructure equipment.

### Claim 13 (depends on 8)

The method of claim 8, wherein the media content includes computer-generated representations of events occurring at the infrastructure equipment.

### Claim 14 (depends on 8)

The method of claim 8, further comprising manipulating the infrastructure equipment in the extended reality playback in accordance with a user selection.

### Claim 15 (depends on 8)

The method of claim 8, wherein manipulating the infrastructure equipment includes reversing at least one event in the series of events.

### Claim 16 (depends on 8)

The method of claim 8, further comprising displaying a characteristic of the infrastructure equipment in a virtual layer over the equipment in the extended reality playback.

### Claim 17 (depends on 16)

The method of claim 16, wherein the characteristics include an internal temperature, a voltage, or a presence of flammable gas within the infrastructure equipment.

### Claim 18 (depends on 8)

The method of claim 8, wherein further assembling is stopped once the infrastructure equipment has reached a steady state or no changes have taken place in the infrastructure equipment after a predetermined time period.

### Claim 19 (depends on 1)

The method of claim 1, wherein access to the media player is granted by a user terminal scan of a visual code, a scan of the equipment, or a location selection.

### Claim 20 (depends on 8)

A non-transitory computer-readable medium having computer-readable instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform a method according to claim 8.