# US 12,216,464 B2 · Augmented reality system for diagnosing system operation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,216,464 B2
- Document ID: US12216464B2
- Status: Granted
- Kind: B2
- Application number: 17/746,613
- Filed: 2022-05-17
- Published: 2023-11-23
- Granted: 2025-02-04
- Earliest claimed: 2022-05-17
- Assignees: Honeywell International Inc.
- Applicants: Honeywell International Inc.
- Inventors: James Junior Nkhata
- Web page: https://patentgrep.com/patent/US12216464B2

## CPC

- G05B 23/0267: {Fault communication, e.g. human machine interface [HMI]}
- G05B 23/0272: {Presentation of monitored results, e.g. selection of status reports to be displayed; Filtering information to the user}
- G06F 3/147: using display panels
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A camera of a mobile device may be used to capture an image of at least part of a component. The component is identified based at least in part on the captured image. Once the component has been identified, the mobile device communicates with a remote server to receive one or more real-time electrical parameter values that correspond to the component. A diagnostic image of the component is displayed on a display of the mobile device, the diagnostic image showing one or more wiring terminals of the component. One or more of the real-time electrical parameter values received from the remote server are superimposed on the diagnostic image, wherein each of the one or more real-time electrical parameter values that is superimposed is positioned adjacent to the corresponding one or more of the wiring terminals.

## Claims

### Claim 1

A method of using a mobile device for diagnosing a system, the mobile device including a display and a camera, the system including a component having a plurality of wiring terminals for wiring the component to one or more other components of the system, the system configured to identify and report to a remote server one or more real-time electrical parameter values that each correspond to one or more of the wiring terminals of the component, the method comprising:
using the camera of the mobile device to capture an image of at least part of the component;
identifying the component based at least in part on the captured image;
once the component has been identified, the mobile device communicating with the remote server to receive one or more real-time electrical parameter values that correspond to the component;
displaying a diagnostic image of the component on the display of the mobile device, the diagnostic image showing one or more wiring terminals of the component; and
superimposing on the diagnostic image one or more of the real-time electrical parameter values received from the remote server, wherein each of the one or more real-time electrical parameter values that is superimposed is positioned adjacent to the corresponding one or more of the wiring terminals.

### Claim 2 (depends on 1)

The method of claim 1, wherein using the camera of the mobile device to capture the image of at least part of the component includes capturing an image of a unique identifier displayed on the component.

### Claim 3 (depends on 2)

The method of claim 2, wherein the unique identifier comprises a bar code, a QR code and/or an alphanumeric identifier code.

### Claim 4 (depends on 1)

The method of claim 1, wherein the diagnostic image comprises a video frame of a real-time video stream, and the method includes repeating the display and superimposing steps over a plurality of video frames.

### Claim 5 (depends on 1)

The method of claim 1, wherein the system comprises a security system, and the component comprises a security component of the security system.

### Claim 6 (depends on 1)

The method of claim 1, wherein at least one of the real-time electrical parameter values comprises a real-time resistance value.

### Claim 7 (depends on 1)

The method of claim 1, wherein at least one of the real-time electrical parameter values comprises a real-time voltage value.

### Claim 8 (depends on 1)

The method of claim 1, wherein at least one of the real-time electrical parameter values comprises a real-time current value.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
applying a filter to the real-time electrical parameter values using filter criteria; and
superimpose only the real-time electrical parameter values that meet the filter criteria.

### Claim 10 (depends on 9)

The method of claim 9, further comprising allowing a user to change the filter criteria.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
identifying an access level of a user;
identifying the real-time electrical parameter values that are available to the user based on the access level of the user; and
superimpose only the real-time electrical parameter values that are available to the user.

### Claim 12

A method of using a mobile device for diagnosing a system, the mobile device including a display and a camera, the system including a component having a plurality of wiring terminals for wiring the component to one or more other components of the system, the method comprising:
identifying the component to the mobile device;
once the component has been identified to the mobile device, the mobile device requesting and receiving one or more electrical parameter values that each correspond to one or more of the wiring terminals of the component;
displaying a diagnostic image of the component on the display of the mobile device, the diagnostic image including one or more wiring terminals of the component; and
superimposing on the diagnostic image one or more of the electrical parameter values, wherein each of the one or more electrical parameter values is superimposed adjacent to the corresponding one or more of the wiring terminals.

### Claim 13 (depends on 12)

The method of claim 12, wherein at least one of the electrical parameter values comprises a real-time resistance value.

### Claim 14 (depends on 12)

The method of claim 12, wherein at least one of the electrical parameter values comprises a real-time voltage value.

### Claim 15 (depends on 12)

The method of claim 12, wherein at least one of the electrical parameter values comprises a real-time current value.

### Claim 16 (depends on 12)

The method of claim 12, wherein the mobile device requests and receives the one or more electrical parameter values that each correspond to one or more of the wiring terminals of the component from a remote server.

### Claim 17 (depends on 12)

The method of claim 12, wherein the mobile device requests and receives the one or more electrical parameter values that each correspond to one or more of the wiring terminals of the component from the component via a wired or wireless connection with the component.

### Claim 18 (depends on 12)

The method of claim 12, wherein the mobile device requests and receives the one or more electrical parameter values that each correspond to one or more of the wiring terminals of the component from another component of the system.

### Claim 19

A method of using a mobile device for diagnosing a system, the mobile device including a display and a camera, the system including a component having a plurality of wiring terminals for wiring the component to one or more other components of the system, the method comprising:
identifying the component to the mobile device;
once the component has been identified to the mobile device, the mobile device requesting and receiving one or more real-time electrical parameter values that each correspond to one or more of the wiring terminals of the component; and
displaying an augmented reality video feed captured by the camera of the mobile device on the display of the mobile device, the augmented reality video feed showing one or more wiring terminals of the component with one or more of the real-time electrical parameter values superimposed adjacent to the corresponding one or more of the wiring terminals.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
superimposing a real-time status of one or more sub-components of the component in the augmented reality video feed, wherein the real-time status is superimposed adjacent to a corresponding sub-component in the augmented reality video feed.