# US 2015/0146008 A1 · Maintenance assistant system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0146008 A1
- Document ID: US20150146008A1
- Status: Published application
- Kind: A1
- Application number: 14/090,106
- Filed: 2013-11-26
- Published: 2015-05-28
- Earliest claimed: 2013-11-26
- Assignees: Honeywell International Inc.
- Applicants: Honeywell International Inc.
- Inventors: Kevin J. Conner; Santosh Mathan
- Web page: https://patentgrep.com/patent/US20150146008A1

## CPC

- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06Q 10/20: Administration of product repair or maintenance
- G06V 10/245: {by locating a pattern; Special marks for positioning}
- G06V 20/20: in augmented reality scenes
- G09B 9/006: {for locating or ranging of objects}
- G05B 2219/32014: Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A maintenance assistance system and method of operating are provided. The maintenance assistance system may include, but is not limited to, a camera, a speaker, a technician marker, a memory configured to store a location of a plurality of tags and a location of at least one component relative to at least one of the plurality of tags, and a processor communicatively coupled to the camera, the speaker and the memory, the processor configured to receive an indication of one of the components, determine a location of the technician marker relative to the location of the indicated component based upon data from the camera and the location of the indicated component relative to the at least one of the plurality of tags, and output position information to the speaker based upon the determined location of the technician marker relative to the location of the indicated component.

## Claims

### Claim 1

A maintenance assistance system, comprising:
a camera;
a speaker;
a technician marker;
a memory configured to store a location of each of a plurality of tags and a location of at least one component relative to at least one of the plurality of tags; and
a processor communicatively coupled to the camera, the speaker and the memory, the processor configured to:
receive an indication of one of the at least one components;
determine a location of the technician marker relative to the location of the indicated component based upon data from the camera and the location of the indicated component relative to the locations of at least one of the plurality of tags stored in the memory; and
output position information to the speaker based upon the determined location of the technician marker relative to the location of the indicated component.

### Claim 2 (depends on 1)

The maintenance assistance system of claim 1, wherein the position information includes instructions for locating the indicated component relative to the determined location of the technician marker.

### Claim 3 (depends on 1)

The maintenance assistance system of claim 1, wherein the position information includes an affirmation that the technician marker is within a predetermined distance of the indicated component.

### Claim 4 (depends on 1)

The maintenance assistance system of claim 1, wherein the technician marker is on a glove.

### Claim 5 (depends on 1)

The maintenance assistance system of claim 1, wherein the processor is further configured to determine a location of the technician marker relative to the location of the indicated component by:
determining the location of the indicated component relative to at least one of the plurality of tags via triangulation; and
comparing the determined location of the indicated component to a position of the technician marker.

### Claim 6 (depends on 1)

The maintenance assistance system of claim 1, wherein the processor is further configured to output the location of the indicated component to the speaker based upon the location of the indicated component stored in the memory.

### Claim 7 (depends on 1)

The maintenance assistance system of claim 1, wherein the processor is further configured to:
receive a request for a maintenance procedure from an input system; and
output instructions for the maintenance procedure to the speaker.

### Claim 8 (depends on 7)

The maintenance assistance system of claim 7, wherein the processor is further configured to store data from at least one of the camera and the input system corresponding to an execution of the maintenance procedure in the memory.

### Claim 9

A method for operating a maintenance assistance system comprising a camera, a technician marker, a speaker, a memory and a processor communicatively coupled to the camera, the speaker and the memory, the method comprising:
receiving, by the processor, an indication of a component;
determining, by the processor, a location of the technician marker relative to the location of the indicated component based upon data from the camera and a location of the indicated component relative to at least one tag stored in the memory; and
outputting position information to the speaker based upon the determined location of the technician marker relative to the location of the indicated component.

### Claim 10 (depends on 9)

The method of claim 9, wherein the outputted position information includes instructions for locating the indicated component relative to the determined location of the technician marker.

### Claim 11 (depends on 9)

The method of claim 9, wherein the determining further comprises:
determining, by the processor, the location of the indicated component relative to the at least one tag via triangulation; and
comparing, by the processor, the determined location of the indicated component to a position of the technician marker.

### Claim 12 (depends on 9)

The method of claim 9, further comprising outputting, by the processor, the location of the indicated component to the speaker based upon a location of the indicated component stored in the memory.

### Claim 13 (depends on 9)

The method of claim 9, further comprising:
receiving, by the processor, a request for maintenance procedure instructions; and
outputting, by the processor, the maintenance procedure instructions to the speaker.

### Claim 14 (depends on 9)

The method of claim 9, further comprising:
recording, by the camera, a technician executing a maintenance procedure; and
storing the recording in the memory.

### Claim 15

A computer-readable medium storing instructions, which when executed by a processor, operate a maintenance assistance system comprising the processor, a camera, a technician marker, a speaker, and a memory, the instructions causing the processor to:
receive an indication of a component;
determine a location of the technician marker relative to the location of the indicated component based upon data from the camera and a location of the indicated component relative to at least one tag stored in the memory; and
output position information to the speaker based upon the determined location of the technician marker relative to the location of the indicated component.

### Claim 16 (depends on 15)

The computer-readable medium of claim 15, wherein the outputted position information includes instructions for locating the indicated component relative to the determined location of the technician marker.

### Claim 17 (depends on 15)

The computer-readable medium of claim 15 wherein the instructions further cause the processor to:
determine the location of the indicated component relative to the at least one tag via triangulation; and
compare the determined location of the indicated component to a position of the technician marker.

### Claim 18 (depends on 15)

The computer-readable medium of claim 15 wherein the instructions further cause the processor to output the location of the indicated component to the speaker based upon a location of the indicated component stored in the memory.

### Claim 19 (depends on 15)

The computer-readable medium of claim 15 wherein the instructions further cause the processor to:
receive a request for maintenance procedure instructions; and
output the maintenance procedure instructions to the speaker.

### Claim 20 (depends on 19)

The computer-readable medium of claim 19 wherein the instructions further cause the processor to
record a technician executing a maintenance procedure via the camera; and
store the recording in the memory.
