# US 2021/0279914 A1 · Systems and methods for augmented reality application

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279914 A1
- Document ID: US20210279914A1
- Status: Published application
- Kind: A1
- Application number: 17/193,373
- Filed: 2021-03-05
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: Oshkosh Corporation
- Applicants: Oshkosh Corporation
- Inventors: Douglas A. Shoup; Korry D. Kobel
- Web page: https://patentgrep.com/patent/US20210279914A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 11/00: Two-dimensional [2D] image generation
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06Q 10/20: Administration of product repair or maintenance
- G06Q 50/08: Construction
- G06V 20/20: in augmented reality scenes
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for optimizing equipment inspection for a machine using augmented reality (AR) is shown. The method includes providing an AR application to a user interface. The method further includes displaying one or more selectable widgets on the user interface via AR functionality. The method further includes receiving, via the user interface, a selection of the selectable widget, the selection providing instructions to provide additional material based on the selectable widget. The method further includes displaying a detailed model of the machine based on the additional material received.

## Claims

### Claim 1

A method for optimizing equipment inspection for a machine using augmented reality (AR), the method comprising:
providing an AR application to a user interface;
displaying one or more selectable widgets on the user interface via AR functionality;
receiving, via the user interface, a selection of the selectable widget, the selection providing instructions to provide additional material based on the selectable widget; and
displaying a detailed model of the machine based on the additional material received.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
detecting, via the user interface, an equipment decal on the machine;
matching the equipment decal with a first set of stored equipment information;
overlaying a first selectable widget with the equipment decal on the user interface of the AR application; and
providing the first set of stored equipment information to the user interface.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
detecting, via the user interface, an equipment control panel on the machine;
matching the equipment control panel with a second set of stored equipment information;
overlaying a second selectable widget with the equipment control panel on the user interface of the AR application; and
providing the second set of stored equipment information to the user interface.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
displaying an annual inspection process on the user interface;
receiving, via the user interface, at least one of a machine, owner, or technician information;
directing a user to inspect the machine based on the at least one of a machine, owner, or technician information; and
receiving an update based on the inspection, wherein the update comprises an indication of passing inspection, failing inspection, or needing correction.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
selecting a machine profile from a database of machine profiles;
overlaying a virtual machine in the real world, wherein the virtual machine is based on the machine profile; and
adjusting the virtual machine based on instructions received via the user interface.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
selecting an attachment profile for a machine from a database of attachment profiles;
overlaying a virtual attachment on an equipment device in the real world, wherein the virtual attachment is based on the attachment profile; and
adjusting the virtual attachment based on instructions received via the user interface.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
accessing a machine software information database;
collecting a set of machine software information for a chosen machine;
transmitting a support request based on the collected set of machine software information; and
transmitting a video feed to a support workstation to receive live AR support.

### Claim 8

A system for optimizing equipment inspection for a machine using augmented reality (AR), the system comprising:
a user device configured to display a user interface and the machine; and
a server comprising a processing circuit comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
providing an AR application to a user interface;
displaying one or more selectable widgets on the user interface via AR functionality;
receiving, via the user interface, a selection of the selectable widget, the selection providing instructions to provide additional material based on the selectable widget; and
displaying a detailed model of the machine based on the additional material received.

### Claim 9 (depends on 8)

The method of claim 8, wherein the processing circuit is further configured to:
detect, via the user interface, an equipment decal on the machine;
match the equipment decal with a first set of stored equipment information;
overlay a first selectable widget with the equipment decal on the user interface of the AR application; and
provide the first set of stored equipment information to the user interface.

### Claim 10 (depends on 8)

The method of claim 8, wherein the processing circuit is further configured to:
detect, via the user interface, an equipment control panel on the machine;
match the equipment control panel with a second set of stored equipment information;
overlay a second selectable widget with the equipment control panel on the user interface of the AR application; and
provide the second set of stored equipment information to the user interface.

### Claim 11 (depends on 8)

The method of claim 8, wherein the processing circuit is further configured to:
display an annual inspection process on the user interface;
receive, via the user interface, at least one of a machine, owner, or technician information;
direct a user to inspect the machine based on the at least one of a machine, owner, or technician information; and
receive an update based on the inspection, wherein the update comprises an indication of passing inspection, failing inspection, or needing correction.

### Claim 12 (depends on 8)

The method of claim 8, wherein the processing circuit is further configured to:
select a machine profile from a database of machine profiles;
overlay a virtual machine in the real world, wherein the virtual machine is based on the machine profile; and
adjust the virtual machine based on instructions received via the user interface.

### Claim 13 (depends on 8)

The method of claim 8, wherein the processing circuit is further configured to:
select an attachment profile for a machine from a database of attachment profiles;
overlay a virtual attachment on an equipment device in the real world, wherein the virtual attachment is based on the attachment profile; and
adjust the virtual attachment based on instructions received via the user interface.

### Claim 14 (depends on 8)

The method of claim 8, wherein the processing circuit is further configured to:
access a machine software information database;
collect a set of machine software information for a chosen machine;
transmit a support request based on the collected set of machine software information; and
transmit a video feed to a support workstation to receive live AR support.

### Claim 15

A building equipment inspection tool for inspecting equipment via augmented reality (AR), the tool comprising a non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to implement operations comprising:
providing an AR application to a user interface;
displaying one or more selectable widgets on the user interface via AR functionality;
receiving, via the user interface, a selection of the selectable widget, the selection providing instructions to provide additional material based on the selectable widget; and
displaying a detailed model of the machine based on the additional material received.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein operations further cause the one or more processors to:
detect, via the user interface, an equipment decal on the machine;
match the equipment decal with a first set of stored equipment information;
overlay a first selectable widget with the equipment decal on the user interface of the AR application; and
provide the first set of stored equipment information to the user interface.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein operations further cause the one or more processors to:
detect, via the user interface, an equipment control panel on the machine;
match the equipment control panel with a second set of stored equipment information;
overlay a second selectable widget with the equipment control panel on the user interface of the AR application; and
provide the second set of stored equipment information to the user interface.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein operations further cause the one or more processors to:
display an annual inspection process on the user interface;
receive, via the user interface, at least one of a machine, owner, or technician information;
direct a user to inspect the machine based on the at least one of a machine, owner, or technician information; and
receive an update based on the inspection, wherein the update comprises an indication of passing inspection, failing inspection, or needing correction.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein operations further cause the one or more processors to:
select a machine profile from a database of machine profiles;
overlay a virtual machine in the real world, wherein the virtual machine is based on the machine profile; and
adjust the virtual machine based on instructions received via the user interface.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein operations further cause the one or more processors to:
access a machine software information database;
collect a set of machine software information for a chosen machine;
transmit a support request based on the collected set of machine software information; and
transmit a video feed to a support workstation to receive live AR support.
