# US 2016/0260261 A1 · Sensor assisted head mounted displays for welding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0260261 A1
- Document ID: US20160260261A1
- Status: Published application
- Kind: A1
- Application number: 15/061,360
- Filed: 2016-03-04
- Published: 2016-09-08
- Earliest claimed: 2015-03-06
- Assignees: Illinois Tool Works Inc.
- Applicants: Illinois Tool Works Inc.
- Inventors: Christopher Hsu
- Web page: https://patentgrep.com/patent/US20160260261A1

## CPC

- A42B 3/225: {with full face protection, e.g. for industrial safety applications}
- A61F 9/06: Masks, shields or hoods for welders
- A61F 9/067: {with variable transmission}
- B23K 9/0953: {using computing means}
- B23K 9/0956: {using sensing means, e.g. optical}
- B23K 9/1087: {Arc welding using remote control}
- B23K 9/173: and of a consumable electrode
- B23K 9/32: Accessories
- G02B 27/01: Head-up displays
- G02B 27/0172: {characterised by optical features}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0174: {holographic}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G05B 2219/32014: Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
- A42B: HATS; HEAD COVERINGS (headbands A41D20/00; head-scarves A41D23/00)
- A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C)
- B23K: SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Sensor assisted head mounted displays for welding are disclosed. Disclosed example head mounted devices include an optical sensor, an augmented reality controller, a graphics processing unit, and a semi-transparent display. The optical sensor collects an image of a weld environment. The augmented reality controller determines a simulated object to be presented in a field of view, a position in the field of view, and a perspective of the simulated object in the field of view. The graphics processing unit renders the simulated object based on the perspective to represent the simulated object being present in the field of view and in the weld environment. The display presents the rendered simulated object within the field of view based on the position. At least a portion of the weld environment is observable through the display and the lens when the display is presenting the rendered simulated object.

## Claims

### Claim 1

A welding interface device, comprising:
an optical sensor to collect an image of a weld environment;
an augmented reality controller to, based on the image of the weld environment and first instructions that correspond to a weld operation in the weld environment, determine a simulated object to be presented in a field of view, a position of the simulated object in the field of view, and a perspective of the simulated object in the field of view;
a graphics processing unit to render the simulated object based on the perspective to represent the simulated object being present in the field of view and in the weld environment; and
a semi-transparent display to present the rendered simulated object within the field of view based on the position determined by the augmented reality controller, at least a portion of the weld environment being observable through the semi-transparent display when the display is presenting the rendered simulated object.

### Claim 2 (depends on 1)

The welding user interface device as defined in claim 1, further comprising a body to house the optical sensor and the display.

### Claim 3 (depends on 2)

The welding user interface device as defined in claim 2, further comprising a lens attached to the body, the lens to reduce an intensity of light occurring in the weld environment, the lens being arranged to provide the field of view to a wearer of the welding user interface device when the welding user interface device is worn by the wearer, the display being a near-eye display that is positioned between the lens and the wearer of the welding user interface device when worn by the wearer.

### Claim 4 (depends on 2)

The welding user interface device as defined in claim 2, wherein the body comprises at least one of a weld operator personal protective equipment, a hard hat, an eye protector, or a face protector.

### Claim 5 (depends on 1)

The welding user interface device as defined in claim 1, further comprising a body, the optical sensor, the graphics processing unit, and the display being attached to the body, the body dimensioned to, when worn by a wearer, enable the wearer to further wear a welding helmet.

### Claim 6 (depends on 1)

The welding user interface device as defined in claim 1, wherein the augmented reality controller is to render a view of the simulated object when viewing of a second object present in the weld environment is obscured to the field of view.

### Claim 7 (depends on 1)

The welding user interface device as defined in claim 1, wherein the augmented reality controller is to generate the simulated object to identify a next weld to be performed based on a current weld being performed and a specified weld sequence.

### Claim 8 (depends on 1)

The welding user interface device as defined in claim 1, wherein the optical sensor includes a filter configured to mitigate radiation at an arc radiation wavelength.

### Claim 9 (depends on 8)

The welding user interface device as defined in claim 8, further comprising an illuminator to output second radiation at a second wavelength outside of an arc radiation spectrum, the optical sensor comprising a time-of-flight sensor to collect third signals at the second wavelength to generate the image, the filter comprising a bandpass filter to mitigate light at wavelengths other than the second wavelength.

### Claim 10 (depends on 1)

The welding user interface device as defined in claim 1, wherein the graphics processing unit comprises a holographic processing unit.

### Claim 11 (depends on 1)

The welding user interface device as defined in claim 1, wherein the simulated object includes at least one of a weld pool, a finished weld bead, an electrode, an electrical arc, a laser path, a shielding gas path, a powder path, a weld tool, a weld bead, a weld joint, a weld fixture, a weldment or parts to be fitted or assembled onto the weldment after welding.

### Claim 12 (depends on 1)

The welding user interface device as defined in claim 1, wherein the simulated object comprises an electrical arc, the graphics processing unit to render the simulated object to be at least partially opaque and the position corresponding to a portion of the field of view in which an actual electrical arc is present in the weld environment, the display to display the rendered simulated object to reduce an intensity of light from the actual electrical arc that is observable to a wearer of the welding user interface device.

### Claim 13 (depends on 1)

The welding user interface device as defined in claim 1, wherein the graphics processing unit is to render a zoomed view of a portion of the field of view, the augmented reality controller to determine the portion of the field of view based on at least one of the image or input from a gaze tracker.

### Claim 14 (depends on 13)

The welding user interface device as defined in claim 13, wherein the augmented reality controller determines the position of the zoomed view of the portion of the field of view as a corner of the field of view.

### Claim 15 (depends on 1)

The welding user interface device as defined in claim 1, wherein the optical sensor is to collect a third image of the weld environment, the augmented reality controller is to update the position and the perspective of the simulated object based on the third image of the weld environment, the graphics processing unit to render the simulated object based on the update to the position and the perspective by the augmented reality controller.

### Claim 16 (depends on 1)

The welding user interface device as defined in claim 1, further including a communications controller to communicate a disable command in response to identifying a deviation from the first instructions, the disable command to cause at least one of a welding torch, a welding power source, or a welding wire feeder to be disabled.

### Claim 17 (depends on 1)

The welding user interface device as defined in claim 1, wherein the augmented reality controller is to determine a status of a weld being performed and compare the status to the first instructions, the augmented reality controller to determine the position and the perspective of the simulated object based on the comparison.

### Claim 18 (depends on 1)

The welding user interface device as defined in claim 1, wherein the optical sensor comprises at least one of a three-dimensional laser scanner, structured light sensor, time of flight camera, or stereo vision cameras, and the first image is a three-dimensional depth map.

### Claim 19 (depends on 1)

The welding user interface device as defined in claim 1, further comprising a communications interface to receive data representative of a weld being performed, the graphics processing unit to render a graphic representing the data and the display to present the graphic.

### Claim 20 (depends on 1)

The welding user interface device as defined in claim 1, wherein the graphics processing unit comprises a holographic processor.

### Claim at position 21

21-34. (canceled)
