# US 2023/0410385 A1 · Spatially-Aware Tool System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410385 A1
- Document ID: US20230410385A1
- Status: Published application
- Kind: A1
- Application number: 18/458,776
- Filed: 2023-08-30
- Published: 2023-12-21
- Earliest claimed: 2018-10-31
- Assignees: Milwaukee Electric Tool Corporation
- Applicants: Milwaukee Electric Tool Corporation
- Inventors: Samuel A. Gould; Kellen Carey; Michael John Caelwaerts; Gareth J. Mueckl; Christopher S. Hoppe; Benjamin T. Jones
- Web page: https://patentgrep.com/patent/US20230410385A1

## CPC

- G06T 7/73: using feature-based methods
- G06T 11/00: Two-dimensional [2D] image generation
- G06F 30/13: Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 9/3179: {Video signal processing therefor}
- H04N 9/3194: {including sensor feedback}
- G06T 2207/30204: Marker
- G06T 2207/30244: Camera pose
- G06T 2210/04: Architectural design, interior design
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The systems described in this disclosure can be used in construction settings to facilitate the tasks being performed. The location of projectors and augmented reality headsets can be calculated and used to determine what images to display to a worker, based on a map of work to be performed, such as a construction plan. Workers can use spatially-aware tools to make different locations be plumb, level, or equidistant with other locations. Power to tools can be disabled if they are near protected objects.

## Claims

### Claim 1

A method of tracking a construction tool within a frame of reference comprising:
receiving a first indication to calculate a first location of a construction tool;
calculating the first location of the construction tool as a result of receiving the first indication;
receiving a second indication to calculate a second location of the construction tool;
calculating the second location of the construction tool as a result of receiving the second indication;
generating a positional frame of reference based on the first location and the second location of the construction tool;
tracking a third location and an orientation of the construction tool within the positional frame of reference;
comparing the third location and the orientation of the construction tool relative to the positional frame of reference;
calculating relative characteristics of the construction tool based on the comparing;
generating augmentation data based on the relative characteristics; and
providing the augmentation data to a user.

### Claim 2 (depends on 1)

The method of claim 1, the method further comprising:
calibrating the construction tool before calculating the first location and the second location.

### Claim 3 (depends on 1)

The method of claim 1, wherein providing the augmentation data comprises a headset displaying an image comprising an augmented image based on the augmented data.

### Claim 4 (depends on 1)

The method of claim 1, wherein providing the augmentation data comprises projecting an augmented image based on the augmented data.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
receiving a plurality of indications to mark a plurality of locations of the construction tool;
generating supplemental augmentation data based on the plurality of locations; and
providing the supplemental augmentation data to the user.

### Claim 6 (depends on 5)

The method of claim 5, the method further comprising:
storing drill point information about a first location of the plurality of locations.

### Claim 7 (depends on 6)

The method of claim 6, wherein the drill point information is selected from the group consisting of an angle, a diameter, and a depth of a hole to be drilled at the first location.

### Claim 8 (depends on 5)

The method of claim 5, the method further comprising:
displaying a distance between a first location of the plurality of locations and a location of the construction tool.

### Claim 9

A method of displaying information from a construction plan comprising:
electronically tracking a location and an orientation of a headset being worn by a user;
calculating a spatial relationship between a construction plan and the location and the orientation of the headset;
generating augmentation data based on the spatial relationship between the construction plan and the location and the orientation of the headset; and
displaying an augmented image in the headset, the augmented image comprising a first image that is representative of the augmentation data and a second image that is an actual view of the environment of the headset.

### Claim 10 (depends on 9)

The method of claim 9, wherein the first image comprises a representative image of a construction object within the construction plan, and wherein the construction object is selected from a group consisting of a door and a window.

### Claim 11 (depends on 10)

The method of claim 10, wherein the representative image is selected from a group consisting of a generic image of the construction object and a schematic representation of the construction object.

### Claim 12 (depends on 11)

The method of claim 11, the method further comprising:
receiving a command to toggle between displaying the generic image of the construction object and displaying the schematic representation of the construction object.

### Claim 13 (depends on 9)

The method of claim 9, wherein the first image comprises a representative image of a construction object within the construction plan, and wherein the construction object is selected from a group consisting of an appliance and an electrical outlet.

### Claim 14 (depends on 9)

The method of claim 9, wherein the first image comprises a representative image of a construction object within the construction plan, and wherein the construction object is selected from a group consisting of a wall, a lighting, a conduit, a datacom component, and a hole in a floor where the hole is configured to receive a pipe.

### Claim 15 (depends on 14)

The method of claim 14, wherein the representative image is selected from a group consisting of a generic image of the construction object and a schematic representation of the construction object.

### Claim 16 (depends on 15)

The method of claim 15, the method further comprising:
receiving a command to toggle between displaying the generic image of the construction object and displaying the schematic representation of the construction object.

### Claim 17

A method of displaying information from a construction plan comprising:
electronically tracking a location of an object being carried by a first user;
calculating a spatial relationship between a construction plan and the location of the object;
generating augmentation data based on the spatial relationship between the construction plan and the location of the object; and
displaying an augmented image to a second user located remotely relative to the first user, the augmented image comprising a first image representative of objects near the first user based on the augmentation data, and a second image representative of the first user.

### Claim 18 (depends on 17)

The method of claim 17, the method further comprising:
transmitting a signal to disable a function of the object based on the spatial relationship between the construction plan and the location of the object.

### Claim 19 (depends on 17)

The method of claim 17, wherein the augmented image comprises a representative image of a construction object within the construction plan, and wherein the construction object is selected from a group consisting of a door, a window, an appliance, and an electrical outlet.

### Claim 20 (depends on 17)

The method of claim 17, wherein the augmented image comprises a representative image of a construction object within the construction plan, and wherein the construction object is selected from a group consisting of a wall, a lighting, a conduit, a datacom component, and a hole in a floor where the hole is configured to receive a pipe.
