# US 2019/0170510 A1 · Augmented reality measurement and schematic system including tool having relatively movable fiducial markers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0170510 A1
- Document ID: US20190170510A1
- Status: Published application
- Kind: A1
- Application number: 15/830,906
- Filed: 2017-12-04
- Published: 2019-06-06
- Earliest claimed: 2017-12-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Jonathan Steven ROBINSON
- Web page: https://patentgrep.com/patent/US20190170510A1

## CPC

- G01B 21/00: Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant
- G01C 9/00: Measuring inclination, e.g. by clinometers, by levels
- G01C 15/06: Surveyors' staffs; Movable markers
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 7/73: using feature-based methods
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G02B 2027/0141: {characterised by the informative content of the display}
- G06T 2207/30208: Marker matrix
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Aspects of the present disclosure include apparatus, method, and computer readable medium for identifying a first geometric position of a first point in space and one or more second geometric positions of one or more respective second points in space associated with placement of the first tool member and a second tool member of a measurement tool, generating a final dimension of a part of the real-world structure based on at least the one or more second geometric positions according to relative geometric relationship with respect to the first point, and displaying a virtual schematic diagram of the real-world structure having the final dimension.

## Claims

### Claim 1

A measurement and schematic-generating system, comprising:
a measuring tool including:
a first tool member having a first plurality of fiducial markers that indicate a first point in space relative to the first tool member;
a second tool member having a second plurality of fiducial markers, wherein each of the second plurality of fiducial markers indicates a respective second point in space relative to the second tool member;
wherein the first tool member and the second tool member have a plurality of relative positions each having a corresponding relative geometric relationship between the first point and the second point; and
a head-mounted display device including:
one or more sensors configured to identify a first geometric position of the first point and one or more second geometric positions of one or more of the respective second points associated with placement of the first tool member and the second tool member relative to a surface of a real-world structure, wherein the first geometric position and the one or more second geometric positions are in a coordinate system relative to the head-mounted display device;
one or more processors configured to generate a final dimension of a part of the real-world structure based on at least the one or more second geometric positions according to relative geometric relationship with respect to the first point; and
a near-eye display configured to display a virtual schematic diagram of the real-world structure having the final dimension.

### Claim 2 (depends on 1)

The measurement and schematic-generating system of claim 1, wherein each of the second plurality of fiducial markers indicates a measurement from an end of the second tool member to each of the second plurality of fiducial markers.

### Claim 3 (depends on 1)

The measurement and schematic-generating system of claim 1, further comprising a connector configured to restrain a relative movement between the first tool member and the second tool member to a known relative movement that defines the relative geometric relationship.

### Claim 4 (depends on 3)

The measurement and schematic-generating system of claim 3, wherein the connector comprises at least one internal wall within a body of a first one of the first tool member and the second tool member, wherein the at least one internal wall defines a slot sized to fit at least a part of a body of a remaining one of a second one of the first tool member and the second tool member.

### Claim 5 (depends on 1)

The measurement and schematic-generating system of claim 1, further comprising a geometric reference on at least one of the first tool member and the second tool member, wherein the head-mounted display device is configured to identify a current one of the plurality of relative positions of the first tool member and the second tool member based on identifying a position of the geometric reference.

### Claim 6 (depends on 5)

The measurement and schematic-generating system of claim 5, wherein the geometric reference comprises one or more internal walls defining a window within a body of the first tool member, wherein the window is sized for displaying one or more of the second plurality of fiducial markers of the second tool member when the second tool member is placed underneath or within a body of the first tool member.

### Claim 7 (depends on 1)

The measurement and schematic-generating system of claim 1, wherein the second tool member is substantially straight or substantially arched.

### Claim 8 (depends on 1)

The measurement and schematic-generating system of claim 1, further comprising an inclinometer configured to measure an angle of the measuring tool or a surface being measured by the measuring tool.

### Claim 9 (depends on 1)

The measurement and schematic-generating system of claim 1, further comprising:
a memory configured to store a profile of a physical geometry of the measuring tool;
wherein the one or more processors are configured to use the profile and an identified one or more of each of the first plurality of fiducial markers and the second plurality of fiducial markers to calculate a current one of the plurality of relative positions between the first tool member and the second tool member.

### Claim 10 (depends on 1)

The measurement and schematic-generating system of claim 1, wherein each of the first plurality of fiducial markers and the second plurality of fiducial markers comprise unique markers.

### Claim 11

A computer-readable medium having instructions stored in a memory that, when executed by one or more processors associated with a head-mounted display device, cause the one or more processors to:
cause one or more sensors to identify:
a first geometric position of a first point in space indicated by a first plurality of fiducial markers of a first tool member of a measurement tool;
one or more second geometric positions of one or more respective second points in space associated with placement of the first tool member and a second tool member of the measurement tool relative to a surface of a real-world structure, wherein the second tool member includes a second plurality of fiducial markers each indicating a respective second point in space relative to the second tool member; and
wherein the first geometric position and the one or more second geometric positions are in a coordinate system relative to the head-mounted display device;
generate a final dimension of a part of the real-world structure based on at least the one or more second geometric positions according to relative geometric relationship with respect to the first point; and
cause a near-eye display configured to display a virtual schematic diagram of the real-world structure having the final dimension.

### Claim 12 (depends on 11)

The computer-readable medium of claim 11, wherein each of the second plurality of fiducial markers indicates a measurement from an end of the second tool member to each of the second plurality of fiducial markers.

### Claim 13 (depends on 11)

The computer-readable medium of claim 11, wherein a connector is configured to restrain a relative movement between the first tool member and the second tool member to a known relative movement that defines the relative geometric relationship.

### Claim 14 (depends on 11)

The computer-readable medium of claim 11, further comprising instructions stored in the memory that, when executed by one or more processors, cause the one or more processors to identify a current one of the plurality of relative positions of the first tool member and the second tool member based on identifying a position of a geometric reference on at least one of the first tool member and the second tool member.

### Claim 15 (depends on 11)

The computer-readable medium of claim 11, further comprising instructions stored in the memory that, when executed by one or more processors, cause the one or more processors to:
calculate a current one of the plurality of relative positions between the first tool member and the second tool member based on a profile of a physical geometry and design of the measuring tool in a memory and based on the head-mounted display device identifying one or more of each of the first plurality of fiducial markers and the second plurality of fiducial markers.

### Claim 16

A method of measuring a dimension of a real-world structure and generating a schematic using a head-mounted display device, comprising:
identifying a first geometric position of a first point in space indicated by a first plurality of fiducial markers of a first tool member of a measurement tool;
identifying one or more second geometric positions of one or more respective second points in space associated with placement of the first tool member and a second tool member of the measurement tool relative to a surface of a real-world structure, wherein the second tool member includes a second plurality of fiducial markers each indicating a respective second point in space relative to the second tool member and wherein the first geometric position and the one or more second geometric positions are in a coordinate system relative to the head-mounted display device;
generating a final dimension of a part of the real-world structure based on at least the one or more second geometric positions according to relative geometric relationship with respect to the first point; and
displaying a virtual schematic diagram of the real-world structure having the final dimension.

### Claim 17 (depends on 16)

The method of claim 16, wherein each of the second plurality of fiducial markers indicates a measurement from an end of the second tool member to each of the second plurality of fiducial markers.

### Claim 18 (depends on 16)

The method of claim 16, wherein a connector is configured to restrain a relative movement between the first tool member and the second tool member to a known relative movement that defines the relative geometric relationship.

### Claim 19 (depends on 16)

The method of claim 16, further comprising, prior to generating the final dimension, identifying a current one of the plurality of relative positions of the first tool member and the second tool member based on identifying a position of a geometric reference on at least one of the first tool member and the second tool member.

### Claim 20 (depends on 16)

The method of claim 16, further comprising calculating a current one of the plurality of relative positions between the first tool member and the second tool member based on a profile of a physical geometry of the measuring tool stored in a memory and based on the head-mounted display device identifying one or more of each of the first plurality of fiducial markers and the second plurality of fiducial markers.
