# US 2020/0272227 A1 · Mixed reality intelligent tether for dynamic attention direction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0272227 A1
- Document ID: US20200272227A1
- Status: Published application
- Kind: A1
- Application number: 16/423,881
- Filed: 2019-05-28
- Published: 2020-08-27
- Earliest claimed: 2019-02-22
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Kevin Thomas MATHER; Sean Robert PULLER; Tsz Fung WAN
- Web page: https://patentgrep.com/patent/US20200272227A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G02B 27/017: {Head mounted}
- G02B 27/0172: {characterised by optical features}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip
- G09B 19/003: {Repetitive work cycles; Sequence of movements}
- G09G 5/37: Details of the operation on graphic patterns (G09G5/38 takes precedence)
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0174: {holographic}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06T 2200/04: involving 3D image data
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06V 20/20: in augmented reality scenes
- G09G 2354/00: Aspects of interface with display user
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The disclosed technology is generally directed to mixed reality devices. In one example of the technology, a mixed-reality view is caused to be provided to an operator. The mixed-reality view includes both a real-world environment of the operator and holographic aspects. The mixed-reality view is caused to include a visual tether to a tether location. The tether location is a real-world location at which work is to be performed. A gaze determination associated with a gaze of the operator is made. Responsive to a positive gaze determination, the visual tether is caused to pulse.

## Claims

### Claim 1

An apparatus, comprising:
a device including at least one memory adapted to store run-time data for the device, and at least one processor that is adapted to execute processor-executable code that, in response to execution, enables the device to perform actions, including:
causing a mixed-reality view to be provided to an operator, wherein the mixed-reality view includes both a real-world environment of the operator and holographic aspects;
causing the mixed-reality view to include a visual tether to a tether location, wherein the tether location is a real-world location at which work is to be performed;
making a gaze determination associated with a gaze of the operator; and
responsive to a positive gaze determination, causing the visual tether to pulse.

### Claim 2 (depends on 1)

The apparatus of claim 1, the actions further including, enabling the operator to change the tether location in the mixed-reality view.

### Claim 3 (depends on 1)

The apparatus of claim 1, the actions further including, responsive to the positive gaze determination, causing a sound that originates from the tether location.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the device is a wearable mixed-reality device that includes a head-mounted display.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the real-world location is a fixed real-world location.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the real-world location is a location of a hologram situated at the real-world location, such that the tether location is tied to the location of the hologram in the mixed-reality view.

### Claim 7 (depends on 1)

The apparatus of claim 1, the actions further including causing the mixed-reality view to include an instruction, and wherein causing the mixed-reality view to include the visual tether to the tether location includes causing the mixed-reality view to include the visual tether to connect the instruction to the tether location.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the gaze determination is a determination as to whether the operator's gaze has remained at the step card without having viewed the tether location for a pre-determined period of time.

### Claim 9 (depends on 7)

The apparatus of claim 7, the actions further including routing the visual tether, such that routing the visual tether includes ensuring that the tether avoids the operator's body.

### Claim 10 (depends on 7)

The apparatus of claim 7, the actions further including routing the visual tether, such that routing the visual tether includes ensuring that the tether avoids going through walls.

### Claim 11 (depends on 7)

The apparatus of claim 7, the actions further including routing the visual tether, such that routing the visual tether includes ensuring that the visual tether avoids going through solid objects in the real-world environment.

### Claim 12 (depends on 7)

The apparatus of claim 7, the actions further including routing the visual tether, such that routing the visual tether includes keeping the visual tether in a vertical sight line of the operator.

### Claim 13

A method, comprising:
causing a mixed-reality view to be provided to an operator, wherein the mixed-reality view includes both a real-world environment of the operator and virtual aspects;
causing the mixed-reality view to include a visual tether to a tether location, wherein the tether location is a real-world location at least one action is to be performed;
making a gaze determination associated with a gaze of the operator; and
responsive to a positive gaze determination, dynamically drawing the operator's attention to the tether location.

### Claim 14 (depends on 13)

The method of claim 13, wherein dynamically drawing the operator's attention to the tether location includes causing the visual tether to pulse.

### Claim 15 (depends on 13)

The method of claim 13, wherein dynamically drawing the operator's attention to the tether location includes causing a sound that originates from the tether location.

### Claim 16 (depends on 13)

The method of claim 13, further comprising causing the mixed-reality view to include an instruction, and wherein causing the mixed-reality view to include the visual tether to the tether location includes causing the mixed-reality view to include the visual tether to connect the instruction to the tether location.

### Claim 17 (depends on 16)

The method of claim 16, further comprising routing the visual tether, such that routing the visual tether includes ensuring that the visual tether avoids going through solid objects in the real-world environment.

### Claim 18

A processor-readable storage medium, having stored thereon process-executable code for computer network design, that, upon execution by at least one processor, enables actions, comprising:
providing a mixed-reality view that includes a visual tether to a tether location, wherein the tether location is a real-world location at which work is to be performed; and
responsive to a determination that an operator is unaware of the tether location, causing the visual tether to pulse.

### Claim 19 (depends on 18)

The processor-readable storage medium of claim 18, the actions further including:
causing the mixed-reality view to include an instruction, wherein providing the mixed-reality view that includes the visual tether includes providing the mixed-reality view such that the visual tether connects the instruction to the tether location; and
routing the visual tether, such that routing the visual tether includes ensuring that the visual tether avoids going through solid objects in a real-world environment of the mixed-reality view.

### Claim 20 (depends on 19)

The processor-readable storage medium of claim 19, wherein routing the visual tether further includes keeping the visual tether in a vertical sight line of the operator.
