# US 2016/0259404 A1 · Systems and methods for augmented reality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0259404 A1
- Document ID: US20160259404A1
- Status: Published application
- Kind: A1
- Application number: 15/062,104
- Filed: 2016-03-05
- Published: 2016-09-08
- Earliest claimed: 2015-03-05
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Michael J. Woods
- Web page: https://patentgrep.com/patent/US20160259404A1

## CPC

- G06F 3/012: {Head tracking input arrangements}
- G01S 19/42: Determining position
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G02B 27/017: {Head mounted}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. An augmented reality display system comprises a handheld component housing an electromagnetic field emitter, the electromagnetic field emitter emitting a known magnetic field, the head mounted component coupled to one or more electromagnetic sensors that detect the magnetic field emitted by the electromagnetic field emitter housed in the handheld component, wherein a head pose is known, and a controller communicatively coupled to the handheld component and the head mounted component, the controller receiving magnetic field data from the handheld component, and receiving sensor data from the head mounted component, wherein the controller determining a hand pose based at least in part on the received magnetic field data and the received sensor data.

## Claims

### Claim 1

An augmented reality (AR) display system, comprising:
an electromagnetic field emitter to emit a known magnetic field;
an electromagnetic sensor to measure a parameter related to a magnetic flux at the electromagnetic sensor as a result of the emitted known magnetic field, wherein world coordinates of the electromagnetic sensor are known;
a controller to determine pose information relative to the electromagnetic field emitter based at least in part on the measured parameter related to the magnetic flux at the electromagnetic sensor; and
a display system to display virtual content to a user based at least in part on the determined pose information relative to the electromagnetic field emitter.

### Claim 2 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic field emitter resides in a mobile component of the AR display system.

### Claim 3 (depends on 2)

The AR display system of claim 2, wherein the mobile component is a hand-held component.

### Claim 4 (depends on 2)

The AR display system of claim 2, wherein the mobile component is a totem.

### Claim 5 (depends on 2)

The AR display system of claim 2, wherein the mobile component is a head-mounted component of the AR display system.

### Claim 6 (depends on 1)

The AR display system of claim 1, further comprising a head-mounted component that houses the display system, wherein the electromagnetic sensor is operatively coupled to the head-mounted component.

### Claim 7 (depends on 1)

The AR display system of claim 1, wherein the world coordinates of the electromagnetic sensor is known based at least in part on SLAM analysis performed to determine head pose information, wherein the electromagnetic sensor is operatively coupled to a head-mounted component that houses the display system.

### Claim 8 (depends on 7)

The AR display system of claim 7, further comprising one or more cameras operatively coupled to the head-mounted component, and wherein the SLAM analysis is performed based at least on data captured by the one or more cameras.

### Claim 9 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic sensors comprise one or more inertial measurement units (IMUs).

### Claim 10 (depends on 1)

The AR display system of claim 1, wherein the pose information corresponds to at least a position and orientation of the electromagnetic field emitter relative to the world.

### Claim 11 (depends on 1)

The AR display system of claim 1, wherein the pose information is analyzed to determine world coordinates corresponding to the electromagnetic field emitter.

### Claim 12 (depends on 1)

The AR display system of claim 1, wherein the controller detects an interaction with one or more virtual contents based at least in part on the pose information corresponding to the electromagnetic field emitter.

### Claim 13 (depends on 12)

The AR display system of claim 12, wherein the display system displays virtual content to the user based at least in part on the detected interaction.

### Claim 14 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic sensor comprises at least three coils to measure magnetic flux in three directions.

### Claim 15 (depends on 14)

The AR display system of claim 14, wherein the at least three coils are housed together at substantially the same location, the electromagnetic sensor being coupled to a head-mounted component of the AR display system.

### Claim 16 (depends on 14)

The AR display system of claim 14, wherein the at least three coils are housed at different locations of the head-mounted component of the AR display system.

### Claim 17 (depends on 1)

The AR display system of claim 1, further comprising a control and quick release module to decouple the magnetic field emitted by the electromagnetic field emitter.

### Claim 18 (depends on 1)

The AR display system of claim 1, further comprising additional localization resources to determine the world coordinates of the electromagnetic field emitter.

### Claim 19 (depends on 18)

The AR display system of claim 18, wherein the additional localization resources comprises a GPS receiver.

### Claim 20 (depends on 18)

The AR display system of claim 18, wherein the additional localization resources comprises a beacon.

### Claim 21 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic sensor comprises a non-solid ferrite cube.

### Claim 22 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic sensor comprises a stack of ferrite disks.

### Claim 23 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic sensor comprises a plurality of ferrite rods each having a polymer coating.

### Claim 24 (depends on 1)

The AR display system of claim 1, wherein the electromagnetic sensor comprises a time division multiplexing switch.

### Claim 25

A method to display augmented reality, comprising:
emitting, through an electromagnetic field emitter, a known magnetic field;
measuring, through an electromagnetic sensor, a parameter related to a magnetic flux at the electromagnetic sensor as a result of the emitted known magnetic field, wherein world coordinates of the electromagnetic sensor are known;
determining pose information relative to the electromagnetic field emitter based at least in part on the measured parameter related to the magnetic flux at the electromagnetic sensor; and
displaying virtual content to a user based at least in part on the determined pose information relative to the electromagnetic field emitter.

### Claim 26 (depends on 25)

The method of claim 25, wherein the electromagnetic field emitter resides in a mobile component of the AR display system.

### Claim 27 (depends on 26)

The method of claim 26, wherein the mobile component is a hand-held component.

### Claim 28 (depends on 26)

The method of claim 26, wherein the mobile component is a totem.

### Claim 29 (depends on 26)

The method of claim 26, wherein the mobile component is a head-mounted component of the AR display system.

### Claim 30 (depends on 25)

The method of claim 25, further comprising housing the display system in a head-mounted component, wherein the electromagnetic sensor is operatively coupled to the head-mounted component.

### Claim 31 (depends on 25)

The method of claim 25, wherein the world coordinates of the electromagnetic sensor is known based at least in part on SLAM analysis performed to determine head pose information, wherein the electromagnetic sensor is operatively coupled to a head-mounted component that houses the display system.

### Claim 32 (depends on 31)

The method of claim 31, capturing image data through one or more cameras that are operatively coupled to the head-mounted component, and wherein the SLAM analysis is performed based at least on data captured by the one or more cameras.

### Claim 33 (depends on 25)

The method of claim 25, wherein the electromagnetic sensors comprise one or more inertial measurement units (IMUs).

### Claim 34 (depends on 25)

The method of claim 25, wherein the pose information corresponds to at least a position and orientation of the electromagnetic field emitter relative to the world.

### Claim 35 (depends on 25)

The method of claim 25, wherein the pose information is analyzed to determine world coordinates corresponding to the electromagnetic field emitter.

### Claim 36 (depends on 25)

The method of claim 25, further comprising detecting an interaction with one or more virtual contents based at least in part on the pose information corresponding to the electromagnetic field emitter.

### Claim 37 (depends on 36)

The method of claim 36, further comprising displaying virtual content to the user based at least in part on the detected interaction.

### Claim 38 (depends on 25)

The method of claim 25, wherein the electromagnetic sensor comprises at least three coils to measure magnetic flux in three directions.

### Claim 39 (depends on 38)

The method of claim 38, wherein the at least three coils are housed together at substantially the same location, the electromagnetic sensor being coupled to a head-mounted component of the AR display system.

### Claim 40 (depends on 38)

The method of claim 38, wherein the at least three coils are housed at different locations of the head-mounted component of the AR display system.

### Claim 41 (depends on 25)

The method of claim 25, further comprising decoupling the magnetic field emitted by the electromagnetic field emitter through a control and quick release module.

### Claim 42 (depends on 25)

The method of claim 25, further comprising determining the world coordinates of the electromagnetic field emitter through additional localization resources.

### Claim 43 (depends on 42)

The method of claim 42, wherein the additional localization resources comprises a GPS receiver.

### Claim 44 (depends on 42)

The method of claim 42, wherein the additional localization resources comprises a beacon.

### Claim 45 (depends on 25)

The method of claim 25, wherein the electromagnetic sensor comprises a non-solid ferrite cube.

### Claim 46 (depends on 25)

The method of claim 25, wherein the electromagnetic sensor comprises a stack of ferrite disks.

### Claim 47 (depends on 25)

The method of claim 25, wherein the electromagnetic sensor comprises a plurality of ferrite rods each having a polymer coating.

### Claim 48 (depends on 25)

The method of claim 25, wherein the electromagnetic sensor comprises a time division multiplexing switch.

### Claim 49

An augmented reality display system, comprising:
a handheld component housing an electromagnetic field emitter, the electromagnetic field emitter emitting a known magnetic field;
a head mounted component having a display system that displays virtual content to a user, the head mounted component coupled to one or more electromagnetic sensors that detect the magnetic field emitted by the electromagnetic field emitter housed in the handheld component, wherein a head pose is known; and
a controller communicatively coupled to the handheld component and the head mounted component, the controller receiving magnetic field data from the handheld component, and receiving sensor data from the head mounted component, wherein the controller determines a hand pose based at least in part on the received magnetic field data and the received sensor data,
wherein the display system modifies the virtual content displayed to the user based at least in part on the determined hand pose.

### Claim 50 (depends on 49)

The AR display system of claim 49, wherein the handheld component is mobile.

### Claim 51 (depends on 50)

The AR display system of claim 50, wherein the handheld component is a totem.

### Claim 52 (depends on 50)

The AR display system of claim 50, wherein the handheld component is a gaming component.

### Claim 53 (depends on 49)

The AR display system of claim 49, wherein the head pose is known based at least in part on SLAM analysis.

### Claim 54 (depends on 53)

The AR display system of claim 53, further comprising one or more cameras operatively coupled to the head-mounted component, and wherein the SLAM analysis is performed based at least on data captured by the one or more cameras.

### Claim 55 (depends on 49)

The AR display system of claim 49, wherein the electromagnetic sensor comprises one or more inertial measurement units (IMUs).

### Claim 56 (depends on 49)

The AR display system of claim 49, wherein the head pose corresponds to at least a position and orientation of the electromagnetic sensor relative to the world.

### Claim 57 (depends on 49)

The AR display system of claim 49, wherein the hand pose is analyzed to determine world coordinates corresponding to the handheld component.

### Claim 58 (depends on 49)

The AR display system of claim 49, wherein the controller detects an interaction with one or more virtual contents based at least in part on the determined hand pose.

### Claim 59 (depends on 58)

The AR display system of claim 58, wherein the display system displays the virtual content to the user based at least in part on the detected interaction.

### Claim 60 (depends on 49)

The AR display system of claim 49, wherein the electromagnetic sensor comprises at least three coils to measure magnetic flux in three directions.

### Claim 61 (depends on 60)

The AR display system of claim 60, wherein the at least three coils are housed together at substantially the same location.

### Claim 62 (depends on 60)

The AR display system of claim 60, wherein the at least three coils are housed at different locations of the head-mounted component.

### Claim 63 (depends on 49)

The AR display system of claim 49, further comprising a control and quick release module to decouple the magnetic field emitted by the electromagnetic field emitter.

### Claim 64 (depends on 49)

The AR display system of claim 49, further comprising additional localization resources to determine the hand pose.

### Claim 65 (depends on 64)

The AR display system of claim 64, wherein the additional localization resources comprises a GPS receiver.

### Claim 66 (depends on 64)

The AR display system of claim 64, wherein the additional localization resources comprises a beacon.

### Claim 67 (depends on 49)

The AR display system of claim 49, wherein at least one of the one or more electromagnetic sensors comprises a non-solid ferrite cube.

### Claim 68 (depends on 49)

The AR display system of claim 49, wherein at least one of the one or more electromagnetic sensors comprises a stack of ferrite disks.

### Claim 69 (depends on 49)

The AR display system of claim 49, wherein at least one of the one or more electromagnetic sensors comprises a plurality of ferrite rods each having a polymer coating.

### Claim 70 (depends on 49)

The AR display system of claim 49, wherein at least one of the one or more electromagnetic sensors comprises a time division multiplexing switch.
