# US 11,079,598 B2 · Augmented reality spectroscopy

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,079,598 B2
- Document ID: US11079598B2
- Status: Granted
- Kind: B2
- Application number: 16/775,123
- Filed: 2020-01-28
- Published: 2020-05-28
- Granted: 2021-08-03
- Earliest claimed: 2016-09-22
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Nicole Elizabeth Samec; Nastasja U. Robaina; Adrian Kaehler; Mark Baerenrodt; Eric Baerenrodt; Christopher M. Harrises; Tammy Sherri Powers
- Web page: https://patentgrep.com/patent/US11079598B2

## CPC

- G02B 27/0172: {characterised by optical features}
- A61B 5/0075: {by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy (A61B5/0071 takes precedence)}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/1114: {Tracking parts of the body}
- A61B 5/1121: {Determining geometric values, e.g. centre of rotation or angular range of movement}
- A61B 5/14551: {for measuring blood gases}
- A61B 5/14555: {specially adapted for the eye fundus}
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots}
- A61B 5/744: {Displaying an avatar, e.g. an animated cartoon character}
- G01J 3/0248: {using a sighting port, e.g. camera or human eye}
- G01J 3/0256: {Compact construction}
- G01J 3/0264: {Electrical interface; User interface}
- G01J 3/10: Arrangements of light sources specially adapted for spectrometry or colorimetry
- G01J 3/108: {for measurement in the infrared range}
- G01J 3/2823: {Imaging spectrometer}
- G01J 3/42: Absorption spectrometry; Double beam spectrometry; Flicker spectrometry; Reflection spectrometry (beam switching arrangements G01J3/08)
- G01J 3/427: Dual wavelengths spectrometry
- G01N 21/359: using near infrared light
- G01N 33/483: Physical analysis of biological material
- G02B 6/0076: {Stacked arrangements of multiple light guides of the same or different cross-sectional area}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- A61B 2017/00216: {with eye tracking or head position tracking control}
- A61B 2090/309: {using white LEDs}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- G01J 2003/106: {the two sources being alternating or selectable, e.g. in two ranges or line:continuum}
- G01J 2003/2826: {Multispectral imaging, e.g. filter imaging}
- G01J 2003/425: {Reflectance}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 2027/0125: {Field-of-view increase by wavefront division}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F 3/012: {Head tracking input arrangements}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In some embodiments, a system comprises a head-mounted frame removably coupleable to the user's head; one or more light sources coupled to the head-mounted frame and configured to emit light with at least two different wavelengths toward a target object in an irradiation field of view of the light sources; one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive light reflected after encountering the target object; and a controller operatively coupled to the one or more light sources and detectors and configured to determine and display an output indicating the identity or property of the target object as determined by the light properties measured by the detectors in relation to the light properties emitted by the light sources.

## Claims

### Claim 1

A wearable spectroscopy system comprising:
a frame removably coupleable to a user's head;
a head-mountable display attached to the frame, the head-mountable display comprising a waveguide stack configured to output light with selectively variable levels of wavefront divergence to an eye of the user;
at least one outward-facing light source coupled to the frame and configured to emit light with at least two different wavelengths;
at least one electromagnetic radiation detector coupled to the frame and configured to receive reflected light from a target object irradiated by the emitted light;
an absorption database of light absorption properties of at least one material;
a controller operatively coupled to the at least one outward-facing light source and the at least one electromagnetic radiation detector, the controller configured to cause:
emission of pulses of light by the at least one outward-facing light source;
detection, by the at least one electromagnetic radiation detector, of levels of light absorption related to the emitted pulses of light and reflected light from the target object; and
matching of detected levels of light absorption with light absorption properties stored in the absorption database; and
a graphics processor unit configured to display an output to the user.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one outward-facing light source comprises a plurality of light emitting diodes.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one outward-facing light source is configured to emit electromagnetic radiation at two or more predetermined wavelengths.

### Claim 4 (depends on 3)

The system of claim 3, wherein the at least one outward-facing light source is configured to emit electromagnetic radiation at a first wavelength of about 660 nanometers, and a second wavelength of about 940 nanometers.

### Claim 5 (depends on 3)

The system of claim 3, wherein the at least one outward-facing light source is configured to emit electromagnetic radiation at the two predetermined wavelengths sequentially.

### Claim 6 (depends on 3)

The system of claim 3, wherein the at least one outward-facing light source is configured to emit electromagnetic radiation at the two predetermined wavelengths simultaneously.

### Claim 7 (depends on 1)

The system of claim 1, wherein the controller is further configured to cause the at least one outward-facing light source to emit a cyclic pattern of a first wavelength on, then a second wavelength on, then both first and second wavelengths off, such that the at least one electromagnetic radiation detector detects the first and second wavelengths separately.

### Claim 8 (depends on 1)

The system of claim 1, wherein the controller is configured to calculate a ratio of first wavelength light measurement to second wavelength light measurement, and wherein the system is configured to convert the ratio to a tissue property based on the absorption database.

### Claim 9 (depends on 8)

The system of claim 8, wherein the controller is operatively coupled to an optical element coupled to the head-mountable display and viewable by the user, wherein the system is configured to provide an output based on the tissue property, wherein the output is viewable by the user through the optical element.

### Claim 10 (depends on 8)

The system of claim 8, wherein the tissue property comprises at least one property selected from the group consisting of: a tissue type, an estimated blood saturation level, a presence of abnormal cells, a presence of cancerous cells.

### Claim 11 (depends on 1)

The system of claim 1, wherein the controller is configured to calculate a ratio of first wavelength light measurement to second wavelength light measurement, and wherein the system is configured to convert the ratio to a material property based on the absorption database.

### Claim 12 (depends on 11)

The system of claim 11, wherein the material property is a material type of the target object.

### Claim 13 (depends on 1)

The system of claim 1, wherein the at least one electromagnetic radiation detector comprises a device selected from the group consisting of: a photodiode and a photodetector.

### Claim 14 (depends on 1)

The system of claim 1, wherein the at least one electromagnetic radiation detector comprises a digital image sensor.

### Claim 15 (depends on 14)

The system of claim 14, wherein the digital image sensor comprises a plurality of pixels, and wherein the controller is configured to automatically detect a subset of pixels which are receiving the light reflected after encountering a predetermined tissue property and to produce an output that displays the location of the subset of pixels indicating the predetermined tissue property.

### Claim 16 (depends on 1)

The system of claim 1, wherein the head-mountable display further comprises an inertial measurement unit positional system.

### Claim 17 (depends on 16)

The system of claim 16, wherein the inertial measurement unit positional system determines a pose orientation of the user's head.

### Claim 18 (depends on 1)

The system of claim 1, wherein the waveguide stack comprises waveguides having optical power.

### Claim 19 (depends on 1)

The system of claim 1, wherein the selectively variable levels of wavefront divergence correspond to different depth planes.

### Claim 20 (depends on 1)

The system of claim 1, wherein at least some waveguides of the waveguide stack output light with different amounts of wavefront divergence than an other waveguide of the waveguide stack.