# US 11,442,534 B1 · Smart glasses system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,442,534 B1
- Document ID: US11442534B1
- Status: Granted
- Kind: B1
- Application number: 17/226,342
- Filed: 2021-04-09
- Granted: 2022-09-13
- Earliest claimed: 2018-04-10
- Assignees: Red Pacs, LLC
- Applicants: Robert Edwin Douglas; Kathleen Mary Douglas; David Byron Douglas
- Inventors: Robert Edwin Douglas; Kathleen Mary Douglas; David Byron Douglas
- Web page: https://patentgrep.com/patent/US11442534B1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- A61B 1/00097: {Sensors}
- A61B 6/12: Arrangements for detecting or locating foreign bodies
- A61B 6/462: {characterised by constructional features of the display}
- A61B 6/463: {characterised by displaying multiple images or images and diagnostic data on one display}
- A61B 6/466: {adapted to display 3D data}
- A61B 6/547: {involving tracking of position of the device or parts of the device}
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for}
- A61B 90/361: {Image-producing devices, e.g. surgical cameras}
- G01S 5/14: Determining absolute distances from a plurality of spaced points of known location
- G01S 17/89: for mapping or imaging
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- H04N 13/344: with head-mounted left-right displays
- A61B 1/0004: {for electronic operation}
- A61B 1/00048: {Constructional features of the display}
- A61B 1/00055: {for alerting the user}
- A61B 1/2676: {Bronchoscopes}
- A61B 1/3132: {for laparoscopy}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 2017/00119: {alarm; indicating an abnormal situation}
- A61B 2017/00203: {with speech control or speech recognition}
- A61B 2017/00216: {with eye tracking or head position tracking control}
- A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones}
- A61B 2034/107: {Visualisation of planned trajectories or target regions}
- A61B 2034/2048: {using an accelerometer or inertia sensor}
- A61B 2034/2051: {Electromagnetic tracking systems}
- A61B 2034/2068: {using pointers, e.g. pointers having reference marks for determining coordinates of body points}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B 2090/371: {with simultaneous use of two cameras}
- A61B 2090/372: {Details of monitor hardware}
- A61B 2090/502: {Headgear, e.g. helmet, spectacles}
- G06F 2203/013: Force feedback applied to a game
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A physician performing a physical examination can miss subtle abnormalities, such as alterations of respiratory rate or dangerous skin lesion. A surgeon performing surgery can miss small areas of bleeding or tumor implants. This invention comprises head display unit comprising sensors including low light level TV (L3TV) cameras that gather data and in real time analyzes the data for potential hazards and alerts the user of a potential hazardous scenario.

## Claims

### Claim 1

A head display unit (HDU) comprising:
a processor;
a first camera located on a first viewing position on said HDU configured to image an area;
a second camera located on a second viewing position on said HDU configured to image said area wherein said second camera is separated from said first camera by a distance;
a left eye display operably connected to the processor;
a right eye display operably connected to the processor;
a GPS;
an orientation sensor; and
a non-transitory memory having computer-executable instructions stored thereupon which, when executed by the processor of the HDU, cause the HDU to:
use said GPS to obtain a GPS coordinate of said HDU at time points,
use said orientation sensor to obtain a viewing angle of said HDU at said time points;
use said first camera to perform imaging of said area at said time points;
use said second camera to perform imaging of said area at said time points;
for said time points, use said GPS coordinate, said viewing angle, said imaging of said area by said first camera, said imaging of said area by said second camera to generate a dataset wherein said dataset represents said area,
for a stored time point from said time points in said dataset, use a left eye image of said dataset wherein said left eye image is based on a GPS coordinate of said HDU at said stored time point, a viewing angle at said stored time point and an image from said first camera at said stored time point;
for said stored time point from said time points in said dataset, use a right eye image of said dataset wherein said right eye image is based on said GPS coordinate of said HDU at said stored time point, said viewing angle at said stored time point and an image from said second camera at said stored time point;
display said left eye image on said left eye display; and
display said right eye image on said right eye display.

### Claim 2 (depends on 1)

The HDU of claim 1 further comprising performing an analysis of said dataset to predict if an adverse event scenario is present.

### Claim 3 (depends on 2)

The HDU of claim 2 further comprising wherein said adverse event scenario includes at least one of a group of:
a dangerous observation during surgery;
a dangerous observation during driving;
a dangerous event when walking;
a household scenario;
a sports scenario; and
an occupation scenario.

### Claim 4 (depends on 1)

The HDU of claim 1 further comprising use by at least one of a group of:
a plumber;
a forest ranger;
an electrician;
a surveyor;
an emergency medical technician;
a security personnel; and
a mechanic.

### Claim 5 (depends on 1)

The HDU of claim 1 further comprising wherein said first camera comprises a low light level TV (L3TV) camera and said second camera comprises a L3TV camera.

### Claim 6 (depends on 1)

The HDU of claim 1 further comprising wherein said first camera comprises a forward looking infrared (FLIR) camera and said second camera comprises a FLIR camera.

### Claim 7 (depends on 1)

The HDU of claim 1 further comprising wherein said left eye image displayed on said left eye display and said right eye image displayed on said right eye display digitally brighten the scene.

### Claim 8 (depends on 1)

The HDU of claim 1 further comprising wherein said left eye image displayed on said left eye display and said right eye image displayed on said right eye display are zoomed.

### Claim 9 (depends on 1)

The HDU of claim 1 further comprising at least one of a group of:
a laser range finder wherein said laser range finder is used in conjunction with said first camera and said second camera to generate said 3D dataset wherein said 3D dataset comprises voxels; and
a lidar device wherein said lidar device is used in conjunction with said first camera and said second camera to generate said 3D dataset wherein said 3D dataset comprises voxels.

### Claim 10 (depends on 9)

The HDU of claim 9 further comprising using one of a group of:
an inertial measurement unit in conjunction with said laser range finder or said lidar device to generate said 3D dataset;
an inclinometer in conjunction with said laser range finder or said lidar device to generate said 3D dataset; and
a GPS in conjunction with said laser range finder or said lidar device to generate said 3D dataset.

### Claim 11

A method comprising:
performing imaging of an area using a first camera located on a left side of a head display unit (HDU) configured to be in proximity to a left eye of a user wearing said HDU at time points wherein said HDU comprises a GPS system and an orientation sensor;
performing imaging of said area using a second camera located on a right side of said HDU configured to be in proximity to a right eye of said user wearing said HDU at said time points wherein said second camera is separated from said first camera by a distance;
using said GPS to obtain a GPS coordinate of said HDU at said time points;
using said orientation sensor to obtain a viewing angle of said HDU at said time points;
for said time points, using said GPS coordinate, said viewing angle, said imaging of said are from said first camera and, said imaging of said area from said second camera to generate a dataset wherein said dataset represents said area;
for a stored time point from said time points, using a left eye of said dataset wherein said left eye image is based on a GPS coordinate of said HDU at said stored time point, a viewing angle at said stored time point and an image from said first camera at said stored time point;
for said stored time points, using a right eye image of said dataset wherein said right eye image is based on said GPS coordinate of said HDU at said stored time points, said viewing angle at said stored time point and an image from said second camera at said stored time point;
displaying said left eye image onto a left eye display of said HDU; and
displaying said right eye image onto a right eye display of said HDU.

### Claim 12 (depends on 11)

The method of claim 11 further comprising generating a scenario database comprising:
a list of adverse event scenarios; and
imaging finding(s) corresponding to each adverse event scenario in said list of adverse event scenarios.

### Claim 13 (depends on 12)

The method of claim 12 further comprising analyzing said generated dataset to determine if there is an imaging finding in relation to said scenario database.

### Claim 14 (depends on 13)

The method of claim 13 further comprising wherein said analysis uses computer aided detection (CAD) algorithm or an artificial intelligence algorithm.

### Claim 15 (depends on 13)

The method of claim 13 further comprising if said imaging finding is present, presenting a notification to said user.

### Claim 16 (depends on 15)

The method of claim 15 further comprising wherein said notification to said user comprises at least one of a group of:
a visual notification presented on said HDU; and
an auditory notification presented by speaker(s) on said HDU.

### Claim 17 (depends on 11)

The method of claim 11 further comprising using a lidar device to generate metric data to be used in conjunction with imagery from said first camera located on said left side of said HDU and with imagery from said second camera located on said right side of said HDU.

### Claim 18 (depends on 17)

The method of claim 17 further comprising displaying metric data to said user onto at least one of a group of:
said left eye display of said HDU; and
said right eye display of said HDU.

### Claim 19 (depends on 11)

The method of claim 11 further comprising using at least one of a group of:
an inertial measurement unit in conjunction with a laser range finder or a lidar device to generate said dataset;
an inclinometer in conjunction with a laser range finder or a lidar device to generate said dataset;
a laser range finder in conjunction with said first camera and said second camera to generate said dataset; and
a lidar device in conjunction with said first camera and said second camera to generate said dataset.

### Claim 20 (depends on 19)

The method of claim 19 further comprising wherein a relationship between said first camera and said second camera is adjustable.