# US 2023/0079255 A1 · Retinal imaging system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0079255 A1
- Document ID: US20230079255A1
- Status: Published application
- Kind: A1
- Application number: 17/835,825
- Filed: 2022-06-08
- Published: 2023-03-16
- Earliest claimed: 2020-09-30
- Assignees: Ai-Ris LLC
- Applicants: Ai-Ris LLC
- Inventors: Marcus Emilio DeAyala; Tokunbo Said Falohun; Daniel Shafiee Kermany; Harsha Kalkunte Mohan; Uthej Vattipalli; Amir Tofighi Zavareh
- Web page: https://patentgrep.com/patent/US20230079255A1

## CPC

- A61B 3/12: for looking at the eye fundus, e.g. ophthalmoscopes (A61B3/13 takes precedence)
- A61B 3/0008: {provided with illuminating means (A61B3/117, A61B3/14 take precedence)}
- A61B 3/0025: {characterised by electronic signal processing, e.g. eye models}
- A61B 3/0091: {Fixation targets for viewing direction}
- A61B 3/156: {for blocking}
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- A61B 5/7267: {involving training the classification device}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06V 40/18: Eye characteristics, e.g. of the iris
- H04N 23/20: for generating image signals from infrared radiation only
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Provided is a wearable fundus camera configured to be worn as a headset by a human, the wearable fundus camera comprising: an infrared light source configured to output infrared light to be directed at a retina of the human; an image sensor configured to capture infrared images depicting a retina of an eye of the human under illumination from the infrared light source without a pupil of the eye being dilated with mydriatics; and an eye cuff configured to be biased against a face of the human and occlude at least some ambient light from reaching the image sensor.

## Claims

### Claim 1

An system, comprising:
a wearable fundus camera configured to be worn as a headset by a human, the wearable fundus camera comprising:
an infrared light source configured to output infrared light to be directed at a retina of the human;
an image sensor configured to capture infrared images depicting a retina of an eye of the human under illumination from the infrared light source without a pupil of the eye being dilated with mydriatics; and
an eye cuff configured to be biased against a face of the human and occlude at least some ambient light from reaching the image sensor;
a computing system storing computer program instructions that, when executed by the computing system, effectuate operations comprising:
obtaining at least some of the captured infrared images depicting the retina of the human;
obtaining access to a trained computer vision model configured to detect ophthalmologic abnormalities in retinal images;
providing the at least some of the captured infrared images, as input, to the trained computer vision model;
obtaining, from the trained computer vision model, based on the at least some of the captured infrared images, a first score indicating whether the at least some of the captured infrared images depict an ophthalmologic abnormality and storing, in memory, a result of the based on the score.

### Claim 2 (depends on 1)

The system of claim 1, wherein the operations comprise:
pretraining the computer vision model with a first training set of images to form a pre-trained computer vision model, at least 50% of the images in the first training set not being retinal images;
training the pre-trained computer vision model with a second training set of labeled images, at least half of the labeled images in the second training set not being retinal images labeled according to whether the respective labeled images depict retinopathy; and
determining the first score.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the trained computer vision model is trained on a corpus of images comprising images depicting a plurality of objects, each of the plurality of objects being classified into one or more first categories of a first plurality of categories, wherein each image from the corpus of images includes one or more first labels, each of the one or more first labels indicating that a respective image has been classified into one of the one or more first categories,
the trained computer vision model is trained on a set of images comprising images depicting a plurality of retinas, each of the plurality of retinas being classified into one or more second categories of a second plurality of categories, wherein each image from the set of images includes one or more second labels, each of the one or more second labels indicating that a respective image has been classified into one of the one or more second categories, wherein each of the second plurality of categories include a subset of images from the set of images depicting a type of ophthalmologic abnormality or ophthalmologic normality, the type of ophthalmologic abnormality being one of a plurality of ophthalmologic abnormalities, and
the trained computer vision model is trained on a set of infrared images comprising infrared images depicting retinas, wherein each infrared image from the set of infrared images is classified into at least one of the second plurality of categories, wherein each infrared image from the set of infrared images includes at least one of the one or more second labels.

### Claim 4 (depends on 1)

The system of claim 1, wherein the wearable fundus camera weighs less than 2 kilograms and has a center of mass less than 10 centimeters from a portion of the eye cuff configured to be positioned adjacent a bridge of the human's nose when worn by the human.

### Claim 5 (depends on 1)

The system of claim 1, wherein the wearable fundus camera further comprises:
one or more actuators configured to orient the infrared light source, wherein the infrared light source is oriented in response to determining that the infrared light is directed to a pupil of an eye of the human when the wearable fundus camera is worn by the human, wherein the operations further comprise:
identifying, based on at least one captured infrared image from the captured infrared images, using a first classifier configured detect an eye within an infrared image, a set of pixels representing a first portion of the at least one captured infrared image depicting the eye of the human;
identifying, using a second classifier configured to detect the pupil within the at least one captured infrared image, a subset of pixels from the set of pixels representing a second portion of the at least one captured infrared image depicting the pupil of the eye of the human, wherein the first portion of the at least one captured infrared image comprises the second portion of the at least one captured infrared image; and
determining, based on the subset of pixels, a location of the center of the pupil; and
causing the one or more actuators to adjust a position of the infrared light source such that the infrared light output by the infrared light source is directed at the location of the center of the pupil.

### Claim 6 (depends on 1)

The system of claim 1, wherein the operations further comprise:
providing one or more additional captured infrared images to one or more binary classifiers, wherein the one or more additional captured infrared images are captured prior to the image sensor capturing the infrared images, wherein each of the one or more binary classifiers is configured to detect whether the retina depicted by the captured infrared images represents a respective contraindicator from a set of contraindicators;
preventing the one or more additional captured infrared images from being analyzed by the trained computer vision model in response to detecting a given contraindicator; and
causing the image sensor to capture the infrared images.

### Claim 7 (depends on 1)

The system of claim 1, wherein the wearable fundus camera comprises at least part of the computer system.

### Claim 8 (depends on 1)

The system of claim 1, wherein the infrared light includes incoherent light of multiple infrared wavelengths, the operations further comprise:
selecting, for each of the multiple infrared wavelengths, a subset of the captured infrared images, wherein each subset of the captured infrared images includes infrared images of a respective infrared wavelength, wherein providing the captured infrared images to the trained computer vision model comprises:
providing each subset of the captured infrared images to the trained computer vision model, wherein the first score is computed based on a weighted combination of a score output by the trained computer vision model for each subset of the captured infrared images.

### Claim 9 (depends on 1)

The system of claim 1, wherein the image sensor captures a plurality of infrared images, the captured infrared images being some of the plurality of infrared images, wherein the wearable fundus camera further comprises:
memory storing additional computer program instructions; and
one or more processors that, in response to executing the additional computer program instructions, effectuate additional operations comprising:
providing the plurality of infrared images to a classifier trained to detect whether a given image depicts a retina of a human; and
filtering the plurality of infrared images based on results of the classifier to obtain the infrared images.

### Claim 10 (depends on 1)

The system of claim 1, wherein the wearable fundus camera further comprises:
one or more computer processors configured to:
compute a blur score for each infrared image of the captured infrared images, wherein the blur score indicates how blurry a respective infrared image is, the blur score being computed by:
transforming a given infrared image into a grayscale infrared image,
applying a Laplacian kernel to an array of pixel values representing the grayscale infrared image, and
computing a variance of each pixel value from the array of pixel values, and
generating, for the given infrared image, the blur score based the variance of each pixel value from the array of pixel values;
determine whether a respective blur score is satisfies a threshold;
removing one or more infrared images from the captured infrared images in response to determining that the respective blur score of the one or more infrared images satisfies the threshold.

### Claim 11 (depends on 1)

The system of claim 1, wherein the wearable fundus camera further comprises:
means for outputting visible light directed at the retina of the human; and
means for capturing a set of visible-light images depicting the retina of the human, wherein the score is further determined based on the set of visible-light images.

### Claim 12 (depends on 1)

The system of claim 1, wherein the operations further comprise:
steps for filtering the captured infrared images.

### Claim 13 (depends on 1)

The system of claim 1, wherein the trained computer vision model is configured to:
compute the first score based on an aggregation of classification scores respectively corresponding to the captured infrared images;
rank the captured infrared images based on the respective classification scores; and
identify one or more infrared images from the captured infrared images having a largest classification score contributing to the computed first score, wherein the operations further comprise:
obtaining, from the trained computer vision model, the one or more infrared images each including an indication of the respective classification score.

### Claim 14 (depends on 13)

The system of claim 13, wherein the operations further comprise:
extracting, for each of the one or more infrared images, values of gradients generated by a last layer of the trained computer vision model;
encoding the values of the gradients for each respective infrared image to represent the values of the gradients as a heat map; and
generating the heat map representing the respective infrared image based on the encoded values, wherein:
regions representing subsets of the values of the gradients within a first gradient value range are depicted using a first color,
regions representing subsets of the values of the gradients within a second gradient value range are depicted using a second color, and
at least one value included in the first gradient value range is greater than values included in the second gradient value range.

### Claim 15 (depends on 1)

The system of claim 1, wherein the wearable fundus camera further comprises:
a volume defined by a case of the wearable fundus camera when worn by the human;
a head strap configured to bias the eye cuff against the face of the human; and
a visible-light sensor configured to detect visible light leaking into the volume from an ambient environment of the human, wherein responsive to detecting more than a threshold amount of visible light, the visible light sensor outputs a signal to cause the computing system to differentiate a polarization of the visible light backscattered from the retina of the human and included within the captured infrared images.

### Claim 16 (depends on 1)

The system of claim 1, wherein the computing system comprises:
means for executing the trained computer vision model on a mobile computing device.

### Claim 17 (depends on 1)

The system of claim 1, wherein the operations further comprise:
steps for performing data normalization for the captured infrared images.

### Claim 18 (depends on 1)

The system of claim 1, wherein the operations further comprise:
steps for training the computer vision model.

### Claim 19 (depends on 1)

The system of claim 1, wherein the operations further comprise:
steps for determining whether the at least some infrared images depict retinopathy.

### Claim 20 (depends on 1)

The system of claim 1, wherein the wearable fundus camera further comprises:
a light emitting diode (LED) that outputs light of a visible wavelength to direct a focus of a pupil of an eye of the human towards a location of the LED such that the infrared light is directed toward a center of the pupil of the eye; and
a beam splitter positioned to reflect the infrared light onto the retina and transmit light returning from the retina through the beam splitter to the image sensor.
