# US 2021/0240980 A1 · Method and system for correlating an image capturing device to a human user for analyzing gaze information associated with cognitive performance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240980 A1
- Document ID: US20210240980A1
- Status: Published application
- Kind: A1
- Application number: 17/234,523
- Filed: 2021-04-19
- Published: 2021-08-05
- Earliest claimed: 2016-11-10
- Assignees: Neurotrack Technologies, Inc.
- Applicants: Neurotrack Technologies, Inc.
- Inventors: Jian YAO; Nicholas T. BOTT; John J. ANDERSON; Scott ANDREWS
- Web page: https://patentgrep.com/patent/US20210240980A1

## CPC

- G06V 40/19: Sensors therefor
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06N 20/00: Machine learning
- G06V 40/165: {using facial parts and geometric relationships}
- H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)}
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

The present invention provides a method for a finalized processed image and related data to identify a spatial location of each pupil in the region. Each pupil is identified by a two-dimensional spatial coordinate. The method includes processing information associated with each pupil identified by the two-dimensional spatial coordinate to output a plurality of two-dimensional spatial coordinates, each of which is in reference to a time, in a two-dimensional space. The method then includes outputting a gaze information about the human user. The gaze information includes the two-dimensional spatial coordinates each of which is in reference to a time in a two-dimensional space.

## Claims

### Claim 1

A method for identifying a feature of an eye of a human user, the method comprising:
initiating an image capturing device under control of a processing device, the image capturing device comprising a plurality of sensors arranged in an array;
capturing video information from a facial region of the human user using the image capturing device, the video information being a stream of video comprising a plurality of frames;
processing the video information using an image processing device, to parse the video information into the plurality of images, each of the plurality of images having a time stamp from a first time stamp, a second time stamp, to an Nth time stamp, where N is greater than 10;
processing each of images using the image processing device to identify a location of the facial region;
processing each of the images with the location of the facial region to identify a plurality of landmarks associated with the facial region;
processing each of the images with the location of the facial regions and the plurality of landmarks to isolation a region including each of the eyes;
processing each of the regions, frame by frame, using the image processing device, to identify a pupil region for each of the eyes, the processing comprising at least: processing the region using a grayscale conversion to output a grayscale image; processing the grayscale image using an equalization process to output an equalized image; processing the equalized image using a thresholding process to output a thresholded image; processing the thresholded image using a dilation and erosion process to output a dilated and eroded image; and processing the dilated and eroded image using a contour and moment process to output a finalized processed image;
processing the finalized processed image to identify a spatial location of each pupil in the region, each pupil being identified by a two-dimensional spatial coordinate;
processing information associated with each pupil identified by the two d-dimensional spatial coordinate to output a plurality of two-dimensional spatial coordinates, each of which is in reference to a time, in a two-dimensional space; and
outputting a gaze information about the human user.