# US 2017/0287447 A1 · Systems and Methods for Head-Mounted Display Adapted to Human Visual Mechanism

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0287447 A1
- Document ID: US20170287447A1
- Status: Published application
- Kind: A1
- Application number: 15/088,816
- Filed: 2016-04-01
- Published: 2017-10-05
- Earliest claimed: 2016-04-01
- Assignees: Linear Algebra Technologies Limited
- Applicants: Linear Algebra Technologies Limited
- Inventors: Brendan BARRY; David MOLONEY
- Web page: https://patentgrep.com/patent/US20170287447A1

## CPC

- G09G 5/391: Resolution modifying circuits, e.g. variable screen formats
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G09G 5/006: {Details of the interface to the display terminal (specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04)}
- H04N 13/344: with head-mounted left-right displays
- H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
- H04N 13/398: Synchronisation thereof; Control thereof
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/426: using memory downsizing methods
- H04N 19/597: specially adapted for multi-view video sequence encoding
- G02B 2027/0132: {comprising binocular systems}
- G02B 2027/014: {comprising information/image processing systems}
- G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas
- G09G 2354/00: Aspects of interface with display user
- G09G 2360/18: Use of a frame buffer in a display terminal, inclusive of the display panel
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods are provided for rendering of a dual eye-specific display. The system tracks the user's eye movements and/or positions, in some implementations, based on electroencephalography (EEG) of the user, to correctly label the central (foveal) and peripheral (extra-foveal) areas of the display. Foveal data is fully rendered while extra-foveal data is reduced in resolution and, in some implementations, shared between the two displays.

## Claims

### Claim 1

A head-mounted display device with eye-tracking of a user's two eyes, comprising:
a first display for the left eye;
a second display for the right eye;
a memory, coupled to the first display and the second display, comprising a first frame buffer for the first display and a second frame buffer for the second display, wherein the first frame buffer comprises a first foveal frame buffer and a first extra-foveal frame buffer and the second frame buffer comprises a second foveal frame buffer and a second extra-foveal frame buffer; and
one or more processors, coupled to the memory, configured to:
receive eye-tracking information of the user wearing the head mounted display, wherein the eye-tracking information comprises at least one of an eye movement and an eye position of the user,
determine, based on the eye-tracking information, a first foveal region of the first display, a first extra-foveal region of the first display, a second foveal region of the second display, and a second extra-foveal region of the second display, and
load, into the memory, first foveal region pixels of the first foveal region, first extra-foveal region pixels of the first extra-foveal region, second foveal region pixels of the second foveal region, second extra-foveal region pixels of the second extra-foveal region,
wherein:
the first foveal region pixels are represented with full resolution and loaded into the first foveal frame buffer,
the first extra-foveal region pixels are represented with reduced resolution and loaded into the first extra-foveal frame buffer,
the second foveal region pixels are represented with full resolution and loaded into the second foveal frame buffer, and
the second extra-foveal region pixels are represented with reduced resolution and loaded into the second extra-foveal frame buffer.

### Claim 2 (depends on 1)

The device of claim 1, wherein the eye-tracking information is based on at least one of infrared light projection onto the user's two eyes or electroencephalography (EEG) of the user.

### Claim 3 (depends on 1)

The device of claim 1, wherein at least one of the first foveal region, the first extra-foveal region, the second foveal region, or the second extra-foveal region is adjustable.

### Claim 4 (depends on 1)

The device of claim 1, wherein, to represent the first extra-foveal region pixels and the second extra-foveal region pixel in reduced resolution, the one or more processors are configured to duplicate a pixel value.

### Claim 5 (depends on 4)

The device of claim 4, wherein the one or more processors are further configured to duplicate the pixel value across at least one of 2×2 pixels, 3×3 pixels, 4×4 pixels, or 5×5 pixels.

### Claim 6 (depends on 4)

The device of claim 4, further comprising a Display Serial Interface coupled to the memory and at least one of the first display or the second display, wherein the one or more processors are configured to duplicate the pixel value after receiving the pixel value from the Display Serial Interface.

### Claim 7 (depends on 1)

The device of claim 1, further comprising a plurality of Display Serial Interfaces, wherein some of the plurality of Display Serial Interfaces are configured to transfer, in parallel, display data corresponding to at least one of the first foveal region or the second foveal region.

### Claim 8 (depends on 1)

The device of claim 1, wherein the first extra-foveal frame buffer and the second extra-foveal frame buffer are shared.

### Claim 9 (depends on 1)

The device of claim 1, wherein the one or more processors are further configured to load the first foveal frame buffer and the first extra-foveal frame buffer at different rates and the second foveal frame buffer and the second extra-foveal frame buffer at different rates.

### Claim 10 (depends on 1)

The device of claim 1, wherein the one or more processors are further configured to apply alpha-blending between depth-planes of the first foveal region pixels and depth-planes of the first extra-foveal region pixels and between depth-planes of the second foveal region pixels and depth-planes of the second extra-foveal region pixels.

### Claim 11

A method for a head-mounted display device with eye-tracking of a user's two eyes, comprising:
receiving eye-tracking information of the user wearing the head mounted display, wherein the eye-tracking information comprises at least one of an eye movement and an eye position of the user;
determining, based on the eye-tracking information, a first foveal region of a first display for the left eye, a first extra-foveal region of the first display, a second foveal region of a second display for the right eye, and a second extra-foveal region of the second display;
loading, into a first foveal frame buffer of a first frame buffer for the first display, first foveal region pixels that are represented with full resolution of the first foveal region;
loading, into a first extra-foveal frame buffer of the first frame buffer for the first display, first extra-foveal region pixels that are represented with reduced resolution of the first extra-foveal region;
loading, into a second foveal frame buffer of a second frame buffer for the second display, second foveal region pixels that are represented with full resolution of the second foveal region; and
loading, into a second extra-foveal frame buffer of the second frame buffer for the second display, second extra-foveal region pixels that are represented with reduced resolution of the second extra-foveal region.

### Claim 12 (depends on 11)

The method of claim 11, wherein the eye-tracking information is based on at least one of infrared light projection onto the user's eyes or electroencephalography (EEG) of the user.

### Claim 13 (depends on 11)

The method of claim 11, wherein at least one of the first foveal region, the first extra-foveal region, the second foveal region, or the second extra-foveal region is adjustable.

### Claim 14 (depends on 11)

The method of claim 11, wherein, to represent the first extra-foveal region pixels and the second extra-foveal region pixel in reduced resolution, further comprising duplicating a pixel value.

### Claim 15 (depends on 14)

The method of claim 14, further comprising, further comprising duplicating the pixel value after receiving the pixel value from a Display Serial Interface, wherein the Display Serial Interface is coupled to the memory and at least one of the first display or the second display

### Claim 16 (depends on 11)

The method of claim 11, wherein the first extra-foveal frame buffer and the second extra-foveal frame buffer are shared.

### Claim 17 (depends on 11)

The method of claim 11, further comprising loading the first foveal frame buffer and the first extra-foveal frame buffer at different rates and the second foveal frame buffer and the second extra-foveal frame buffer at different rates.

### Claim 18 (depends on 11)

The method of claim 11, further comprising applying alpha-blending between depth-planes of the first foveal region pixels and depth-planes of the first extra-foveal region pixels and between depth-planes of the second foveal region pixels and depth-planes of the second extra-foveal region pixels.

### Claim 19 (depends on 11)

The method of claim 11, further comprising transferring, in parallel, display data corresponding to at least one of the first foveal region or the second foveal region.

### Claim 20

A non-transitory computer readable medium storing a computer-readable program for a head mounted display device with eye-tracking of a user's two eyes, comprising
computer-readable instructions to receive eye-tracking information of the user wearing the head mounted display, wherein the eye-tracking information comprises at least one of an eye movement and an eye position of the user;
computer-readable instructions to determine, based on the eye-tracking information, a first foveal region of a first display for the left eye, a first extra-foveal region of the first display, a second foveal region of a second display for the right eye, and a second extra-foveal region of the second display; and
computer-readable instructions to load, into a first foveal frame buffer of a first frame buffer for the first display, first foveal region pixels that are represented with full resolution of the first foveal region;
computer-readable instructions to load, into a first extra-foveal frame buffer of the first frame buffer for the first display, first extra-foveal region pixels that are represented with reduced resolution of the first extra-foveal region;
computer-readable instructions to load, into a second foveal frame buffer of a second frame buffer for the second display, second foveal region pixels that are represented with full resolution of the second foveal region; and
computer-readable instructions to load, into a second extra-foveal frame buffer of the second frame buffer for the second display, second extra-foveal region pixels that are represented with reduced resolution of the second extra-foveal region.
