# US 2019/0318677 A1 · Display with gaze-adaptive resolution enhancement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318677 A1
- Document ID: US20190318677A1
- Status: Published application
- Kind: A1
- Application number: 16/054,962
- Filed: 2018-08-03
- Published: 2019-10-17
- Earliest claimed: 2018-04-16
- Assignees: Oculus VR, LLC
- Applicants: Oculus VR, LLC
- Inventors: Lu Lu; Fenglin Peng; Michael Sean Mounier; Kevin James MacKenzie; Wai Sze Tiffany Lam
- Web page: https://patentgrep.com/patent/US20190318677A1

## CPC

- G02B 27/1066: {for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view}
- G09G 3/007: {Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/0172: {characterised by optical features}
- G02B 30/25: using polarisation techniques
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G09G 3/001: {using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background (slide projectors per se G03B23/00 = 42 HP)}
- G09G 3/002: {to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT}
- G09G 3/003: {to produce spatial visual effects}
- G09G 3/3611: {Control of matrices with row and column drivers}
- G02B 2027/0147: {comprising a device modifying the resolution of the displayed image}
- G09G 2310/04: Partial updating of the display screen
- G09G 2320/0285: using tables for spatial correction of display data
- G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas
- G09G 2340/0435: Change or adaptation of the frame rate of the video stream
- G09G 2340/0464: Positioning
- G09G 2340/14: Solving problems related to the presentation of information to be displayed
- G09G 2354/00: Aspects of interface with display user
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A display apparatus includes a pixelated display, an eye tracking system, and a segmented resolution enhancing device coupled to the eye tracking system and operable to selectively enhance effective image resolution for the electronic display in the gaze direction. The segmented resolution enhancing device may include a polarization switch in series with a polarization gratings, at least one of which is segmented, for shifting image pixels in the gaze direction between offset positions while displaying a frame.

## Claims

### Claim 1

An apparatus comprising:
an electronic display comprising a pixel array configured to display a frame, the pixel array comprising a plurality of display pixels;
an eye tracking device configured to determine a gaze direction of a user of the electronic display; and,
an optical block for forming a frame image from the frame displayed by the electronic display, the optical block comprising a segmented resolution enhancing device (SRED) operatively coupled to the eye tracking device, the frame image comprising an array of image pixels disposed at an image pixel pitch p;
wherein the SRED comprises a plurality of device segments configured to be individually activated in response to a signal from the eye tracking device, each device segment being optically coupled to a different group of display pixels of the pixel array, for selectively enhancing effective resolution for a portion of the frame image in the gaze direction.

### Claim 2 (depends on 1)

The apparatus of claim 1 further comprising a controller configured to activate a selected device segment in the gaze direction from the array of device segments based on the signal from the eye tracking device.

### Claim 3 (depends on 2)

The apparatus of claim 2 further comprising a display processor configured to:
provide pixel data to the electronic display for displaying the frame;
identify a selected group of display pixels, from the different groups of display pixels of the electronic display, that is optically coupled to the selected device segment; and,
perform a selective pixel update of the selected group of pixels with new pixel data at least once while the frame is being displayed by the electronic display.

### Claim 4 (depends on 3)

The apparatus of claim 3 wherein the selective pixel update is performed without updating a peripheral group of display pixels located outside of the selected group of display pixels with new pixel data while the frame is being displayed.

### Claim 5 (depends on 3)

The apparatus of claim 3 wherein each device segment comprises a light steering switch (LSS) configured to switch, upon activation by the controller, between two or more light steering states while the frame is being displayed, so as to provide a sequence of offset images for the portion of the frame image in the gaze direction.

### Claim 6 (depends on 5)

The apparatus of claim 5 wherein the selective pixel update is performed in coordination with switching the LSS of the selected device segment.

### Claim 7 (depends on 6)

The apparatus of claim 6 wherein the LSS is operable, upon activation by the controller, to provide the sequence of mutually offset images comprising two images having a spatial offset s therebetween equal to a fraction of the image pixel pitch p.

### Claim 8 (depends on 7)

The apparatus of claim 7 wherein the frame image comprises interstitial areas between adjacent image pixels that are at least equal to the image pixels in size, and wherein the spatial offset s is such that at least some of the image pixels of a first of the two mutually offset images are located in the interstitial spaces of the second of the mutually offset images.

### Claim 9 (depends on 7)

The apparatus of claim 7 wherein the display processor is configured to selectively sub-sample input frame data for the selected group of display pixels at two or more sampling grids with a sampling offset therebetween matching the spatial offset s.

### Claim 10 (depends on 1)

The apparatus of claim 1 wherein the SRED comprises:
a first polarization grating, and
a first polarization switch disposed in sequence with the first polarization grating,
wherein at least one of the first polarization grating or the first polarization switch is segmented to define the plurality of device segments, and
wherein at least one of the electronic display or the optics block comprises a polarization element configured to provide polarized pixel light from the electronic display to the SRED.

### Claim 11 (depends on 10)

The apparatus of claim 10 wherein the electronic display is configured to transmit polarized light.

### Claim 12 (depends on 10)

The apparatus of claim 10 comprising polarization device disposed optically between the electronic display and the SRED and configured to provide light from the electronic display to the SRED in a circular polarization state.

### Claim 13 (depends on 10)

The apparatus of claim 10 wherein at least one of the first polarization switch or the first polarization grating comprises an active LC device comprising an LC layer disposed between two electrodes, at least one of which is transparent, and wherein at least one of the two electrodes comprises a plurality of electrically separate electrode segments.

### Claim 14 (depends on 10)

The apparatus of claim 10 wherein the SRED further comprises a second polarization grating disposed in sequence with a second polarization switch, at least one of which is segmented to match segments of the at least one of the first polarization grating or the first polarization switch.

### Claim 15 (depends on 14)

The apparatus of claim 14 wherein the second polarization grating is disposed with a rotational offset relative to the first polarization grating.

### Claim 16 (depends on 14)

The apparatus of claim 14 wherein the rotational offset is substantially 90°.

### Claim 17 (depends on 14)

The apparatus of claim 14 wherein at least one of the first polarization grating or the second polarization grating comprises at least one of: a liquid crystal (LC) Pancharatnam Berry Phase (PBP) grating or an LC volume holographic grating.

### Claim 18 (depends on 14)

The apparatus of claim 14 wherein at least one of the first polarization grating or the second polarization grating comprises at least one of: a segmented active liquid crystal LC PBP grating, or a segmented active LC holographic grating.

### Claim 19 (depends on 14)

The apparatus of claim 14 wherein at least one of the first polarization switch or the second polarization switch comprises an active LC HWP.

### Claim 20

A method for operating a display apparatus, the method comprising:
displaying a frame with an electronic display comprising an array of display pixels;
forming an image of the displayed frame to obtain an image frame comprising an array of image pixels, the array of image pixels comprising image pixels disposed at an image pixel pitch p;
determining, with an eye tracking device, a gaze direction of a user of the electronic display; and,
selectively shifting a portion of the frame image located in the gaze direction, at least once while displaying the frame, by at least a fraction of the image pixel pitch.