# US 11,307,410 B2 · Display apparatus and method incorporating adaptive pose locking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,307,410 B2
- Document ID: US11307410B2
- Status: Granted
- Kind: B2
- Application number: 17/155,621
- Filed: 2021-01-22
- Published: 2021-08-12
- Granted: 2022-04-19
- Earliest claimed: 2020-02-06
- Assignees: Varjo Technologies Oy
- Applicants: Varjo Technologies Oy
- Inventors: Mikko Strandborg; Ville Miettinen; Klaus Melakari
- Web page: https://patentgrep.com/patent/US11307410B2

## CPC

- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G09G 3/001: {using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background (slide projectors per se G03B23/00 = 42 HP)}
- G02B 27/0179: {Display position adjusting means not related to the information to be displayed}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G09G 3/003: {to produce spatial visual effects}
- G09G 3/007: {Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels}
- G09G 3/2092: {Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto (suitable for both CRT and flat panel G09G5/003; specific for a CRT G09G1/165)}
- G09G 3/3611: {Control of matrices with row and column drivers}
- G09G 3/3648: {using an active matrix (G09G3/367 - G09G3/3696 take precedence)}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0147: {comprising a device modifying the resolution of the displayed image}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G09G 2300/0426: Layout of electrodes and connections
- G09G 2300/0452: Details of colour pixel setup, e.g. pixel composed of a red, a blue and two green components
- G09G 2310/04: Partial updating of the display screen
- G09G 2320/028: by changing the viewing angle properties, e.g. widening the viewing angle, adapting the viewing angle to the view direction
- G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas
- G09G 2340/0435: Change or adaptation of the frame rate of the video stream
- G09G 2340/0464: Positioning
- G09G 2354/00: Aspects of interface with display user
- G09G 2360/16: Calculation or use of calculated indices related to luminance levels in display data
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A display apparatus including pose-tracking means; image renderer per eye; liquid-crystal device including liquid-crystal structure and control circuit; and processor. Processor is configured to: process pose-tracking data to determine user's head pose; detect if rate at which head pose is changing is below predefined threshold rate; if yes, switch on lock mode, select head pose for session of lock mode, and generate output image frames according to head pose during session; if no, generate output image frames according to corresponding head poses of user using pose-tracking data; and display output image frames, whilst shifting light emanating to from pixels of image renderer to multiple positions (P1-P9) in sequential and repeated manner, said shifting causes resolution of output image frames to appear higher than display resolution of image renderer.

## Claims

### Claim 1

A display apparatus comprising:
pose-tracking means;
an image renderer per eye;
a liquid-crystal device comprising a liquid-crystal structure and a control circuit, wherein the liquid-crystal structure is arranged in front of an image-rendering surface of the image renderer; and
at least one processor configured to:
process pose-tracking data, collected by the pose-tracking means, to determine a head pose of a user;
detect whether or not a rate at which the head pose of the user is changing is below a predefined threshold rate;
when it is detected that the rate is below the predefined threshold rate, switch on a lock mode of the display apparatus, select a given head pose of the user for a given session of the lock mode, and generate a sequence of output image frames according to the given head pose during the given session of the lock mode;
when it is detected that the rate is not below the predefined threshold rate, generate the sequence of output image frames according to corresponding head poses of the user as determined from the pose-tracking data; and
display the sequence of output image frames via the image renderer, whilst controlling the liquid-crystal structure, via the control circuit, to shift light emanating from a given pixel of the image renderer to a plurality of positions in a sequential and repeated manner, wherein a shift in the light emanating from the given pixel of the image renderer to the plurality of positions causes a resolution of the output image frames to appear higher than a display resolution of the image renderer;
wherein the at least one processor is further configured to:
detect, after the lock mode is switched on, whether or not the rate at which the head pose of the user is changing exceeds the predefined threshold rate; and
when it is detected, after the lock mode is switched on, that the rate exceeds the predefined threshold rate, switch off the lock mode of the display apparatus;
wherein the given head pose of the user is selected for the given session of the lock mode by employing a low-pass filter to filter out changes in the head pose of the user, and wherein the at least one processor is configured to loosen the low-pass filter over a second time period prior to ending the given session of the lock mode.

### Claim 2 (depends on 1)

The display apparatus of claim 1, wherein, when generating the output image frames of said sequence, the at least one processor is configured to generate corresponding pixel values to be displayed by the given pixel of the image renderer, based on a shifting sequence in which the light emanating from the given pixel is to be shifted to the plurality of positions.

### Claim 3 (depends on 1)

The display apparatus of claim 1, wherein the at least one processor is configured to send a drive signal to the control circuit of the liquid-crystal device, to increase a number of positions, in the plurality of positions, to which the light emanating from the given pixel is to be shifted during the given session of the lock mode.

### Claim 4 (depends on 3)

The display apparatus of claim 3, wherein the plurality of positions of the given pixel form an N×N array, wherein the number of positions are to be increased by increasing a value of N by at least one for the given session of the lock mode.

### Claim 5 (depends on 3)

The display apparatus of claim 3, further comprising gaze-tracking means, wherein the at least one processor is configured to:
process gaze-tracking data, collected by the gaze-tracking means, to determine a gaze location and a gaze velocity and/or acceleration of the user;
detect, based on the gaze location and the gaze velocity and/or acceleration of the user, whether or not the user's gaze is in smooth pursuit; and
send the drive signal to the control circuit to increase the number of positions during the given session of the lock mode, when it is detected that the user's gaze is in smooth pursuit.

### Claim 6 (depends on 1)

The display apparatus of claim 1, wherein the at least one processor is configured to select the given head pose of the user for the given session of the lock mode by employing a low-pass filter to filter out changes in the head pose of the user.

### Claim 7 (depends on 6)

The display apparatus of claim 6, wherein the at least one processor is configured to tighten the low-pass filter over a first time period at a beginning of the given session of the lock mode.

### Claim 8 (depends on 1)

The display apparatus of claim 1, wherein, when generating a given output image frame, the at least one processor is configured to:
determine a first region and a second region within the given output image frame, the second region surrounding the first region; and
generate pixel values for the first region and the second region in a manner that an angular resolution of the first region is higher than an angular resolution of the second region.

### Claim 9

A method of displaying via a display apparatus comprising pose-tracking means, an image renderer per eye, and a liquid-crystal device comprising a liquid-crystal structure and a control circuit, wherein the liquid-crystal structure is arranged in front of an image-rendering surface of the image renderer, the method comprising:
processing pose-tracking data, collected by the pose-tracking means, to determine a head pose of a user;
detecting whether or not a rate at which the head pose of the user is changing is below a predefined threshold rate;
when it is detected that the rate is below the predefined threshold rate, switching on a lock mode of the display apparatus, selecting a given head pose of the user for a given session of the lock mode, and generating a sequence of output image frames according to the given head pose during the given session of the lock mode;
when it is detected that the rate is not below the predefined threshold rate, generating the sequence of output image frames according to corresponding head poses of the user as determined from the pose-tracking data; and
displaying the sequence of output image frames via the image renderer, whilst controlling the liquid-crystal structure, via the control circuit, to shift light emanating from a given pixel of the image renderer to a plurality of positions in a sequential and repeated manner, wherein a shift in the light emanating from the given pixel of the image renderer to the plurality of positions causes a resolution of the output image frames to appear higher than a display resolution of the image renderer;
detecting, after the lock mode is switched on, whether or not the rate at which the head pose of the user is changing exceeds the predefined threshold rate; and
when it is detected, after the lock mode is switched on, that the rate exceeds the predefined threshold rate, switching off the lock mode of the display apparatus,
wherein the given head pose of the user is selected for the given session of the lock mode by employing a low-pass filter to filter out changes in the head pose of the user, the method further comprising loosening the low-pass filter over a second time period prior to ending the given session of the lock mode.

### Claim 10 (depends on 9)

The method of claim 9, wherein the step of generating the output image frames of said sequence comprises generating corresponding pixel values to be displayed by the given pixel of the image renderer, based on a shifting sequence in which the light emanating from the given pixel is to be shifted to the plurality of positions.

### Claim 11 (depends on 9)

The method of claim 9, further comprising sending a drive signal to the control circuit of the liquid-crystal device, to increase a number of positions, in the plurality of positions, to which the light emanating from the given pixel is to be shifted during the given session of the lock mode.

### Claim 12 (depends on 11)

The method of claim 11, wherein the plurality of positions of the given pixel form an N×N array, wherein the number of positions are increased by increasing a value of N by at least one for the given session of the lock mode.

### Claim 13 (depends on 11)

The method of claim 11, wherein the display apparatus further comprises gaze-tracking means, the method further comprising:
processing gaze-tracking data, collected by the gaze-tracking means, to determine a gaze location and a gaze velocity and/or acceleration of the user;
detecting, based on the gaze location and the gaze velocity and/or acceleration of the user, whether or not the user's gaze is in smooth pursuit; and
sending the drive signal to the control circuit to increase the number of positions during the given session of the lock mode, when it is detected that the user's gaze is in smooth pursuit.

### Claim 14 (depends on 9)

The method of claim 9, wherein the step of selecting the given head pose of the user for the given session of the lock mode is performed by employing a low-pass filter to filter out changes in the head pose of the user.

### Claim 15 (depends on 14)

The method of claim 14, further comprising tightening the low-pass filter over a first time period at a beginning of the given session of the lock mode.

### Claim 16 (depends on 9)

The method of claim 9, wherein the step of generating a given output image frame comprises:
determining a first region and a second region within the given output image frame, the second region surrounding the first region; and
generating pixel values for the first region and the second region in a manner that an angular resolution of the first region is higher than an angular resolution of the second region.