# US 11,443,718 B2 · Circuit device, electronic apparatus, and mobile body for generating latency compensated display

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,718 B2
- Document ID: US11443718B2
- Status: Granted
- Kind: B2
- Application number: 17/137,732
- Filed: 2020-12-30
- Published: 2021-07-01
- Granted: 2022-09-13
- Earliest claimed: 2019-12-31
- Assignees: Seiko Epson Corporation
- Applicants: Seiko Epson Corporation
- Inventors: Wittmeir Manfred; Yosuke Tanikawa; Kumar anandabairavasamy Anand; Tetsuo Kawamoto
- Web page: https://patentgrep.com/patent/US11443718B2

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G09G 5/37: Details of the operation on graphic patterns (G09G5/38 takes precedence)
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- B60R 2300/10: characterised by the type of camera system used
- B60R 2300/205: using a head-up display
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G09G 2354/00: Aspects of interface with display user
- G09G 2380/10: Automotive applications
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

A circuit device (160) includes an interface (161), a parameter computation unit (140), and a warp processing unit (130). A rendering image and tracking information are input to the interface (161). The parameter computation unit (140) computes a latency compensation parameter for compensating a latency including a rendering processing latency of a rendering image based on tracking information. The warp processing unit (130) performs latency compensation processing for compensating the position of a virtual object in a display region based on the latency compensation parameter.

## Claims

### Claim 1

A circuit device that performs display control of a head up display that displays, in a display region, a virtual object corresponding to a real object in a real space, the circuit device comprising:
an interface into which a rendering image including the virtual object and tracking information are input, the tracking information being obtained by a sensor and being at least one of first information of a mobile body in which the head up display is mounted, second information of a viewer of the head up display, and third information of the real object, the tracking information including first tracking information and second tracking information,
wherein the rendering image and the first tracking information are obtained at a first timing,
the second tracking information is obtained at a second timing, and
the second timing is later than the first timing, and the second timing is prior to a third timing at which a display image is displayed in the display region;
a parameter computation controller configured to compute a latency compensation parameter for compensating a latency including a rendering processing latency of the rendering image based on the first and second tracking information; and
a warp processing controller configured to generate the display image to be displayed in the display region by performing warp processing in which coordinate transformation is performed on the rendering image according to a property of the display region,
wherein the warp processing controller performs, in the warp processing, latency compensation processing for compensating a position of the virtual object in the display region based on the latency compensation parameter.

### Claim 2 (depends on 1)

The circuit device according to claim 1,
wherein the parameter computation controller is configured to output a corrected warp parameter obtained by correcting a warp parameter of the coordinate transformation according to the property of the display region, based on the latency compensation parameter, and
the warp processing controller is configured to perform the latency compensation processing by performing the warp processing using the corrected warp parameter.

### Claim 3 (depends on 2)

The circuit device according to claim 2,
wherein the tracking information indicates a displacement including at least one of a first axis displacement and a second axis displacement that are perpendicular to a running direction of the mobile body, and
the parameter computation controller is configured to output the corrected warp parameter obtained by correcting a position error of the virtual object incurred by the displacement.

### Claim 4 (depends on 2)

The circuit device according to claim 2,
wherein the tracking information indicates a displacement including at least one of a first axis displacement and a second axis displacement that are perpendicular to a running direction of the mobile body, and the displacement in an up-down direction of the mobile body is the second axis displacement,
the tracking information is information indicating the second axis displacement of the head up display or the viewer due to a pitch displacement of the mobile body, and
the parameter computation controller is configured to output the corrected warp parameter obtained by correcting a position error of the virtual object incurred by the second axis displacement.

### Claim 5 (depends on 2)

The circuit device according to claim 2, further comprising:
a memory configured to store a table in which a coordinate in the display image is associated with a movement amount indicating a movement destination of the coordinate, as the warp parameter,
wherein the parameter computation controller is configured to shift the movement amount in the table based on the latency compensation parameter and output the table in which the movement amount has been shifted as the corrected warp parameter.

### Claim 6 (depends on 2)

The circuit device according to claim 2, further comprising:
a memory configured to store a parameter of a transformation formula for transforming a coordinate in the display image to another coordinate in a movement destination, as the warp parameter,
wherein the parameter computation controller is configured to shift the another coordinate in the movement destination based on the latency compensation parameter and output the parameter of the transformation formula for obtaining the shifted another coordinate in the movement destination, as the corrected warp parameter.

### Claim 7 (depends on 1)

The circuit device according to claim 1,
wherein the warp processing controller is configured to perform, in the warp processing, the latency compensation processing in which at least one of rotation and scale of the virtual object in the display region is compensated based on the latency compensation parameter.

### Claim 8 (depends on 1)

The circuit device according to claim 1,
wherein rendering tracking information generated by tracking processing based on an output signal of a rendering sensor, the rendering image generated based on the rendering tracking information, and latency compensation tracking information that is an output signal of a latency compensation sensor different from the rendering sensor are input into the interface, and
the parameter computation controller is configured to compute the latency compensation parameter based on the rendering tracking information and the latency compensation tracking information.

### Claim 9 (depends on 1)

An electronic apparatus comprising the circuit device according to claim 1.

### Claim 10 (depends on 1)

A mobile body comprising the circuit device according to claim 1.