# US 10,504,207 B1 · Rendering composite content on a head-mounted display including a high resolution inset > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,207 B1 - Document ID: US10504207B1 - Status: Granted - Kind: B1 - Application number: 16/433,236 - Filed: 2019-06-06 - Granted: 2019-12-10 - Earliest claimed: 2016-10-04 - Assignees: Facebook Technologies, LLC - Applicants: Facebook Technologies, LLC - Inventors: Austin Wilson; Douglas Robert Lanman; Nicholas Daniel Trail; Scott Charles McEldowney; Stephen James McNally; Yusufu Njoni Bamaxam Sulai - Web page: https://patentgrep.com/patent/US10504207B1 ## CPC - G06T 3/4092: Image resolution transcoding, e.g. by using client-server architectures - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - G06F 3/012: {Head tracking input arrangements} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 7/12: Edge-based segmentation - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 11/00: Two-dimensional [2D] image generation - G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T) - G02B 2027/0132: {comprising binocular systems} - G02B 2027/0134: {of stereoscopic type} - G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display} - G09G 3/002: {to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT} - G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract A head-mounted display (HMD) divides an image into a high resolution (HR) inset portion at a first resolution, a peripheral portion, and a transitional portion. The peripheral portion is downsampled to a second resolution that is less than the first resolution. The transitional portion is blended such that there is a smooth change in resolution that corresponds to a change in resolution between a fovea region and a non-fovea region of a retina. An inset region is generated using the HR inset portion and the blended transitional portion, and a background region is generated using the downsampled peripheral portion. The inset region is provided to a HR inset display, and the background region is provided to a peripheral display. An optics block combines the displayed inset region with the displayed background region to generate composite content. ## Claims ### Claim 1 A device comprising: one or more controllers configured, in combination, to: generate an image with a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the peripheral portion is at a lower resolution than the HR inset portion, and the transitional portion is configured to conceal a change resolution between the HR inset portion and the peripheral portion; a display system coupled to the controller and configured to present the HR inset portion, the peripheral portion, and the transitional portion; and an optics block configured to: connect the HR inset portion to the peripheral portion using the transitional portion to present a multi-resolution composite of the image. ### Claim 2 (depends on 1) The device of claim 1, further comprising: an optical anti-aliasing filter that optically blurs a portion of the peripheral portion. ### Claim 3 (depends on 1) The device of claim 1, wherein the multi-resolution composite of the image has variable resolution that matches a resolution of a retina of a human eye. ### Claim 4 (depends on 1) The device of claim 1, wherein the transitional portion is blended such that there is a smooth change in resolution. ### Claim 5 (depends on 1) The device of claim 1, wherein the optics block includes a steering element that, responsive to instructions received from a controller of the one or more controllers, is configured to adjust a position of the HR inset portion in the multi-resolution composite of the image. ### Claim 6 (depends on 1) The device of claim 1, wherein the optics block includes a beam splitter that combines the HR inset portion, the peripheral portion, and the transitional portion to form the multi-resolution composite of the image. ### Claim 7 (depends on 6) The device of claim 6, wherein the beam splitter is within a ball structure. ### Claim 8 (depends on 7) The device of claim 7, wherein a peripheral display of the display system is a curved display that is coupled directly to a curved surface of the ball structure. ### Claim 9 (depends on 7) The device of claim 7, further comprising a fiber taper having a first surface and a second surface, and a peripheral display of the display system is a flat display that is coupled to the first surface of the fiber taper, and the second surface of the fiber taper is curved and is coupled to a curved surface of the ball structure. ### Claim 10 (depends on 1) The device of claim 1, wherein the display system comprises: a high resolution (HR) inset display that is configured to display the HR inset portion; and a peripheral display that is configured to display the peripheral portion and the transitional portion. ### Claim 11 (depends on 1) The device of claim 1, wherein the display system comprises: a high resolution (HR) inset display that is configured to display the HR inset portion and the transitional portion; and a peripheral display that is configured to display the peripheral portion. ### Claim 12 (depends on 1) The device of claim 1, wherein a resolution of the HR inset portion corresponds to a resolution of a fovea of a human eye. ### Claim 13 (depends on 1) The device of claim 1, wherein a resolution of the HR inset portion is lower than a corresponding resolution of a fovea of a human eye, and the display system comprises: a high resolution (HR) inset display that is a large inset display that is configured to display the HR inset portion, wherein the optics block is configured to de-magnify the HR inset portion displayed by the large inset display to increase its resolution. ### Claim 14 (depends on 1) The device of claim 1, wherein the device is a head-mounted display. ### Claim 15 (depends on 1) The device of claim 1, further comprising: an eye tracking unit that is configured to detect movement of an eye of a user, and wherein at least one of the one or more controllers is further configured to: receive, from the eye tracking unit, information related to movement of the eye, determine, based on the information, a gaze direction of the user, and send steering instructions to a steering element of the optics block to adjust a position of the HR inset portion such that the inset region is centered on the determined gaze direction. ### Claim 16 (depends on 1) The device of claim 1, wherein at least one controller of the one or more controllers is configured to: apply a blurring function that blurs a portion of the transitional portion of the image. ### Claim 17 (depends on 1) The device of claim 1, wherein the display system comprises: a high resolution (HR) inset display that is configured to display the HR inset portion; and a peripheral display that is configured to display the peripheral portion, and wherein at least one controller of the one or more controllers is configured to: calibrate color for the peripheral display and the HR inset display such that they both are within a threshold range of color values. ### Claim 18 (depends on 1) The device of claim 1, wherein the display system comprises: a high resolution (HR) inset display that is configured to display the HR inset portion; and a peripheral display that is configured to display the peripheral portion, and wherein at least one controller of the one or more controllers is configured to: calibrate luminance for the peripheral display and the HR inset display such that they both are within a threshold range of luminance values. ### Claim 19 (depends on 1) The device of claim 1, wherein at least one controller of the one or more controllers is configured to: downsample a portion of the image to first resolution that is lower than a resolution of the HR inset portion to generate the peripheral portion. ### Claim 20 (depends on 19) The device of claim 19, wherein the first resolution corresponds to a resolution of a non-fovea region of a human eye.