# US 10,373,294 B1 · Rendering composite content on a head-mounted display including a high resolution inset > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,294 B1 - Document ID: US10373294B1 - Status: Granted - Kind: B1 - Application number: 15/947,846 - Filed: 2018-04-08 - Granted: 2019-08-06 - Earliest claimed: 2016-10-04 - Assignees: Facebook Technologies, LLC - Applicants: Facebook Technologies, LLC - Inventors: Austin Wilson; Douglas Robert Lanman; Nicholas Daniel Trail; Scott Charles McEldowney; Stephen James McNally; Yusufu Njoni Bamaxam Sulai - Web page: https://patentgrep.com/patent/US10373294B1 ## CPC - G06T 3/4092: Image resolution transcoding, e.g. by using client-server architectures - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - G06F 3/012: {Head tracking input arrangements} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 7/12: Edge-based segmentation - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 11/00: Two-dimensional [2D] image generation - G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T) - G02B 2027/0132: {comprising binocular systems} - G02B 2027/0134: {of stereoscopic type} - G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display} - G09G 3/002: {to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT} - G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract A head-mounted display (HMD) divides an image into a high resolution (HR) inset portion at a first resolution, a peripheral portion, and a transitional portion. The peripheral portion is downsampled to a second resolution that is less than the first resolution. The transitional portion is blended such that there is a smooth change in resolution that corresponds to a change in resolution between a fovea region and a non-fovea region of a retina. An inset region is generated using the HR inset portion and the blended transitional portion, and a background region is generated using the downsampled peripheral portion. The inset region is provided to a HR inset display, and the background region is provided to a peripheral display. An optics block combines the displayed inset region with the displayed background region to generate composite content. ## Claims ### Claim 1 A method comprising: dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the HR inset portion is at a first resolution, and the transitional portion has an inner boundary between the transitional portion and the HR inset portion and an outer boundary between the transitional portion and the peripheral portion, wherein the peripheral portion is at a second resolution that is lower than the first resolution; blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution; generating an inset region using the HR inset portion of the image and the blended transitional portion of the image; and providing, for display, the inset region and a background region to a display. ### Claim 2 (depends on 1) The method of claim 1, wherein dividing the image into the HR inset portion, the peripheral portion, and the transitional portion between the HR inset portion and the peripheral portion, further comprises: applying a first fading function to the image that causes the inset portion and the transitional portion to fade to black, the faded image being the peripheral portion; and applying a second fading function to some of the transitional portion of the image that surrounds the HR inset. ### Claim 3 (depends on 1) The method of claim 1, wherein dividing the image into the HR inset portion, the peripheral portion, and the transitional portion between the HR inset portion and the peripheral portion is based in part on an eye orientation of a user of a head mounted display (HMD). ### Claim 4 (depends on 1) The method of claim 1, wherein blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, comprises: applying a blurring function that blurs a portion of the transitional portion of the image. ### Claim 5 (depends on 1) The method of claim 1, wherein the background region is displayed using the display, and an optical anti-aliasing filter configured to optically blur some of the background region. ### Claim 6 (depends on 1) The method of claim 1, wherein the background region passes through an optical anti-aliasing filter before being combined with the inset portion that is displayed using the HR inset display, the optical anti-aliasing filter optically blurring some of the background region. ### Claim 7 (depends on 1) The method of claim 1, further comprising: calibrating color for the inset portion and the transitional portion of the display such that the display is within a threshold range of color values. ### Claim 8 (depends on 1) The method of claim 1, further comprising: calibrating luminance for the display such that it is within a threshold range of luminance values. ### Claim 9 (depends on 1) The method of claim 1 further including an optics block configured to combine the inset region and the background region. ### Claim 10 A non-transitory computer-readable storage medium storing executable computer program instructions, the instructions executable to perform steps comprising: dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the HR inset portion is at a first resolution, and the transitional portion has an inner boundary between the transitional portion and the HR inset portion and an outer boundary between the transitional portion and the peripheral portion, wherein the peripheral portion is at a second resolution that is lower than the first resolution; blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution; generating an inset region using the HR inset portion of the image and the blended transitional portion of the image; and providing, for display, the inset region and a background region to a display. ### Claim 11 (depends on 10) The computer readable medium of claim 10, wherein dividing the image into the HR inset portion, the peripheral portion, and the transitional portion between the HR inset portion and the peripheral portion, further comprises: applying a first fading function to the image that causes the inset portion and the transitional portion to fade to black, the faded image being the peripheral portion; and applying a second fading function to some of the transitional portion of the image that surrounds the HR inset. ### Claim 12 (depends on 10) The computer readable medium of claim 10, wherein dividing the image into the HR inset portion, the peripheral portion, and the transitional portion between the HR inset portion and the peripheral portion is based in part on an eye orientation of a user of a head mounted display (HMD). ### Claim 13 (depends on 10) The computer readable medium of claim 10, wherein blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, comprises: applying a blurring function that blurs a portion of the transitional portion of the image. ### Claim 14 (depends on 10) The computer readable medium of claim 10, wherein the background region is displayed using the display, and an optical anti-aliasing filter configured to optically blur some of the background region. ### Claim 15 (depends on 10) The computer readable medium of claim 10, wherein the background region passes through an optical anti-aliasing filter before being combined with the inset portion that is displayed using the HR inset display, the optical anti-aliasing filter optically blurring some of the background region. ### Claim 16 (depends on 10) The computer readable medium of claim 10, further comprising: calibrating color for the inset portion and the transitional portion of the display such that the display is within a threshold range of color values. ### Claim 17 (depends on 10) The computer readable medium of claim 10, further comprising: calibrating luminance for the display such that it is within a threshold range of luminance values. ### Claim 18 (depends on 10) The computer readable medium of claim 10, further including an optics block configured to combine the inset region and the background region. ### Claim 19 A head-mounted display (HMD) comprising: a controller configured to: divide an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the HR inset portion is at a first resolution, and the transitional portion has an inner boundary between the transitional portion and the HR inset portion and an outer boundary between the transitional portion and the peripheral portion, wherein the peripheral portion is at a second resolution that is lower than the first resolution, blend the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, generate an inset region using the HR inset portion of the image and the blended transitional portion of the image, and provide, for display, the inset region and a background region to a display; and an optics block configured to: combine the inset region and the background region to create composite content, wherein the inset region is inset into the background region, and direct the composite content to an exit pupil of the HMD corresponding to a location of an eye of a user of the HMD. ### Claim 20 (depends on 19) The HMD of claim 19, wherein the controller is further configured to: apply a first fading function to the image that causes the inset portion and the transitional portion to fade to black, the faded image being the peripheral portion; and apply a second fading function to some of the transitional portion of the image that surrounds the HR inset.