# US 10,373,592 B2 · Adaptive parameters in image regions based on eye tracking information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,592 B2 - Document ID: US10373592B2 - Status: Granted - Kind: B2 - Application number: 15/662,582 - Filed: 2017-07-28 - Published: 2018-02-01 - Granted: 2019-08-06 - Earliest claimed: 2016-08-01 - Assignees: Facebook Technologies, LLC - Applicants: Facebook Technologies, LLC - Inventors: Martin Henrik Tall; Javier San Agustin Lopez; Rasmus Dahl - Web page: https://patentgrep.com/patent/US10373592B2 ## CPC - G09G 5/37: Details of the operation on graphic patterns (G09G5/38 takes precedence) - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/147: using display panels - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - G09G 5/14: Display of multiple viewports - G09G 5/391: Resolution modifying circuits, e.g. variable screen formats - H04N 19/124: Quantisation - H04N 19/162: User input - H04N 19/167: Position within a video image, e.g. region of interest [ROI] - H04N 19/17: the unit being an image region, e.g. an object - G09G 2320/0693: Calibration of display systems - G09G 2340/04: Changes in size, position or resolution of an image - G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A display system divides a screen into regions and applies a different set of rendering/encoding parameters to each region. The system applies a first set of parameters to a first region that is being viewed by a fovea of an eye of a user. The system may also apply a second set of parameters to a second region that is being viewed by a parafovea of the eye, and apply a third set of parameters to a third region that is being viewed by the area of the eye outside of the parafovea. The first set of parameters are selected to yield relatively high image quality, while the second set of parameters are yield intermediate quality, and the third set of parameters yield lower quality. As a result, the second region and the third region can be rendered, encoded, and transmitted with less computing power and less bandwidth. ## Claims ### Claim 1 A method comprising: receiving eye tracking information comprising at least one image of an eye of a user viewing a screen of a display device; determining, based on the eye tracking information, a first screen region corresponding to a region of the screen, the first screen region containing a point of regard of the user, the point of regard representing a point on the screen at which the user is looking at the time the eye tracking information was captured; determining, based on the eye tracking information, a second screen region corresponding to a region of the screen that is separate from the first screen region; determining, based on the eye tracking information, a user's depth of focus; encoding an image for display on the screen, the image comprising an object with at least, a portion of the object to be displayed within the first screen region, the encoding comprising: encoding a first image region of the image based on a first set of encoding parameters, the first image region comprising a portion of the image to be displayed in the first screen region, encoding a second image region of the image based on a second set of encoding parameters, the second image region separate from the first image region and to be displayed in the second screen region, the second set of encoding parameters resulting in a lower quality than the first set of encoding parameters, and responsive to a difference between a depth of the object in the image and the user's depth of focus exceeding a threshold value, encoding the object, including the portion to be displayed within the first screen region, with the second set of encoding parameters; and transmitting the encoded image to the display device to be displayed on the screen. ### Claim 2 (depends on 1) The method of claim 1, wherein the first screen region corresponds to a region of the screen viewed by a fovea of the eye. ### Claim 3 (depends on 1) The method of claim 1, wherein the second screen region corresponds to a region of the screen viewed by a portion of the retina of the eye outside of the fovea. ### Claim 4 (depends on 1) The method of claim 1, wherein the first set of encoding parameters includes a first image resolution, the second set of encoding parameters includes a second image resolution, and the second image resolution is lower than the first image resolution. ### Claim 5 (depends on 4) The method of claim 4, wherein encoding the first image region comprises generating a first encoded image region, wherein encoding the second image region comprises generating a second encoded image region, and further comprising: generating a packed image comprising the first encoded image region and the second encoded image region. ### Claim 6 (depends on 1) The method of claim 1, wherein the first set of encoding parameters includes a first framerate, the second set of encoding parameters includes a second framerate, and the second framerate is lower than the first framerate. ### Claim 7 (depends on 1) The method of claim 1, further comprising: determining, based on the eye tracking information, a third screen region corresponding to a region of the screen that is separate from the first screen region and the second screen region, wherein encoding the image for display on the screen further comprises: encoding a third image region of the image based on a third set of encoding parameters, the third image region separate from the first image region and the second image region and to be displayed in the third screen region, the third set of encoding parameters resulting in a lower quality than the first set of encoding parameters and a higher quality than the second set of encoding parameters. ### Claim 8 (depends on 7) The method of claim 7, wherein the third screen region corresponds to a region of the screen viewed by a parafovea of the eye. ### Claim 9 (depends on 1) The method of claim 1, wherein the display device is a virtual reality headset. ### Claim 10 A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving eye tracking information comprising at least one image of an eye of a user viewing a screen of a display device; determining, based on the eye tracking information, a first screen region corresponding to a region of the screen, the first screen region containing a point of regard of the user, the point of regard representing a point on the screen at which the user is looking at the time the eye tracking information was captured; determining, based on the eye tracking information, a second screen region corresponding to a region of the screen that is separate from the first screen region; determining, based on the eye tracking information, a user's depth of focus, encoding an image for display on the screen, the image comprising an object with at least a portion of the object to be displayed within the first screen region, the encoding comprising: encoding a first image region of the image based on a first set of encoding parameters, the first image region comprising a portion of the image to be displayed in the first screen region, encoding a second image region of the image based on a second set of encoding parameters, the second image region separate from the first image region and to be displayed in the second screen region, the second set of encoding parameters resulting in a lower quality than the first set of encoding parameters, and responsive to a difference between a depth of the object in the image and the user's depth of focus exceeding a threshold value, encoding the object, including the portion to be displayed within the first screen region, with the second set of encoding parameters; and transmitting the encoded image to the display device to be displayed on the screen. ### Claim 11 (depends on 10) The computer-readable storage medium of claim 10, wherein the first screen region corresponds to a region of the screen viewed by a fovea of the eye. ### Claim 12 (depends on 10) The computer-readable storage medium of claim 10, wherein the second screen region corresponds to a region of the screen viewed by a portion of the retina of the eye outside of the fovea. ### Claim 13 (depends on 10) The computer-readable storage medium of claim 10, wherein the first set of encoding parameters includes a first image resolution, the second set of encoding parameters includes a second image resolution, and the second image resolution is lower than the first image resolution. ### Claim 14 (depends on 10) The computer-readable storage medium of claim 10, wherein encoding the first image region comprises generating a first encoded image region, wherein encoding the second image region comprises generating a second encoded image region, and further comprising: generating a packed image comprising the first encoded image region and the second encoded image region. ### Claim 15 (depends on 10) The computer-readable storage medium of claim 10, wherein the first set of encoding parameters includes a first framerate, the second set of encoding parameters includes a second framerate, and the second framerate is lower than the first framerate. ### Claim 16 (depends on 10) The computer-readable storage medium of claim 10, the operations further comprising: determining, based on the eye tracking information, a third screen region corresponding to a region of the screen that is separate from the first screen region and the second screen region, wherein encoding the image for display on the screen further comprises: encoding a third image region of the image based on a third set of encoding parameters, the third image region separate from the first image region and the second image region and to be displayed in the third screen region, the third set of encoding parameters resulting in a lower quality than the first set of encoding parameters and a higher quality than the second set of encoding parameters. ### Claim 17 (depends on 16) The computer-readable storage medium of claim 16, wherein the third screen region corresponds to a region of the screen viewed by a parafovea of the eye. ### Claim 18 (depends on 10) The computer-readable storage medium of claim 10, wherein the display device is a virtual reality headset. ### Claim 19 A system comprising: one or more processors; and a non-transitory computer-readable storage medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving eye tracking information comprising at least one image of an eye of a user viewing a screen of a display device, determining, based on the eye tracking information, a first screen region corresponding to a region of the screen, the first screen region containing a point of regard of the user, the point of regard representing a point on the screen at which the user is looking at the time the eye tracking information was captured, determining, based on the eye tracking information, a second screen region corresponding to a region of the screen that is separate from the first screen region, determining, based on the eye tracking information, a user's depth of focus, encoding an image for display on the screen, the image comprising an object with at least a portion of the object to be displayed within the first screen region, the encoding comprising: encoding a first image region of the image based on a first set of encoding parameters, the first image region comprising a portion of the image to be displayed in the first screen region, encoding a second image region of the image based on a second set of encoding parameters, the second image region separate from the first image region and to be displayed in the second screen region, the second set of encoding parameters resulting in a lower quality than the first set of encoding parameters, and responsive to a difference between a depth of the object in the image and the user's depth of focus exceeding a threshold value, encoding the object, including the portion to be displayed within the first screen region, with the second set of encoding parameters, and transmitting the encoded image to the display device to be displayed on the screen. ### Claim 20 (depends on 19) The system of claim 19, the operations further comprising: determining, based on the eye tracking information, a third screen region corresponding to a region of the screen that is separate from the first screen region and the second screen region, wherein encoding the image for display on the screen further comprises: encoding a third image region of the image based on a third set of encoding parameters, the third image region separate from the first image region and the second image region and to be displayed in the third screen region, the third set of encoding parameters resulting in a lower quality than the first set of encoding parameters and a higher quality than the second set of encoding parameters.