# US 2021/0241676 A1 · Modifying rendered image data based on ambient light from a physical environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241676 A1 - Document ID: US20210241676A1 - Status: Published application - Kind: A1 - Application number: 17/158,402 - Filed: 2021-01-26 - Published: 2021-08-05 - Earliest claimed: 2020-02-03 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Siddharth Shankar Hazra; Ralf Habel; Anselm Grundhoefer; Aleksandr M. Movshovich; Moinul Khan; Marwan Hussein - Web page: https://patentgrep.com/patent/US20210241676A1 ## CPC - G09G 5/10: Intensity circuits - G02B 27/017: {Head mounted} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G09G 5/02: characterised by the way in which colour is displayed {(details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607)} - G09G 5/363: {Graphics controllers} - G01J 1/4204: {with determination of ambient light (solar light G01J2001/4266)} - G02B 27/0172: {characterised by optical features} - G02B 2027/0118: {comprising devices for improving the contrast of the display / brillance control visibility} - G09G 3/22: using controlled light sources - G09G 2360/06: Use of more than one graphics processor to process data before displaying to one or more screens - G09G 2360/14: Detecting light within display terminals, e.g. using a single or a plurality of photosensors - G09G 2360/144: the light being ambient light - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) ## Abstract A method includes obtaining rendered image data that includes a representation of an object for display using a see-through display. The see-through display permits ambient light from a physical environment through the see-through display. The method includes sensing a plurality of light superposition characteristic values associated with the ambient light that quantifies the ambient light. The method includes determining a plurality of display correction values associated with the electronic device based on the plurality of light superposition characteristic values and predetermined display characteristics of the representation of the object. The method includes generating, from the rendered image data, display data for the see-through display in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the representation of the object within a performance threshold. ## Claims ### Claim 1 A method comprising: at an electronic device including one or more processors, a non-transitory memory, and a see-through display: obtaining rendered image data that includes a representation of an object for display using the see-through display, wherein the see-through display permits ambient light from a physical environment through the see-through display; sensing a plurality of light superposition characteristic values associated with the ambient light from the physical environment, wherein the plurality of light superposition characteristic values quantifies the ambient light arriving at the see-through display; determining a plurality of display correction values associated with the electronic device based on the plurality of light superposition characteristic values and predetermined display characteristics of the representation of the object; and generating, from the rendered image data, display data for the see-through display in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the representation of the object within a performance threshold. ### Claim 2 (depends on 1) The method of claim 1, further comprising: transmitting, via the one or more processors, the display data to the see-through display; and displaying the display data on the see-through display. ### Claim 3 (depends on 1) The method of claim 1, wherein sensing the plurality of light superposition characteristic values is performed by an image sensor. ### Claim 4 (depends on 1) The method of claim 1, wherein sensing the plurality of light superposition characteristic values is performed by an ambient light sensor. ### Claim 5 (depends on 1) The method of claim 1, wherein sensing the plurality of light superposition characteristic values includes determining scene content information associated with the physical environment. ### Claim 6 (depends on 1) The method of claim 1, wherein the plurality of display correction values includes grid data. ### Claim 7 (depends on 6) The method of claim 6, further comprising determining interpolated grid values by interpolating the grid data, wherein generating the display data includes adjusting the rendered image data according to the interpolated grid values. ### Claim 8 (depends on 1) The method of claim 1, wherein the plurality of display correction values includes a combination of global luminance values associated with the physical environment and chromaticity values associated with the physical environment. ### Claim 9 (depends on 1) The method of claim 1, wherein the plurality of display correction values includes a combination of local luminance values associated with the physical environment and chromaticity values associated with the physical environment. ### Claim 10 (depends on 1) The method of claim 1, wherein the plurality of display correction values includes a combination of spatial luminance values associated with the physical environment and chromaticity values associated with the physical environment. ### Claim 11 (depends on 1) The method of claim 1, wherein the display data is generated as a further function of scene content information associated with the rendered image data. ### Claim 12 A system comprising: a see-through display, wherein the see-through display permits ambient light from a physical environment through the see-through display; a sensor subsystem to sense a plurality of light superposition characteristic values associated with the ambient light from the physical environment, wherein the plurality of light superposition characteristic values quantifies the ambient light arriving at the see-through display; a correction values generator to determine a plurality of display correction values associated with the electronic device based on the plurality of light superposition characteristic values and predetermined display characteristics of the representation of the object; and a display pipeline to: obtain rendered image data that includes a representation of an object for display using the see-through display, and generate, from the rendered image data, display data for the see-through display in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the representation of the object within a performance threshold. ### Claim 13 (depends on 12) The system of claim 12, further comprising a head and body pose identifier to: obtain head and body pose data, and provide the head and body pose data to the display pipeline in order to generate the display data. ### Claim 14 (depends on 12) The system of claim 12, wherein the sensor subsystem includes an image sensor or an ambient light sensor. ### Claim 15 (depends on 12) The system of claim 12, wherein the correction values generator includes a grid data generator to generate grid data, wherein the plurality of display correction values includes the grid data. ### Claim 16 (depends on 15) The system of claim 15, wherein the display pipeline includes a grid data interpolator to interpolate the grid data, and wherein the display pipeline generates the display data as a function of the interpolated grid data. ### Claim 17 (depends on 12) The system of claim 12, wherein the correction values generator includes a color correction manager and a tint manager to collectively determine a combination of global luminance values associated with the physical environment and chromaticity values associated with the physical environment, wherein the combination of the global luminance values and the chromaticity values are included in the plurality of display correction values. ### Claim 18 (depends on 12) The system of claim 12, wherein the correction values generator includes a color correction manager and a tint manager to collectively determine a combination of local luminance values associated with the physical environment and chromaticity values associated with the physical environment, wherein the combination of the local luminance values and the chromaticity values are included in the plurality of display correction values. ### Claim 19 (depends on 12) The system of claim 12, wherein the correction values generator includes a color correction manager and a tint manager to collectively determine a combination of spatial luminance values associated with the physical environment and chromaticity values associated with the physical environment, wherein the combination of the spatial luminance values and the chromaticity values are included in the plurality of display correction values. ### Claim 20 An electronic device comprising: one or more processors; a non-transitory memory; a see-through display; and one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for: obtaining rendered image data that includes a representation of an object for display using the see-through display, wherein the see-through display permits ambient light from a physical environment through the see-through display; sensing a plurality of light superposition characteristic values associated with the ambient light from the physical environment, wherein the plurality of light superposition characteristic values quantifies the ambient light arriving at the see-through display; determining a plurality of display correction values associated with the electronic device based on the plurality of light superposition characteristic values and predetermined display characteristics of the representation of the object; and generating, from the rendered image data, display data for the see-through display in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the representation of the object within a performance threshold.