# US 11,443,676 B1 · Increasing resolution and luminance of a display > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,676 B1 - Document ID: US11443676B1 - Status: Granted - Kind: B1 - Application number: 17/537,908 - Filed: 2021-11-30 - Granted: 2022-09-13 - Earliest claimed: 2021-11-29 - Assignees: Unity Technologies SF - Applicants: Unity Technologies SF - Inventors: Joseph W. Marks; Luca Fascione; Kimball D. Thurston, III; Millie Maier; Kenneth Gimpelson; Dejan Momcilovic; Keith F. Miller; Peter M. Hillman; Jonathan S. Swartz; Carter Bart Sullivan - Web page: https://patentgrep.com/patent/US11443676B1 ## CPC - G06F 3/1446: {display composed of modules, e.g. video walls} - G09G 3/346: {based on modulation of the reflection angle, e.g. micromirrors (micromirrors devices per se G02B26/0833)} - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition - G09G 3/007: {Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels} - G09G 3/20: for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix {no fixed position being assigned to or needed to be assigned to the individual characters or partial characters} - H04N 5/265: Mixing - H04N 5/76: Television signal recording - H04N 7/0117: {involving conversion of the spatial resolution of the incoming video signal (for graphics images G09G2340/0407)} - H04N 7/0135: {involving interpolation processes (interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59)} - H04N 9/64: Circuits for processing colour signals (H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84) - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/20212: Image combination - G09G 3/001: {using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background (slide projectors per se G03B23/00 = 42 HP)} - G09G 2300/026: Video wall, i.e. juxtaposition of a plurality of screens to create a display screen of bigger dimensions - G09G 2320/0233: Improving the luminance or brightness uniformity across the screen - G09G 2320/0686: with two or more screen areas displaying information with different brightness or colours - G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas - G09G 2340/0464: Positioning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The disclosed system modifies luminance of a display associated with a selective screen. The display provides a camera with an image having resolution higher than the resolution of the display by presenting multiple images while the selective screen enables light from different portions of the multiple images to reach the camera. The resulting luminance of the recorded image is lower than a combination of luminance values of the multiple images. The processor obtains a criterion indicating a property of the input image where image detail is unnecessary. The processor detects a region of the input image satisfying the criterion, and determines a region of the selective screen corresponding to the region of the input image. The processor increases the luminance of the display by disabling the region of the selective screen corresponding to the region of the input image. ## Claims ### Claim 1 A method to modify luminance of a display comprising: obtaining an input image to present on the display, wherein the display is associated with a selective screen, wherein the display is configured to provide a light receiver with an image having resolution higher than resolution of the display by presenting multiple images associated with the input image while the selective screen enables light from different portions of the multiple images to reach the light receiver, thereby causing the light receiver to form the image including the different portions of the multiple images, wherein luminance of the image is lower than a combination of luminance values of the multiple images; obtaining a criterion indicating a property of the input image where image detail is unnecessary; detecting a region of the input image satisfying the criterion; determining a region of the selective screen corresponding to the region of the input image; and increasing the luminance of the display by disabling the region of the selective screen corresponding to the region of the input image, thereby decreasing resolution of a region of the display corresponding to the region of the input image. ### Claim 2 (depends on 1) The method of claim 1, comprising: dividing the region of the selective screen into a first region and a second region, wherein the second region provides a transition region between the second region and a remainder of the display; obtaining a first luminance associated with the first region, and a third luminance associated with the remainder of the display; and adjusting a second luminance associated with the second region according to an interpolation between the third luminance and the first luminance. ### Claim 3 (depends on 1) The method of claim 1, wherein the light receiver, the selective screen, and the display are synchronized in a manner causing the light receiver to perceive the multiple images as a single image. ### Claim 4 (depends on 1) The method of claim 1, comprising: obtaining the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detecting the region including: detecting the region of the input image having an original luminance above the threshold; detecting an edge in the image; determining whether the region of the input image is proximate to the edge in the image; upon determining that the region of the input image is proximate to the edge in the image, enabling the region of the selective screen corresponding to the region of the input image. ### Claim 5 (depends on 1) The method of claim 1, comprising: obtaining the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detecting the region including: detecting the region of the input image having an original luminance above the threshold; detecting an edge in the image; determining whether the region of the input image is proximate to the edge in the image; upon determining that the region of the input image is not proximate to the edge in the image, disabling the region of the selective screen corresponding to the region of the input image. ### Claim 6 (depends on 1) The method of claim 1, comprising: obtaining the criterion including obtaining a region of focus associated with the light receiver; and detecting the region including detecting the region of the input image outside the region of focus associated with the light receiver. ### Claim 7 (depends on 1) The method of claim 1, comprising: obtaining the criterion including obtaining an indication to detect an area associated with the display directly visible to the light receiver; and detecting the region including detecting the region of the input image outside of the area associated with the display directly visible to the light receiver. ### Claim 8 (depends on 1) The method of claim 1, comprising: obtaining the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detecting the region including detecting the region of the input image having an original luminance above the threshold. ### Claim 9 (depends on 1) The method of claim 1, comprising: operating the display at a predetermined frequency by causing the display to present a first image at a first time and a second image at a second time, wherein the display includes a first plurality of light transmitting elements defining the resolution of the display; increasing the resolution of the display by operating the selective screen at the predetermined frequency and causing a first portion of the first image to be shown at the first time, and a second portion of the second image to be shown at the second time, wherein the first portion of the first image and the second portion of the second image are different, wherein the predetermined frequency enables the light receiver to form the image based on the first portion of the first image, and the second portion of the second image, wherein the selective screen includes a second plurality of light transmitting elements defining resolution of the selective screen, wherein the selective screen is disposed between the first plurality of light transmitting elements and the light receiver; wherein a first light transmitting element in the first plurality of light transmitting elements corresponds to multiple light transmitting elements in the second plurality of light transmitting elements; causing a second light transmitting element among the multiple light transmitting elements to redirect light transmitted by the first light transmitting element in the first plurality of light transmitting elements; and causing a third light transmitting element among the multiple light transmitting elements to allow light transmitted by the first light transmitting element in the first plurality of light transmitting elements to reach the light receiver. ### Claim 10 (depends on 1) The method of claim 1, comprising: enabling light from a first portion of a first image among the multiple images to reach the light receiver when the display is presenting the first image; enabling light from a second portion of a second image among the multiple images to reach the light receiver when the display is presenting the second image, wherein the first portion of the first image and the second portion of the second image occupy different regions of the display; wherein the enabling comprises: changing a state of at least a portion of the selective screen at a predetermined frequency; aligning the portion of the selective screen with the first portion of the first image when the display is presenting the first image, and aligning the portion of the selective screen with the second portion of the second image when the display is presenting the second image, wherein the portion of the selective screen allows light from the display to reach the light receiver. ### Claim 11 At least one computer-readable storage medium, excluding transitory signals and carrying instructions, which, when executed by at least one data processor of a system, causes the system to: obtain an input image to present on a display, wherein the display is associated with a selective screen, wherein the display is configured to provide a light receiver with an image having resolution higher than resolution of the display by presenting multiple images associated with the input image while the selective screen enables light from different portions of the multiple images to reach the light receiver, thereby causing the light receiver to form the image including the different portions of the multiple images, wherein luminance of the image is lower than a combination of luminance values of the multiple images; obtain a criterion indicating a property of the input image where image detail is unnecessary; detect a region of the input image satisfying the criterion; determine a region of the selective screen corresponding to the region of the input image; and increase luminance of the display by disabling the region of the selective screen corresponding to the region of the input image, thereby decreasing resolution of a region of the display corresponding to the region of the input image. ### Claim 12 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: divide the region of the selective screen into a first region and a second region, wherein the second region provides a transition region between the second region and a remainder of the display; obtain a first luminance associated with the first region, and a third luminance associated with the remainder of the display; and adjust a second luminance associated with the second region according to an interpolation between the third luminance and the first luminance. ### Claim 13 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to synchronize the light receiver, the selective screen, and the display are in a manner causing the light receiver to perceive the multiple images as a single image. ### Claim 14 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: obtain the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detect the region including: detect the region of the input image having an original luminance above the threshold; detect an edge in the image; determine whether the region of the input image is proximate to the edge in the image; upon determining that the region of the input image is proximate to the edge in the image, enable the region of the selective screen corresponding to the region of the input image. ### Claim 15 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: obtain the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detect the region including: detect the region of the input image having an original luminance above the threshold; detect an edge in the image; determine whether the region of the input image is proximate to the edge in the image; upon determining that the region of the input image is not proximate to the edge in the image, disable the region of the selective screen corresponding to the region of the input image. ### Claim 16 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: obtain the criterion including obtaining a region of focus associated with the light receiver; and detect the region including detecting the region of the input image outside the region of focus associated with the light receiver. ### Claim 17 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: obtain the criterion including obtaining an indication to detect an area associated with the display directly visible to the light receiver; and detect the region including detecting the region of the input image outside of the area associated with the display directly visible to the light receiver. ### Claim 18 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: obtain the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detect the region including detecting the region of the input image having an original luminance above the threshold. ### Claim 19 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: operate the display at a predetermined frequency by causing the display to present a first image at a first time and a second image at a second time, wherein the display includes a first plurality of light transmitting elements defining the resolution of the display; increase the resolution of the display by operating the selective screen at the predetermined frequency and causing a first portion of the first image to be shown at the first time, and a second portion of the second image to be shown at the second time, wherein the first portion of the first image and the second portion of the second image are different, wherein the predetermined frequency enables the light receiver to form the image based on the first portion of the first image, and the second portion of the second image, wherein the selective screen includes a second plurality of light transmitting elements defining the resolution of the selective screen, wherein the selective screen is disposed between the first plurality of light transmitting elements and the light receiver; wherein a first light transmitting element in the first plurality of light transmitting elements corresponds to multiple light transmitting elements in the second plurality of light transmitting elements; cause a second light transmitting element among the multiple light transmitting elements to redirect light transmitted by the first light transmitting element in the first plurality of light transmitting elements; and cause a third light transmitting element among the multiple light transmitting elements to allow light transmitted by the first light transmitting element in the first plurality of light transmitting elements to reach the light receiver. ### Claim 20 (depends on 11) The computer-readable storage medium of claim 11, comprising instructions to: cause a light conductor associated with a first portion of the selective screen to guide a light beam blocked by the first portion of the selective screen to a second portion of the selective screen, thereby increasing brightness of light emanating from the display, wherein the first portion of the selective screen redirects light from the display, and wherein the second portion of the selective screen allows light from the display to reach the light receiver. ### Claim 21 A system comprising: a display; a selective screen disposed between the display and a light receiver; the display configured to provide the light receiver with an image having resolution higher than resolution of the display by presenting multiple images associated with the image while the selective screen enables light from different portions of the multiple images to reach the light receiver, thereby causing the light receiver to form the image including the different portions of the multiple images, wherein luminance of the image is lower than a combination of luminance values of the multiple images; at least one hardware processor; and at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to: obtain an input image to present on the display, obtain a criterion indicating a property of the input image where image detail is unnecessary; detect a region of the input image satisfying the criterion; determine a region of the selective screen corresponding to the region of the input image; and increase luminance of the display by disabling the region of the selective screen corresponding to the region of the input image, thereby decreasing resolution of a region of the display corresponding to the region of the input image. ### Claim 22 (depends on 21) The system of claim 21, comprising instructions to: divide the region of the selective screen into a first region and a second region, wherein the second region provides a transition region between the second region and a remainder of the display; obtain a first luminance associated with the first region, and third luminance associated with the remainder of the display; and adjust a second luminance associated with the second region according to an interpolation between the third luminance and the first luminance. ### Claim 23 (depends on 21) The system of claim 21, comprising instructions to synchronize the light receiver, the selective screen, and the display in a manner causing the light receiver to perceive the multiple images as a single image. ### Claim 24 (depends on 21) The system of claim 21, comprising instructions to: obtain the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detect the region including: detect the region of the input image having an original luminance above the threshold; detect an edge in the image; determine whether the region of the input image is proximate to the edge in the image; upon determining that the region of the input image is proximate to the edge in the image, enable the region of the selective screen corresponding to the region of the input image. ### Claim 25 (depends on 21) The system of claim 21, comprising instructions to: obtain the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detect the region including: detect the region of the input image having an original luminance above the threshold; detect an edge in the image; determine whether the region of the input image is proximate to the edge in the image; upon determining that the region of the input image is not proximate to the edge in the image, disable the region of the selective screen corresponding to the region of the input image. ### Claim 26 (depends on 21) The system of claim 21, comprising instructions to: obtain the criterion including obtaining a region of focus associated with the light receiver; and detect the region including detecting the region of the input image outside the region of focus associated with the light receiver. ### Claim 27 (depends on 21) The system of claim 21, comprising instructions to: obtain the criterion including obtaining an indication to detect an area associated with the display directly visible to the light receiver; and detect the region including detecting the region of the input image outside of the area associated with the display directly visible to the light receiver. ### Claim 28 (depends on 21) The system of claim 21, comprising instructions to: obtain the criterion including obtaining a threshold between a minimum luminance of the display and a maximum luminance of the display; and detect the region including detecting the region of the input image having an original luminance above the threshold. ### Claim 29 (depends on 21) The system of claim 21, comprising: the display including a first plurality of light transmitting elements arranged in a first pattern defining the resolution of the display, wherein the display is configured to operate at a predetermined frequency by displaying a first image at a first time and a second image at a second time; the selective screen including a second plurality of light transmitting elements arranged in a second pattern defining resolution of the selective screen, wherein the selective screen is disposed between the first plurality of light transmitting elements and the light receiver, wherein a first light transmitting element in the first plurality of light transmitting elements corresponds to multiple light transmitting elements in the second plurality of light transmitting elements; a second light transmitting element among the multiple light transmitting elements configured to redirect a first portion of light transmitted by the first light transmitting element in the first plurality of light transmitting elements; a third light transmitting element among the multiple light transmitting elements configured to allow a second portion of light transmitted by the first light transmitting element in the first plurality of light transmitting elements to reach the light receiver; and the selective screen configured to increase the resolution of the display by operating at the predetermined frequency and causing a first portion of the first image to be shown at the first time, and a second portion of the second image to be shown at the second time, wherein the first portion of the first image and the second portion of the second image are different, wherein the predetermined frequency enables the light receiver to form the image based on the first portion of the first image, and the second portion of the second image. ### Claim 30 (depends on 21) The system of claim 21, comprising instructions to: enable light from a first portion of a first image among the multiple images to reach the light receiver when the display is presenting the first image; enable light from a second portion of a second image among the multiple images to reach the light receiver when the display is presenting the second image, wherein the first portion of the first image and the second portion of the second image occupy different regions of the display; wherein the instructions to enable comprise instructions to: change a state of at least a portion of the selective screen at a predetermined frequency; align the portion of the selective screen with the first portion of the first image when the display is presenting the first image, and align the portion of the selective screen with the second portion of the second image when the display is presenting the second image, wherein the portion of the selective screen allows light from the display to reach the light receiver.