# US 11,122,243 B2 · Method and apparatus for remapping pixel locations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,243 B2 - Document ID: US11122243B2 - Status: Granted - Kind: B2 - Application number: 16/688,795 - Filed: 2019-11-19 - Published: 2020-06-04 - Granted: 2021-09-14 - Earliest claimed: 2018-11-19 - Assignees: Flightsafety International Inc. - Applicants: FlightSafety International - Inventors: David Michael Emig; Brock Allen Ebert; Terry Richard Linn - Web page: https://patentgrep.com/patent/US11122243B2 ## CPC - H04N 9/317: {Convergence or focusing systems (electronic adjustment of convergence H04N9/3185; convergence or focusing arrangements for cathode ray tubes H04N9/28; means for automatic focusing of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/53)} - G02B 13/08: Anamorphotic objectives - G02B 27/0911: {Anamorphotic systems} - G03B 21/10: Projectors with built-in or built-on screen - G03B 37/06: involving anamorphosis (G03B37/02, G03B37/04 take precedence) - G06T 3/18: Image warping, e.g. rearranging pixels individually - G09B 9/32: by projected image ({G09B9/307,} G09B9/36 take precedence) - H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)} - H04N 9/3188: {Scale or resolution adjustment (scaling in general G06T3/40; resolution modifying circuits for displays in general G09G5/391)} - G02B 26/0825: {the reflecting element being a flexible sheet or membrane, e.g. for varying the focus (flexible mirrors for cosmetic use A45D42/24)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract An apparatus and method for optically remapping projected pixels to maximize the utilization and to optimize the distribution of remapped projection pixels to achieve optimal visual performance (generally uniform resolution and luminance). A device interposed between a projector and an imaging surface for optically remapping projected pixel locations with minimal aberration. When this device is interposed between a projector and an imaging surface, it changes the terminal location of each focused pixel such that it maximally coincides with the imaging surface, which is often a surface of complex curvature and very different from the native focal surface of the projector. One implementation of the technology includes a device that uses multiple optical surfaces. ## Claims ### Claim 1 A method for optically remapping projected pixel locations, comprising: defining a design eye point (DEP), a field of view corresponding to the DEP, and an image resolution for a plurality of portions of the field of view when viewing a projected image; generating a set of viewing vectors from the DEP representative of a projection of the field of view onto an image formation surface, whereby the viewing vectors intersect a curved screen whereupon the projected image will be filmed; determining, for each of the plurality of portions of the field of view, screen intersection locations to which projector pixels will be substantially mapped; determining an optimal density of pixels within each of the plurality of portions of field of view based on the image resolution for a given portion of the field of view; interposing an optical device within an optical path between an image source formation point within a projector and a projected image formation point upon the curved screen, wherein the optical device includes at least a refractive element and a reflective element positioned in optical subsequence, where focal surfaces of the refractive and reflective elements each have a longest dimension, and said focal surface longest dimensions are oriented substantially orthogonal one with respect to the other; refracting and reflecting the projected image with the optical device; adjusting the optical effect of the optical device in order to support one or more of various projectors, various projector configurations, various display systems, various display system configurations, variations in display system components, variation in observer location, variation in field of view, and variation in image resolution; and astigmatizing the projected image's focus by way of refracting with the refractive element of the optical device and reflecting the astigmatized refracted image with the reflective element of the optical device to supplementally optically redirect light from the astigmatized refracted image to the screen intersection locations with optimal pixel density, whereby, the optical device thereby optimizes the utilization of the projector's pixels by satisfying the image resolution for the given portion of the field of view with a minimum of projector pixels. ### Claim 2 (depends on 1) The method for optically remapping as recited in claim 1, comprising: interposing a collimating mirror between the DEP and the curved screen whereby a viewer at the DEP is viewing the curved screen through the collimating mirror. ### Claim 3 (depends on 2) The method for optically remapping as recited in claim 2, where the viewing vectors reflect off of the collimating mirror before intersecting the curved screen. ### Claim 4 (depends on 1) The method for optically remapping as recited in claim 1, where a horizontal field of view exceeds 180 degrees, the image resolution is less than 6.1 arcminutes per optical line pair, and one or more of the reflective element and the refractive element is adjustable. ### Claim 5 (depends on 1) The method for optically remapping as recited in claim 1, where substantially orthogonal means between approximately 70 degrees and 110 degrees one with respect to the other. ### Claim 6 (depends on 1) The method for optically remapping as recited in claim 1, where the projector includes a projection lens through which the projector projects the image. ### Claim 7 (depends on 6) The method for optically remapping as recited in claim 6, where the refractive element is positioned along the optical path at one of before, after and within the projection lens. ### Claim 8 (depends on 1) The method for optically remapping as recited in claim 1, further comprising: determining a compromise DEP for each of the plurality of portions of the field of view when there are two or more observers. ### Claim 9 (depends on 1) The method for optically remapping as recited in claim 1, where the curved screen is a section of one of an ellipsoid, a toroid, and a minor variation from the ellipsoid or toroid, where said minor variation does not exceed about approximately 0.25 times a maximum radius of curvature of the curved screen. ### Claim 10 (depends on 1) The method for optically remapping as recited in claim 1, where the curved screen includes a radius of curvature which is approximately infinite. ### Claim 11 (depends on 1) The method for optically remapping as recited in claim 1, where pixel density is the number of pixels per unit screen area and where a resolution of the projector may exceed a pixel count of the projector by way of pixel shifting. ### Claim 12 (depends on 1) The method for optically remapping as recited in claim 1, where one or more planar fold mirrors are employed throughout the optical path, about or between any optical elements, from the image source formation point within the projector to image observation at the DEP. ### Claim 13 (depends on 1) The method for optically remapping as recited in claim 1, where the refractive element incorporates multiple sub-elements and multiple ref alive optical surfaces. ### Claim 14 (depends on 13) The method for optically remapping as recited in claim 13, where the refractive element includes one or more optical surfaces substantially having a shape of a generalized cylinder with a focal surface substantially near, or encompassing; a location of a vertex of a projected light frustum of the projector. ### Claim 15 (depends on 13) The method for optically remapping as recited in claim 13, where the refractive element includes one or more optical surfaces substantially having a shape of a freeform with a focal surface substantially near, or encompassing, a location of a vertex of a projected light frustum of the projector. ### Claim 16 (depends on 13) The method for optically remapping as recited in claim 13, where the refractive element includes one or more optical surfaces substantially having a Gaussian curvature of approximately zero with a focal surface substantially near, or encompassing, a location of a vertex of a projected light frustum of the projector. ### Claim 17 (depends on 13) The method for optically remapping as recited in claim 13, where adjusting the optical effect of the optical device includes one or more of mechanical deformation and thermal deformation of any of the refractive optical surfaces of the refractive element. ### Claim 18 (depends on 13) The method for optically remapping as recited in claim 13, where adjusting the optical effect of the optical device includes adjustment of relative locations of sub-elements of the refractive element. ### Claim 19 (depends on 1) The method for optically remapping as recited in claim 1, where adjusting the optical effect of the optical device includes adjustment of one or more of: a location of the reflective element relative to the projector; a location of the reflective element relative to the refractive element; and the relative locations of sub-elements of the reflective element. ### Claim 20 (depends on 1) The method for optically remapping as recited in claim 1, further comprising: adjusting the refractive element by altering its optical effect. ### Claim 21 (depends on 20) The method for optically remapping as recited in claim 20, where the refractive element is adjusted to a freeform shape. ### Claim 22 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element is positioned along the optical path, after the refractive element and a projection lens. ### Claim 23 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element includes one or more of multiple sub-elements and multiple reflective optical surfaces. ### Claim 24 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element has a shape of a generalized cylinder. ### Claim 25 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element has a shape of a freeform. ### Claim 26 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element substantially has a Gaussian curvature of approximately zero. ### Claim 27 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element is adjustable in its optical effect. ### Claim 28 (depends on 1) The method for optically remapping as recited in claim 1, where the reflective element is adjustable to a substantially freeform shape. ### Claim 29 (depends on 1) The method for optically remapping as recited in claim 1, where adjusting the optical effect of the optical device includes one or more of mechanical deformation and thermal deformation of a reflective surface of the reflective element. ### Claim 30 (depends on 1) The method for optically remapping as recited in claim 1, where the refractive element is sufficiently close to the reflective element that the projected light traverses the refractive element before and after reflection from the reflective element. ### Claim 31 An apparatus for optically remapping projected pixels for an observer of a display system, comprising: a projector and a curved screen oriented and positioned for a design eye point (DEP) and a corresponding field of view and for projection of an image from the projector onto the curved screen; an optical device interposed within an optical path between an image source formation point within the projector and a projected image formation point upon the curved screen, where the optical device includes at least a refractive element and a reflective element which are positioned in optical subsequence, and where a focal surface of the refractive element and a focal surface of the reflective element each have a longest dimension, and the longest dimensions are oriented substantially orthogonal to one with respect to the other; said refractive element configured to substantially astigmatize the projected image's focus to a degree which is substantially inverse to a focal astigmatism introduced by optically subsequent elements using refraction by the refractive element; said reflective element configured to supplementally optically redirect light from the projector to optimal locations on the curved screen thereby optically remapping pixels, wherein the reflective element uses reflection to optically redirect the light; and said optical device configured to be adjustable in its optical effect in order to support one or more of various projectors, various projector configurations, various display systems, various display system configurations, variations in display system components, variation in observer location, variation in field of view, and variation in image resolution, whereby, achieving substantially maximal pixel utilization and substantially optimal image resolution for one or more observers of the display system. ### Claim 32 (depends on 31) The apparatus for optically remapping as recited in claim 31, where a collimating mirror is positioned between the DEP and the curved screen whereby the curved screen is viewed through the collimating mirror and a set of viewing vectors from the DEP reflect off of the collimating mirror before intersecting the curved screen. ### Claim 33 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the projector and the curved screen are configured with respect to the DEP such that a horizontal held of view exceeds 180 degrees, the image resolution is less than 6.1 arcminutes per optical line pair, and at least the reflective element is adjustable. ### Claim 34 (depends on 31) The apparatus for optically remapping as recited in claim 31, where substantially orthogonal means between 70 degrees and 110 degrees. ### Claim 35 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the projector includes a projection lens. ### Claim 36 (depends on 35) The apparatus for optically remapping as recited in claim 35, where the reflective element is positioned along the optical path after the refractive element and the projection lens. ### Claim 37 (depends on 31) The dcvicc apparatus for optically remapping as recited in claim 31, where the projector and curved screen are oriented and positioned for a compromise DEP for each portion of the field of view when there are two or more observers. ### Claim 38 (depends on 31) The dcvicc apparatus for optically remapping as recited in claim 31, where the curved screen is a section of one of an ellipsoid, a torpid, a minor variation from the ellipsoid, and a minor variation from the torrid, where said minor variation does not exceed 0.25 times a maximum radius of curvature of the curved screen. ### Claim 39 (depends on 38) The apparatus for optically remapping as recited in claim 38, where the curved screen has one radius of curvature which is approximately infinite. ### Claim 40 (depends on 35) The apparatus for optically remapping as recited in claim 35, where the refractive element is positioned along the optical path at one of before, after and within and after the projection lens. ### Claim 41 (depends on 31) The apparatus for optically remapping as recited in claim 31, where planar told mirrors are employed throughout the optical path, about or between any optical elements from the image source formation point within the projector to image observation at DEP. ### Claim 42 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the refractive element includes multiple refractive sub-elements and multiple refractive optical surfaces. ### Claim 43 (depends on 42) The apparatus for optically remapping as recited in claim 42, where the refractive element includes one or more optical surfaces substantially having a shape of a generalized cylinder with a focal surface substantially near, or encompassing, a location of a vertex of a projected light frustum of the projector. ### Claim 44 (depends on 42) The apparatus for optically remapping as recited in claim 42, where the refractive element includes one or more optical surfaces substantially having a shape of a freeform with a focal surface substantially near, or encompassing, a location of a vertex of a projected light frustum of the projector. ### Claim 45 (depends on 42) The apparatus for optically remapping as recited in claim 42, where the refractive element includes one or more optical surfaces substantially having a Gaussian curvature of approximately zero with a focal surface substantially near, or encompassing, a location of a vertex of a projected light frustum of the projector. ### Claim 46 (depends on 42) The apparatus for optically remapping as recited in claim 42, where the optical effect of the optical device is adjustable by adjustment of relative locations of sub-elements of the refractive element. ### Claim 47 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the refractive element is configured to be adjustable in its optical effect. ### Claim 48 (depends on 47) The apparatus for optically remapping as recited in claim 47, where the refractive element is configured to be adjustable to a freeform shape. ### Claim 49 (depends on 48) The apparatus for optically remapping as recited in claim 48, where adjustment of the refractive element is achieved by one or more of mechanical deformation and thermal deformation of a refractive optical surfaces of the refractive element. ### Claim 50 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the reflective element includes one or more of multiple reflective sub-elements and multiple reflective optical surfaces. ### Claim 51 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the reflective element has a shape of a generalized cylinder. ### Claim 52 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the reflective element has a shape of a freeform. ### Claim 53 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the reflective element has an optical surface with a Gaussian curvature of approximately zero. ### Claim 54 (depends on 31) The apparatus for optically remapping as recited in claim 31, where reflective element is adjustable in its optical effect. ### Claim 55 (depends on 54) The apparatus for optically remapping as recited in claim 54, where the reflective element may be is adjustable to a substantially freeform shape. ### Claim 56 (depends on 54) The apparatus for optically remapping as recited in claim 54, where adjustment of the reflective element is achieved by one or more of mechanical deformation and thermal deformation of a reflective surface of the reflective element. ### Claim 57 (depends on 54) The apparatus for optically remapping as recited in claim 54, where adjustment of the reflective element is achieved by adjustment of one or more of: its location relative to the projector; its location relative to the refractive element; and relative locations of reflective sub-element of the reflective element. ### Claim 58 (depends on 31) The apparatus for optically remapping as recited in claim 31, where the refractive element is positioned sufficiently close to the reflective element such that light projected by the projector traverses the refractive element before and after reflection from the reflective element.