# US 2024/0201512 A1 · Lenticular image generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0201512 A1 - Document ID: US20240201512A1 - Status: Published application - Kind: A1 - Application number: 18/067,604 - Filed: 2022-12-16 - Published: 2024-06-20 - Earliest claimed: 0001-01-01 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Felipe Bacim De Araujo E Silva; Noah D. Bedard; Bosheng Zhang; Brett D. Miller; Seung Wook Kim - Web page: https://patentgrep.com/patent/US20240201512A1 ## CPC - G02B 30/27: involving lenticular arrays - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - G02B 27/017: {Head mounted} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 15/005: {General purpose rendering architectures} - H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking - H04N 13/275: from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals - H04N 13/279: the virtual viewpoint locations being selected by the viewers or determined by tracking - H04N 13/305: using lenticular lenses, e.g. arrangements of cylindrical lenses - H04N 13/351: for displaying simultaneously - H04N 13/366: using viewer tracking - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - H04N 13/261: with monoscopic-to-stereoscopic image conversion - H04N 13/344: with head-mounted left-right displays - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Methods and apparatus for generating images to be displayed on lenticular displays. In these methods, a fixed mesh is generated offline, and in real-time texture information is mapped to the fixed mesh. In an offline process, texture and 3D mesh information for an object is used to render UV map views for multiple viewpoints of the object, view maps are generated from display calibration data, and a lenticular to UV map is generated from the UV map views and view maps. In real-time, texture information is captured, and a composite process is performed that generates a lenticular image for multiple viewpoints by sampling pixels from the texture based on the lenticular to UV map. The lenticular image is then displayed on the lenticular display. Detected positions of persons in the environment may be used to limit the number of viewpoints that are generated during the real-time composite process. ## Claims ### Claim 1 A device, comprising: a lenticular display; one or more sensors; memory; and a controller comprising one or more processors configured to: generate fixed mapping information for an object from texture and mesh information for the object obtained from the one or more sensors; store the fixed mapping information to the memory; generate a plurality of lenticular images from texture information for the object obtained from at least one of the one or more sensors and the fixed mapping information; and provide the lenticular images to the lenticular display. ### Claim 2 (depends on 1) The device as recited in claim 1, wherein the lenticular display comprises: a display panel; and a lenticular lens attached to a surface of the display panel. ### Claim 3 (depends on 2) The device as recited in claim 2, wherein the display panel is one of a liquid crystal display (LCD), an organic light-emitting diode (OLED) technology display, a DLP (digital light processing) technology display, or an LCoS (liquid crystal on silicon) technology display. ### Claim 4 (depends on 2) The device as recited in claim 2, wherein the lenticular display displays individual ones of the lenticular images on the display panel, and wherein the lenticular lens is configured to provide three-dimensional virtual views of the object displayed on the display panel from multiple different viewing angles. ### Claim 5 (depends on 1) The device as recited in claim 1, wherein, to generate fixed mapping information for an object from texture and mesh information for the object, the controller is configured to: render UV map views for multiple viewing angles of the lenticular display from the obtained texture and mesh information; generate view maps from a ray tracing model and calibration data for the lenticular display; and generate a lenticular to UV map for the object from the UV map views and the view maps; wherein the fixed mapping information includes the lenticular to UV map. ### Claim 6 (depends on 1) The device as recited in claim 1, wherein, to generate a plurality of lenticular images from texture information for the object obtained from at least one of the one or more sensors and the fixed mapping information, the controller is configured to, for each lenticular image to be generated: obtain current texture information for the object from the at least one sensor; and generate the lenticular image by sampling pixels from the current texture information based on the fixed mapping information. ### Claim 7 (depends on 6) The device as recited in claim 6, wherein the fixed mapping information includes a lenticular to UV map that maps subpixels in the current texture information to subpixels of the lenticular display. ### Claim 8 (depends on 7) The device as recited in claim 7, wherein the controller is further configured to: detect one or more persons in front of the lenticular display; and generate the lenticular image by sampling only subpixels from the current texture information that correspond to viewing angles of the detected one or more persons. ### Claim 9 (depends on 1) The device as recited in claim 1, wherein the device is a head-mounted device (HMD) of a computer-generated reality (CGR) system, and wherein the object is at least a portion of a face of a user of the HMD. ### Claim 10 (depends on 9) The device as recited in claim 9, wherein the controller is further configured to: detect movement of the HMD on the user's head; and in response to the movement, regenerate fixed mapping information for the user's face from texture and mesh information for the user's face obtained from the one or more sensors. ### Claim 11 (depends on 9) The device as recited in claim 9, wherein the controller is further configured to: detect movement of the HMD on the user's head; and move a window within the fixed mapping information based on the detected movement. ### Claim 12 A method, comprising: performing, by one or more processors of a device in an off-line process: generating fixed mapping information for an object from texture and mesh information for the object obtained from one or more sensors; storing the fixed mapping information to the memory; performing, by the one or more processors of the device in a real-time process: generating a plurality of lenticular images from texture information for the object obtained from at least one of the one or more sensors and the fixed mapping information; and providing the lenticular images to a lenticular display. ### Claim 13 (depends on 12) The method as recited in claim 12, wherein the lenticular display includes a display panel and a lenticular lens attached to a surface of the display panel, the method further comprising displaying, by the lenticular display, individual ones of the lenticular images on the display panel, wherein the lenticular lens is configured to provide three-dimensional virtual views of the object displayed on the display panel from multiple different viewing angles. ### Claim 14 (depends on 12) The method as recited in claim 12, wherein generating fixed mapping information for an object from texture and mesh information for the object obtained from one or more sensors comprises: rendering UV map views for multiple viewing angles of the lenticular display from the obtained texture and mesh information; generating view maps from a ray tracing model and calibration data for the lenticular display; and generating a lenticular to UV map for the object from the UV map views and the view maps; wherein the fixed mapping information includes the lenticular to UV map. ### Claim 15 (depends on 12) The method as recited in claim 12, wherein generating a plurality of lenticular images from texture information for the object obtained from at least one of the one or more sensors and the fixed mapping information comprises, for each lenticular image to be generated: obtaining current texture information for the object from the at least one sensor; and generating the lenticular image by sampling pixels from the current texture information based on the fixed mapping information. ### Claim 16 (depends on 15) The method as recited in claim 15, wherein the fixed mapping information includes a lenticular to UV map that maps subpixels in the current texture information to subpixels of the lenticular display. ### Claim 17 (depends on 16) The method as recited in claim 16, further comprising: detecting one or more persons in front of the lenticular display; and generating the lenticular image by sampling only subpixels from the current texture information that correspond to viewing angles of the detected one or more persons. ### Claim 18 (depends on 12) The method as recited in claim 12, wherein the device is a head-mounted device (HMD) of a computer-generated reality (CGR) system, and, wherein the object is at least a portion of a face of a user of the HMD. ### Claim 19 (depends on 18) The method as recited in claim 18, further comprising: detecting movement of the HMD on a user's head; and in response to the movement, regenerating the fixed mapping information for the user's face from texture and mesh information for the user's face obtained from the one or more sensors. ### Claim 20 (depends on 18) The method as recited in claim 18, further comprising: detecting movement of the HMD on a user's head; and moving a window within the fixed mapping information based on the detected movement.