# US 2014/0193068 A1 · Using infrared imaging to create digital images for use in product customization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0193068 A1 - Document ID: US20140193068A1 - Status: Published application - Kind: A1 - Application number: 13/736,844 - Filed: 2013-01-08 - Published: 2014-07-10 - Earliest claimed: 2013-01-08 - Assignees: Zazzle.com, Inc. - Inventors: Young Harvill; Robert I. Beaver, III; Jeffrey J. Beaver - Web page: https://patentgrep.com/patent/US20140193068A1 ## CPC - H04N 1/387: Composing, repositioning or otherwise {geometrically} modifying originals - G06T 7/11: Region-based segmentation - G06T 11/00: Two-dimensional [2D] image generation - G09G 5/36: characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04N 23/95: Computational photography systems, e.g. light-field imaging systems - G06T 2207/10024: Color image - G06T 2207/10048: Infrared image - G09G 2320/0693: Calibration of display systems - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Techniques for using infrared imaging to create digital images for use in product customization are described. In an embodiment, an infrared photograph of a product with imprinted markup is received and a visible light photograph of the product with the imprinted markup is received. The imprinted markup is visible in the visible light photograph but is not visible in the infrared photograph. Instructions for rendering a customization image of the product depicting a particular customization are determined based in part on the infrared photograph and visible light photograph, where the particular customization is not in the infrared photograph or the visible light photograph. ## Claims ### Claim 1 A computer-implemented method, comprising: receiving an infrared photograph of a tangible product having an imprinted markup on the product; receiving a visible light photograph of the same product with the same imprinted markup, wherein the imprinted markup is visible in the visible light photograph but is not visible in the infrared photograph; determining, based, in part, on the infrared photograph and visible light photograph, instructions for rendering a customization image of the product depicting a particular customization, wherein the particular customization is not in the infrared photograph or the visible light photograph; wherein the method is performed by one or more computing devices. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the customization specifies one or more designs for imprinting on one or more design areas of the product or a color for at least a portion of the product. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising determining the particular customization after determining the instructions for rendering the customization image and providing the particular customization as an input to the instructions for rendering the customization image. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein at least a portion of a shape and a shading of the particular customization image correspond to a shape and a shading of the imprinted markup. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: determining, based in part on the infrared photograph, luminance information; determining, based in part on the visible light photograph, design geometry information; wherein the instructions for rendering the customization image of the product are determined based in part on the luminance information and the design geometry information. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising performing a color segmentation process on an image based on the visible light photograph, resulting in a digitally stored color segmented image, wherein the instructions for rendering the customization image of the product are determined based, in part, on the color segmented image. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising determining a normalized visible light image by removing luminance from an image based on the visible light photograph, wherein the instructions for rendering the customization image of the product are determined based, in part, on the normalized visible light image. ### Claim 8 (depends on 5) The computer-implemented method of claim 5, further comprising: determining, automatically, based on the visible light photograph and the infrared photograph, a stored digital image of the product without markup; wherein the instructions for rendering the customization image of the product are determined based in part on the image of the product without markup and the design geometry information. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising: determining a first color-selected image and a second color-selected image based, at least in part on particular image, wherein the particular image is the visible light photograph or an image based on the visible light photograph, wherein the markup comprises a plurality of colors; wherein, for each region of the particular image that is colored a first color of the plurality of colors, a same region of the first color-selected image is colored a particular color and remaining regions of the first color-selected image is colored a different color than the particular color; wherein, for each region of the particular image that is colored a second color of the plurality of colors, a same region of the second color-selected image is colored the particular color and remaining regions of the first color-selected image is colored a different color than the particular color. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising: determining a design mask image based on the first color-selected image and the second color-selected image, wherein each region of the design mask image that is colored the particular color in either the first color-selected image or the second color-selected image is of a particular mask color in the design mask image and remaining portions of the design mask image are a different color than the particular mask color. ### Claim 11 (depends on 9) The computer-implemented method of claim 9, further comprising determining a grid image based on the first color-selected image and the second color-selected image wherein the grid image indicates locations at which one or more regions of the first color are adjacent to one or more regions of the second color in the particular image. ### Claim 12 (depends on 3) The computer-implemented method of claim 3, further comprising: receiving a customization specification specifying the particular customization; causing performance of the instructions for rendering the customization image of the product depicting the particular customization; causing display of the customization image of the product depicting the particular customization. ### Claim 13 (depends on 1) The computer-implemented method of claim 1, wherein at least a portion of the instructions for rendering the customization image are associated with a particular viewpoint; the method further comprising: receiving a customization specification specifying, at least in part, a customization associated with the particular viewpoint; selecting, based on the customization specification, the portion of the instructions for rendering the customization image associated with the particular viewpoint for execution. ### Claim 14 (depends on 1) The computer-implemented method of claim 1, wherein the product is the first product, the customization image is the first customization image, the method further comprising: receiving a customization specification specifying the particular customization; in response to the receiving the customization specification: causing performance of the instructions for rendering the customization image of the product depicting the particular customization, wherein the particular customization is specified by the customization specification; causing performance of second instructions for rendering a second customization images of a second product different from the first product depicting the particular customization; causing display of the first customization image of the product depicting the particular customization and the second customization of the second product depicting the particular customization. ### Claim 15 A non-transitory computer-readable storage medium storing one or more instructions which, when executed by one or more processors, cause the one or more processors to perform: receiving an infrared photograph of a tangible product having an imprinted markup on the product; receiving a visible light photograph of the same product with the same imprinted markup, wherein the imprinted markup is visible in the visible light photograph but is not visible in the infrared photograph; determining, based, in part, on the infrared photograph and visible light photograph, instructions for rendering a customization image of the product depicting a particular customization, wherein the particular customization is not in the infrared photograph or the visible light photograph. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the customization specifies one or more designs for imprinting on one or more design areas of the product or a color for at least a portion of the product. ### Claim 17 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the one or more processors to further perform: determining the particular customization after determining the instructions for rendering the customization image and providing the particular customization as an input to the instructions for rendering the customization image. ### Claim 18 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein at least a portion of a shape and a shading of the particular customization image correspond to a shape and a shading of the imprinted markup. ### Claim 19 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the one or more processors to further perform: determining, based in part on the infrared photograph, luminance information; determining, based in part on the visible light photograph, design geometry information; wherein the instructions for rendering the customization image of the product are determined based in part on the luminance information and the design geometry information. ### Claim 20 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the one or more processors to further perform: performing a color segmentation process on an image based on the visible light photograph, resulting in a digitally stored color segmented image, wherein the instructions for rendering the customization image of the product are determined based, in part, on the color segmented image. ### Claim 21 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the one or more processors to further perform determining a normalized visible light image by removing luminance from an image based on the visible light photograph, wherein the instructions for rendering the customization image of the product are determined based, in part, on the normalized visible light image. ### Claim 22 (depends on 19) The non-transitory computer-readable storage medium of claim 19, wherein the instructions cause the one or more processors to further perform determining, automatically, based on the visible light photograph and the infrared photograph, a stored digital image of the product without markup; wherein the instructions for rendering the customization image of the product are determined based in part on the image of the product without markup and the design geometry information. ### Claim 23 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the one or more processors to further perform: determining a first color-selected image and a second color-selected image based, at least in part on particular image, wherein the particular image is the visible light photograph or an image based on the visible light photograph, wherein the markup comprises a plurality of colors; wherein, for each region of the particular image that is colored a first color of the plurality of colors, a same region of the first color-selected image is colored a particular color and remaining regions of the first color-selected image is colored a different color than the particular color; wherein, for each region of the particular image that is colored a second color of the plurality of colors, a same region of the second color-selected image is colored the particular color and remaining regions of the first color-selected image is colored a different color than the particular color. ### Claim 24 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the instructions cause the one or more processors to further perform: determining a design mask image based on the first color-selected image and the second color-selected image, wherein each region of the design mask image that is colored the particular color in either the first color-selected image or the second color-selected image is of a particular mask color in the design mask image and remaining portions of the design mask image are a different color than the particular mask color. ### Claim 25 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the instructions cause the one or more processors to further perform: determining a grid image based on the first color-selected image and the second color-selected image wherein the grid image indicates locations at which one or more regions of the first color are adjacent to one or more regions of the second color in the particular image. ### Claim 26 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the instructions cause the one or more processors to further perform: receiving a customization specification specifying the particular customization; causing performance of the instructions for rendering the customization image of the product depicting the particular customization; causing display of the customization image of the product depicting the particular customization. ### Claim 27 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein at least a portion of the instructions for rendering the customization image are associated with a particular viewpoint; wherein the instructions cause the one or more processors to further perform: receiving a customization specification specifying, at least in part, a customization associated with the particular viewpoint; selecting, based on the customization specification, the portion of the instructions for rendering the customization image associated with the particular viewpoint for execution. ### Claim 28 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the product is the first product, the customization image is the first customization image, and wherein the instructions cause the one or more processors to further perform: receiving a customization specification specifying the particular customization; in response to the receiving the customization specification: causing performance of the instructions for rendering the customization image of the product depicting the particular customization, wherein the particular customization is specified by the customization specification; causing performance of second instructions for rendering a second customization images of a second product different from the first product depicting the particular customization; causing display of the first customization image of the product depicting the particular customization and the second customization of the second product depicting the particular customization.