# US 12,008,704 B2 · System for photo-realistic reflections in augmented reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,704 B2 - Document ID: US12008704B2 - Status: Granted - Kind: B2 - Application number: 17/934,735 - Filed: 2022-09-23 - Published: 2023-01-19 - Granted: 2024-06-11 - Earliest claimed: 2016-01-28 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Reuven Bakalash; Elad Haviv - Web page: https://patentgrep.com/patent/US12008704B2 ## CPC - G06T 15/06: Ray-tracing - G06T 15/005: {General purpose rendering architectures} - G06T 15/04: Texture mapping - G06T 15/405: {using Z-buffer} - G06T 15/506: {Illumination models} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present disclosure describes a system for fast generation of ray traced reflections of virtually augmented objects into a real-world image, specifically on reflective surfaces. The system utilizes a standard raster graphics pipeline. ## Claims ### Claim 1 A system comprising: at least one processor configured to perform operations comprising: projecting primary rays at reflective surfaces in an image to generate a cluster of primary hit points; generating projections of carrier rays through the cluster of generated primary hit points; generating secondary rays from carrier rays that intersect an augmented object; generating a render target texture of intersections between the secondary rays and the augmented object; and reconstructing reflections of the augmented object on one or more of the reflective surfaces in the image from light values sampled in the render target texture. ### Claim 2 (depends on 1) The system of claim 1, wherein the operations comprise: identifying reflective surface areas in the image that may reflect the augmented object; and feeding sampled light values of points of intersection between the secondary rays and augmented object into their respective primary hit points. ### Claim 3 (depends on 1) The system of claim 1, wherein a location and boundaries of an area of reflection is determined according to location of a camera, distance and size of the augmented object, and consideration of a principal direction. ### Claim 4 (depends on 1) The system of claim 1, wherein projection of primary rays is implemented by a hardware graphics pipeline of a graphics processor. ### Claim 5 (depends on 1) The system of claim 1, wherein the projection of primary rays is implemented by a software graphics pipeline. ### Claim 6 (depends on 5) The system of claim 5, wherein in case of generating a semi-reflective reflection, each successive projection of carrier rays is deviated from a principal direction. ### Claim 7 (depends on 1) The system of claim 1, wherein secondary rays commence at an area of reflection. ### Claim 8 (depends on 1) The system of claim 1, wherein rays that miss hit points are entirely discarded. ### Claim 9 (depends on 1) The system of claim 1, wherein rays that miss hit points are considered as early segments in their entirety. ### Claim 10 (depends on 1) The system of claim 1, wherein the projection of carrier rays is implemented by a hardware raster pipeline. ### Claim 11 (depends on 1) The system of claim 1, wherein intersection points between secondary rays and the augmented object are configured for use as a start point for a next generation of hit points. ### Claim 12 A method comprising: projecting primary rays at reflective surfaces in a image to generate a cluster of primary hit points; generating projections of carrier rays through the cluster of generated primary hit points; generating secondary rays from carrier rays that intersect an augmented object; generating a render target texture of intersections between the secondary rays and the augmented object; and reconstructing reflections of the augmented object on one or more of the reflective surfaces in the image from light values sampled in the render target texture. ### Claim 13 (depends on 12) The method of claim 12, further comprising: identifying reflective surface areas in the image that may reflect the augmented object; and feeding the sampled light values into their respective primary hit points. ### Claim 14 (depends on 12) The method of claim 12, wherein a location and boundaries of an area of reflection is determined according to location of a camera, distance and size of the augmented objects, and consideration of a principal direction. ### Claim 15 (depends on 12) The method of claim 12, wherein projection of primary rays is implemented by a hardware graphics pipeline of a graphics processor. ### Claim 16 (depends on 12) The method of claim 12, wherein the projection of primary rays is implemented by a software graphics pipeline. ### Claim 17 (depends on 16) The method of claim 16, wherein in case of generating a semi-reflective reflection, each successive projection of carrier rays is deviated from a principal direction. ### Claim 18 (depends on 12) The method of claim 12, wherein secondary rays commence at an area of reflection. ### Claim 19 (depends on 12) The method of claim 12, wherein rays that miss hit points are entirely discarded. ### Claim 20 A non-transitory computer readable medium comprising computer readable instructions that, when executed by at least one processor, configure the processor to perform operations comprising: projecting primary rays at reflective surfaces in a image to generate a cluster of primary hit points; generating projections of carrier rays through the cluster of generated primary hit points; generating secondary rays from carrier rays that intersect an augmented object; generating a render target texture of intersections between the secondary rays and the augmented object; and reconstructing reflections of the augmented object on one or more of the reflective surfaces in the image from light values sampled in the render target texture.