# US 9,530,242 B2 · Point and click lighting for image based lighting surfaces > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,530,242 B2 - Document ID: US9530242B2 - Status: Granted - Kind: B2 - Application number: 14/386,452 - Filed: 2013-02-28 - Published: 2015-02-12 - Granted: 2016-12-27 - Earliest claimed: 2012-03-20 - Assignees: Lightmap Limited - Applicants: Lightmap Limited - Inventors: Mark Segasby; Simon C. Smith - Web page: https://patentgrep.com/patent/US9530242B2 ## CPC - G06T 15/06: Ray-tracing - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06T 5/70: Denoising; Smoothing - G06T 15/50: Lighting effects - G06T 15/506: {Illumination models} - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20092: Interactive image processing based on input by user - G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A computer implemented method of interacting with a three dimensional image based lighting surface to adjust its lighting properties, comprising defining an image plane and a user viewpoint for the lighting surface; rendering and displaying a scene containing an object in situ within the lighting surface; by way of a user interaction with the displayed scene, receiving an identification of a point on the image plane; tracing a ray from the user viewpoint through the identified point on the image plane and either, determining a surface intersection point of the ray with the surface or determining an object intersection point of the ray with the object and tracing a further ray, and determining a surface intersection point of the further ray with the surface, and adjusting the lighting properties of the surface at or in the region of the surface intersection point. ## Claims ### Claim 1 A computer implemented method of interacting with a three dimensional image based lighting surface in order to adjust the lighting properties of the surface, the method comprising: defining an image plane and a user viewpoint for the lighting surface; rendering and displaying on a display of a computer, a three-dimensional scene containing an object in situ within the lighting surface, taking into account said image plane and said user viewpoints; by way of user interaction with the displayed three-dimensional scene, receiving an identification of a selected point on the image plane; tracing a ray from the user viewpoint through the identified selected point on the image plane and either, determining a surface intersection point of the ray with said lighting surface, or determining an object intersection point of the ray with said object and tracing a further ray either being a reflection of the ray from the object or being normal to the surface of the object at said object intersection point, and determining a surface intersection point of the further ray with said lighting surface; generating coordinate data representative of a location within a canvas, being a two-dimensional mapping of said lighting surface, of said surface intersection point, and using said coordinate data to adjust, within said canvas, the lighting properties of the lighting surface in the region of the surface intersection point either by adding, at coordinates within said two-dimensional canvas corresponding to the region of said surface intersection point, a light source having a geometry including mapping the geometry of the light source onto the three dimensional image based lighting surface, or selecting, at coordinates within said two-dimensional canvas corresponding to the region of said surface intersection point, a light source having a geometry and subtracting the selected light source or modifying the properties of the selected light source; and updating in substantially real-time, said three-dimensional scene displayed on said display of said computer using the lighting surface having adjusted lighting properties within the canvas. ### Claim 2 (depends on 1) A method as claimed in claim 1, wherein said lighting surface is a sphere or a cuboid. ### Claim 3 (depends on 1) A method as claimed in claim 1, wherein said lighting surface comprises a photographic or computer generated image. ### Claim 4 (depends on 1) A method as claimed in claim 1, wherein, if more than one light source is located at or in the region of said surface intersection point, said step of selecting a light source comprises providing the user with a means to select the desired light source from the more than one light source located at or in the region of said surface intersection point. ### Claim 5 (depends on 1) A method as claimed in claim 1, wherein said step of mapping comprises tracing rays from an added light source to an origin of the lighting surface. ### Claim 6 (depends on 1) A method as claimed in claim 1 and comprising displaying on the display of the computer, the canvas. ### Claim 7 (depends on 1) A method as claimed in claim 1, wherein said step of receiving an identification of a point on the image plane comprises receiving an input on the rendered scene from a user operated pointing device such as a mouse, tablet or touch screen. ### Claim 8 A system configured to interact with a three dimensional image based lighting surface in order to adjust the lighting properties of the lighting surface, the system comprising: a display configured to display a three-dimensional rendered scene containing an object in situ within the lighting surface taking into account an image plane and a user viewpoint; a graphical user interface configured to receive an identification of a selected point on the image plane by way of a user interaction with the displayed three-dimensional scene; and a processor configured to: render the scene to be displayed on the display; trace a ray from the user viewpoint through the identified selected point on the image plane and either, determine a surface intersection point of the ray with said lighting surface, or determine an object intersection point of the ray with said object and trace a further ray either being a reflection of the ray from the object or being normal to the surface of the object at said object intersection point, and determine a surface intersection point of the further ray with said lighting surface, generate coordinate data representative of a location within a canvas, being a two-dimensional mapping of said lighting surface, of said surface intersection point, and using said coordinate data to adjust, within said canvas, the lighting properties of the lighting surface in the region of the surface intersection point, either by adding, at coordinates within said two-dimensional canvas corresponding to the region of said surface intersection point, a light source having a geometry, and mapping the geometry of the light source onto the three dimensional image based lighting surface, or selecting, at coordinates within said two-dimensional canvas corresponding to the regiof of said surface intersection points, a light source having a geometry and subtracting the selected light source or modifying the properties of the selected light source; and update in substantially real-time, said three-dimensional scene displayed on said display of said computer using the lighting surface having adjusted lighting properties within the canvas. ### Claim 9 (depends on 8) A system as claimed in claim 8, wherein the user interaction with the displayed scene is by way of a user operated pointing device such as a mouse, tablet or touch screen. ### Claim 10 A computer implemented method of interacting with a three dimensional image based lighting surface in order to adjust the lighting properties of the surface, the method comprising: defining an image plane and a user viewpoint for the lighting surface; rendering and displaying on a display of a computer a three-dimensional scene, receiving an identification of a selected point on the image plane; performing a ray tracing operation from the user viewpoint through said selected point on the image plane in order to identify a lighting surface intersection point; generating coordinate data representative of a location within a canvas, being a two-dimensional mapping of said lighting surface, of said surface intersection point, and using said coordinate data to adjust, within said canvas, the lighting properties of the lighting surface in the region of the surface intersection point, either by adding, at coordinates within said two-dimensional canvas corresponding to the region of said surface intersection point, a light source having a geometry, and mapping the geometry of the light source onto the three dimensional image based lighting surface represented by said two-dimensional canvas, or selecting, at coordinates within said two-dimensional canvas corresponding to the region of said surface intersection point, a light source having a geometry and subtracting the selected light source or modifying the properties of the selected light source; and updating in substantially real-time, said three-dimensional scene displayed on said display of said computer using the lighting surface having adjusted lighting properties within the canvas.