# US 2016/0148432 A1 · Systems and methods for projecting planar and 3d images through water or liquid onto a surface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0148432 A1
- Document ID: US20160148432A1
- Status: Published application
- Kind: A1
- Application number: 14/935,594
- Filed: 2015-11-09
- Published: 2016-05-26
- Earliest claimed: 2014-11-24
- Assignees: Pointcloud Media, LLC
- Applicants: Pointcloud Media, LLC
- Inventors: Jacobus G. Hattingh
- Web page: https://patentgrep.com/patent/US20160148432A1

## CPC

- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G09G 3/003: {to produce spatial visual effects}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Systems and methods for projecting planar and 3D images through water or liquid onto a surface include creating a 3D model of the body of liquid and surface and 3D models of creative elements to be used in scenes. Animating the 3D models of the creative elements, placing them inside the 3D model of the body of liquid. Lighting the animated creative elements, rendering planar animations of the modeled creative elements and, using projection and texturing software, virtually projecting the planar animations back onto the surface of the 3D model of the body of liquid from the same camera position in order to “bake in” a warped transformation of the digitally rendered planar animations. Digitally rendering a 3D animation of the warped, transformed planar animations, and playing or looping the digitally rendered 3D animation through the body of liquid on a digital video player or digital server.

## Claims

### Claim 1

A method comprising:
a) selecting a body of liquid for projecting images through, the body of liquid contained within a surface;
b) gathering measurements of the body of liquid and surface;
c) creating a digital virtual 3D model of the body of liquid and the surface by loading the measurements into a memory of a digital computer, the computer including at least one processor programmed by computer software sufficient to create virtual 3D images of the body of liquid and surface;
d) creating digital virtual 3D models of creative elements to be used in one or more scenes using the at least one processor programmed by computer software sufficient to create the digital virtual 3D models of the creative elements;
e) digitally animating the digital virtual 3D models of the creative elements using the at least one processor programmed by computer software sufficient to digitally animate the digital virtual 3D models of the creative elements;
f) placing the digitally animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid using the at least one processor programmed by computer software sufficient to place the digitally animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid, using the virtual surface to virtually place objects and cast shadows onto the virtual surface;
g) lighting the animated digital virtual 3D modeled creative elements using the at least one processor programmed with one or more virtual 3D lighting programs;
h) creating digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements using the at least one processor programmed by digital rendering software;
i) using the at least one processor programmed by digital projection and texturing software, virtually projecting (and optionally texturing) the digitally rendered planar animations back onto the virtual surface of the now empty virtual 3D model of the body of liquid from the same virtual camera position as in the previous step in order to “bake in” a warped transformation of the digitally rendered planar animations; and
j) using the at least one processor programmed by digital rendering software, digitally rendering a 3D animation of the warped, transformed projected rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of the physical projector to be installed at the site of the body of liquid.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
k) playing and/or looping the digitally rendering a 3D animation on a digital video player or digital server at the physical location on a digital projector installed on a structure above or to one side of the body of liquid in the planned angle and location as the virtual position.

### Claim 3 (depends on 1)

The method of claim 1 wherein the gathering of measurements is by a method selected from the group consisting of by hand, from CAD drawings, from blueprints, using a laser scanner, and combinations of two or more of these.

### Claim 4 (depends on 1)

The method of claim 1 wherein the creating of digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements comprises:
A) digitally rendering the lighted, animated digital virtual 3D modeled creative elements from a digital virtual camera directly above the virtual body of liquid in the digital 3D virtual model if the body of liquid and surface are located in the center of a viewing area; or
B) digitally rendering the lighted, animated digital virtual 3D modeled creative elements from a digital virtual camera located at a center of a side vantage point in the digital 3D virtual model if the body of liquid and surface are to be viewed from one side only.

### Claim 5 (depends on 1)

The method of claim 1 wherein the digitally rendering of the 3D animation of the warped, transformed projected rendered planar animations comprises:
A) in the case of an internal installed projector, digitally rendering the animations from the same angle as the internal installed projector using a spherical, fisheye or mirrored lens; or
B) in the case of an internal (existing light replacement) projector, replacing the glass lens with a manufactured bubble lens to accommodate a wide, fisheye or mirror lens.

### Claim 6 (depends on 1)

The method of claim 1 wherein the body of liquid is selected from the group consisting of a swimming pool, a reflecting pool, a pond, and a fountain.

### Claim 7 (depends on 1)

The method of claim 1 wherein the computer software sufficient to create virtual 3D images, to create the digital virtual 3D models of the creative elements, digitally animate creative elements, place the digitally animated creative elements inside the digital virtual 3D model of the body of liquid, and the one or more virtual 3D lighting programs includes one or more:
i) options for 3D unwrapping, shading, physics, dynamics and particles, real time 3D/game creation, camera projection shading, 2D and 3D procedural brushes, edge rendering, and/or collision simulation;
ii) integration software built into the program, allowing seamless integration of 3D scenes into the digital rendering, projection, and texturing software;
iii) allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software; and
iv) a software toolset enabling creation of flying logos and similar simple effects.

### Claim 8 (depends on 1)

The method of claim 1 wherein the digital rendering, projection and texturing software is selected from software that allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software.

### Claim 9

A method comprising:
a) selecting a swimming pool or pond for projecting images through, the swimming pool or pond contained within a surface;
b) gathering exact measurements of an outer rim as well as the inside surface area of the pool or pond by hand or from CAD drawings, blueprints or a laser scanner;
c) creating a digital virtual 3D model of the swimming pool or pond and the surface by loading the measurements into a memory of a digital computer, the computer including at least one processor programmed by computer software sufficient to make the digital virtual 3D images of the body of water and surface;
d) creating digital virtual 3D models of creative elements to be used in one or more scenes using the digital computer and the computer software;
e) animating the digital virtual 3D models making sure that moveable creative elements (for example, human characters and non-human creatures) travel within the digital virtual 3D model of the swimming pool or pond;
f) placing the animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the swimming pool or pond using the digital virtual surface to place objects and cast shadows onto the digital virtual surface;
g) lighting the animated digital virtual 3D modeled creative elements using the programmed digital computer programmed with one or more digital virtual 3D lighting programs in such a way to allow shadows of the animated digital virtual 3D modeled creative elements to fall on the digital virtual surface, causing enhanced 3D augmentation in the swimming pool or pond when the animation is played therein;
h) creating digital rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements and/or stills by:
A) rendering the lighted, animated digital virtual 3D modeled creative elements and/or stills from a virtual camera directly above the virtual swimming pool or pond in the 3D virtual model if the swimming pool or pond and surface are located in the center of a viewing area; or
B) rendering the lighted, animated digital virtual 3D modeled creative elements and/or stills from a virtual camera located at a center of a side vantage point in the 3D virtual model if the swimming pool or pond and surface are to be viewed from one side only;
i) using the programmed digital computer, virtually projecting (and optionally texturing) the digital rendered planar animations back onto the digital virtual surface of the now empty digital virtual 3D model of the swimming pool or pond from the same virtual camera position as in the previous step in order to “bake in” a warped transformation of the digitally rendered planar animations;
j) digitally rendering a 3D animation of the warped, transformed projected digital rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of the physical projector to be installed at the site of the swimming pool or pond; and
k) at the physical location, installing a projector as planned on a structure above or to one side of the swimming pool or pond in the same position as the virtual position, connecting the projector to a video player or server capable of playing digital and video images, loading the completed animations or stills into the video player or server, waiting until the sun sets or reasonable darkness if the swimming pool or pond is outdoors, focusing the projector and lining up the digital virtual swimming pool or pond with the physical swimming pool or pond while projecting the digital virtual 3D model of the swimming pool or pond onto the physical swimming pool or pond, and playing and/or looping the animation.

### Claim 10 (depends on 9)

The method of claim 9 wherein the computer software sufficient to create virtual 3D images, to create the digital virtual 3D models of the creative elements, digitally animate creative elements, place the digitally animated creative elements inside the digital virtual 3D model of the of the swimming pool or pond, and the one or more virtual 3D lighting programs includes one or more:
i) options for 3D unwrapping, shading, physics, dynamics and particles, real time 3D/game creation, camera projection shading, 2D and 3D procedural brushes, edge rendering, and/or collision simulation;
ii) integration software built into the program, allowing seamless integration of 3D scenes into the digital rendering, projection, and texturing software;
iii) allows changes to the 3D animation to automatically be updated inside of the digital rendering; projection, and texturing software; and
iv) a software toolset enabling creation of flying logos and similar simple effects.

### Claim 11 (depends on 9)

The method of claim 9 wherein the digital rendering, projection and texturing software is selected from software that allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software.

### Claim 12

A system comprising:
a) a digital computer, the digital computer including a memory and at least one processor programmed by computer software sufficient to make virtual 3D images of a body of liquid and a surface containing the body of liquid,
b) the memory sufficient to hold measurements of the body of liquid and the surface, the at least one processor programmed by computer software sufficient to:
create digital virtual 3D models of creative elements;
animate the digital virtual 3D models of the creative elements;
place at least some of the animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid using the virtual surface to place objects and cast shadows onto the virtual surface;
light at least some of the animated digital virtual 3D modeled creative elements with one or more virtual 3D lighting programs;
create digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements;
virtually project (and optionally texture) the digitally rendered planar animations back onto the virtual surface of the now empty virtual 3D model of the body of liquid from the same virtual camera position as in the previous step, in order to “bake in” a warped transformation of the digitally rendered planar animations; and
digitally render a 3D animation of the warped transformation of the projected rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of the physical projector to be installed at the site of the body of liquid.

### Claim 13 (depends on 12)

The system of claim 12 further comprising a digital video player or digital server computer, and one or more digital projectors, the digital video player or digital server computer configured sufficiently to play and/or loop the 3D animation on the one or more digital projectors installed on a structure above or to one side of the body of liquid in the planned angle and location as the virtual position, the one or more digital projectors projecting at least a portion of the digitally rendered 3D animation of the warped transformation of the projected rendered planar animations through the body of liquid and onto the surface, completing an illusion of at least some of the creative elements in the body of liquid.

### Claim 14 (depends on 12)

The system of claim 12 wherein the body of liquid is selected from the group consisting of a swimming pool, a reflecting pool, a pond, and a fountain.

### Claim 15 (depends on 13)

The system of claim 13 wherein the one or more digital projectors transmit an image at a resolution, represented by horizontal pixel count, of at least 2K (2048×1080 or 2.2 megapixels), more preferably at least 4K (4096×2160 or 8.8 megapixels).

### Claim 16 (depends on 15)

The system of claim 15 wherein the one or more digital projectors are selected from the group consisting of DLP and laser cinema projectors.

### Claim 17 (depends on 13)

The system of claim 13 wherein the server computer supports JPEG2000 and MPEG2 files, and dual-projector 3D playback.

### Claim 18 (depends on 12)

The system of claim 12 wherein the computer software sufficient to create virtual 3D images, to create the digital virtual 3D models of the creative elements, digitally animate creative elements, place the digitally animated creative elements inside the digital virtual 3D model of the of the body of liquid and surface, and the one or more virtual 3D lighting programs includes one or more:
i) options for 3D unwrapping, shading, physics, dynamics and particles, real time 3D/game creation, camera projection shading, 2D and 3D procedural brushes, edge rendering, and/or collision simulation;
ii) integration software built into the program, allowing seamless integration of 3D scenes into the digital rendering, projection, and texturing software;
iii) allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software; and
iv) a software toolset enabling creation of flying logos and similar simple effects.

### Claim 19 (depends on 12)

The method of claim 12 wherein the digital rendering, projection and texturing software is selected from software that allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software.

### Claim 20

A system comprising:
a) a swimming pool contained within a surface;
b) a digital computer, the digital computer including a memory and at least one processor programmed by computer software sufficient to make virtual 3D images of the swimming pool and surface, the memory sufficient to hold measurements of the swimming pool and surface;
c) the at least one processor programmed by computer software sufficient to create digital virtual 3D models of creative elements to be used in one or more scenes using the computer and the computer software;
d) the at least one processor programmed by computer software sufficient to animate the digital virtual 3D models of the creative elements;
e) the at least one processor programmed by computer software sufficient to place at least some of the animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the swimming pool using the virtual surface to place objects and cast shadows onto the virtual surface;
f) the at least one processor programmed by computer software sufficient to light the animated digital virtual 3D modeled creative elements with one or more virtual 3D lighting programs;
g) the at least one processor programmed by computer software sufficient to create digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements;
h) the at least one processor programmed by computer software sufficient to virtually project (and optionally texture) the digitally rendered planar animations back onto the virtual surface of an empty virtual 3D model of the swimming pool from the same virtual camera position as in the previous step;
i) the at least one processor programmed by computer software sufficient to digitally render a 3D animation of the projected rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of the physical projector to be installed at the site of the swimming pool and surface;
j) a digital video player or digital server computer sufficient to play and/or loop the 3D animation; and
k) one or more digital projectors installed on a structure above or to one side of the swimming pool and surface in the planned angle and location as the virtual position, the one or more projectors sufficient to project the digitally rendered 3D animation of the warped transformation of the projected rendered planar animations through the swimming pool and onto the surface, completing an illusion of at least some of the creative elements in the swimming pool.

### Claim 21

A kit comprising:
a) a structure for containing a body of liquid comprising a rear wall, a front wall, a floor, and left and right side walls;
b) a digital projector for projecting a digital image through a body of liquid held by the structure;
c) a digital computer, the digital computer including a memory and at least one processor programmed by computer software sufficient to make virtual 3D images of the structure and body of liquid, the memory sufficient to hold measurements of the structure and the body of liquid, the at least one processor programmed by computer software sufficient to:
create digital virtual 3D models of creative elements;
animate the digital virtual 3D models of the creative elements;
place the animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid using the virtual structure to place objects and cast shadows onto at least a portion of the virtual structure;
light the animated digital virtual 3D modeled creative elements with one or more virtual 3D lighting programs;
create digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements;
virtually project (and optionally texture) the digitally rendered planar animations back onto at least a portion of the virtual structure of the now empty virtual 3D model of the virtual structure from the same virtual camera position as in the previous step, in order to “bake in” a warped transformation of the digitally rendered planar animations; and
digitally render a 3D animation of the warped transformation of the projected rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of the physical projector to be installed at the site of the body of liquid; and
d) a digital video player or digital server computer sufficient to control playing and/or looping the digitally rendered 3D animation;
e) optionally one or more digital projectors for playing and/or looping the digitally rendered 3D animation, the one or more digital projectors sufficient to project the digitally rendered 3D animation of the warped transformation of the projected rendered planar animations through the body of liquid and onto at least a portion of the virtual structure, completing an illusion of the creative elements in the body of liquid;
f) optionally a support structure (portable or non-portable, modular or non-modular) for supporting the one or more digital projectors above or to one side of the body of liquid in the planned angle and location as the virtual position.

### Claim 22 (depends on 21)

The kit of claim 21 further comprising one or more digital projectors.

### Claim 23 (depends on 22)

The kit of claim 22 further comprising a support structure (portable or non-portable, modular or non-modular) for the one or more digital projectors.

### Claim 24 (depends on 21)

The kit of claim 21 wherein at least one of the structure walls is transparent.

### Claim 25 (depends on 24)

The kit of claim 24 further comprising a light-reflecting screen.