# US 9,898,861 B2 · Systems and methods for projecting planar and 3D images through water or liquid onto a surface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,898,861 B2
- Document ID: US9898861B2
- Status: Granted
- Kind: B2
- Application number: 14/935,594
- Filed: 2015-11-09
- Published: 2016-05-26
- Granted: 2018-02-20
- Earliest claimed: 2014-11-24
- Assignees: Pointcloud Media, LLC
- Applicants: Pointcloud Media, LLC
- Inventors: Jacobus G. Hattingh
- Web page: https://patentgrep.com/patent/US9898861B2

## CPC

- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G09G 3/003: {to produce spatial visual effects}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Systems and methods for projecting planar and 3D images through water or liquid onto a surface include creating a 3D model of the body of liquid and surface and 3D models of creative elements to be used in scenes. Animating the 3D models of the creative elements, placing them inside the 3D model of the body of liquid. Lighting the animated creative elements, rendering planar animations of the modeled creative elements and, using projection and texturing software, virtually projecting the planar animations back onto the surface of the 3D model of the body of liquid from the same camera position in order to “bake in” a warped transformation of the digitally rendered planar animations. Digitally rendering a 3D animation of the warped, transformed planar animations, and playing or looping the digitally rendered 3D animation through the body of liquid on a digital video player or digital server.

## Claims

### Claim 1

A method comprising:
a) selecting a body of liquid for projecting images through, the body of liquid contained within a surface;
b) gathering measurements of the body of liquid and surface;
c) creating a digital virtual 3D model of the body of liquid and the surface by loading the measurements into a memory of a digital computer, the computer including at least one processor programmed by computer software sufficient to create virtual 3D images of the body of liquid and surface;
d) creating digital virtual 3D models of creative elements to be used in one or more scenes using the at least one processor programmed by computer software sufficient to create the digital virtual 3D models of the creative elements;
e) digitally animating the digital virtual 3D models of the creative elements using the at least one processor programmed by computer software sufficient to digitally animate the digital virtual 3D models of the creative elements;
f) placing the digitally animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid using the at least one processor programmed by computer software sufficient to place the digitally animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid, using the digital virtual 3D model of the body of liquid and surface to virtually place the digitally animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the body of liquid and cast shadows onto the digital virtual 3D model of the surface;
g) lighting the animated digital virtual 3D modeled creative elements using the at least one processor programmed with one or more virtual 3D lighting programs to create the cast shadows;
h) creating digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements using the at least one processor programmed by digital rendering software;
i) using the at least one processor programmed by digital projection software, virtually projecting the digitally rendered planar animations back onto the digital virtual 3D model of the surface of an empty digital virtual 3D model of the body of liquid from the same virtual camera position as in the previous step, thereby electronically baking forced perspective images of the digitally rendered planar animations onto the digital virtual 3D model of the surface; and
j) using the at least one processor programmed by digital rendering software, digitally rendering a 3D animation of the electronically baked forced perspective images of the digitally rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of a digital physical projector to be installed at a site of the body of liquid.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
k) playing and/or looping the digitally rendered 3D animation on a digital video player or digital server at the site and projecting the digitally rendered 3D animation from the digital physical projector installed on a structure above or to one side of the body of liquid in the planned angle and location as the virtual position.

### Claim 3 (depends on 1)

The method of claim 1 wherein the gathering of measurements is by a method selected from the group consisting of by hand, from CAD drawings, from blueprints, using a laser scanner, and combinations of two or more of these.

### Claim 4 (depends on 1)

The method of claim 1 wherein the creating of digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements comprises:
A) digitally rendering the lighted, animated digital virtual 3D modeled creative elements from a digital virtual camera directly above the virtual body of liquid in the digital 3D virtual model if the body of liquid and surface are located in the center of a viewing area; or
B) digitally rendering the lighted, animated digital virtual 3D modeled creative elements from a digital virtual camera located at a center of a side vantage point in the digital 3D virtual model if the body of liquid and surface are to be viewed from one side only.

### Claim 5 (depends on 1)

The method of claim 1 wherein the digitally rendering of the 3D animation of the electronically baked forced perspective images of the digitally rendered planar animations comprises:
A) in the case of an internal installed projector, digitally rendering the animations from the same angle as the internal installed projector using a spherical, fisheye or mirrored lens; or
B) in the case of an internal (existing light replacement) projector, replacing the glass lens with a manufactured bubble lens to accommodate a wide, fisheye or mirror lens.

### Claim 6 (depends on 1)

The method of claim 1 wherein the body of liquid is selected from the group consisting of a swimming pool, a reflecting pool, a pond, and a fountain.

### Claim 7 (depends on 1)

The method of claim 1 wherein the computer software sufficient to create virtual 3D images, to create the digital virtual 3D models of the creative elements, digitally animate creative elements, place the digitally animated creative elements inside the digital virtual 3D model of the body of liquid, and the one or more virtual 3D lighting programs includes one or more:
i) options for 3D unwrapping, shading, physics, dynamics and particles, real time 3D/game creation, camera projection shading, 2D and 3D procedural brushes, edge rendering, and/or collision simulation;
ii) integration software built into the program, allowing seamless integration of 3D scenes into the digital rendering, projection, and texturing software;
iii) allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software; and
iv) a software toolset enabling creation of flying logos and similar simple effects.

### Claim 8 (depends on 1)

The method of claim 1 wherein the digital rendering and projection software is selected from software that allows changes to the 3D animation to automatically be updated inside of the digital rendering and projection software.

### Claim 9 (depends on 1)

The method of claim 1 comprising virtually texturing the digitally rendered planar animations using the at least one processor programmed by digital projection and texturing software.

### Claim 10

A method comprising:
a) selecting a swimming pool or pond for projecting images through, the swimming pool or pond contained within a surface;
b) gathering exact measurements of an outer rim as well as the inside surface area of the pool or pond by hand or from CAD drawings, blueprints or a laser scanner;
c) creating a digital virtual 3D model of the swimming pool or pond and the surface by loading the measurements into a memory of a digital computer, the computer including at least one processor programmed by computer software sufficient to make the digital virtual 3D images of the body of water and surface;
d) creating digital virtual 3D models of creative elements to be used in one or more scenes using the digital computer and the computer software;
e) animating the digital virtual 3D models making sure that moveable creative elements (for example, human characters and non-human creatures) travel within the digital virtual 3D model of the swimming pool or pond;
f) placing the animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the swimming pool or pond using the digital virtual 3D model of the swimming pool or pond and surface to virtually place the digitally animated digital virtual 3D modeled creative elements inside the digital virtual 3D model of the swimming pool or pond and cast shadows onto the digital virtual 3D model of the surface;
g) lighting the animated digital virtual 3D modeled creative elements using the programmed digital computer programmed with one or more digital virtual 3D lighting programs in such a way to cast the shadows of the animated digital virtual 3D modeled creative elements on the digital virtual 3D model of the surface, causing enhanced 3D augmentation in the swimming pool or pond when the animation is played therein;
h) creating digital rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements and/or stills by:
A) rendering the lighted, animated digital virtual 3D modeled creative elements and/or stills from a virtual camera directly above the virtual swimming pool or pond in the 3D virtual model if the swimming pool or pond and surface are located in the center of a viewing area; or
B) rendering the lighted, animated digital virtual 3D modeled creative elements and/or stills from a virtual camera located at a center of a side vantage point in the 3D virtual model if the swimming pool or pond and surface are to be viewed from one side only;
i) using the programmed digital computer, virtually projecting the digital rendered planar animations back onto the digital virtual 3D model of the surface of an empty digital virtual 3D model of the swimming pool or pond from the same virtual camera position as in the previous step, thereby electronically baking forced perspective images of the digital rendered planar animations onto the digital virtual 3D model of the surface;
j) digitally rendering a 3D animation of the electronically baked forced perspective images of the digital rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of a digital physical projector to be installed at a site of the swimming pool or pond; and
k) at the site, installing the digital physical projector as planned on a structure above or to one side of the swimming pool or pond in the same position as the virtual position, connecting the digital physical projector to a video player or server capable of playing digital and video images, loading the digitally rendering a 3D animation into the video player or server, waiting until the sun sets or reasonable darkness if the swimming pool or pond is outdoors, focusing the digital physical projector and lining up the digital virtual 3D model of the swimming pool or pond with the swimming pool or pond while projecting the digital virtual 3D model of the swimming pool or pond onto the surface containing the swimming pool or pond, and playing and/or looping the digitally rendering a 3D animation.

### Claim 11 (depends on 10)

The method of claim 10 wherein the computer software sufficient to create virtual 3D images, to create the digital virtual 3D models of the creative elements, digitally animate creative elements, place the digitally animated creative elements inside the digital virtual 3D model of the of the swimming pool or pond, and the one or more virtual 3D lighting programs includes one or more:
i) options for 3D unwrapping, shading, physics, dynamics and particles, real time 3D/game creation, camera projection shading, 2D and 3D procedural brushes, edge rendering, and/or collision simulation;
ii) integration software built into the program, allowing seamless integration of 3D scenes into the digital rendering, projection, and texturing software;
iii) allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software; and
iv) a software toolset enabling creation of flying logos and similar simple effects.

### Claim 12 (depends on 10)

The method of claim 10 wherein the digital rendering and projection software is selected from software that allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software.

### Claim 13 (depends on 10)

The method of claim 10 comprising virtually texturing the digitally rendered planar animations using the at least one processor programmed by digital projection and texturing software.

### Claim 14

A system comprising:
a) a digital computer, the digital computer including a memory and at least one processor programmed by computer software sufficient to make digital virtual 3D images of a body of liquid and a surface containing the body of liquid,
b) the memory sufficient to hold measurements of the body of liquid and the surface, the at least one processor programmed by computer software sufficient to:
create digital virtual 3D models of creative elements;
animate the digital virtual 3D models of the creative elements;
place at least some of the animated digital virtual 3D modeled creative elements inside the digital virtual 3D image of the body of liquid using the digital virtual 3D images of the body of liquid and surface to place the at least some of the animated digital virtual 3D modeled creative elements into the digital virtual 3D image of the body of liquid and cast shadows onto the digital virtual 3D image of the surface;
light at least some of the animated digital virtual 3D modeled creative elements with one or more virtual 3D lighting programs to create the cast shadows;
create digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements;
virtually project the digitally rendered planar animations back onto the digital virtual 3D image of the surface of an empty digital virtual 3D image of the body of liquid from the same virtual camera position as in the previous step, thereby electronically baking forced perspective images of the digitally rendered planar animations onto the digital virtual 3D image of the surface; and
digitally render a 3D animation of the electronically baked forced perspective images of the digitally rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of a digital physical projector to be installed at a site of the body of liquid.

### Claim 15 (depends on 14)

The system of claim 14 further comprising a digital video player or digital server computer, and one or more digital physical projectors, the digital video player or digital server computer configured sufficiently to play and/or loop the 3D animation, the one or more digital physical projectors installed on a structure above or to one side of the body of liquid in the planned angle and location as the virtual position, the one or more digital physical projectors configured to project at least a portion of the digitally rendered 3D animation through the body of liquid and onto the surface, completing an illusion of at least some of the creative elements in the body of liquid.

### Claim 16 (depends on 15)

The system of claim 15 wherein the one or more digital physical projectors transmit an image at a resolution, represented by horizontal pixel count, of at least 4K (4096×2160 or 8.8 megapixels).

### Claim 17 (depends on 16)

The system of claim 16 wherein the one or more digital projectors are selected from the group consisting of DLP and laser cinema projectors.

### Claim 18 (depends on 15)

The system of claim 15 wherein the server computer supports JPEG2000 and MPEG2 files, and dual-projector 3D playback.

### Claim 19 (depends on 14)

The system of claim 14 wherein the body of liquid is selected from the group consisting of a swimming pool, a reflecting pool, a pond, and a fountain.

### Claim 20 (depends on 14)

The system of claim 14 wherein the computer software sufficient to create virtual 3D images, to create the digital virtual 3D models of the creative elements, digitally animate creative elements, place the digitally animated creative elements inside the digital virtual 3D model of the of the body of liquid and surface, and the one or more virtual 3D lighting programs includes one or more:
i) options for 3D unwrapping, shading, physics, dynamics and particles, real time 3D/game creation, camera projection shading, 2D and 3D procedural brushes, edge rendering, and/or collision simulation;
ii) integration software built into the program, allowing seamless integration of 3D scenes into the digital rendering, projection, and texturing software;
iii) allows changes to the 3D animation to automatically be updated inside of the digital rendering, projection, and texturing software; and
iv) a software toolset enabling creation of flying logos and similar simple effects.

### Claim 21 (depends on 14)

The system of claim 14 wherein the digital rendering and projection software is selected from software that allows changes to the 3D animation to automatically be updated inside of the digital rendering and projection software.

### Claim 22 (depends on 14)

The system of claim 14 wherein the at least one processor programmed by computer software is sufficient to virtually texture the digitally rendered planar animations.

### Claim 23

A system comprising:
a) a swimming pool contained within a surface;
b) a digital computer, the digital computer including a memory and at least one processor programmed by computer software sufficient to make virtual 3D images of the swimming pool and surface, the memory sufficient to hold measurements of the swimming pool and surface;
c) the at least one processor programmed by computer software sufficient to create digital virtual 3D models of creative elements to be used in one or more scenes using the computer and the computer software;
d) the at least one processor programmed by computer software sufficient to animate the digital virtual 3D models of the creative elements;
e) the at least one processor programmed by computer software sufficient to place at least some of the animated digital virtual 3D modeled creative elements inside the digital virtual 3D image of the swimming pool using the digital virtual 3D image of the swimming pool and surface to place objects inside the digital virtual 3D image of the swimming pool and cast shadows onto the digital virtual 3D image of the surface;
f) the at least one processor programmed by computer software sufficient to light the animated digital virtual 3D modeled creative elements with one or more virtual 3D lighting programs to create the cast shadows;
g) the at least one processor programmed by computer software sufficient to create digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements;
h) the at least one processor programmed by computer software sufficient to virtually project the digitally rendered planar animations back onto the digital virtual 3D image of the surface of an empty virtual 3D image of the swimming pool from the same virtual camera position as in the previous step, thereby electronically baking forced perspective images of the digitally rendered planar animations onto the digital virtual 3D image of the surface;
i) the at least one processor programmed by computer software sufficient to digitally render a 3D animation of the electronically baked forced perspective images of the virtually projected rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of a digital physical projector to be installed at a site of the swimming pool and surface;
j) a digital video player or digital server computer sufficient to play and/or loop the 3D animation; and
k) one or more digital physical projectors installed on a structure above or to one side of the swimming pool and surface in the planned angle and location as the virtual position, the one or more digital physical projectors sufficient to project the digitally rendered 3D animation of the electronically baked forced perspective images of the virtually projected rendered planar animations through the swimming pool and onto the surface, completing an illusion of at least some of the lighted, animated digital virtual 3D modeled creative elements in the swimming pool.

### Claim 24 (depends on 23)

The system of claim 23 wherein the at least one processor programmed by computer software is sufficient to virtually texture the digitally rendered planar animations.

### Claim 25

A kit comprising:
a) a structure for containing a body of liquid comprising a rear wall, a front wall, a floor, and left and right side walls;
b) a digital physical projector for projecting a digital image through a body of liquid held by the structure;
c) a digital computer, the digital computer including a memory and at least one processor programmed by computer software sufficient to make digital virtual 3D images of the structure and body of liquid, the memory sufficient to hold measurements of the structure and the body of liquid, the at least one processor programmed by computer software sufficient to:
create digital virtual 3D models of creative elements;
animate the digital virtual 3D models of the creative elements;
place the animated digital virtual 3D modeled creative elements inside the digital virtual 3D image of the body of liquid using the digital virtual 3D image of the body of liquid and structure to place the animated digital virtual 3D modeled creative elements inside the digital virtual 3D image of the body of liquid and cast shadows onto at least a portion of the digital virtual 3D image of the structure;
light the animated digital virtual 3D modeled creative elements with one or more virtual 3D lighting programs to create the cast shadows;
create digitally rendered planar animations of the lighted, animated digital virtual 3D modeled creative elements;
virtually project the digitally rendered planar animations back onto at least a portion of the digital virtual 3D image of the structure of an empty virtual 3D image of the digital virtual 3D image of the structure from the same virtual camera position as in the previous step, thereby electronically baking forced perspective images of the digitally rendered planar animations onto the digital virtual 3D image of the structure; and
digitally render a 3D animation of the electronically baked forced perspective images of the projected rendered planar animations from a virtual position that is at the same angle and position as a planned angle and location of a physical projector to be installed at a site of the body of liquid; and
d) a digital video player or digital server computer sufficient to control playing and/or looping the digitally rendered 3D animation;
e) optionally one or more digital physical projectors for projecting the digitally rendered 3D animation, the one or more digital physical projectors sufficient to project the digitally rendered 3D animation of the electronically baked forced perspective images of the projected rendered planar animations through the body of liquid and onto at least a portion of the virtual structure, completing an illusion of the lighted, animated digital virtual 3D modeled creative elements in the body of liquid;
f) optionally a support structure (portable or non-portable, modular or non-modular) for supporting the one or more digital physical projectors above or to one side of the body of liquid in the planned angle and location as the virtual position.

### Claim 26 (depends on 25)

The kit of claim 25 further comprising one or more digital physical projectors.

### Claim 27 (depends on 26)

The kit of claim 26 further comprising a support structure (portable or non-portable, modular or non-modular) for the one or more digital physical projectors.

### Claim 28 (depends on 25)

The kit of claim 25 wherein at least one of the structure walls is transparent.

### Claim 29 (depends on 28)

The kit of claim 28 further comprising a light-reflecting screen.

### Claim 30 (depends on 25)

The kit of claim 25 wherein the at least one processor programmed by computer software is sufficient to virtually texture the digitally rendered planar animations.