# US 10,372,920 B2 · Non-rasterized image streaming system that uses ray tracing samples

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,920 B2
- Document ID: US10372920B2
- Status: Granted
- Kind: B2
- Application number: 16/118,070
- Filed: 2018-08-30
- Published: 2019-02-28
- Granted: 2019-08-06
- Earliest claimed: 2017-08-30
- Assignees: Go Ghost, LLC
- Applicants: Go Ghost, LLC
- Inventors: Erik Anderson; Mathew Fride
- Web page: https://patentgrep.com/patent/US10372920B2

## CPC

- G06F 21/602: {Providing cryptographic facilities or services}
- G06F 21/10: Protecting distributed programs or content, e.g. vending or licensing of copyrighted material (protection in video systems or pay television H04N7/16) {; Digital rights management [DRM]}
- G06T 15/005: {General purpose rendering architectures}
- G06T 15/06: Ray-tracing
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G09C 5/00: Ciphering apparatus or methods not provided for in the preceding groups, e.g. involving the concealment or deformation of graphic data such as designs, written or printed messages
- H04L 9/30: Public key, i.e. encryption algorithm being computationally infeasible to invert or user's encryption keys not requiring secrecy
- G06T 15/50: Lighting effects
- G06T 2210/08: Bandwidth reduction
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- H04L 2209/16: Obfuscation or hiding, e.g. involving white box
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09C: CIPHERING OR DECIPHERING APPARATUS FOR CRYPTOGRAPHIC OR OTHER PURPOSES INVOLVING THE NEED FOR SECRECY
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Secure rendering system that creates ray tracing samples with obfuscated positions, so that images can only be viewed by an authorized consumer able to recover the sample positions. Obfuscation of ray directions is integrated into the rendering process, for example by incorporating encryption into a lens shader. The rendering system never stores or transmits an image without obfuscating positions, so even the rendering system cannot see the image it is rendering. Embodiments may use public key cryptography, so that encryption of sample positions is done with a public key, and only the owner of the secret private key can view the rendered image. Since keys are asymmetric, the rendering system cannot decrypt the obfuscated samples. Piracy of rendered images is therefore mitigated. Some compositing operations may be performed on the secure rendering output prior to decrypting sample positions; for example, colors may be modified globally or for selected objects.

## Claims

### Claim 1

A non-rasterized image streaming system that uses ray tracing samples, comprising:
a memory;
a processor coupled with said memory, wherein said processor is configured to
obtain a plurality of ray tracing samples, each ray tracing sample of said plurality of ray tracing samples comprising
a ray direction associated with a ray in a three-dimensional model of a scene or a sample location corresponding to an intersection of said ray direction with said three-dimensional model of a scene; and
a sample color corresponding to said intersection;
obtain device information associated with each receiving device of a plurality of receiving devices, said device information comprising one or more of
a display resolution of said each receiving device;
a network quality of a network connection between said processor and said each receiving device;
a desired stream quality associated with said each receiving device;
transform said plurality of ray tracing samples into a stream content associated with each receiving device of said plurality of receiving devices, wherein
said stream content associated with said each receiving device does not include a rasterized image comprising a plurality of pixels;
said stream content associated with said each receiving device comprises said plurality of ray tracing samples or a subset of said plurality of ray tracing samples; and
said stream content associated with said each receiving device is based on said plurality of ray tracing samples and on said device information that corresponds to said each receiving device; and,
transmit said stream content associated with each receiving device of said plurality of receiving devices to said each receiving device over said network connection between said processor and said each receiving device;
a first receiving device of said plurality of receiving devices, said first device comprising an associated first device information comprising a first display resolution; and,
a second receiving device of said plurality of receiving devices, said second device comprising an associated second device information comprising a second display resolution;
wherein
said first display resolution comprises more pixels than said second display resolution; and,
a size of said stream content associated with said first receiving device is larger than a size of said stream content associated with said second receiving device.

### Claim 2 (depends on 1)

The system of claim 1, wherein said processor is further configured to compress said plurality of ray tracing samples.

### Claim 3 (depends on 1)

The system of claim 1 wherein said processor is further configured to apply one or more compositing operations to modify said plurality of ray tracing samples, without obtaining or generating said rasterized image comprising said plurality of pixels.

### Claim 4 (depends on 1)

The system of claim 1, wherein said network quality of said network connection comprises a bandwidth of said network connection.

### Claim 5 (depends on 1)

The system of claim 1, wherein said processor is further configured to
obtain an update to said network quality associated with said first receiving device of said plurality of receiving devices; and,
modify said stream content associated with said first receiving device based on said update to said network quality, wherein said stream content that is modified does not include said rasterized image comprising said plurality of pixels.

### Claim 6 (depends on 1)

The system of claim 1, wherein
said first device further comprises an associated first device information comprising a first network quality; and,
said second device further comprises an associated second device information comprising a second network quality;
wherein
said first network quality is higher than said second network quality; and,
a size of said stream content associated with said first receiving device is larger than a size of said stream content associated with said second receiving device.

### Claim 7 (depends on 1)

The system of claim 1, wherein
said first device further comprises an associated first device information comprising a first desired stream quality; and,
said second device further comprises an associated second device information comprising a second desired stream quality;
wherein
said first desired stream quality is higher than said second desired stream quality; and,
a size of said stream content associated with said first receiving device is larger than a size of said stream content associated with said second receiving device.

### Claim 8 (depends on 1)

The system of claim 1, wherein
said first receiving device is configured to generate said rasterized image comprising said plurality of pixels, based on ray tracing samples in said stream content received by said first receiving device.

### Claim 9 (depends on 8)

The system of claim 8, wherein said first receiving device further comprises a filter configured to select a subset of said ray tracing samples in said stream content received by said first receiving device.

### Claim 10 (depends on 8)

The system of claim 8, wherein
said ray direction associated with each ray tracing sample of said plurality of ray tracing samples comprises a scrambled ray direction derived from an original ray direction;
said sample location associated with each ray tracing sample corresponds to said scrambled ray direction; and
said first receiving device is further configured to
transform said plurality of ray tracing samples in said stream content received by said first receiving device into a corresponding plurality of unscrambled ray tracing samples, wherein each unscrambled ray tracing sample of said plurality of unscrambled ray tracing samples comprises
said sample color; and
said original ray direction or a sample location corresponding to said original ray direction; and,
generate said rasterized image based on said plurality of unscrambled ray tracing samples.

### Claim 11 (depends on 10)

The system of claim 10, wherein
said scrambled ray direction is derived from said original ray direction through encryption of said original ray direction with a cryptography algorithm and through use of an encryption key; and,
said first receiving device computes said original ray direction or said sample location corresponding to said original ray direction through decryption of said scrambled ray direction with said cryptography algorithm through use of a decryption key that matches said encryption key.

### Claim 12 (depends on 11)

The system of claim 11, wherein
said cryptography algorithm comprises a public key cryptography algorithm;
said encryption key comprises a public key of said public key cryptography algorithm; and,
said decryption key comprises a private key of said public key cryptography algorithm corresponding to said public key;
or
said decryption key comprises a plurality of private keys and wherein
said cryptography algorithm decrypts said scrambled ray direction using any one of said plurality of private keys or using a combination of two or more of said plurality of private keys.

### Claim 13 (depends on 11)

The system of claim 11, wherein
said cryptography algorithm comprises a symmetric key cryptography algorithm; and,
said encryption key and said decryption key are identical keys.

### Claim 14 (depends on 10)

The system of claim 10, wherein
said original ray direction of each ray tracing sample of said plurality of ray tracing samples is generated using a pseudorandom number generator and a seed;
said scrambled ray direction comprises an order in which said pseudorandom number generator generated said original ray direction;
said first receiving device further comprises said pseudorandom number generator and said seed; and,
said first receiving device uses said pseudorandom number generator, said seed, and said order to generate said original ray direction.

### Claim 15 (depends on 10)

The system of claim 10 wherein said processor is further configured to apply one or more compositing operations to modify said plurality of ray tracing samples, without obtaining or generating said rasterized image comprising said plurality of pixels, and without generating said original ray direction.

### Claim 16 (depends on 15)

The system of claim 15 wherein said apply one or more compositing operations comprises:
modify said sample color of one or more ray tracing samples of said plurality of ray tracing samples.

### Claim 17

A non-rasterized image streaming system that uses ray tracing samples, comprising:
a memory; and,
a processor coupled with said memory, wherein said processor is configured to
obtain a plurality of ray tracing samples, each ray tracing sample of said plurality of ray tracing samples comprising
a scrambled ray direction associated with a ray in a three-dimensional model of a scene, or a sample location corresponding to an intersection of said scrambled ray direction with said three-dimensional model of a scene, wherein said scrambled ray direction is derived from an original ray direction; and
a sample color corresponding to said intersection;
transform said plurality of ray tracing samples into a stream content associated with each receiving device of a plurality of receiving devices, wherein
said stream content associated with said each receiving device does not include a rasterized image comprising a plurality of pixels; and
said stream content associated with said each receiving device comprises said plurality of ray tracing samples or a subset of said plurality of ray tracing samples;
transmit said stream content associated with each receiving device of said plurality of receiving devices to said each receiving device over a network connection between said processor and said each receiving device;
wherein each receiving device of said plurality of receiving devices is configured to
transform said plurality of ray tracing samples in said stream content received by
said each receiving device into a corresponding plurality of unscrambled ray tracing samples, wherein each unscrambled ray tracing sample of said plurality of unscrambled ray tracing samples comprises
said sample color; and
said original ray direction or a sample location corresponding to said original ray direction; and,
generate said rasterized image based on said plurality of unscrambled ray tracing samples.

### Claim 18 (depends on 17)

The system of claim 17, wherein
said scrambled ray direction is derived from said original ray direction through encryption of said original ray direction with a cryptography algorithm and through use of an encryption key; and,
each receiving device of said plurality of receiving devices computes said original ray direction or said sample location corresponding to said original ray direction through decryption of said scrambled ray direction with said cryptography algorithm through use of a decryption key that matches said encryption key.

### Claim 19 (depends on 18)

The system of claim 18, wherein
said cryptography algorithm comprises a public key cryptography algorithm;
said encryption key comprises a public key of said public key cryptography algorithm; and,
said decryption key comprises a private key of said public key cryptography algorithm corresponding to said public key;
or
said decryption key comprises a plurality of private keys and wherein said cryptography algorithm decrypts said scrambled ray direction using any one of said plurality of private keys or using a combination of two or more of said plurality of private keys.

### Claim 20 (depends on 18)

The system of claim 18, wherein
said cryptography algorithm comprises a symmetric key cryptography algorithm; and,
said encryption key and said decryption key are identical keys.

### Claim 21 (depends on 17)

The system of claim 17, wherein
said original ray direction of each ray tracing sample of said plurality of ray tracing samples is generated using a pseudorandom number generator and a seed;
said scrambled ray direction comprises an order in which said pseudorandom number generator generated said original ray direction;
each receiving device of said plurality of receiving devices comprises said pseudorandom number generator and said seed; and,
each receiving device of said plurality of receiving devices uses said pseudorandom number generator, said seed, and said order to generate said original ray direction.

### Claim 22 (depends on 17)

The system of claim 17 wherein said processor is further configured to apply one or more compositing operations to modify said plurality said plurality of ray tracing samples, without obtaining or generating said rasterized image comprising said plurality of pixels, and without generating said original ray direction.

### Claim 23 (depends on 22)

The system of claim 22 wherein said apply one or more compositing operations comprises:
modify said sample color of one or more ray tracing samples of said plurality of ray tracing samples.

### Claim 24 (depends on 17)

The system of claim 17 wherein said processor is further configured to compress said plurality of ray tracing samples.

### Claim 25

A non-rasterized image streaming system that uses ray tracing samples, comprising:
a processor configured to
obtain a plurality of ray tracing samples, each ray tracing sample of said plurality of ray tracing samples comprising
a scrambled ray direction associated with a ray in a three-dimensional model of a scene, or a sample location corresponding to an intersection of said scrambled ray direction with said three-dimensional model of a scene, wherein said scrambled ray direction is derived from an original ray direction; and
a sample color corresponding to said intersection;
apply one or more compositing operations to modify said plurality of ray tracing samples, without obtaining or generating a rasterized image comprising a plurality of pixels, and without generating said original ray direction;
compress said plurality of ray tracing samples;
obtain device information associated with each receiving device of a plurality of receiving devices, said device information comprising one or more of
a display resolution of said each receiving device;
a network quality of a network connection between said processor and said each receiving device, wherein said network quality comprises a bandwidth of said network connection; and
a desired stream quality associated with said each receiving device;
wherein
a first device information associated with a first receiving device of said plurality of receiving devices comprises one or more of
a first display resolution;
a first network quality; and
a first desired stream quality;
a second device information associated with a second receiving device of said plurality of receiving devices comprises one or more of
a second display resolution;
a second network quality; and
a second desired stream quality;
and
said first display resolution comprises more pixels than said second display resolution; or
said first network quality is higher than said second network quality; or
said first desired stream quality is higher than said second desired stream quality; or
any combination thereof;
transform said plurality of ray tracing samples into a stream content associated with each receiving device of said plurality of receiving devices, wherein
said stream content associated with said each receiving device does not include said rasterized image comprising said plurality of pixels;
said stream content associated with said each receiving device comprises said plurality of ray tracing samples or a subset of said plurality of ray tracing samples; and
said stream content associated with said each receiving device is based on said plurality of ray tracing samples and on said device information that corresponds to said each receiving device; and
a size of said stream content associated with said first receiving device is larger than a size of said stream content associated with said second receiving device; and,
transmit said stream content associated with each receiving device of said plurality of receiving devices to said each receiving device over said network connection between said processor and said each receiving device;
wherein
each receiving device of said plurality of receiving devices is configured to
transform said plurality of ray tracing samples in said stream content received by said each receiving device into a corresponding plurality of unscrambled ray tracing samples, wherein each unscrambled ray tracing sample of said plurality of unscrambled ray tracing samples comprises
said sample color; and
said original ray direction or a sample location corresponding to said original ray direction; and,
generate said rasterized image based on said plurality of unscrambled ray tracing samples.

### Claim 26 (depends on 25)

The system of claim 25, wherein
said scrambled ray direction is derived from said original ray direction through encryption of said original ray direction with a cryptography algorithm and through use of an encryption key; and,
each receiving device of said plurality of receiving devices computes said original ray direction or said sample location corresponding to said original ray direction through decryption of said scrambled ray direction with said cryptography algorithm through use of a decryption key that matches said encryption key.

### Claim 27 (depends on 26)

The system of claim 26, wherein
said cryptography algorithm comprises a public key cryptography algorithm;
said encryption key comprises a public key of said public key cryptography algorithm; and,
said decryption key comprises a private key of said public key cryptography algorithm corresponding to said public key;
or
said decryption key comprises a plurality of private keys and wherein
said cryptography algorithm decrypts said scrambled ray direction using any one of said plurality of private keys or using a combination of two or more of said plurality of private keys.

### Claim 28 (depends on 26)

The system of claim 26, wherein
said cryptography algorithm comprises a symmetric key cryptography algorithm; and,
said encryption key and said decryption key are identical keys.

### Claim 29 (depends on 25)

The system of claim 25, wherein
said original ray direction of each ray tracing sample of said plurality of ray tracing samples is generated using a pseudorandom number generator and a seed;
said scrambled ray direction comprises an order in which said pseudorandom number generator generated said original ray direction;
each receiving device of said plurality of receiving devices comprises said pseudorandom number generator and said seed; and,
each receiving device of said plurality of receiving devices uses said pseudorandom number generator, said seed, and said order to generate said original ray direction.

### Claim 30

A non-rasterized image streaming system that uses ray tracing samples, comprising:
a memory;
a processor coupled with said memory, wherein said processor is configured to
obtain a plurality of ray tracing samples, each ray tracing sample of said plurality of ray tracing samples comprising
a ray direction associated with a ray in a three-dimensional model of a scene or a sample location corresponding to an intersection of said ray direction with said three-dimensional model of a scene; and
a sample color corresponding to said intersection;
obtain device information associated with each receiving device of a plurality of receiving devices, said device information comprising one or more of
a display resolution of said each receiving device;
a network quality of a network connection between said processor and said each receiving device;
a desired stream quality associated with said each receiving device;
transform said plurality of ray tracing samples into a stream content associated with each receiving device of said plurality of receiving devices, wherein
said stream content associated with said each receiving device does not include a rasterized image comprising a plurality of pixels;
said stream content associated with said each receiving device comprises said plurality of ray tracing samples or a subset of said plurality of ray tracing samples; and
said stream content associated with said each receiving device is based on said plurality of ray tracing samples and on said device information that corresponds to said each receiving device; and,
transmit said stream content associated with each receiving device of said plurality of receiving devices to said each receiving device over said network connection between said processor and said each receiving device;
a first receiving device of said plurality of receiving devices, said first device comprising an associated first device information comprising a first network quality; and,
a second receiving device of said plurality of receiving devices, said second device comprising an associated second device information comprising a second network quality;
wherein
said first network quality is higher than said second network quality; and,
a size of said stream content associated with said first receiving device is larger than a size of said stream content associated with said second receiving device.

### Claim 31

A non-rasterized image streaming system that uses ray tracing samples, comprising:
a memory;
a processor coupled with said memory, wherein said processor is configured to
obtain a plurality of ray tracing samples, each ray tracing sample of said plurality of ray tracing samples comprising
a ray direction associated with a ray in a three-dimensional model of a scene or a sample location corresponding to an intersection of said ray direction with said three-dimensional model of a scene; and
a sample color corresponding to said intersection;
obtain device information associated with each receiving device of a plurality of receiving devices, said device information comprising one or more of
a display resolution of said each receiving device;
a network quality of a network connection between said processor and said each receiving device;
a desired stream quality associated with said each receiving device;
transform said plurality of ray tracing samples into a stream content associated with each receiving device of said plurality of receiving devices, wherein
said stream content associated with said each receiving device does not include a rasterized image comprising a plurality of pixels;
said stream content associated with said each receiving device comprises said plurality of ray tracing samples or a subset of said plurality of ray tracing samples; and
said stream content associated with said each receiving device is based on said plurality of ray tracing samples and on said device information that corresponds to said each receiving device; and,
transmit said stream content associated with each receiving device of said plurality of receiving devices to said each receiving device over said network connection between said processor and said each receiving device;
a first receiving device of said plurality of receiving devices, said first device comprising an associated first device information comprising a first desired stream quality; and,
a second receiving device of said plurality of receiving devices, said second device comprising an associated second device information comprising a second desired stream quality;
wherein
said first desired stream quality is higher than said second desired stream quality; and,
a size of said stream content associated with said first receiving device is larger than a size of said stream content associated with said second receiving device.