# US 2019/0378305 A1 · Image compression method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378305 A1
- Document ID: US20190378305A1
- Status: Published application
- Kind: A1
- Application number: 16/310,756
- Filed: 2017-06-14
- Published: 2019-12-12
- Earliest claimed: 2016-06-17
- Assignees: Immersive Robotics Pty Ltd
- Applicants: Immersive Robotics Pty Ltd
- Inventors: Daniel Liam Fitzgerald; Timothy Simon LUCAS
- Web page: https://patentgrep.com/patent/US20190378305A1

## CPC

- G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00)
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- H04N 19/124: Quantisation
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/18: the unit being a set of transform coefficients
- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- G06T 2200/16: involving adaptation to the client's capabilities
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of compressing image data from one or more images forming part of digital reality content, the method including obtaining pixel data from the image data, the pixel data representing an array of pixels within the one or more images; determining a position of the array of pixels within the one or more images relative to a defined position, the defined position being at least partially indicative of a point of gaze of the user; and compressing the pixel data at least partially in accordance the determined position so that a degree of compression depends on the determined position of the array of pixels.

## Claims

### Claim at position 1

1-122. (canceled)

### Claim 123

A method of compressing image data representing one or more images, the method including:
a) obtaining pixel data from the image data, the pixel data representing an array of pixels within the one or more images;
b) applying a transformation to the pixel data to determine a set of frequency coefficients indicative of frequency components of the array of pixels;
c) selecting one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression and wherein the bit encoding scheme is selected at least in part based on:
i) a desired degree of compression; and,
ii) a position of the array of pixels in the one or more images;
d) selectively encoding at least some of the frequency coefficients using the selected bit encoding scheme to thereby generate a set of encoded frequency coefficients, wherein the bit encoding scheme defines the number of bits used to encode each of the frequency coefficients so that when the frequency coefficients are selectively encoded:
i) at least some of the encoded frequency coefficients are encoded with a different numbers of bits; and,
ii) at least one frequency coefficient is discarded so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and,
e) generating compressed image data using the encoded frequency coefficients.

### Claim 124 (depends on 123)

A method according to claim 123, wherein the method includes selecting the bit encoding scheme based on at least one of:
a) a transmission bandwidth of a wireless communications link used to transmit the compressed image data to a display device;
b) a transmission quality of service of a wireless communications link used to transmit the compressed image data to a display device;
c) a lens arrangement of a display device;
d) movement of a display device;
e) image display requirements of a display device;
f) a target display resolution of a display device.

### Claim 125 (depends on 123)

A method according to claim 123, wherein the frequency coefficients are selectively encoded so that at least one of:
a) a number of bits used to encode each frequency coefficient is defined irrespective of the value of the respective frequency coefficient;
b) a smaller number of bits are used to encode frequency coefficients corresponding to higher frequencies;
c) a progressively smaller number of bits are used to encode frequency coefficients corresponding to progressively higher frequencies;
d) at least one frequency coefficient is discarded corresponding to higher frequencies.

### Claim 126 (depends on 123)

A method according to claim 123, wherein the method includes applying a scaling factor to at least some of the frequency coefficients so that scaled frequency coefficients are encoded and wherein at least one of:
a) different scaling factors are applied to at least some frequency coefficients;
b) the same scaling factor is applied to each frequency coefficient; and,
c) the scaling factor is used to reduce a magnitude of each frequency coefficient.

### Claim 127 (depends on 123)

A method according to claim 123, wherein the frequency components are arranged in a plurality of levels and wherein each bit encoding scheme defines a respective number of bits to be used to encode the frequency coefficients in each of the plurality of levels.

### Claim 128 (depends on 123)

A method according to claim 123, wherein the method includes:
a) determining a point of gaze of an observer of the one or more images;
b) selectively encoding frequency coefficients at least partially in accordance with the point of gaze by:
i) determining a distance between the point of gaze and a position of the array of pixels in the one or more images; and,
ii) selectively encoding frequency coefficients in accordance with the distance so that less frequency coefficients are encoded for greater distances.

### Claim 129 (depends on 123)

A method according to claim 123, wherein the image data defines a plurality of channels, and wherein the method includes selectively encoding frequency coefficients for each channel in parallel by;
a) converting the RGB channels into YCbCr channels; and,
b) transforming the YCbCr channels by selectively encoding more frequency coefficients for the Y channel than the Cb or Cr channels.

### Claim 130 (depends on 123)

A method according to claim 123, wherein the method includes obtaining pixel data from image data by:
a) buffering image data corresponding to a next n−1 rows of pixels of the image;
b) buffering image data for a next n pixels of the next row of pixels;
c) obtaining pixel data for a next n×n block of pixels from the buffered image data;
d) repeating steps b) and c) until pixel data has been obtained from all of the n rows of pixels; and,
e) repeating steps a) to d) until pixel data has been obtained from each row of pixels of the image.

### Claim 131 (depends on 123)

A method according to claim 123, wherein the method includes:
a) selectively encoding frequency coefficients in parallel; and,
b) generating compressed image data at least in part by parallel to serial byte encoding.

### Claim 132 (depends on 123)

A method according to claim 123, wherein the digital reality is at least one of:
a) augmented reality;
b) virtual reality; and,
c) mixed reality.

### Claim 133 (depends on 123)

A method according to claim 123, wherein the method is used for displaying image data in a wearable digital reality headset by receiving the compressed image data from a computing device via at least one of:
a) a communication network; and,
b) a wireless communications link.

### Claim 134 (depends on 123)

A method according to claim 123, wherein the method is used for at least one of:
a) transmitting virtual reality video data; and,
b) wirelessly transmitting virtual reality video data.

### Claim 135

A method of decompressing compressed image data representing one or more images, the method including:
a) obtaining compressed image data;
b) determining a selected one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression, wherein the bit encoding scheme was selected at least in part based on:
i) a desired degree of compression; and,
ii) a position of the array of pixels in the one or more images;
c) decoding the encoded frequency coefficients in accordance with the selected bit encoding scheme by:
i) determining a set of encoded frequency coefficients from the compressed image data in accordance with the selected bit encoding scheme;
ii) performing bit decoding of the encoded frequency coefficients in accordance with the selected bit encoding scheme to thereby generate a set of frequency coefficients, wherein at least one frequency coefficient is generated so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and,
iii) applying an inverse transformation to the set of frequency coefficients to determine pixel data representing an array of pixels within the one or more images.

### Claim 136 (depends on 135)

A method according to claim 135, wherein the bit encoding scheme is selected based on at least one of:
a) a transmission bandwidth of a wireless communications link used to transmit the compressed image data to a display device;
b) a transmission quality of service of a wireless communications link used to transmit the compressed image data to a display device;
c) a lens arrangement of a display device;
d) movement of a display device;
e) image display requirements of a display device;
f) a target display resolution of a display device.

### Claim 137

Apparatus for compressing image data representing one or more images, the apparatus including at least one electronic encoder processing device that:
a) obtains pixel data from the image data, the pixel data representing an array of pixels within the one or more images;
b) applies a transformation to the pixel data to determine a set of frequency coefficients indicative of frequency components of the array of pixels;
c) selecting one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression and wherein the bit encoding scheme is selected at least in part based on:
i) a desired degree of compression; and,
ii) a position of the array of pixels in the one or more images;
d) selectively encodes at least some of the frequency coefficients using the selected bit encoding scheme to thereby generate a set of encoded frequency coefficients, wherein the bit encoding scheme defines the number of bits used to encode the frequency coefficients, and wherein the frequency coefficients are selectively encoded so that:
i) at least some of the encoded frequency coefficients have different numbers of bits; and,
ii) at least one frequency coefficient is discarded so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and,
e) generates compressed image data using the encoded frequency coefficients.

### Claim 138 (depends on 137)

Apparatus according to claim 137, wherein the at least one electronic encoder processing device selects the bit encoding scheme based on at least one of:
a) a transmission bandwidth of a wireless communications link used to transmit the compressed image data to a display device;
b) a transmission quality of service of a wireless communications link used to transmit the compressed image data to a display device;
c) a lens arrangement of a display device;
d) movement of a display device;
e) image display requirements of a display device;
f) a target display resolution of a display device.

### Claim 139 (depends on 137)

Apparatus according to claim 137, wherein the apparatus includes an encoder in wireless communication with a decoder allowing image data to be transferred between the encoder and decoder as compressed image data, wherein the encoder is at least one of coupled to and part of a suitably programmed processing system and the decoder is at least one of coupled to and part of a wearable display device.

### Claim 140 (depends on 137)

Apparatus according to claim 137, wherein the at least one encoder processing device includes:
a) a suitably programmed field programmable gate array;
b) an Application-Specific Integrated Circuit; and,
c) a Graphics Processing Unit.

### Claim 141 (depends on 137)

Apparatus according to claim 137, wherein the pixel data defines a plurality of channels, and wherein the apparatus includes:
a) a respective processing device for each channel; and,
b) a parallel processing device for processing each channel in parallel.

### Claim 142

Apparatus for decompressing compressed image data representing one or more images, the apparatus including at least one electronic decoder processing device that:
a) obtains compressed image data;
b) determines a selected one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression, wherein the bit encoding scheme was selected at least in part based on:
i) a desired degree of compression; and,
ii) a position of the array of pixels in the one or more images;
c) decodes the encoded frequency coefficients in accordance with the selected bit encoding scheme by:
i) determining a set of encoded frequency coefficients from the compressed image data in accordance with the selected bit encoding scheme that defines the number of bits used in each encoded frequency coefficient;
ii) performing bit decoding of the encoded frequency coefficients in accordance with the selected bit encoding scheme to thereby generate a set of frequency coefficients, wherein at least one frequency coefficient is generated so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and,
iii) applying an inverse transformation to the set of frequency coefficients to determine pixel data representing an array of pixels within the one or more images.