# US 12,010,329 B2 · Embedding data within transformed coefficients using bit partitioning operations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,010,329 B2
- Document ID: US12010329B2
- Status: Granted
- Kind: B2
- Application number: 17/770,114
- Filed: 2020-10-23
- Published: 2022-12-22
- Granted: 2024-06-11
- Earliest claimed: 2019-10-25
- Assignees: V-Nova International Limited
- Applicants: V-Nova International Limited
- Inventors: Lorenzo Ciccarelli; Simone Ferrara; Guido Meardi
- Web page: https://patentgrep.com/patent/US12010329B2

## CPC

- H04N 19/162: User input
- H04N 19/467: characterised by the embedded information being invisible, e.g. watermarking
- H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N 19/124: Quantisation
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/18: the unit being a set of transform coefficients
- H04N 19/184: the unit being bits, e.g. of the compressed video stream
- H04N 19/33: in the spatial domain
- H04N 19/34: Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
- H04N 19/463: by compressing encoding parameters before transmission
- H04N 19/48: using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data (motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513)
- H04N 19/60: using transform coding
- H04N 19/88: involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Examples described herein relate to decoding and encoding signals. Certain examples described herein encapsulate custom data that is not signal data within a stream of encoded signal data. The custom data may comprise a wide variety of metadata that annotates the signal data, or provides additional information relating to the signal data. Certain examples described herein encapsulate custom data within a set of transformed coefficient values that represent data derived from a transform operation that forms part of the signal encoding. The encapsulation is may be performed by applying a bit shift operation to coefficient bits representing the set of transformed coefficient values.

## Claims

### Claim 1

A method of encoding signal data, comprising:
obtaining coefficient bits representing values for a set of transformed coefficients, the values being generated by applying at least a transform operation to blocks of signal data derived from an input signal being encoded;
obtaining userdata bits representing custom data to add to an encoded signal bitstream, wherein the userdata bits are arranged as groups of bits, where each group of bits has a length that is equal to a size of a bit shift performed as a part of a bit shift operation;
applying the bit shift operation to the coefficient bits, the bit shift operation shifting the coefficient bits by a predefined number of bits, where that predefined number of bits is based on the lengths of the groups of bits for the userdata bits;
setting values of a set of additional bits added to the coefficient bits based on the userdata bits to generate a modified set of coefficient bits;
instructing generation of an encoded bitstream using the modified set of coefficient bits, wherein the encoded bitstream carries both the custom data and an encoding of the signal data, and wherein the encoded bitstream is an encoded enhancement bitstream for a first enhancement sub-layer at a first level of quality;
obtaining further coefficient bits representing values for a set of transformed coefficients at a second level of quality, the second level of quality being higher than the first level of quality, the values being generated by applying at least the transform operation to blocks of signal data at the second level of quality;
instructing generation of an encoded enhancement bitstream for a second enhancement sub-layer at the second level of quality using the further coefficient bits; and
applying the bit shift operation.

### Claim 2 (depends on 1)

The method of claim 1, comprising, prior to obtaining coefficient bits:
obtaining the blocks of signal data derived from an input signal being encoded;
applying the transform operation to data from each of the blocks of signal data to generate initial transformed coefficients; and
quantising the initial transformed coefficients to generate the set of transformed coefficients.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
encoding the modified set of coefficient bits using one or more of entropy encoding and run-length encoding to generate the encoded bitstream.

### Claim 4 (depends on 1)

The method of claim 1, wherein the blocks of signal data comprise residual data generated by:
comparing data derived from the input signal being encoded and data derived from a reconstruction of the input signal,
wherein a first level of quality of a first representation of the input signal used to generate the data derived from the reconstruction of the input signal is lower than a second level of quality of a second representation of the input signal used to generate the data derived from the input signal being encoded.

### Claim 5 (depends on 4)

The method of claim 4, wherein the first representation of the input signal used to generate the data derived from the reconstruction of the input signal comprises a representation of an input signal at a resolution lower than that of the resolution of the second representation of the input signal used to generate the data derived from the input signal being encoded.

### Claim 6 (depends on 4)

The method of claim 4, wherein the encoded bitstream is an encoded enhancement bitstream to enhance an encoded base bitstream, the encoded base bitstream being an encoded representation of the input signal at the first level of quality.

### Claim 7 (depends on 1)

The method of claim 1, wherein the blocks of signal data comprise n by n blocks of signal data, and the transform operation implements a matrix multiplication applied to flattened vectors of length n2 representing the blocks of signal data, wherein the matrix multiplication comprises a multiplication with an n2 by n2 Hadamard matrix, wherein n may be an integer value greater than or equal to 2.

### Claim 8 (depends on 1)

The method of claim 1, wherein the transform operation outputs values for a set of data elements for each block of signal data, and the coefficient bits represent transformed coefficient values for a predefined one of the set of data elements.

### Claim 9

A method of decoding signal data, the method comprising:
obtaining an encoded bitstream;
decoding the encoded bitstream to obtain an initial set of coefficient bits representing values for a set of transformed coefficients, the values being generated during encoding by applying at least a transform operation to blocks of signal data derived from an input signal;
extracting userdata from a set of end bits of the initial set of coefficient bits, wherein the userdata is arranged as groups of bits, where each group of bits has a length that is equal to a size of a bit shift performed as a part of a bit shift operation;
applying the bit shift operation to the initial set of coefficient bits, the bit shift operation being in a direction that is opposite to a direction of a bit shift operation applied during encoding, the bit shift operation generating a reconstructed set of coefficient bits, the bit shift operation shifting the coefficient bits by a predefined number of bits, which is based on the lengths of the groups of bits for the userdata; and
instructing further decoding of the reconstructed set of coefficient bits, the further decoding comprising applying at least an inverse transform operation to values represented by the reconstructed set of coefficient bits, wherein the further decoding is used to generate a reconstruction of the input signal, wherein decoding the encoded bitstream comprises at least run-length decoding, and wherein further decoding of the reconstructed set of coefficient bits comprises applying an inverse quantisation operation prior to the inverse transform operation.

### Claim 10 (depends on 9)

The method of claim 9, wherein the reconstructed set of coefficient bits comprise transformed residual data, and the method further comprises:
instructing a combination of residual data obtained from the further decoding of the reconstructed set of coefficient bits with a reconstruction of the input signal generated from a representation of the input signal at a lower level of quality to generate a reconstruction of the input signal at a first level of quality, wherein the lower level of quality is lower in quality than the first level of quality.

### Claim 11 (depends on 10)

The method of claim 10, wherein the encoded bitstream is an encoded enhancement bitstream to enhance an encoded base bitstream, the reconstruction of the input signal being derived from a decoding of the encoded base bitstream.

### Claim 12 (depends on 10)

The method of claim 10, wherein the encoded bitstream is an encoded enhancement bitstream for a first enhancement sub-layer at the first level of quality and the method further comprises:
obtaining an encoded enhancement bitstream for a second enhancement sub-layer at a second level of quality;
decoding the encoded enhancement bitstream for the second enhancement sub-layer to obtain a second set of residual data for the second level of quality; and
instructing a combination of the second set of residual data with a reconstruction at the second level of quality derived from the reconstruction of the input signal at the first level of quality to generate a reconstruction of the input signal at the second level of quality.

### Claim 13 (depends on 9)

The method of claim 9, wherein extracting userdata comprises:
obtaining a parameter indicating a number of bits—D—used for userdata values;
obtaining bit values for a set of D bits that are located at one end of the initial set of coefficient bits, the set of D bits being added during the bit shift operation applied during encoding; and
post-processing the bit values to reconstruct a set of user datauserdata values.

### Claim 14 (depends on 9)

The method of claim 9, wherein the input signal is encoded using MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC).

### Claim 15

An encoder comprising:
a coefficient interface to obtain coefficient bits representing values for a set of transformed coefficients, the values being generated by applying at least a transform operation to blocks of signal data derived from an input signal being encoded;
a userdata interface to obtain userdata bits representing custom data to add to an encoded signal bitstream, wherein the userdata bits are arranged as groups of bits, where each group of bits has a length that is equal to a size of a bit shift performed as a part of a bit shift operation;
a userdata insertion component to apply a-the bit shift operation to the coefficient bits and to set values of a set of additional bits added to the coefficient bits based on the userdata, the bit shift operation shifting the coefficient bits by a predefined number of bits, where that predefined number of bits is based on the lengths of the groups of bits for the userdata bits; and
a bitstream encoding interface to output a modified set of coefficient bits as generated by the userdata insertion component for generation of an encoded bitstream,
wherein a bitstream encoder is communicatively coupled to the bitstream encoding interface comprising at least a run-length encoder to generate an encoded bitstream.

### Claim 16 (depends on 15)

The encoder of claim 15, the encoder further comprising:
a transformation component to obtain residual data organised in blocks and to transform the residual data on a block-by-block basis to generate values for a set of data elements for each block of signal data, the residual data being generated based on a comparison of data derived from the input signal and data reconstructed from a base encoding of the input signal; and
a quantisation component to quantise data derived from the transformation component,
wherein the coefficient interface is communicatively coupled to an output of the quantisation component,
wherein the userdata insertion component is applied to coefficient bits representing values for at least one of the set of data elements for a set of blocks forming at least part of a frame of signal data,
wherein the coefficient interface, the userdata interface, the userdata insertion component, the bitstream encoding interface, the transformation component, the quantisation component and the bitstream encoder form part of a first sub-layer of the encoder, the first sub-layer encoding a first residual data stream at a first spatial resolution, and
wherein the encoder comprises a second sub-layer comprising a transformation component, a quantisation component, and a bitstream encoder to encode a second residual data stream at a second spatial resolution, the second residual data stream being generated based on a comparison of data derived from the input signal and data reconstructed from the base encoding of the input signal and the first residual data stream.

### Claim 17

A decoder comprising:
a decoded bitstream interface to obtain an initial set of coefficient bits representing values for a set of transformed coefficients, the initial set of coefficient bits being decoded from an obtained encoded bitstream, the values being generated during encoding by applying at least a transform operation to blocks of signal data derived from an input signal;
a userdata extraction component to extract userdata from a set of end bits of the initial set of coefficient bits and then to apply a bit shift operation to the initial set of coefficient bits, the bit shift operation generating a reconstructed set of coefficient bits, wherein the userdata is arranged as groups of bits, where each group of bits has a length that is equal to a size of a bit shift performed as a part of the bit shift operation;
a coefficient decoding interface to output the reconstructed set of coefficient bits for further decoding, the further decoding comprising applying at least an inverse transform operation to values represented by the reconstructed set of coefficient bits, wherein the further decoding is used to generate a reconstruction of the input signal; and
a bitstream decoder communicatively coupled to the decoded bitstream interface comprising at least a run-length encoder to decode the obtained encoded bitstream.

### Claim 18 (depends on 17)

The decoder of claim 17, the decoder further comprising:
an inverse quantisation component to apply an inverse quantisation operation to data derived from the coefficient decoding interface; and
an inverse transformation component to apply the inverse transform operation on a block-by-block basis to data derived from the inverse quantisation component to obtain residual data organised in a set of signal blocks,
wherein the residual data being are combinable with a decoded base encoding of the input signal to generate a reconstruction of the input signal,
wherein the bitstream decoder, the decoded bitstream interface, the userdata extraction component, the coefficient decoding interface, the inverse quantisation component, and the inverse transformation component form part of a first sub-layer of the decoder, the first sub-layer decoding a first residual data stream at a first spatial resolution,
wherein the decoder comprises a second sub-layer comprising a bitstream decoder, an inverse quantisation component, and an inverse transformation component decode a second residual data stream at a second spatial resolution, the second residual data stream being generated based on a comparison of data derived from the input signal and data reconstructed from the base encoding of the input signal and the first residual data stream, and
wherein the second residual data stream is combinable with a reconstruction of the input signal generated based on the decoded base encoding and the first residual data stream to output a reconstruction of the input signal at the second spatial resolution.