# US 11,445,218 B2 · Image processing apparatus and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,218 B2 - Document ID: US11445218B2 - Status: Granted - Kind: B2 - Application number: 16/764,707 - Filed: 2018-11-12 - Published: 2021-06-17 - Granted: 2022-09-13 - Earliest claimed: 2017-11-24 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Takeshi Tsukuba - Web page: https://patentgrep.com/patent/US11445218B2 ## CPC - H04N 19/61: in combination with predictive coding - H04N 19/88: involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks - H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264 - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/184: the unit being bits, e.g. of the compressed video stream - H04N 19/426: using memory downsizing methods - H04N 19/96: Tree coding, e.g. quad-tree coding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract There is provided an image processing apparatus and an image processing method for enabling suppression of an increase in a memory capacity required for orthogonal transform and inverse orthogonal transform. A second transformation matrix is derived using a first transformation matrix, a prediction residual of an image is orthogonally transformed using the derived second transformation matrix, and coefficient data obtained by orthogonally transforming the prediction residual is encoded to generate a bit stream. The present disclosure can be applied to, for example, an image processing apparatus, an image encoding device, an image decoding device, or the like. ## Claims ### Claim 1 An image processing apparatus comprising: a derivation unit configured to derive a second transformation matrix using a first transformation matrix; an orthogonal transform unit configured to orthogonally transform a prediction residual of an image, using the second transformation matrix derived by the derivation unit; and an encoding unit configured to encode coefficient data obtained by orthogonally transforming the prediction residual by the orthogonal transform unit to generate a bit stream, wherein the derivation unit derives the second transformation matrix using a transform type identifier of the first transformation matrix in each one of a horizontal direction and a vertical direction, and wherein the derivation unit, the orthogonal transform unit, and the encoding unit are each implemented via at least one processor. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the derivation unit derives the second transformation matrix of a transform type different from that of the first transformation matrix, using the first transformation matrix. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the derivation unit derives the second transformation matrix having a same number of rows and a same number of columns as the first transformation matrix, using the first transformation matrix. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein the derivation unit derives the second transformation matrix by an operation for an element of the first transformation matrix. ### Claim 5 (depends on 4) The image processing apparatus according to claim 4, wherein the derivation unit derives the second transformation matrix by performing the operation a plurality of times. ### Claim 6 (depends on 1) The image processing apparatus according to claim 1, wherein the derivation unit derives the second transformation matrix, using the first transformation matrix to be stored in a lookup table. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the orthogonal transform unit performs primary transform for the prediction residual, using the second transformation matrix derived by the derivation unit, and further performs secondary transform for a result of the primary transform. ### Claim 8 (depends on 1) The image processing apparatus according to claim 1, wherein the derivation unit derives the second transformation matrix in which a lowest-order row vector has a waveform of a desired type, using the first transformation matrix. ### Claim 9 (depends on 1) The image processing apparatus according to claim 1, wherein the orthogonal transform unit orthogonally transforms the prediction residual, using a transform unit (TU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 10 (depends on 1) The image processing apparatus according to claim 1, wherein the encoding unit encodes the coefficient data, using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 11 An image processing method comprising: deriving a second transformation matrix using a first transformation matrix; orthogonally transforming a prediction residual of an image, using the derived second transformation matrix; and encoding coefficient data obtained by orthogonally transforming the prediction residual to generate a bit stream, wherein the second transformation matrix is derived using a transform type identifier of the first transformation matrix in each one of a horizontal direction and a vertical direction. ### Claim 12 An image processing apparatus comprising: a decoding unit configured to decode a bit stream to obtain coefficient data that is an orthogonally transformed prediction residual of an image; a derivation unit configured to derive a second transformation matrix, using a first transformation matrix; and an inverse orthogonal transform unit configured to inversely orthogonally transform the coefficient data obtained by the decoding unit, using the second transformation matrix derived by the derivation unit, wherein the derivation unit derives the second transformation matrix using a transform type identifier of the first transformation matrix in each one of a horizontal direction and a vertical direction, and wherein the decoding unit, the derivation unit, and the inverse orthogonal transform unit are each implemented via at least one processor. ### Claim 13 (depends on 12) The image processing apparatus according to claim 12, wherein the inverse orthogonal transform unit performs inverse secondary transform for the coefficient data, and further performs inverse primary transform for a result of the inverse secondary transform, using the second transformation matrix derived by the derivation unit. ### Claim 14 (depends on 12) The image processing apparatus according to claim 12, wherein the decoding unit decodes the bit stream, using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 15 (depends on 12) The image processing apparatus according to claim 12, wherein the inverse orthogonal transform unit inversely orthogonally transforms the coefficient data, using a transform unit (TU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 16 An image processing method comprising: decoding a bit stream to obtain coefficient data that is an orthogonally transformed prediction residual of an image; deriving a second transformation matrix, using a first transformation matrix; and inversely orthogonally transforming the obtained coefficient data, using the derived second transformation matrix, wherein the second transformation matrix is derived using a transform type identifier of the first transformation matrix in each one of a horizontal direction and a vertical direction. ### Claim 17 An image processing apparatus comprising: an operation unit configured to perform an operation for permutation for a prediction residual of an image; an orthogonal transform unit configured to orthogonally transform the prediction residual operated for permutation by the operation unit, using a transformation matrix serving as a base; and an encoding unit configured to encode coefficient data obtained by orthogonally transforming the prediction residual by the orthogonal transform unit to generate a bit stream, wherein the transformation matrix is a second transformation matrix derived using a transform type identifier of a first transformation matrix in each one of a horizontal direction and a vertical direction, and wherein the operation unit, the orthogonal transform unit, and the encoding unit are each implemented via at least one processor. ### Claim 18 (depends on 17) The image processing apparatus according to claim 17, wherein the operation unit flips the prediction residual in a spatial symmetry direction of during one-dimensional orthogonal transform, and the orthogonal transform unit orthogonally transforms the prediction residual flipped by the operation unit, using the transformation matrix. ### Claim 19 (depends on 17) The image processing apparatus according to claim 17, further comprising: a derivation unit configured to derive the second transformation matrix using the first transformation matrix, wherein the orthogonal transform unit is configured to orthogonally transform the prediction residual flipped by the operation unit, using the second transformation matrix derived by the derivation unit, and wherein the derivation unit is implemented via at least one processor. ### Claim 20 (depends on 19) The image processing apparatus according to claim 19, wherein the derivation unit inverts a sign of an odd-numbered row vector, of the first transformation matrix, and derives the second transformation matrix. ### Claim 21 (depends on 17) The image processing apparatus according to claim 17, wherein the orthogonal transform unit orthogonally transforms the prediction residual, using a transform unit (TU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 22 (depends on 17) The image processing apparatus according to claim 17, wherein the encoding unit encodes the coefficient data, using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 23 An image processing method comprising: performing an operation for permutation for a prediction residual of an image; orthogonally transforming the prediction residual operated for permutation, using a transformation matrix serving as a base; and encoding coefficient data obtained by orthogonally transforming the prediction residual to generate a bit stream, wherein the transformation matrix is a second transformation matrix derived using a transform type identifier of a first transformation matrix in each one of a horizontal direction and a vertical direction. ### Claim 24 An image processing apparatus comprising: a decoding unit configured to decode a bit stream to obtain coefficient data that is an orthogonally transformed prediction residual of an image; an inverse orthogonal transform unit configured to inversely orthogonally transform the coefficient data obtained by the decoding unit, using a transformation matrix; and an operation unit configured to perform an operation for permutation for an inverse orthogonal transform result of the coefficient data obtained by the inverse orthogonal transform unit, wherein the transformation matrix is a second transformation matrix derived using a transform type identifier of a first transformation matrix in each one of a horizontal direction and a vertical direction, and wherein the decoding unit, the inverse orthogonal transform unit, and the operation unit are each implemented via at least one processor. ### Claim 25 (depends on 24) The image processing apparatus according to claim 24, further comprising: a derivation unit configured to derive the second transformation matrix using the first transformation matrix, wherein the inverse orthogonal transform unit is configured to inversely orthogonally transform the coefficient data obtained by the decoding unit, using the second transformation matrix derived by the derivation unit, wherein the operation unit is configured to perform an operation for permutation for the inverse orthogonal transform result of the coefficient data obtained by the inverse orthogonal transform unit, and wherein the derivation unit is implemented via at least one processor. ### Claim 26 (depends on 24) The image processing apparatus according to claim 24, wherein the decoding unit decodes the bit stream, using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 27 (depends on 24) The image processing apparatus according to claim 24, wherein the inverse orthogonal transform unit inversely orthogonally transforms the coefficient data, using a transform unit (TU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a unit of processing. ### Claim 28 An image processing method comprising: decoding a bit stream to obtain coefficient data that is an orthogonally transformed prediction residual of an image; inversely orthogonally transforming the obtained coefficient data, using a transformation matrix; and performing an operation for permutation for an inverse orthogonal transform result of the obtained coefficient data, wherein the transformation matrix is a second transformation matrix derived using a transform type identifier of a first transformation matrix in each one of a horizontal direction and a vertical direction. ### Claim 29 (depends on 1) The image processing apparatus according to claim 1, wherein the derivation unit derives the second transformation matrix using at least one of a flip operation or a transposition operation.