# US 2021/0279912 A1 · Method, apparatus, and storage medium using padding/trimming in compression neural network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279912 A1 - Document ID: US20210279912A1 - Status: Published application - Kind: A1 - Application number: 17/192,480 - Filed: 2021-03-04 - Published: 2021-09-09 - Earliest claimed: 2020-03-05 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics and Telecommunications Research Institute - Inventors: Joo-Young Lee; Se-Yoon Jeong; Hyoung-Jin Kwon; Dong-Hyun Kim; Youn-Hee Kim; Jong-Ho Kim; Tae-Jin Lee; Jin-Soo Choi - Web page: https://patentgrep.com/patent/US20210279912A1 ## CPC - G06T 9/002: {using neural networks} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06T 3/4023: based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels - G06T 3/4046: using neural networks - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An encoding apparatus extracts features of an image by applying multiple padding operations and multiple downscaling operations to an image represented by data and transmits feature information indicating the features to a decoding apparatus. The multiple padding operations and the multiple downscaling operations are applied to the image in an order in which one padding operation is applied and thereafter one downscaling operation corresponding to the padding operation is applied. A decoding method receives feature information from an encoding apparatus, and generates a to reconstructed image by applying multiple upscaling operations and multiple trimming operations to an image represented by the feature information. The multiple upscaling operations and the multiple trimming operations are applied to the image in an order in which one upscaling operation is applied and thereafter one trimming operation corresponding to the upscaling operation is applied. ## Claims ### Claim 1 An encoding method performed by an encoding apparatus, comprising: extracting features of an image by applying multiple padding operations and multiple downscaling operations to an image represented by data; and transmitting feature information indicating the features to a decoding apparatus. ### Claim 2 (depends on 1) The encoding method of claim 1, wherein the multiple padding operations and the multiple downscaling operations are applied to the image in an order in which one padding operation is applied to the image and thereafter one downscaling operation corresponding to the padding operation is applied to the image. ### Claim 3 (depends on 1) The encoding method of claim 1, wherein each of the multiple downscaling operations is configured to decrease a size of the image to 1/n in a horizontal direction and to 1/n in a vertical direction, where n is an integer equal to or greater than 2. ### Claim 4 (depends on 1) The encoding method of claim 1, wherein, in each of the multiple padding operations and each of the multiple downscaling operations, each padding operation is configured to adjust a size of the image to a multiple of 2, and a ratio of a downscaling operation corresponding to each padding operation is ½. ### Claim 5 (depends on 1) The encoding method of claim 1, wherein each of the multiple downscaling operations comprises processing of a convolutional layer and/or nonlinear processing of the image. ### Claim 6 (depends on 1) The encoding method of claim 1, wherein ratios of the multiple downscaling operations are different from each other. ### Claim 7 (depends on 1) The encoding method of claim 1, wherein ranges of one or more lines that are capable of being added through the multiple padding operations are different from each other. ### Claim 8 (depends on 1) The encoding method of claim 1, wherein information about an original size of the image is transmitted to the decoding apparatus. ### Claim 9 (depends on 1) The encoding method of claim 1, wherein number-of-lines information indicating numbers of one or more lines that are added through the multiple padding operations is transmitted to the decoding apparatus. ### Claim 10 A decoding method performed by a decoding apparatus, comprising: receiving feature information from an encoding apparatus; and generating a reconstructed image by applying multiple upscaling operations and multiple trimming operations to an image represented by the feature information. ### Claim 11 (depends on 10) The decoding method of claim 10, wherein the multiple upscaling operations and the multiple trimming operations are applied to the image in an order in which one upscaling operation is applied to the image and thereafter one trimming operation corresponding to the upscaling operation is applied to the image. ### Claim 12 (depends on 10) The decoding method of claim 10, wherein a number of pairs of the multiple upscaling operations and the multiple trimming operations is identical to a number of pairs of multiple padding operations and multiple downscaling operations performed by the encoding apparatus. ### Claim 13 (depends on 10) The decoding method of claim 10, wherein each of the multiple upscaling operations is configured to adjust a size of the image to n times, where n is an integer equal to or greater than 2. ### Claim 14 (depends on 10) The decoding method of claim 10, wherein: each of the multiple upscaling operations is configured to increase a size of the image to n times, each of the multiple trimming operations is configured to remove one or more lines, identical to a number of one or more lines that are added through a padding operation performed by the encoding apparatus, from the image to which the upscaling operation is applied, and n is an integer equal to or greater than 2. ### Claim 15 (depends on 10) The decoding method of claim 10, wherein number-of-lines information indicating numbers of one or more lines that are added through multiple padding operations performed by the encoding apparatus is received from the encoding apparatus. ### Claim 16 (depends on 10) The decoding method of claim 10, wherein: to ratio information is received from the encoding apparatus, the ratio information includes ratios of multiple downscaling operations performed by the encoding apparatus or reciprocals of the ratios of the multiple downscaling operations performed by the encoding apparatus. ### Claim 17 (depends on 16) The decoding method of claim 16, wherein: a k-th upscaling operation, among the multiple upscaling operations, is configured to adjust a size of the image to Sk times, k is an integer that is equal or greater than 1 and less than or equal to m, m is a number of the multiple downscaling operations, and Sk is determined based on the ratio information. ### Claim 18 (depends on 10) The decoding method of claim 10, wherein: a k-th trimming operation on the image, among the multiple trimming operations, is configured to remove, from the image, a number of lines identical to a number of lines that are added through a k-th padding operation corresponding to the k-th trimming operation, among multiple padding operations performed by the encoding apparatus, k is an integer that is equal to or greater than 1 and less than or equal to m, and m is a number of the multiple downscaling operations. ### Claim 19 (depends on 18) The decoding method of claim 18, wherein: a padding operation performed in a k-th order, among m padding operations performed by the encoding apparatus, corresponds to a trimming operation performed in an (m−k+1)-th order, among m trimming operations performed by the decoding apparatus, k is an integer that is equal to or greater than 1 and less than or equal to m, and m is a number of the multiple downscaling operations. ### Claim 20 A computer-readable storage medium storing a bitstream, the bitstream comprising feature information, wherein: a reconstructed image is generated by applying multiple upscaling operations and multiple trimming operations to an image represented by the feature information.