# US 11,122,267 B2 · Method and apparatus for encoding image by using quantization table adaptive to image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,267 B2 - Document ID: US11122267B2 - Status: Granted - Kind: B2 - Application number: 16/671,286 - Filed: 2019-11-01 - Published: 2020-05-07 - Granted: 2021-09-14 - Earliest claimed: 2018-11-01 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Pilkyu Park; Kiljong Kim; Kwangpyo Choi - Web page: https://patentgrep.com/patent/US11122267B2 ## CPC - G06N 3/08: Learning methods - G06F 18/241: relating to the classification model, e.g. parametric or non-parametric approaches - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/063: using electronic means - G06N 3/09: Supervised learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks - H04N 19/124: Quantisation - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/18: the unit being a set of transform coefficients - H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Provided is a method of encoding an image, the method including: obtaining a plurality of patches from the image; obtaining a plurality of transform coefficient groups respectively corresponding to the plurality of patches; inputting, to a machine learning model, input values corresponding to transform coefficients included in each of the plurality of transform coefficient groups; quantizing transform coefficients corresponding to the image by using a quantization table output from the machine learning model; and generating a bitstream including data generated as a result of the quantizing and information about the quantization table. ## Claims ### Claim 1 A method of encoding an image, the method comprising: obtaining a plurality of patches from the image; obtaining a plurality of transform coefficient groups respectively corresponding to the plurality of patches; inputting, to a machine learning model trained to output a quantization table, input values corresponding to transform coefficients comprised in each of the plurality of transform coefficient groups; quantizing transform coefficients corresponding to the image by using the quantization table output from the machine learning model; and generating a bitstream comprising data generated as a result of the quantizing and information about the quantization table. ### Claim 2 (depends on 1) The method of claim 1, wherein: the machine learning model comprises a first fully connected layer (FCL) receiving the input values and a second FCL outputting element values comprised in the quantization table; and a number of input nodes of the first FCL corresponds to a number of the input values, and a number of output nodes of the second FCL corresponds to a number of elements comprised in the quantization table. ### Claim 3 (depends on 1) The method of claim 1, wherein the obtaining the plurality of patches comprises obtaining the plurality of patches by splitting the image into split regions of a predetermined size. ### Claim 4 (depends on 3) The method of claim 3, wherein adjacent split regions, among the split regions of the predetermined size, adjacent to each other in the image partially overlap. ### Claim 5 (depends on 1) The method of claim 1, wherein the obtaining the plurality of patches comprises obtaining the plurality of patches by alternately allocating pixels comprised in the image to the plurality of patches, respectively, in a predetermined order. ### Claim 6 (depends on 1) The method of claim 1, further comprising: classifying the transform coefficients comprised in each of the plurality of transform coefficient groups according to locations of the transform coefficients within each of the transform coefficient groups and obtaining the input values based on the classified transform coefficients. ### Claim 7 (depends on 6) The method of claim 6, wherein the obtaining the input values comprises obtaining at least one of a maximum value, a minimum value, or an average value of the classified transform coefficients from each of the locations in the transformation coefficient groups as the input values. ### Claim 8 (depends on 1) The method of claim 1, wherein: the machine learning model is trained to decrease a loss value calculated, after quantizing and inversely quantizing training transform coefficients corresponding to a training image by using a training quantization table, based on a difference between the training transform coefficients before the quantizing and the inversely quantized training transform coefficients; and the training quantization table is obtained by inputting training input values corresponding to the training image to the machine learning model. ### Claim 9 (depends on 1) The method of claim 1, wherein: the machine learning model is trained to decrease a loss value calculated, after quantizing and inversely quantizing training transform coefficients corresponding to a training image by using a training quantization table and reconstructing the training image by inversely transforming the inversely quantized training transform coefficients into pieces of data in a spatial domain, based on a difference between the reconstructed training image and the training image; and the training quantization table is obtained by inputting training input values corresponding to the training image to the machine learning model. ### Claim 10 (depends on 1) The method of claim 1, wherein: the machine learning model is trained to decrease a loss value calculated, after quantizing training transform coefficients corresponding to a training image by using a training quantization table, based on whether the quantized training transform coefficients are 0 or not; and the training quantization table is obtained by inputting training input values corresponding to the training image to the machine learning model. ### Claim 11 (depends on 10) The method of claim 10, wherein, based on the quantized training transform coefficients not being 0, the loss value increases in a direction from low frequency to high frequency. ### Claim 12 (depends on 1) The method of claim 1, wherein the machine learning model comprises: a first sub-model configured to receive the input values and output a predictive quantization table; and a second sub-model configured to receive a result of quantizing and inversely quantizing the input values with the predictive quantization table and the input values and outputting the quantization table. ### Claim 13 An apparatus for encoding an image, the apparatus comprising: at least one memory storing a machine learning model, trained to output a quantization table, and at least one instruction; and at least one processor configured to execute the at least one instruction to: obtain a plurality of patches from the image; obtain a plurality of transform coefficient groups respectively corresponding to the plurality of patches; input, to the machine learning model, input values corresponding to transform coefficients comprised in each of the plurality of transform coefficient groups; quantize transform coefficients corresponding to the image by using the quantization table output from the machine learning model; and generate a bitstream comprising data generated as a result of the quantizing and information about the quantization table. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein: the machine learning model comprises a first fully connected layer (FCL) receiving the input values and a second FCL outputting element values comprised in the quantization table; and a number of input nodes of the first FCL corresponds to a number of the input values, and a number of output nodes of the second FCL corresponds to a number of elements comprised in the quantization table. ### Claim 15 (depends on 13) The apparatus of claim 13, wherein the at least one processor is further configured to execute the at least one instruction to obtain the plurality of patches by splitting the image into split regions of a predetermined size. ### Claim 16 (depends on 13) The apparatus of claim 13, wherein the at least one processor is further configured to execute the at least one instruction to obtain the plurality of patches by alternately allocating pixels comprised in the image to the plurality of patches, respectively, in a predetermined order. ### Claim 17 (depends on 13) The apparatus of claim 13, wherein the at least one processor is further configured to execute the at least one instruction to classify the transform coefficients comprised in each of the plurality of transform coefficient groups according to locations of the transform coefficients within each of the transform coefficient groups and obtain the input values based on the classified transform coefficients.