# US 11,308,650 B2 · Display apparatus, image providing apparatus, and methods of controlling the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,650 B2 - Document ID: US11308650B2 - Status: Granted - Kind: B2 - Application number: 16/730,291 - Filed: 2019-12-30 - Published: 2020-07-09 - Granted: 2022-04-19 - Earliest claimed: 2019-01-03 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Doohyun Kim; Minyong Sung; Sangkwon Na; Kiwon Yoo; Sangjo Lee - Web page: https://patentgrep.com/patent/US11308650B2 ## CPC - G06T 9/002: {using neural networks} - H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06T 5/60: using machine learning, e.g. neural networks - G06T 5/70: Denoising; Smoothing - G06T 5/73: Deblurring; Sharpening - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/172: the region being a picture, frame or field - H04N 19/184: the unit being bits, e.g. of the compressed video stream - H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513) - H04N 19/20: using video object coding - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 19/85: using pre-processing or post-processing specially adapted for video compression - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Disclosed are a display apparatus, an image providing apparatus, and methods of controlling the same, the display apparatus including: a display; and a processor configured to: decode an encoded video stream, decompress the video stream through a neural network including a plurality of channels and a plurality of layers with a parameter set based on learning, and perform image compensation determined based on learning about the decompression with respect to the video stream. ## Claims ### Claim 1 A display apparatus comprising: a display; and a processor configured to: decode an encoded video stream through a decoding module, wherein the decoded video stream comprises a first image, decompress the decoded video stream through a neural network, wherein the neural network comprises a plurality of channels and a plurality of layers, the neural network is configured to operate with a parameter set, and the parameter set is identified based on a learning operation about the neural network, wherein an output performance of the neural network with regard to another input image is evaluated based on the learning operation and a configuration of the neural network is changeable based on an evaluation result on the output performance of the neural network to identify a final configuration, and perform a function corresponding to an image compensation through at least one of a preprocessing module connected to a front-end of the neural network or a postprocessing module connected to a back-end of the neural network, wherein the image compensation is identified based on a problem of the neural network, the problem of the neural network corresponding to a predetermined image pattern with regard to the neural network with the final configuration changed based on a final evaluation result on the output performance of the neural network, and wherein the decoding module, the preprocessing module, the neural network and the postprocessing module are configured to connected serially so that the video stream passes sequentially to output the first image, and the image compensation based on the problem of the neural network is pre-processed by the preprocessing module or post-processed by the postprocessing module. ### Claim 2 (depends on 1) The display apparatus according to claim 1, wherein the image compensation is at least one of preprocessing performed through the preprocessing module or postprocessing performed through the postprocessing module. ### Claim 3 (depends on 2) The display apparatus according to claim 2, wherein the processor is further configured to: identify features of the first image, and determine whether to perform the preprocessing or the postprocessing with regard to the first image based on the identified features. ### Claim 4 (depends on 2) The display apparatus according to claim 2, wherein the processor is further configured to: obtain, from the decoded video stream, a flag, wherein the flag is configured to indicate whether the preprocessing or the postprocessing is to be performed. ### Claim 5 (depends on 2) The display apparatus according to claim 2, wherein the processor is further configured to determine whether to perform the preprocessing or the postprocessing, based on one among a frame unit, a block unit and a pixel unit of the first image. ### Claim 6 (depends on 1) The display apparatus according to claim 1, wherein the image compensation comprises at least one of dejagging, decontouring, or deblurring corresponding to the predetermined image pattern. ### Claim 7 An image providing apparatus comprising: a processor configured to: compress an input image through a neural network, wherein the neural network comprises a plurality of channels and a plurality of layers, the neural network is configured to operate with a parameter set, and the parameter set is identified based on a learning operation about the neural network, wherein an output performance of the neural network with regard to another input image is evaluated based on the learning operation and a configuration of the neural network is changeable based on an evaluation result on the output performance of the neural network to identify a final configuration, perform a function corresponding to an image compensation through at least one of a preprocessing module connected to a front-end of the neural network or a postprocessing module connected to a back-end of the neural network, wherein the image compensation is identified based on a problem of the neural network, the problem of the neural network corresponding to a predetermined image pattern with regard to the neural network with the final configuration changed based on a final evaluation result on the output performance of the neural network, and generate an encoded video stream through an encoding module configured to encode the input image compressed and compensated; and data interface circuitry configured to output the encoded video stream, and wherein the preprocessing module, the neural network, the postprocessing module and the encoding module are configured to connected serially so that the input image passes sequentially to output the encoded video stream, and the image compensation based on the problem of the neural network is pre-processed by the preprocessing module or post-processed by the postprocessing module. ### Claim 8 (depends on 7) The image providing apparatus according to claim 7, wherein the image compensation is at least one of preprocessing performed through the preprocessing module or postprocessing performed through the postprocessing module. ### Claim 9 (depends on 7) The image providing apparatus according to claim 7, wherein the encoded video stream comprises a flag indicating whether second preprocessing or second postprocessing is to be performed in a display apparatus configured to decode the encoded video stream. ### Claim 10 (depends on 7) The image providing apparatus according to claim 7, wherein the image compensation comprises noise making corresponding to the predetermined image pattern, and the noise making is configured to compensate for at least one of jagging, contouring, or blurring. ### Claim 11 A method of controlling a display apparatus, the method comprising: decoding an encoded video stream through a decoding module, wherein the decoded video stream includes a first image; decompressing the decoded video stream through a neural network, wherein the neural network comprises a plurality of channels and a plurality of layers, the neural network is configured to operate with a parameter set, and the parameter set is identified based on a learning operation about the neural network, wherein an output performance of the neural network with regard to another input image is evaluated based on the learning operation and a configuration of the neural network is changeable based on an evaluation result on the output performance of the neural network to identify a final configuration; and performing a function corresponding to an image compensation through at least one of a preprocessing module connected to a front-end of the neural network or a postprocessing module connected to a back-end of the neural network, wherein the image compensation is identified based on a problem of the neural network, the problem of the neural network corresponding to a predetermined image pattern with regard to the neural network with the final configuration changed based on a final evaluation result on the output performance of the neural network, and wherein the decoding module, the preprocessing module, the neural network and the postprocessing module are configured to connected serially so that the video stream passes sequentially to output the first image, and the image compensation based on the problem of the neural network is pre-processed by the preprocessing module or post-processed by the postprocessing module. ### Claim 12 (depends on 11) The method according to claim 11, wherein the image compensation is at least one of preprocessing performed through the preprocessing module or postprocessing performed through the postprocessing module. ### Claim 13 (depends on 12) The method according to claim 12, further comprising: identifying features of the first image; and determining whether to perform the preprocessing or the postprocessing with regard to the first image based on the identified features. ### Claim 14 (depends on 12) The method according to claim 12, further comprising: obtaining, from the decoded video stream, a flag, wherein the flag is configured to indicate whether the preprocessing or the postprocessing is to be performed. ### Claim 15 (depends on 11) The method according to claim 11, wherein the image compensation comprises at least one of dejagging, decontouring, or deblurring corresponding to the predetermined image pattern. ### Claim 16 A method of controlling an image providing apparatus, the method comprising: compressing an input image through a neural network, wherein the neural network comprises a plurality of channels and a plurality of layers, the neural network is configured to operate with a plurality of parameters, and the plurality of parameters are identified based on a learning operation about the neural network, wherein an output performance of the neural network with regard to another input image is evaluated based on the learning operation and a configuration of the neural network is changeable based on an evaluation result on the output performance of the neural network to identify a final configuration; performing a function corresponding to an image compensation through at least one of a preprocessing module connected to a front-end of the neural network or a postprocessing module connected to a back-end of the neural network, wherein the image compensation is identified based on a problem of the neural network, the problem of the neural network corresponding to a predetermined image pattern with regard to the neural network with the final configuration changed based on a final evaluation result on the output performance of the neural network; generating an encoded video stream through an encoding module configured to encode the input image compressed and compensated; and outputting the encoded video stream, and wherein the preprocessing module, the neural network, the postprocessing module and the encoding module are configured to connected serially so that the input image passes sequentially to output the encoded video stream, and the image compensation based on the problem of the neural network is pre-processed by the preprocessing module or post-processed by the postprocessing module. ### Claim 17 (depends on 16) The method according to claim 16, wherein the image compensation is at least one of preprocessing or postprocessing performed through the preprocessing module or postprocessing performed through the postprocessing module. ### Claim 18 (depends on 16) The method according to claim 16, wherein the performing of the image compensation comprises making noise corresponding to the predetermined image pattern, and wherein the noise making is configured to compensate for at least one of jagging, contouring, or blurring.