# US 11,907,835 B2 · Signal enhancement and manipulation using a signal-specific deep network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,835 B2 - Document ID: US11907835B2 - Status: Granted - Kind: B2 - Application number: 16/765,886 - Filed: 2018-11-26 - Published: 2020-11-19 - Granted: 2024-02-20 - Earliest claimed: 2017-11-26 - Assignees: Yeda Research and Development Co. Ltd. - Applicants: Yeda Research and Development Co. Ltd. - Inventors: Michal Irani; Assaf Shocher - Web page: https://patentgrep.com/patent/US11907835B2 ## CPC - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution - G06T 5/60: using machine learning, e.g. neural networks - G06N 3/048: Activation functions - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20132: Image cropping - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Given an input image, an image enhancement task, and no external examples available to train on, an Image-Specific Deep Network is constructed tailored to solve the task for this specific image. Since there are no external examples available to train on, the network is trained on examples extracted directly from the input image itself. The current solution solves the problem of Super-Resolution (SR), whereas the framework is more general and is not restricted to SR. ## Claims ### Claim 1 A computer implemented method for performing a task on a specific single test signal; the task is at least one selected from: super-resolution, deblurring, denoising, completion of missing data, distortion correction, correcting compression artifacts, dehazing, signal-enhancement, signal-manipulation, and degradation removal; the method comprising: constructing a signal-specific deep learning neural network (DLNN), by selecting number of nodes, number of layers, and their inter connections, configured for said task and said specific single test signal; generating training target outputs from said specific single test signal, via transformations thereof; generating a corresponding training input for each of said training target outputs, by degrading said training target outputs; training said DLNN to perform said task, using a training set comprising pairs of said training target outputs and their said corresponding training inputs, whereby the DLNN is trained to perform said task only on said specific single test signal and signals originated therefrom; and applying said trained DLNN on said specific single test signal, wherein said specific single test signal is provided as an input to said trained DLNN, to obtain an output signal in accordance with said task. ### Claim 2 (depends on 1) The method of claim 1, wherein said training target outputs are generated from at least one of: different parts of said specific single test signal, different dimensions of said specific single test signal. ### Claim 3 (depends on 1) The method according to claim 1, wherein each of said transformations comprises at least one selected from: rotating said specific single test signal to at least one angle, around at least one axis; mirror reflecting said specific single test signal, around at least one axis; scaling said specific single test signal along at least one axis; cropping part of said specific single test signal; applying an intensity transformation on said specific single test signal; applying a filter on said specific single test signal; applying a linear transformation on said specific single test signal; applying a non-linear transformation on said specific single test signal; applying identity transformation on said specific single test signal; and any combination thereof. ### Claim 4 (depends on 1) The method according to claim 1, wherein at least one of the following holds true: said specific single test signal is one selected from the group consisting of: image, video sequence, medical signal, MRI, fMRI, audio, an N-dimensional signal; said method further comprises a step of training said signal-specific DLNN using an external training set; and said method is implemented via at least one device selected from a group consisting of: a computer, a camera, a mobile phone, a screen, a photo editing software, a projector, a computer chip, a hardware platform. ### Claim 5 (depends on 1) The method according to claim 1, further comprising using said trained DLNN to estimate a degradation underlying said specific single test signal. ### Claim 6 (depends on 5) The method according to claim 5, wherein said estimated degradation is at least one of: a blur kernel, a non-uniform blur-field, a super-resolution downscaling kernel, a noise type, a compression type. ### Claim 7 (depends on 1) The method of claim 1, wherein the DLNN is a convolutional neural network (CNN). ### Claim 8 A computer implemented method for increasing a resolution of a specific single test image, the method comprising: constructing an image-specific deep learning neural network (DLNN), by selecting number of nodes, number of layers, and their inter connections, configured specifically for increasing the resolution of said specific single test image; generating training target outputs from said specific single test image, via transformations thereof; generating a corresponding training input for each of said training target outputs, by downscaling resolution of said training target outputs; training said DLNN to increase image resolution, using a training set comprising pairs of said training target outputs and their said corresponding training inputs, whereby the DLNN is trained to increase resolution only of said specific single test image and images originated therefrom; and applying said trained DLNN on said specific single test image, wherein said specific single test image is provided as an input to said trained DLNN, to obtain an output image with increased-resolution thereof. ### Claim 9 (depends on 8) The method according to claim 8, wherein said training target outputs are extracted from at least one of: different parts of said specific single test image, different dimensions of said specific single test image. ### Claim 10 (depends on 8) The method according to claim 8, wherein said generating of said training target outputs is configured to at least partially preserve cross-scale internal recurrence within said specific single test image. ### Claim 11 (depends on 8) The method according to claim 8, wherein each of said transformations comprises at least one selected from: rotating said specific single test image to at least one angle, around at least one axis; mirror reflecting said specific single test image, around at least one axis; scaling said specific single test image along at least one axis; cropping part of said specific single test image; applying a color transformation on said specific single test image; applying an intensity transformation on said specific single test image; applying a filter on said specific single test image; applying a linear transformation on said specific single test image; applying a non-linear transformation on said specific single test image; applying identity transformation; and any combination thereof. ### Claim 12 (depends on 8) The method according to claim 8, wherein said downscaling comprising lowering resolution of said training target outputs, using a desired super resolution (SR) scale factor (s). ### Claim 13 (depends on 12) The method according to claim 12 wherein said downscaling is gradual, provided via several intermediate SR scale factors. ### Claim 14 (depends on 8) The method according to claim 8, wherein said training is further provided using crops of at least some of said training target outputs and their corresponding training input crops. ### Claim 15 (depends on 8) The method according to claim 8, wherein at least one of the following holds true: said method further comprises a step of training said image-specific DLNN using an external training set; and said method is implemented via at least one device selected from a group consisting of: a computer, a camera, a mobile phone, a screen, a photo editing software, a projector, a computer chip, a hardware platform. ### Claim 16 (depends on 8) The method according to claim 8, further comprising using said trained DLNN to estimate a degradation underlying said specific single test image. ### Claim 17 (depends on 16) The method according to claim 16, wherein said estimated degradation is at least one of: a blur kernel, a non-uniform blur-field, a super-resolution downscaling kernel, a noise type, a compression type. ### Claim 18 (depends on 8) The method of claim 8, wherein the DLNN is a convolutional neural network (CNN).