Semantic-Based Image Signal Processing

A non-transitory computer-readable medium having stored thereon instructions that, when executed by a computing device, cause the computing device to perform operations comprising:

• obtaining an input image having a first spatial resolution;
• generating, based on the input image and by a bilateral grid model comprising a machine learning (ML) model, a bilateral grid representing, at a second spatial resolution that is smaller than the first spatial resolution, a plurality of transform coefficients configured to map pixel values of the input image to filter guide values, wherein each respective transform coefficient of the plurality of transform coefficients is based on a corresponding class of an image feature represented by a corresponding pixel of the input image, and wherein the corresponding class is one of a plurality of classes of image features represented by the input image;
• generating, based on the bilateral grid, one or more transform coefficients corresponding to a selected pixel of the input image;
• applying the one or more transform coefficients to a pixel value of the selected pixel to generate a filter guide value for the selected pixel at the first spatial resolution;
• and
• generating an output image by applying a corresponding operator to the pixel value of the selected pixel based on the filter guide value for the selected pixel, wherein at least one parameter of the corresponding operator differs across the plurality of classes as a function of the filter guide value.

A computer-implemented method comprising:

• obtaining an input image having a first spatial resolution;
• generating, based on the input image and by a bilateral grid model comprising a machine learning (ML) model, a bilateral grid representing, at a second spatial resolution that is smaller than the first spatial resolution, a plurality of transform coefficients configured to map pixel values of the input image to filter guide values, wherein each respective transform coefficient of the plurality of transform coefficients is based on a corresponding class of an image feature represented by a corresponding pixel of the input image, and wherein the corresponding class is one of a plurality of classes of image features represented by the input image;
• generating, based on the bilateral grid, one or more transform coefficients corresponding to a selected pixel of the input image;
• applying the one or more transform coefficients to a pixel value of the selected pixel to generate a filter guide value for the selected pixel at the first spatial resolution;
• and
• generating an output image by applying a corresponding operator to the pixel value of the selected pixel based on the filter guide value for the selected pixel, wherein at least one parameter of the corresponding operator differs across the plurality of classes as a function of the filter guide value.