\Imagick noise reduction using masks to preserve details

Recently when I tried to do some noise-reduction with \Imagick I noticed that whatever function/filter I use too much detail is lost. Then I had the idea to use a mask which then is used to preserve details.

For the examples here, I used an 8-bit JPG-to-PNG-converted 4608x2592px photo. Cropped to 576x324px at two different positions.

<?php
$im = new Imagick();
$im->readImage('input.png');
$im->transformImageColorspace(Imagick::COLORSPACE_RGB);
// your denoise-filter belongs here :)
//$im->despeckleImage();
$im2 = clone $im;
$im->cropImage(576, 324, 2140, 1200);
$im2->cropImage(576, 324, 2130, 1450);
$im->transformImageColorspace(Imagick::COLORSPACE_SRGB);
$im2->transformImageColorspace(Imagick::COLORSPACE_SRGB);
$im->writeImage('src1.png');
$im2->writeImage('src2.png');

First of all, let’s take a look at some of the functions which might be used for noise reduction:

original
blurImage(0,1) While probably not specific for noise removal, blurring a photo will also reduce noise. Similar methods would be adaptiveBlurImage(), gaussianBlurImage() and radialBlurImage()
medianFilterImage(1) This one is pretty effective in removing the sort of noise most of my photos have. I do like this one.
despeckleImage() This one removes speckles; in reality/with my photos I noticed that it does blur the whole image somewhat. does not have any parameters to tune
enhanceImage() Well, the documentation states this one improves the quality of a noisy picture. does not have any parameters to tune
reduceNoiseImage(0) You can easily notice, that this looks similar to what blurImage(0, 1) does – not sure what reduceNoiseImage does under the hood.

To sum it up, none of the above preserves details as much as I would love to: How to reduce the effect of the above filters? I had two ideas for that, the first one is about averaging the filtered variant with the original and the second one is about just limiting the areas in which noise reduction is applied using an edge mask.

For the averaging part I do use this code:

<?php
$im = new Imagick();
$im->readImage('input.png');
$im->transformImageColorspace(Imagick::COLORSPACE_RGB);
$nr = clone $im;
$nr->blurImage(0, 1);
$nr->cropImage(576, 324, 2140, 1200);
$im->cropImage(576, 324, 2140, 1200);
 
$im->addImage($nr);
$im->resetIterator();
$avg = $im->averageImages();
$avg->transformImageColorspace(Imagick::COLORSPACE_SRGB);
$avg->writeImage('avg_blurimage.png');
original filtered averaged
blurImage(0,1)
medianFilterImage(1)
despeckleImage()
enhanceImage()
reduceNoiseImage(0)

This allows some sort of light filtering. Wouldn’t it be nice to use this light filtering for parts of the photo which contain edges/details and strong filtering for the remaining parts? A mask and some compositing will allow that.

The mask I want should contain areas which contain details; using an edge mask created by morphology leads to better results than using simple convolution with convolveImage() or creating a mask out of charcoalImage() for my use case. Anyway, to hand out three examples:

$mask1 = clone $im;
$mask1->morphology(Imagick::MORPHOLOGY_EDGE, 3, $kernel);
$mask1->thresholdImage(0.5 * $mask1->getQuantum());
$mask1->writeImage('mask1b.png');
$mask1->cropImage(576, 324, 2140, 1200);
$mask1->writeImage('mask1.png');
$mask2 = clone $im;
$matrix = array(-1, -1, -1, -1, 8, -1, -1, -1, -1);
$mask2->convolveImage($matrix);
$mask2->thresholdImage(0.5 * $mask2->getQuantum());
$mask2->writeImage('mask2b.png');
$mask2->cropImage(576, 324, 2140, 1200);
$mask2->writeImage('mask2.png');
$mask3 = clone $im;
$kernel = ImagickKernel::fromBuiltin(Imagick::KERNEL_OCTAGON, "3");
$mask3->charcoalImage(5, 3);
$mask3->thresholdImage(0.5 * $mask3->getQuantum());
$mask3->negateImage(false);
$mask3->writeImage('mask3b.png');
$mask3->cropImage(576, 324, 2140, 1200);
$mask3->writeImage('mask3.png');

If you do play around with the parameters (0.5 at thresholdImage) and if you add some softening before the mask creation (e.g. blurImage(0, 2)) you’ll get better results. However, the first masks does exactly what I want.

The generated mask is used with COPYOPACITY; black in the mask refers to opaque and white is transparent. Because I want to take the edges from the original onto the denoised/filtered image, the edges needs to be white (just as in the above example). Putting it together in PHP is somewhat weird / I would have loved something like overlay($src_im, $dst_im, $mask, $overlayVariant…) but that does not exist. Hence:

<?php
$im = new Imagick;
$im->readImage('input.png');
 
$kernel = ImagickKernel::fromBuiltin(Imagick::KERNEL_OCTAGON, "3");
 
$mask = clone $im;
$mask->morphology(Imagick::MORPHOLOGY_EDGE, 3, $kernel);
$mask->thresholdImage(0.5 * $mask->getQuantum());
 
$nmask = clone $mask;
$nmask->negateImage(false);
 
$denoised = clone $im;
$denoised->blurImage(0,5);
$denoised->compositeImage($nmask, imagick::COMPOSITE_COPYOPACITY, 0, 0);
 
$merged = clone $im;
$merged->compositeImage($mask, imagick::COMPOSITE_COPYOPACITY, 0, 0);
$merged->compositeImage($denoised, imagick::COMPOSITE_ADD, 0, 0);
 
$merged->cropImage(576, 324, 2140, 1200);
$merged->writeImage('composite_7_merged.png');

I guess, this requires some explanation. And well, this was a lot of trial and error. Basically, I am using the masks on the specific (original or filtered) picture so that the remaining parts are transparent. At the end I merge them together. Let me show that:

original
+ mask
= result
 
blurred
+ mask (negated)
= result
 
result mask1
+ result mask2
= final image

The issues with that approach are easily noticeable. While the gradient / heaven looks pretty good, some parts of the houses look worse. That is due to a few reasons:

  1. My edge-mask should not just contain edges but all the surrounding parts and in this case it isn’t thick enough. So I’ll get better results with tuning the mask.
  2. This variant of merging two images will always cause issues with filters that are „stretching“ like a gaussian blur. Sharpening might produce unwanted effects as well – at the borders of both images. I might improve that even more by averaging with the original.
  3. I used a pretty strong blurImage() – if you try the same with medianFilterImage(2) the result is way better. I’ll get better results with less blurry filters.
  4. alternatively: I need another way to merge everything.

Here’s the same image with ->despeckleImage() and ->medianFilterImage(2):


despeckle+median

despeckle+median+avg with original

orginal

despeckle+median+avg with original

Instead of averaging with the original, you might also average with the denoised variant (as long as you don’t use too blurry filters) to not re-introduce noise from the original while still keeping the image sharp. A tiny bit sharpening to the edge-containing picture might look good, too. I’m still experimenting to find good settings.

Helpful? Let me know! You’ve got a better way doing such stuff (with \Imagick or plain ImageMagick)? Please share it with me 🙂

Anyway, here’s the whole script in a function:

    /**
     * reducedDenoising
     *
     * this method combines a denoised photo with the original without touching the edges/details (and noise within
     * that edges) and averages the result with the original. If no denoised photo is given, despeckleImage,
     * medianFilterImage(3) and reduceNoiseImage(0) is used to create one.
     *
     * @author Jean-Michel Bruenn <himself@jeanbruenn.info>
     * @see https://jeanbruenn.info/2018/10/20/imagick-noise-reduction-using-masks-to-preserve-details/
     * @param \Imagick $im Source photo
     * @param int $edgeIterations The number of iteration to apply the morphology function
     * @param float $edgeThreshold 0.1 to 1.0 are useful; less = more detail (and noise) preserved
     * @param \Imagick $denoised optional denoised photo
     * @return \Imagick returns the processed photo
     */
    function reducedDenoising(\Imagick $im, $edgeIterations = 3, $edgeThreshold = 0.15, \Imagick $denoised = null)
    {
        $kernel = ImagickKernel::fromBuiltin(Imagick::KERNEL_OCTAGON, "3");
 
        $mask = clone $im;
        $mask->blurImage(0, 1);
        $mask->morphology(Imagick::MORPHOLOGY_EDGE, $edgeIterations, $kernel);
        $mask->thresholdImage($edgeThreshold * $mask->getQuantum());
 
        $nmask = clone $mask;
        $nmask->negateImage(false);
 
        if (is_null($denoised)) {
            $denoised = clone $im;
            $denoised->despeckleImage();
            $denoised->reduceNoiseImage(0);
            $denoised->medianfilterimage(3);
        }
 
        $denoised->compositeImage($nmask, imagick::COMPOSITE_COPYOPACITY, 0, 0);
 
        $merged = clone $im;
        $merged->compositeImage($mask, imagick::COMPOSITE_COPYOPACITY, 0, 0);
        $merged->compositeImage($denoised, imagick::COMPOSITE_ADD, 0, 0);
 
        $merged->addImage($im);
        $merged->resetIterator();
 
        return $merged->averageImages();
    }

No Comments

Post a Comment