Upscaling in Avisynth – Comparison of resizers

I made this tests because I was searching for a pretty good upscaler. Checking pictures manually is a very subjective thing and so I wanted a way to “calculate” how good a/an scaler/upscaler is. The easiest approach is to check, how similar the upscaled variant is to the original variant. Which is, what my tests do.

  1. used software/material
  2. preparation
  3. execution
  4. results
    1. testcase1 – big buck bunny 1920×1080 » lanczos 854×480 » 1920×1080
    2. testcase2 – sintel 1920×818 » lanczos 1126×480 » 1920×1080
    3. testcase3 – elephants dream 1920×1080 » bicubic 640×360 » 1920×1080
    4. overall result
    5. visual results
  5. conclusion

 
used software/material
MPlayer, Avisynth, Bash, Gimp, ImageMagick, OpenMovie Sintel, OpenMovie Big Buck Bunny, OpenMovie Elephants Dream

 
preparation
I took screenshots of a movie, stored the original highres image and created a lowscaled version of it. Then I created upscaled (to the same resolution as the highres image) variants of that lowscaled variant. And finally I compared the upscaled variant with the original highres variant.

 
execution
To create screenshots I have written a little Bashscript which gets screenshots using mplayer. The quality of the jpegs is set to 100 to have “lossless” images (more or less). The script needs ffmpeg to get the duration of the movie (pretty sure that might work with mplayer aswell, just didn’t find out how to easily parse that):

#!/bin/bash

infile=$1;
amount=$2;

# getting the length of the movie in seconds
ffmpeg -i "$infile" 2>/tmp/video.log
h=$(grep Duration /tmp/video.log | awk '{ print $2 }' | cut -d: -f1);
m=$(grep Duration /tmp/video.log | awk '{ print $2 }' | cut -d: -f2);
s=$(echo "($h * 60 * 60) + ($m * 60) - 60" | bc);

for (( x=1; x<=$amount; x++)); do
  stime=$(echo "$x * ($s / $amount)" | bc);
  echo $stime;
  tmp=$(mplayer -frames 1 -ss $stime -vo jpeg:quality=100:outdir=screens $infile > /dev/null);
  mv screens/00000001.jpg screens/$x.jpg
done

Then I’ve used gimp to create low-res variants of that screens – For the first two testcases I used lanczos and downscaled to a height of 480px. For the third testcase I used bicubic and downscaled to a height of 360px.

Then I’ve used lots of avisynth scripts to do the upscaling. An example for blackman and nnedi3:

source = ImageSource("3.jpg").Trim(0,-1)
us3 = BlackmanResize(source, 1920, 1080)
us13 = nnedi3_rpow2(source, rfactor=4, cshift="spline36resize", fwidth=1920, fheight=1080)

#ImageWriter(us3, file="3_blackman", type="jpeg")
ImageWriter(us13, file="3_nnedi3", type="jpeg")

The resulting upscaled image I’ve compared to the original image using ImageMagick:

convert originalimage.jpg upscaledimage.jpg -fx "abs(u[0]-u[1])" pgm:- \
  | identify -verbose pgm:- \
  | grep -E "mean|max|devi"

 
results

 
Big Buck Bunny

1920×1080 » (lanczos) 854×480 » 1920×1080

screenthumbnail method used peak average standard deviation filesize
1 original - - - 640K
bicubicResize 63 (0.247059) 1.39138 (0.0054564) 2.02799 (0.00795292) 564K
bilinearResize 66 (0.258824) 1.41046 (0.00553122) 2.1246 (0.00833176) 568K
blackmanResize 60 (0.235294) 1.32716 (0.00520453) 1.7842 (0.00699688) 584K
lanczosResize 59 (0.231373) 1.32695 (0.00520371) 1.76561 (0.00692397) 588K
lanczos4resize 58 (0.227451) 1.31528 (0.00515796) 1.73246 (0.00679397) 592K
pointresize 138 (0.541176) 1.93816 (0.00760062) 3.38685 (0.0132818) 688K
sincresize 97 (0.380392) 1.94942 (0.00764479) 3.12623 (0.0122597) 712K
spline16resize 60 (0.235294) 1.34655 (0.0052806) 1.85531 (0.00727572) 580K
spline36resize 58 (0.227451) 1.32948 (0.00521366) 1.78602 (0.00700402) 584K
spline64resize 59 (0.231373) 1.3242 (0.00519292) 1.77446 (0.00695865) 584K
nnedi3 (cshift=spline36) 59 (0.231373) 1.34151 (0.00526083) 1.76169 (0.00690858) 616K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 56 (0.219608) 1.34097 (0.0052587) 1.76592 (0.00692518) 620K
spline36resize(hq4x) 81 (0.317647) 1.61154 (0.00631977) 2.08531 (0.00817769) 568K
screenthumbnail method used peak average standard deviation filesize
2 original - - - 1100K
bicubicresize 131 (0.513725) 2.01965 (0.0079202) 2.58451 (0.0101353) 716K
bilinearresize 129 (0.505882) 2.06215 (0.00808685) 2.66154 (0.0104374) 724K
blackmanresize 128 (0.501961) 1.92173 (0.0075362) 2.37666 (0.00932023) 744K
lanczosresize 128 (0.501961) 1.91967 (0.0075281) 2.36059 (0.00925722) 752K
lanczos4resize 128 (0.501961) 1.89933 (0.00744836) 2.33001 (0.00913729) 756K
pointresize 173 (0.678431) 3.04469 (0.01194) 3.97362 (0.0155828) 924K
sincresize 167 (0.654902) 2.94485 (0.0115484) 3.62896 (0.0142312) 920K
spline16resize 128 (0.501961) 1.95451 (0.00766474) 2.44045 (0.00957039) 740K
spline36resize 128 (0.501961) 1.92393 (0.00754484) 2.37922 (0.00933026) 744K
spline64resize 129 (0.505882) 1.91636 (0.00751512) 2.36858 (0.00928855) 744K
nnedi3 (cshift=spline36) 119 (0.466667) 1.94348 (0.00762151) 2.34163 (0.00918287) 776K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 112 (0.439216) 1.94203 (0.00761579) 2.34288 (0.00918778) 776K
spline36resize(hq4x) 122 (0.478431) 2.19204 (0.00859625) 2.6589 (0.0104271) 744K
screenthumbnail method used peak average standard deviation filesize
3 original - - - 732K
bicubicresize 49 (0.192157) 1.32602 (0.00520009) 1.51821 (0.00595376) 572K
bilinearresize 53 (0.207843) 1.33697 (0.00524301) 1.57944 (0.0061939) 572K
blackmanresize 43 (0.168627) 1.29434 (0.00507583) 1.3829 (0.00542314) 592K
lanczosresize 41 (0.160784) 1.29722 (0.00508715) 1.37439 (0.00538977) 596K
lanczos4resize 41 (0.160784) 1.28806 (0.00505121) 1.35241 (0.00530357) 596K
pointresize 144 (0.564706) 1.86225 (0.00730295) 2.74764 (0.010775) 692K
sincresize 110 (0.431373) 1.85213 (0.00726325) 2.47998 (0.00972541) 708K
spline16resize 44 (0.172549) 1.306 (0.00512158) 1.42527 (0.00558931) 584K
spline36resize 43 (0.168627) 1.29596 (0.00508218) 1.38413 (0.00542798) 588K
spline64resize 42 (0.164706) 1.29306 (0.00507082) 1.37745 (0.00540178) 592K
nnedi3 (cshift=spline36) 39 (0.152941) 1.30354 (0.00511192) 1.36729 (0.00536193) 620K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 39 (0.152941) 1.30032 (0.00509928) 1.36227 (0.00534222) 624K
spline36resize(hq4x) 77 (0.301961) 1.52609 (0.00598468) 1.58047 (0.00619794) 572K

 
Sintel

1920×818 » (lanczos) 1126×480 » 1920×1080

screenthumbnail method used peak average standard deviation filesize
1_0
original - - - 620K
bicubicresize 76 (0.298039) 1.22795 (0.00481549) 1.64521 (0.00645182) 444K
bilinearresize 78 (0.305882) 1.22377 (0.0047991) 1.66566 (0.00653199) 440K
blackmanresize 72 (0.282353) 1.23057 (0.00482575) 1.57433 (0.00617384) 464K
lanczosresize 71 (0.278431) 1.2383 (0.00485609) 1.57933 (0.00619346) 468K
lanczos4resize 70 (0.27451) 1.24825 (0.00489511) 1.59137 (0.00624068) 472K
pointresize 106 (0.415686) 1.48345 (0.00581747) 2.42587 (0.00951323) 508K
sincresize 88 (0.345098) 1.47682 (0.00579147) 2.15301 (0.00844318) 492K
spline16resize 73 (0.286275) 1.2257 (0.00480666) 1.58467 (0.00621438) 460K
spline36resize 72 (0.282353) 1.23204 (0.00483153) 1.57656 (0.00618259) 464K
spline64resize 71 (0.278431) 1.23371 (0.00483806) 1.57686 (0.00618377) 464K
nnedi3 (cshift=spline36) 70 (0.27451) 1.2102 (0.00474587) 1.53526 (0.00602061) 484K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 76 (0.298039) 1.20684 (0.00473269) 1.53277 (0.00601086) 484K
spline36resize(hq4x) 77 (0.301961) 1.4199 (0.00556825) 1.62984 (0.00639155) 416K
screenthumbnail method used peak average standard deviation filesize
2_0
original - - - 1200K
bicubicresize 68 (0.266667) 3.90677 (0.0153207) 4.63845 (0.01819) 756K
bilinearresize 72 (0.282353) 3.93428 (0.0154286) 4.70239 (0.0184407) 772K
blackmanresize 70 (0.27451) 3.75073 (0.0147087) 4.45619 (0.0174752) 824K
lanczosresize 69 (0.270588) 3.77025 (0.0147853) 4.47537 (0.0175505) 832K
lanczos4resize 70 (0.27451) 3.80719 (0.0149302) 4.51547 (0.0177077) 832K
pointresize 141 (0.552941) 5.41135 (0.021221) 7.10313 (0.0278554) 992K
sincresize 113 (0.443137) 5.13697 (0.020145) 6.00752 (0.0235589) 880K
spline16resize 70 (0.27451) 3.76273 (0.0147558) 4.47469 (0.0175478) 812K
spline36resize 70 (0.27451) 3.75862 (0.0147397) 4.46286 (0.0175014) 820K
spline64resize 70 (0.27451) 3.7603 (0.0147463) 4.46454 (0.017508) 824K
nnedi3 (cshift=spline36) 75 (0.294118) 3.69705 (0.0144982) 4.36505 (0.0171178) 836K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 72 (0.282353) 3.68776 (0.0144618) 4.35658 (0.0170846) 836K
spline36resize(hq4x) 68 (0.266667) 3.9761 (0.0155926) 4.64142 (0.0182016) 760K
screenthumbnail method used peak average standard deviation filesize
3_0
original - - - 652K
bicubicresize 102 (0.4) 1.61719 (0.00634193) 3.41673 (0.013399) 516K
bilinearresize 113 (0.443137) 1.62747 (0.00638224) 3.50645 (0.0137508) 516K
blackmanresize 103 (0.403922) 1.57874 (0.00619113) 3.1816 (0.0124769) 544K
lanczosresize 107 (0.419608) 1.58978 (0.00623445) 3.1924 (0.0125192) 548K
lanczos4resize 118 (0.462745) 1.60308 (0.00628658) 3.20413 (0.0125652) 552K
pointresize 212 (0.831373) 2.33259 (0.00914742) 5.96943 (0.0234095) 612K
sincresize 141 (0.552941) 2.18924 (0.00858526) 4.94182 (0.0193797) 588K
spline16resize 102 (0.4) 1.58086 (0.00619945) 3.22538 (0.0126486) 536K
spline36resize 104 (0.407843) 1.58196 (0.00620377) 3.18951 (0.0125079) 544K
spline64resize 106 (0.415686) 1.58257 (0.00620615) 3.18302 (0.0124824) 544K
nnedi3 (cshift=spline36) 99 (0.388235) 1.52185 (0.00596802) 2.96699 (0.0116352) 560K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 101 (0.396078) 1.52184 (0.00596802) 2.98155 (0.0116923) 560K
spline36resize(hq4x) 106 (0.415686) 1.80159 (0.00706505) 3.38661 (0.0132808) 496K

Averaged

method average peak average mean average standard deviatation
bicubicresize 81.5 1.91 2.63
bilinearresize 85.16 1.93 2.70
blackmanresize 79.33 1.85 2.45
lanczosresize 79.16 1.85 2.45
lanczos4resize 80.83 1.86 2.45
pointresize 152.33 2.67 4.26
sincresize 119.33 2.59 3.72
spline16resize 79.5 1.86 2.5
spline36resize 79.16 1.85 2.46
spline64resize 79.5 1.85 2.45
nnedi3 (cshift=spline36) 76.83 1.83 2.38
nnedi3 (cshift=spline36, pscnr=4, etype=1) 76 1.83 2.39
spline36resize(hq4x) 88.5 2.08 2.66

 
Elephants Dream

1920×1080 » (bicubic) 640×360 » 1920×1080

screenthumbnail method used peak average standard deviation filesize
1_1
original - - - 696K
blackmanresize 182 (0.713725) 5.15758 (0.0202258) 9.2996 (0.036469) 672K
lanczosresize 181 (0.709804) 5.17115 (0.020279) 9.25688 (0.0363015) 676K
lanczos4resize 180 (0.705882) 5.19959 (0.0203906) 9.21027 (0.0361187) 684K
spline16resize 183 (0.717647) 5.21444 (0.0204488) 9.46563 (0.0371201) 672K
spline36resize 182 (0.713725) 5.1683 (0.0202678) 9.30998 (0.0365097) 672K
spline64resize 182 (0.713725) 5.16437 (0.0202524) 9.28429 (0.036409) 672K
nnedi3 (cshift=spline36) 174 (0.682353) 4.71817 (0.0185026) 8.52777 (0.0334422) 724K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 176 (0.690196) 4.70834 (0.0184641) 8.60432 (0.0337424) 732K
screenthumbnail method used peak average standard deviation filesize
2_1
original - - - 328K
blackmanresize 62 (0.243137) 1.10186 (0.00432103) 1.69206 (0.00663553) 496K
lanczosresize 63 (0.247059) 1.10822 (0.00434597) 1.68183 (0.0065954) 676K
lanczos4resize 62 (0.243137) 1.11478 (0.00437168) 1.67341 (0.00656238) 504K
spline16resize 62 (0.243137) 1.10619 (0.00433802) 1.73069 (0.00678702) 488K
spline36resize 62 (0.243137) 1.10259 (0.00432386) 1.69271 (0.00663806) 496K
spline64resize 63 (0.247059) 1.10177 (0.00432066) 1.68822 (0.00662046) 496K
nnedi3 (cshift=spline36) 56 (0.219608) 1.08019 (0.00423605) 1.54706 (0.00606689) 524K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 52 (0.203922) 1.07437 (0.00421321) 1.53682 (0.00602673) 524K
screenthumbnail method used peak average standard deviation filesize
3_1
original - - - 648K
blackmanresize 131 (0.513725) 3.69918 (0.0145066) 6.33968 (0.0248615) 688K
lanczosresize 128 (0.501961) 3.70108 (0.014514) 6.31411 (0.0247612) 696K
lanczos4resize 127 (0.498039) 3.72408 (0.0146042) 6.30599 (0.0247294) 700K
spline16resize 132 (0.517647) 3.74263 (0.014677) 6.43383 (0.0252307) 684K
spline36resize 129 (0.505882) 3.7027 (0.0145204) 6.3419 (0.0248702) 688K
spline64resize 129 (0.505882) 3.70227 (0.0145187) 6.33526 (0.0248442) 688K
nnedi3 (cshift=spline36) 127 (0.498039) 3.39755 (0.0133237) 5.79667 (0.022732) 736K
nnedi3 (cshift=spline36, pscnr=4, etype=1) 125 (0.490196) 3.39746 (0.0133234) 5.7985 (0.0227392) 740K

 
Averaged #2

first average result + picture 1 result + picture 2 result + picture 3 result / 4

method average peak average mean average standard deviatation
blackmanresize 113.58 2.95 4.94
lanczosresize 112.79 2.95 4.92
lanczos4resize 112.45 2.97 4.90
spline16resize 114.12 2.98 5.03
spline36resize 113.04 2.95 4.95
spline64resize 113.37 2.95 4.93
nnedi3 (cshift=spline36) 108.45 2.75 4.56
nnedi3 (cshift=spline36, pscnr=4, etype=1) 107.25 2.75 4.58

 
visual results
At the top of this long article I already explained that visual results are very subjective; that is because things look differently on different screens (tft/crt/etc), also due to different brightness and similar settings. However, sometimes the visual difference is obvious. Here are some examples:

original
original
lanczos
lanczos
spline36
spline36
nnedi3
nnedi_pscrn

 
Conclusion
What we can see, looking at the numbers, is that the higher the resolution of your source (or better, the more details the source has) the better upscaling works. If your source is too blurry, upscaling won’t work as good as it could. All scalers perform quite good (except for point and sinc, but that was to be expected). Spline16 performs worse than spline36 and spline64, for upscaling spline36 performs better than spline64. Lanczos and spline are quite similar the difference is very minimal, according to the numbers, lanczos performs better than spline; overall best resizer for upscaling is nnedi3. Another thing which you can see in the visual results above is that the nnedi3 image doesn’t have the stairs-effect.

If you have to decide which resizer (in avisynth) to use for upscaling, go this route:

nnedi3 > lanczos/lanczos4 > spline36 > everything else

3 Comments

  1. tt (1 comments) says:

    awesome test,

    i made my own nnedi comparisons tests with avisynth,

    spline36 in nnedi cshift worked even better than lanczos3 or 4 on its own or in cshift,

    will only need to test spline64 in cshift now…

  2. De-M-oN (1 comments) says:

    Why you used jpg and not PNG?

    JPG has already lossy compression. Also @ 100%.

  3. Mexxxi (1 comments) says:

    Interesting comparison, but it doesn’t work for every material. Unfortunately you didn’t use real live, 2D animated or game footage.

    I used Lanczos4Resize to upscale 320×224 emulator game captures (lossless) to 1280×896. The results were subpar since the sharpening effect suddenly showed of lines of tiles that weren’t visible in either the emulator or the lossless video. With LanczosResize the lines were less visible, but still perceivable. I then tried GaussResize with its sharpening setting set to 100 and the picture looked beautiful. It was color rich which was important since the Lanczos filters seemed to wash out the colors a bit. Plus encoding sizes shrunk by 60%.

    I would have loved to see how GaussResize would have fared in your test, but I guess it was implented much later.

0 Trackbacks

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>