I made this tests because I was searching for a pretty good upscaler. Checking pictures manually is a very subjective thing and so I wanted a way to “calculate” how good a/an scaler/upscaler is. The easiest approach is to check, how similar the upscaled variant is to the original variant. Which is, what my tests do.
used software/material
MPlayer, Avisynth, Bash, Gimp, ImageMagick, OpenMovie Sintel, OpenMovie Big Buck Bunny, OpenMovie Elephants Dream
preparation
I took screenshots of a movie, stored the original highres image and created a lowscaled version of it. Then I created upscaled (to the same resolution as the highres image) variants of that lowscaled variant. And finally I compared the upscaled variant with the original highres variant.
execution
To create screenshots I have written a little Bashscript which gets screenshots using mplayer. The quality of the jpegs is set to 100 to have “lossless” images (more or less). The script needs ffmpeg to get the duration of the movie (pretty sure that might work with mplayer aswell, just didn’t find out how to easily parse that):
#!/bin/bash
infile=$1;
amount=$2;
# getting the length of the movie in seconds
ffmpeg -i "$infile" 2>/tmp/video.log
h=$(grep Duration /tmp/video.log | awk '{ print $2 }' | cut -d: -f1);
m=$(grep Duration /tmp/video.log | awk '{ print $2 }' | cut -d: -f2);
s=$(echo "($h * 60 * 60) + ($m * 60) - 60" | bc);
for (( x=1; x<=$amount; x++)); do
stime=$(echo "$x * ($s / $amount)" | bc);
echo $stime;
tmp=$(mplayer -frames 1 -ss $stime -vo jpeg:quality=100:outdir=screens $infile > /dev/null);
mv screens/00000001.jpg screens/$x.jpg
done
Then I’ve used gimp to create low-res variants of that screens – For the first two testcases I used lanczos and downscaled to a height of 480px. For the third testcase I used bicubic and downscaled to a height of 360px.
Then I’ve used lots of avisynth scripts to do the upscaling. An example for blackman and nnedi3:
source = ImageSource("3.jpg").Trim(0,-1)
us3 = BlackmanResize(source, 1920, 1080)
us13 = nnedi3_rpow2(source, rfactor=4, cshift="spline36resize", fwidth=1920, fheight=1080)
#ImageWriter(us3, file="3_blackman", type="jpeg")
ImageWriter(us13, file="3_nnedi3", type="jpeg")
The resulting upscaled image I’ve compared to the original image using ImageMagick:
convert originalimage.jpg upscaledimage.jpg -fx "abs(u[0]-u[1])" pgm:- \ | identify -verbose pgm:- \ | grep -E "mean|max|devi"
1920×1080 » (lanczos) 854×480 » 1920×1080
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
original | - | - | - | 640K |
| bicubicResize | 63 (0.247059) | 1.39138 (0.0054564) | 2.02799 (0.00795292) | 564K | |
| bilinearResize | 66 (0.258824) | 1.41046 (0.00553122) | 2.1246 (0.00833176) | 568K | |
| blackmanResize | 60 (0.235294) | 1.32716 (0.00520453) | 1.7842 (0.00699688) | 584K | |
| lanczosResize | 59 (0.231373) | 1.32695 (0.00520371) | 1.76561 (0.00692397) | 588K | |
| lanczos4resize | 58 (0.227451) | 1.31528 (0.00515796) | 1.73246 (0.00679397) | 592K | |
| pointresize | 138 (0.541176) | 1.93816 (0.00760062) | 3.38685 (0.0132818) | 688K | |
| sincresize | 97 (0.380392) | 1.94942 (0.00764479) | 3.12623 (0.0122597) | 712K | |
| spline16resize | 60 (0.235294) | 1.34655 (0.0052806) | 1.85531 (0.00727572) | 580K | |
| spline36resize | 58 (0.227451) | 1.32948 (0.00521366) | 1.78602 (0.00700402) | 584K | |
| spline64resize | 59 (0.231373) | 1.3242 (0.00519292) | 1.77446 (0.00695865) | 584K | |
| nnedi3 (cshift=spline36) | 59 (0.231373) | 1.34151 (0.00526083) | 1.76169 (0.00690858) | 616K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 56 (0.219608) | 1.34097 (0.0052587) | 1.76592 (0.00692518) | 620K | |
| spline36resize(hq4x) | 81 (0.317647) | 1.61154 (0.00631977) | 2.08531 (0.00817769) | 568K | |
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
original | - | - | - | 1100K |
| bicubicresize | 131 (0.513725) | 2.01965 (0.0079202) | 2.58451 (0.0101353) | 716K | |
| bilinearresize | 129 (0.505882) | 2.06215 (0.00808685) | 2.66154 (0.0104374) | 724K | |
| blackmanresize | 128 (0.501961) | 1.92173 (0.0075362) | 2.37666 (0.00932023) | 744K | |
| lanczosresize | 128 (0.501961) | 1.91967 (0.0075281) | 2.36059 (0.00925722) | 752K | |
| lanczos4resize | 128 (0.501961) | 1.89933 (0.00744836) | 2.33001 (0.00913729) | 756K | |
| pointresize | 173 (0.678431) | 3.04469 (0.01194) | 3.97362 (0.0155828) | 924K | |
| sincresize | 167 (0.654902) | 2.94485 (0.0115484) | 3.62896 (0.0142312) | 920K | |
| spline16resize | 128 (0.501961) | 1.95451 (0.00766474) | 2.44045 (0.00957039) | 740K | |
| spline36resize | 128 (0.501961) | 1.92393 (0.00754484) | 2.37922 (0.00933026) | 744K | |
| spline64resize | 129 (0.505882) | 1.91636 (0.00751512) | 2.36858 (0.00928855) | 744K | |
| nnedi3 (cshift=spline36) | 119 (0.466667) | 1.94348 (0.00762151) | 2.34163 (0.00918287) | 776K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 112 (0.439216) | 1.94203 (0.00761579) | 2.34288 (0.00918778) | 776K | |
| spline36resize(hq4x) | 122 (0.478431) | 2.19204 (0.00859625) | 2.6589 (0.0104271) | 744K | |
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
original | - | - | - | 732K |
| bicubicresize | 49 (0.192157) | 1.32602 (0.00520009) | 1.51821 (0.00595376) | 572K | |
| bilinearresize | 53 (0.207843) | 1.33697 (0.00524301) | 1.57944 (0.0061939) | 572K | |
| blackmanresize | 43 (0.168627) | 1.29434 (0.00507583) | 1.3829 (0.00542314) | 592K | |
| lanczosresize | 41 (0.160784) | 1.29722 (0.00508715) | 1.37439 (0.00538977) | 596K | |
| lanczos4resize | 41 (0.160784) | 1.28806 (0.00505121) | 1.35241 (0.00530357) | 596K | |
| pointresize | 144 (0.564706) | 1.86225 (0.00730295) | 2.74764 (0.010775) | 692K | |
| sincresize | 110 (0.431373) | 1.85213 (0.00726325) | 2.47998 (0.00972541) | 708K | |
| spline16resize | 44 (0.172549) | 1.306 (0.00512158) | 1.42527 (0.00558931) | 584K | |
| spline36resize | 43 (0.168627) | 1.29596 (0.00508218) | 1.38413 (0.00542798) | 588K | |
| spline64resize | 42 (0.164706) | 1.29306 (0.00507082) | 1.37745 (0.00540178) | 592K | |
| nnedi3 (cshift=spline36) | 39 (0.152941) | 1.30354 (0.00511192) | 1.36729 (0.00536193) | 620K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 39 (0.152941) | 1.30032 (0.00509928) | 1.36227 (0.00534222) | 624K | |
| spline36resize(hq4x) | 77 (0.301961) | 1.52609 (0.00598468) | 1.58047 (0.00619794) | 572K |
1920×818 » (lanczos) 1126×480 » 1920×1080
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
|||||
| original | - | - | - | 620K | |
| bicubicresize | 76 (0.298039) | 1.22795 (0.00481549) | 1.64521 (0.00645182) | 444K | |
| bilinearresize | 78 (0.305882) | 1.22377 (0.0047991) | 1.66566 (0.00653199) | 440K | |
| blackmanresize | 72 (0.282353) | 1.23057 (0.00482575) | 1.57433 (0.00617384) | 464K | |
| lanczosresize | 71 (0.278431) | 1.2383 (0.00485609) | 1.57933 (0.00619346) | 468K | |
| lanczos4resize | 70 (0.27451) | 1.24825 (0.00489511) | 1.59137 (0.00624068) | 472K | |
| pointresize | 106 (0.415686) | 1.48345 (0.00581747) | 2.42587 (0.00951323) | 508K | |
| sincresize | 88 (0.345098) | 1.47682 (0.00579147) | 2.15301 (0.00844318) | 492K | |
| spline16resize | 73 (0.286275) | 1.2257 (0.00480666) | 1.58467 (0.00621438) | 460K | |
| spline36resize | 72 (0.282353) | 1.23204 (0.00483153) | 1.57656 (0.00618259) | 464K | |
| spline64resize | 71 (0.278431) | 1.23371 (0.00483806) | 1.57686 (0.00618377) | 464K | |
| nnedi3 (cshift=spline36) | 70 (0.27451) | 1.2102 (0.00474587) | 1.53526 (0.00602061) | 484K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 76 (0.298039) | 1.20684 (0.00473269) | 1.53277 (0.00601086) | 484K | |
| spline36resize(hq4x) | 77 (0.301961) | 1.4199 (0.00556825) | 1.62984 (0.00639155) | 416K | |
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
|||||
| original | - | - | - | 1200K | |
| bicubicresize | 68 (0.266667) | 3.90677 (0.0153207) | 4.63845 (0.01819) | 756K | |
| bilinearresize | 72 (0.282353) | 3.93428 (0.0154286) | 4.70239 (0.0184407) | 772K | |
| blackmanresize | 70 (0.27451) | 3.75073 (0.0147087) | 4.45619 (0.0174752) | 824K | |
| lanczosresize | 69 (0.270588) | 3.77025 (0.0147853) | 4.47537 (0.0175505) | 832K | |
| lanczos4resize | 70 (0.27451) | 3.80719 (0.0149302) | 4.51547 (0.0177077) | 832K | |
| pointresize | 141 (0.552941) | 5.41135 (0.021221) | 7.10313 (0.0278554) | 992K | |
| sincresize | 113 (0.443137) | 5.13697 (0.020145) | 6.00752 (0.0235589) | 880K | |
| spline16resize | 70 (0.27451) | 3.76273 (0.0147558) | 4.47469 (0.0175478) | 812K | |
| spline36resize | 70 (0.27451) | 3.75862 (0.0147397) | 4.46286 (0.0175014) | 820K | |
| spline64resize | 70 (0.27451) | 3.7603 (0.0147463) | 4.46454 (0.017508) | 824K | |
| nnedi3 (cshift=spline36) | 75 (0.294118) | 3.69705 (0.0144982) | 4.36505 (0.0171178) | 836K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 72 (0.282353) | 3.68776 (0.0144618) | 4.35658 (0.0170846) | 836K | |
| spline36resize(hq4x) | 68 (0.266667) | 3.9761 (0.0155926) | 4.64142 (0.0182016) | 760K | |
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
|||||
| original | - | - | - | 652K | |
| bicubicresize | 102 (0.4) | 1.61719 (0.00634193) | 3.41673 (0.013399) | 516K | |
| bilinearresize | 113 (0.443137) | 1.62747 (0.00638224) | 3.50645 (0.0137508) | 516K | |
| blackmanresize | 103 (0.403922) | 1.57874 (0.00619113) | 3.1816 (0.0124769) | 544K | |
| lanczosresize | 107 (0.419608) | 1.58978 (0.00623445) | 3.1924 (0.0125192) | 548K | |
| lanczos4resize | 118 (0.462745) | 1.60308 (0.00628658) | 3.20413 (0.0125652) | 552K | |
| pointresize | 212 (0.831373) | 2.33259 (0.00914742) | 5.96943 (0.0234095) | 612K | |
| sincresize | 141 (0.552941) | 2.18924 (0.00858526) | 4.94182 (0.0193797) | 588K | |
| spline16resize | 102 (0.4) | 1.58086 (0.00619945) | 3.22538 (0.0126486) | 536K | |
| spline36resize | 104 (0.407843) | 1.58196 (0.00620377) | 3.18951 (0.0125079) | 544K | |
| spline64resize | 106 (0.415686) | 1.58257 (0.00620615) | 3.18302 (0.0124824) | 544K | |
| nnedi3 (cshift=spline36) | 99 (0.388235) | 1.52185 (0.00596802) | 2.96699 (0.0116352) | 560K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 101 (0.396078) | 1.52184 (0.00596802) | 2.98155 (0.0116923) | 560K | |
| spline36resize(hq4x) | 106 (0.415686) | 1.80159 (0.00706505) | 3.38661 (0.0132808) | 496K |
Averaged
| method | average peak | average mean | average standard deviatation |
| bicubicresize | 81.5 | 1.91 | 2.63 |
| bilinearresize | 85.16 | 1.93 | 2.70 |
| blackmanresize | 79.33 | 1.85 | 2.45 |
| lanczosresize | 79.16 | 1.85 | 2.45 |
| lanczos4resize | 80.83 | 1.86 | 2.45 |
| pointresize | 152.33 | 2.67 | 4.26 |
| sincresize | 119.33 | 2.59 | 3.72 |
| spline16resize | 79.5 | 1.86 | 2.5 |
| spline36resize | 79.16 | 1.85 | 2.46 |
| spline64resize | 79.5 | 1.85 | 2.45 |
| nnedi3 (cshift=spline36) | 76.83 | 1.83 | 2.38 |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 76 | 1.83 | 2.39 |
| spline36resize(hq4x) | 88.5 | 2.08 | 2.66 |
1920×1080 » (bicubic) 640×360 » 1920×1080
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
|||||
| original | - | - | - | 696K | |
| blackmanresize | 182 (0.713725) | 5.15758 (0.0202258) | 9.2996 (0.036469) | 672K | |
| lanczosresize | 181 (0.709804) | 5.17115 (0.020279) | 9.25688 (0.0363015) | 676K | |
| lanczos4resize | 180 (0.705882) | 5.19959 (0.0203906) | 9.21027 (0.0361187) | 684K | |
| spline16resize | 183 (0.717647) | 5.21444 (0.0204488) | 9.46563 (0.0371201) | 672K | |
| spline36resize | 182 (0.713725) | 5.1683 (0.0202678) | 9.30998 (0.0365097) | 672K | |
| spline64resize | 182 (0.713725) | 5.16437 (0.0202524) | 9.28429 (0.036409) | 672K | |
| nnedi3 (cshift=spline36) | 174 (0.682353) | 4.71817 (0.0185026) | 8.52777 (0.0334422) | 724K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 176 (0.690196) | 4.70834 (0.0184641) | 8.60432 (0.0337424) | 732K | |
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
|||||
| original | - | - | - | 328K | |
| blackmanresize | 62 (0.243137) | 1.10186 (0.00432103) | 1.69206 (0.00663553) | 496K | |
| lanczosresize | 63 (0.247059) | 1.10822 (0.00434597) | 1.68183 (0.0065954) | 676K | |
| lanczos4resize | 62 (0.243137) | 1.11478 (0.00437168) | 1.67341 (0.00656238) | 504K | |
| spline16resize | 62 (0.243137) | 1.10619 (0.00433802) | 1.73069 (0.00678702) | 488K | |
| spline36resize | 62 (0.243137) | 1.10259 (0.00432386) | 1.69271 (0.00663806) | 496K | |
| spline64resize | 63 (0.247059) | 1.10177 (0.00432066) | 1.68822 (0.00662046) | 496K | |
| nnedi3 (cshift=spline36) | 56 (0.219608) | 1.08019 (0.00423605) | 1.54706 (0.00606689) | 524K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 52 (0.203922) | 1.07437 (0.00421321) | 1.53682 (0.00602673) | 524K | |
| screenthumbnail | method used | peak | average | standard deviation | filesize |
 |
|||||
| original | - | - | - | 648K | |
| blackmanresize | 131 (0.513725) | 3.69918 (0.0145066) | 6.33968 (0.0248615) | 688K | |
| lanczosresize | 128 (0.501961) | 3.70108 (0.014514) | 6.31411 (0.0247612) | 696K | |
| lanczos4resize | 127 (0.498039) | 3.72408 (0.0146042) | 6.30599 (0.0247294) | 700K | |
| spline16resize | 132 (0.517647) | 3.74263 (0.014677) | 6.43383 (0.0252307) | 684K | |
| spline36resize | 129 (0.505882) | 3.7027 (0.0145204) | 6.3419 (0.0248702) | 688K | |
| spline64resize | 129 (0.505882) | 3.70227 (0.0145187) | 6.33526 (0.0248442) | 688K | |
| nnedi3 (cshift=spline36) | 127 (0.498039) | 3.39755 (0.0133237) | 5.79667 (0.022732) | 736K | |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 125 (0.490196) | 3.39746 (0.0133234) | 5.7985 (0.0227392) | 740K |
first average result + picture 1 result + picture 2 result + picture 3 result / 4
| method | average peak | average mean | average standard deviatation |
| blackmanresize | 113.58 | 2.95 | 4.94 |
| lanczosresize | 112.79 | 2.95 | 4.92 |
| lanczos4resize | 112.45 | 2.97 | 4.90 |
| spline16resize | 114.12 | 2.98 | 5.03 |
| spline36resize | 113.04 | 2.95 | 4.95 |
| spline64resize | 113.37 | 2.95 | 4.93 |
| nnedi3 (cshift=spline36) | 108.45 | 2.75 | 4.56 |
| nnedi3 (cshift=spline36, pscnr=4, etype=1) | 107.25 | 2.75 | 4.58 |
visual results
At the top of this long article I already explained that visual results are very subjective; that is because things look differently on different screens (tft/crt/etc), also due to different brightness and similar settings. However, sometimes the visual difference is obvious. Here are some examples:
original |
lanczos |
spline36 |
nnedi3 |
Conclusion
What we can see, looking at the numbers, is that the higher the resolution of your source (or better, the more details the source has) the better upscaling works. If your source is too blurry, upscaling won’t work as good as it could. All scalers perform quite good (except for point and sinc, but that was to be expected). Spline16 performs worse than spline36 and spline64, for upscaling spline36 performs better than spline64. Lanczos and spline are quite similar the difference is very minimal, according to the numbers, lanczos performs better than spline; overall best resizer for upscaling is nnedi3. Another thing which you can see in the visual results above is that the nnedi3 image doesn’t have the stairs-effect.
If you have to decide which resizer (in avisynth) to use for upscaling, go this route:
nnedi3 > lanczos/lanczos4 > spline36 > everything else
awesome test,
i made my own nnedi comparisons tests with avisynth,
spline36 in nnedi cshift worked even better than lanczos3 or 4 on its own or in cshift,
will only need to test spline64 in cshift now…