Selecting a compression method for zfs send

Recently I was wondering why I do only get 1-2 mbit/s on a gbit/s link while using zfs send. As it turned out xz caused that. xz with default settings is just too slow for this sort of data. Due to that I compared the available compression methods in linux. Well, at least some of them.

My test-case is simple. I’ve created a snapshot called backup and now I do time zfs send -R storage@backup | xz > test.xz. The dataset on the celeron machine is 1665576 K and the same dataset on the xeon machine is 1765767 K big (and I have no idea about why there’s a difference. However, for my tests that shouldn’t matter)

MB/s is calculated as original-size / time-in-seconds
Comp is calculated as 100 – (compressed-size * 100) / original-size

Method Time Size MB/s Comp Time Size MB/s Comp
# Intel® Celeron® CPU J1900 @ 1.99GHz Intel® Xeon® CPU E5-1620 v3 @ 3.50GHz
lz4 0m38.959s 1427208 41,74 14,32 0m12.288s 1425488 140,33 19,28
compress 5m16.839s 1923672 5,13 -15,49 1m29.687s 1904584 19,22 -7,84
xz 41m1.661s 912628 0,66 45,21 13m42.812s 912624 2,09 48,32
pixz 10m54.021s 924296 2,48 44,51 2m34.995s 923200 11,12 47,72
gzip 3m40.056s 1151348 7,39 30,88 1m21.117s 1150004 21,25 34,88
pigz 1m5.636s 1148772 24,78 31,03 0m15.499s 1147436 111,25 35,02
bzip2 12m38.862s 1086556 2,14 34,77 3m43.735s 1085972 7,70 38,50
pbzip2 3m59.354s 1086544 6,79 34,77 0m47.477s 1085824 36,32 38,51
lzop 1m8.186s 1417160 23,85 14,92 0m12.368s 1415664 139,42 19,83
Brotli1 461m42.891s 936088 0,05 43,80 92m6.691s 935184 0,31 47,04
Brotli –window 24 –quality 1 1m9.604s 1104568 23,36 33,69 0m16.917s 1104528 101,93 37,45
Zstandard2 0m44.748s 1175912 25,66 36,34 0m12.955s 1174588 133,10 33,49

1 Github: Brotli
2 Github: Zstandard

I think this table shows some interesting results, though keep in mind that (a) you are able to tune the compression tools. For example you could use xz -3 to get more speed for lesser compression. I’ve used the default settings. (b) this comparison is just for my specific use case.

You can easily see that brotli and xz are not well suited for this data-stream due to their slowness You can see that the default settings for brotli and xz are not useful for my use-case. As expected the ancient compress-utility just increases the size and is as such not very useful neither. lz4 is the fastest of the tested compression tools offering okay’ish compression (14-19% smaller size). Zstandard looks very promising. It’s speed is similar to pigz (parallel gzip) and lzop just somewhat slower than lz4, offering average compressibility similar to gzip. The default compression level for Zstandard is 1 which means you might get even more compressibility (at the cost of speed) by tuning that.

Not sure if Zstandard should be used in production now, though. If I’d go for standard utilities I’d decide between lz4, lzo and pigz. From these 3, looking for a compromise between speed and compression, pigz would be my choice.

Update checked with brotli –window 24 –quality 1 according to Jyrki’s comment. Thanks. To be fair I should pick good settings for the other compression tools, too. Updating if I have time for that.


  • Jyrki Alakuijala

    24. Januar 2016 at 1:49 Antworten

    You can more interesting results for your use case with brotli –window 24 –quality 1, possibly 1000x faster than the default setting.

    • jean

      24. Januar 2016 at 10:32 Antworten

      Thank you. Updated.

Post a Comment