Recently I was wondering why I do only get 1-2 mbit/s on a gbit/s link while using zfs send. As it turned out xz caused that. xz with default settings is just too slow for this sort of data. Due to that I compared the available compression methods in linux. Well, at least some of them.
My test-case is simple. I’ve created a snapshot called backup and now I do time zfs send -R storage@backup | xz > test.xz. The dataset on the celeron machine is 1665576 K and the same dataset on the xeon machine is 1765767 K big (and I have no idea about why there’s a difference. However, for my tests that shouldn’t matter)
MB/s is calculated as original-size / time-in-seconds
Comp is calculated as 100 – (compressed-size * 100) / original-size
| Method | Time | Size | MB/s | Comp | Time | Size | MB/s | Comp |
|---|---|---|---|---|---|---|---|---|
| # | Intel® Celeron® CPU J1900 @ 1.99GHz | Intel® Xeon® CPU E5-1620 v3 @ 3.50GHz | ||||||
| lz4 | 0m38.959s | 1427208 | 41,74 | 14,32 | 0m12.288s | 1425488 | 140,33 | 19,28 |
| compress | 5m16.839s | 1923672 | 5,13 | -15,49 | 1m29.687s | 1904584 | 19,22 | -7,84 |
| xz | 41m1.661s | 912628 | 0,66 | 45,21 | 13m42.812s | 912624 | 2,09 | 48,32 |
| pixz | 10m54.021s | 924296 | 2,48 | 44,51 | 2m34.995s | 923200 | 11,12 | 47,72 |
| gzip | 3m40.056s | 1151348 | 7,39 | 30,88 | 1m21.117s | 1150004 | 21,25 | 34,88 |
| pigz | 1m5.636s | 1148772 | 24,78 | 31,03 | 0m15.499s | 1147436 | 111,25 | 35,02 |
| bzip2 | 12m38.862s | 1086556 | 2,14 | 34,77 | 3m43.735s | 1085972 | 7,70 | 38,50 |
| pbzip2 | 3m59.354s | 1086544 | 6,79 | 34,77 | 0m47.477s | 1085824 | 36,32 | 38,51 |
| lzop | 1m8.186s | 1417160 | 23,85 | 14,92 | 0m12.368s | 1415664 | 139,42 | 19,83 |
| Brotli1 | 461m42.891s | 936088 | 0,05 | 43,80 | 92m6.691s | 935184 | 0,31 | 47,04 |
| Brotli –window 24 –quality 1 | 1m9.604s | 1104568 | 23,36 | 33,69 | 0m16.917s | 1104528 | 101,93 | 37,45 |
| Zstandard2 | 0m44.748s | 1175912 | 25,66 | 36,34 | 0m12.955s | 1174588 | 133,10 | 33,49 |
1 Github: Brotli
2 Github: Zstandard
I think this table shows some interesting results, though keep in mind that (a) you are able to tune the compression tools. For example you could use xz -3 to get more speed for lesser compression. I’ve used the default settings. (b) this comparison is just for my specific use case.
You can easily see that brotli and xz are not well suited for this data-stream due to their slowness You can see that the default settings for brotli and xz are not useful for my use-case. As expected the ancient compress-utility just increases the size and is as such not very useful neither. lz4 is the fastest of the tested compression tools offering okay’ish compression (14-19% smaller size). Zstandard looks very promising. It’s speed is similar to pigz (parallel gzip) and lzop just somewhat slower than lz4, offering average compressibility similar to gzip. The default compression level for Zstandard is 1 which means you might get even more compressibility (at the cost of speed) by tuning that.
Not sure if Zstandard should be used in production now, though. If I’d go for standard utilities I’d decide between lz4, lzo and pigz. From these 3, looking for a compromise between speed and compression, pigz would be my choice.
Update checked with brotli –window 24 –quality 1 according to Jyrki’s comment. Thanks. To be fair I should pick good settings for the other compression tools, too. Updating if I have time for that.
No Comments