Benchmarks using a SSD to speed up ext4 with external journal and enhanceio

I’ve just made some tests to see how EnhanceIO performs and to see how data=journal and data=ordered perform when using a SSD as external journal device..

Tests have been made on the following hardware

  • CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
  • SSD: SAMSUNG MZ7PD120HAFV-000DA (120G)
  • MD0: Software Raid 1 over a partition (sdb3 and sdc3) on the following two HDDs.
  • HDD1: WDC WD1003FBYZ-010FB0
  • HDD2: Seagate Constellation ES.3
  • RAM: 16 GB
  • OS: Linux nyota 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u1 (2017-02-22) x86_64 GNU/Linux, Debian Jessie

Test #1

Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=ordered, Journal Device Size 1GB

Enhance IO: Policy lru, Mode read-only, Size 8GB

writes rewrites random reads mixed workload random writes pwrites fwrites
min
throughput
per thread
42980.63 kB/sec 34438.25 kB/sec 108361.57 kB/sec 75899.74 kB/sec 31737.90 kB/sec 42651.92 kB/sec 59742.36 kB/sec
max
throughput
per thread
43066.30 kB/sec 66467.18 kB/sec 109245.78 kB/sec 235433.14 kB/sec 67148.04 kB/sec 42739.32 kB/sec 59996.75 kB/sec
avg
throughput
per thread
43023.47 kB/sec 50452.71 kB/sec 108803.68 kB/sec 155666.44 kB/sec 49442.97 kB/sec 42695.62 kB/sec 59869.55 kB/sec

Test #2

Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=ordered, Journal Device Size 1GB

Enhance IO: Disabled

writes rewrites random reads mixed workload random writes pwrites fwrites
min
throughput
per thread
41,44 MB/s
(-0,52 MB/s)
33,17 MB/s
(-0,45 MB/s)
113,23 MB/s
(+7,41 MB/s)
63,61 MB/s
(-10,50 MB/s)
41,84 MB/s
(+10,84 MB/s)
41,25 MB/s
(-0,39 MB/s)
58,85 MB/s
(+0,51 MB/s)
max
throughput
per thread
41,48 MB/s
(-0,57 MB/s)
66,40 MB/s
(+1,49 MB/s)
118,59 MB/s
(+11,91 MB/s)
72,21 MB/s
(-157,69 MB/s)
55,35 MB/s
(-10,21 MB/s)
41,29 MB/s
(-0,43 MB/s)
59,60 MB/s
(+1,01 MB/s)
avg
throughput
per thread
41,46 MB/s
(-0,55 MB/s)
49,78 MB/s
(+0,51 MB/s)
115,91 MB/s
(+9,66 MB/s)
67,91 MB/s
(-84,10 MB/s)
48,60 MB/s
(+0,31 MB/s)
41,27 MB/s
(-0,41 MB/s)
59,23 MB/s
(+0,76 MB/s)

I divided the results with 1024 here to make the difference to the first test obvious. Differences of +- 1 MB/s can safely be ignored in my opinion. Also I couldn’t believe the -157,69 MB/s and ran the test another time with and without enhanceio. The results stayed same. So enhanceio seems to offer some improvement for specific workloads.

Test #3

Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=journal, Journal Device Size 1GB

Enhance IO: Policy lru, Mode read-only, Size 8GB

writes rewrites random reads mixed workload random writes pwrites fwrites
min
throughput
per thread
41,75 MB/s
(-0,21 MB/s)
41,39 MB/s
(+7,76 MB/s)
3,78 GB/s
(+3,68 GB/s)
50,54 MB/s
(-23,58 MB/s)
36,12 MB/s
(+5,13 MB/s)
38,71 MB/s
(-2,93 MB/s)
56,63 MB/s
(-1,70 MB/s)
max
throughput
per thread
42,29 MB/s
(+0,23 MB/s)
41,67 MB/s
(-23,23 MB/s)
4,63 GB/s
(+4,53 GB/s)
6,82 GB/s
(+6,59 GB/s)
36,15 MB/s
(-29,42 MB/s)
39,38 MB/s
(-2,35 MB/s)
58,76 MB/s
(+0,17 MB/s)
avg
throughput
per thread
42,02 MB/s
(+0 MB/s)
41,53 MB/s
(-7,73 MB/s)
4,21 GB/s
(+4,10 GB/s)
3,43 GB/s
(+3,28 GB/s)
36,13 MB/s
(-12,14 MB/s)
39,05 MB/s
(-2,64 MB/s)
57,70 MB/s
(-0,76 MB/s)

I couldn’t believe these results, neither. Note that the bold values are in GB/s. So I issued a reboot and re-checked. After a Reboot I am presented with the same values. Trying with enhanceio disabled

Test #4

Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=journal, Journal Device Size 1GB

Enhance IO: Disabled

writes rewrites random reads mixed workload random writes pwrites fwrites
min
throughput
per thread
41,84 MB/s
(-0,12 MB/s)
41,82 MB/s
(+8,19 MB/s)
3,45 GB/s
(+3,35 GB/s)
52,76 MB/s
(-21,36 MB/s)
36,46 MB/s
(+5,46 MB/s)
39,49 MB/s
(-2,15 MB/s)
54,88 MB/s
(-3,45 MB/s)
max
throughput
per thread
41,98 MB/s
(-0,06 MB/s)
42,05 MB/s
(-22,85 MB/s)
4,32 GB/s
(+4,22 GB/s)
6,83 GB/s
(+6,76 GB/s)
36,53 MB/s
(-29,04 MB/s)
40,10 MB/s
(-1,63 MB/s)
55,76 MB/s
(-2,82 MB/s)
avg
throughput
per thread
41,91 MB/s
(-0,09 MB/s)
41,93 MB/s
(-7,33 MB/s)
3,88 GB/s
(+3,78 GB/s)
3,44 GB/s
(+3,29 GB/s)
36,49 MB/s
(-11,78 MB/s)
39,80 MB/s
(-1,89 MB/s)
55,32 MB/s
(-3,13 MB/s)

So, while I am wondering about how to achieve that high numbers (mind the GB/s in the previous results) we can clearly see that there is an improvement when using data=journal combined with an external journal lying on a SSD over data=ordered. The difference between using EnhanceIO is not so clear. Let’s take another look at the information I gathered using EnhanceIO by taking a look at some min/max values of with and without:

min random reads max random reads min mixed workload max mixed workload
Without EnhanceIO #2 113,23 MB/s 118,59 MB/s 63,61 MB/s 72,21 MB/s
Without EnhanceIO #4 3,45 GB/s 4,32 GB/s 52,76 MB/s 6,83 GB/s
With EnhanceIO #1 105,82 MB/s 106,68 MB/s 74,12 MB/s 229,91 MB/s
With EnhanceIO #3 3,78 GB/s 4,63 GB/s 50,54 MB/s 6,82 GB/s

The difference between using EnhanceIO (if you compare #2 with #1 and #4 with #3) is not that obvious; I guess this highly depends on the work load and since my journal is on a SSD already, I would need a faster device for EnhanceIO to see a notable difference. From what I can tell now, it seems to help more to place the journal onto an SSD and use data=journal than using enhanceIO.

No Comments

Post a Comment