I’ve just made some tests to see how EnhanceIO performs and to see how data=journal and data=ordered perform when using a SSD as external journal device..
Tests have been made on the following hardware
- CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
- SSD: SAMSUNG MZ7PD120HAFV-000DA (120G)
- MD0: Software Raid 1 over a partition (sdb3 and sdc3) on the following two HDDs.
- HDD1: WDC WD1003FBYZ-010FB0
- HDD2: Seagate Constellation ES.3
- RAM: 16 GB
- OS: Linux nyota 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u1 (2017-02-22) x86_64 GNU/Linux, Debian Jessie
Test #1
Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=ordered, Journal Device Size 1GB
Enhance IO: Policy lru, Mode read-only, Size 8GB
| writes | rewrites | random reads | mixed workload | random writes | pwrites | fwrites | |
|---|---|---|---|---|---|---|---|
| min throughput per thread |
42980.63 kB/sec | 34438.25 kB/sec | 108361.57 kB/sec | 75899.74 kB/sec | 31737.90 kB/sec | 42651.92 kB/sec | 59742.36 kB/sec |
| max throughput per thread |
43066.30 kB/sec | 66467.18 kB/sec | 109245.78 kB/sec | 235433.14 kB/sec | 67148.04 kB/sec | 42739.32 kB/sec | 59996.75 kB/sec |
| avg throughput per thread |
43023.47 kB/sec | 50452.71 kB/sec | 108803.68 kB/sec | 155666.44 kB/sec | 49442.97 kB/sec | 42695.62 kB/sec | 59869.55 kB/sec |
Test #2
Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=ordered, Journal Device Size 1GB
Enhance IO: Disabled
| writes | rewrites | random reads | mixed workload | random writes | pwrites | fwrites | |
|---|---|---|---|---|---|---|---|
| min throughput per thread |
41,44 MB/s (-0,52 MB/s) |
33,17 MB/s (-0,45 MB/s) |
113,23 MB/s (+7,41 MB/s) |
63,61 MB/s (-10,50 MB/s) |
41,84 MB/s (+10,84 MB/s) |
41,25 MB/s (-0,39 MB/s) |
58,85 MB/s (+0,51 MB/s) |
| max throughput per thread |
41,48 MB/s (-0,57 MB/s) |
66,40 MB/s (+1,49 MB/s) |
118,59 MB/s (+11,91 MB/s) |
72,21 MB/s (-157,69 MB/s) |
55,35 MB/s (-10,21 MB/s) |
41,29 MB/s (-0,43 MB/s) |
59,60 MB/s (+1,01 MB/s) |
| avg throughput per thread |
41,46 MB/s (-0,55 MB/s) |
49,78 MB/s (+0,51 MB/s) |
115,91 MB/s (+9,66 MB/s) |
67,91 MB/s (-84,10 MB/s) |
48,60 MB/s (+0,31 MB/s) |
41,27 MB/s (-0,41 MB/s) |
59,23 MB/s (+0,76 MB/s) |
I divided the results with 1024 here to make the difference to the first test obvious. Differences of +- 1 MB/s can safely be ignored in my opinion. Also I couldn’t believe the -157,69 MB/s and ran the test another time with and without enhanceio. The results stayed same. So enhanceio seems to offer some improvement for specific workloads.
Test #3
Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=journal, Journal Device Size 1GB
Enhance IO: Policy lru, Mode read-only, Size 8GB
| writes | rewrites | random reads | mixed workload | random writes | pwrites | fwrites | |
|---|---|---|---|---|---|---|---|
| min throughput per thread |
41,75 MB/s (-0,21 MB/s) |
41,39 MB/s (+7,76 MB/s) |
3,78 GB/s (+3,68 GB/s) |
50,54 MB/s (-23,58 MB/s) |
36,12 MB/s (+5,13 MB/s) |
38,71 MB/s (-2,93 MB/s) |
56,63 MB/s (-1,70 MB/s) |
| max throughput per thread |
42,29 MB/s (+0,23 MB/s) |
41,67 MB/s (-23,23 MB/s) |
4,63 GB/s (+4,53 GB/s) |
6,82 GB/s (+6,59 GB/s) |
36,15 MB/s (-29,42 MB/s) |
39,38 MB/s (-2,35 MB/s) |
58,76 MB/s (+0,17 MB/s) |
| avg throughput per thread |
42,02 MB/s (+0 MB/s) |
41,53 MB/s (-7,73 MB/s) |
4,21 GB/s (+4,10 GB/s) |
3,43 GB/s (+3,28 GB/s) |
36,13 MB/s (-12,14 MB/s) |
39,05 MB/s (-2,64 MB/s) |
57,70 MB/s (-0,76 MB/s) |
I couldn’t believe these results, neither. Note that the bold values are in GB/s. So I issued a reboot and re-checked. After a Reboot I am presented with the same values. Trying with enhanceio disabled
Test #4
Mount options: noatime,user_xattr,errors=remount-ro,commit=300,journal_checksum,journal_async_commit,data=journal, Journal Device Size 1GB
Enhance IO: Disabled
| writes | rewrites | random reads | mixed workload | random writes | pwrites | fwrites | |
|---|---|---|---|---|---|---|---|
| min throughput per thread |
41,84 MB/s (-0,12 MB/s) |
41,82 MB/s (+8,19 MB/s) |
3,45 GB/s (+3,35 GB/s) |
52,76 MB/s (-21,36 MB/s) |
36,46 MB/s (+5,46 MB/s) |
39,49 MB/s (-2,15 MB/s) |
54,88 MB/s (-3,45 MB/s) |
| max throughput per thread |
41,98 MB/s (-0,06 MB/s) |
42,05 MB/s (-22,85 MB/s) |
4,32 GB/s (+4,22 GB/s) |
6,83 GB/s (+6,76 GB/s) |
36,53 MB/s (-29,04 MB/s) |
40,10 MB/s (-1,63 MB/s) |
55,76 MB/s (-2,82 MB/s) |
| avg throughput per thread |
41,91 MB/s (-0,09 MB/s) |
41,93 MB/s (-7,33 MB/s) |
3,88 GB/s (+3,78 GB/s) |
3,44 GB/s (+3,29 GB/s) |
36,49 MB/s (-11,78 MB/s) |
39,80 MB/s (-1,89 MB/s) |
55,32 MB/s (-3,13 MB/s) |
So, while I am wondering about how to achieve that high numbers (mind the GB/s in the previous results) we can clearly see that there is an improvement when using data=journal combined with an external journal lying on a SSD over data=ordered. The difference between using EnhanceIO is not so clear. Let’s take another look at the information I gathered using EnhanceIO by taking a look at some min/max values of with and without:
| min random reads | max random reads | min mixed workload | max mixed workload | |
|---|---|---|---|---|
| Without EnhanceIO #2 | 113,23 MB/s | 118,59 MB/s | 63,61 MB/s | 72,21 MB/s |
| Without EnhanceIO #4 | 3,45 GB/s | 4,32 GB/s | 52,76 MB/s | 6,83 GB/s |
| With EnhanceIO #1 | 105,82 MB/s | 106,68 MB/s | 74,12 MB/s | 229,91 MB/s |
| With EnhanceIO #3 | 3,78 GB/s | 4,63 GB/s | 50,54 MB/s | 6,82 GB/s |
The difference between using EnhanceIO (if you compare #2 with #1 and #4 with #3) is not that obvious; I guess this highly depends on the work load and since my journal is on a SSD already, I would need a faster device for EnhanceIO to see a notable difference. From what I can tell now, it seems to help more to place the journal onto an SSD and use data=journal than using enhanceIO.
No Comments