Storage cluster performance on Exoscale
This page documents the performance of the VSHN Managed Storage Cluster on Exoscale.
The performance numbers presented on this page have been obtained from a storage cluster instance deployed on OpenShift 4.7. All performance numbers on this page have been measured on an otherwise idle storage cluster instance.
Block storage performance
Block storage is provided by Ceph RBD.
Read IOPS
Block storage provides 36303 read IOPS on average with a standard deviation of 2254IOPS over 10 benchmark iterations.
![ceph rbd read iops](../_images/storage_performance/exoscale/ceph_rbd_read_iops.png)
Read bandwidth
Block storage provides an average read bandwidth of 1.425 GB/s with a standard deviation of 60MB/s over 10 benchmark iterations.
![ceph rbd read bandwidth](../_images/storage_performance/exoscale/ceph_rbd_read_bandwidth.png)
Write IOPS
Block storage write IOPS vary depending on the frequency of fsync
calls.
The graph below shows write IOPS for different fsync
frequencies.
The shaded areas in the graph represent the standard deviation of the measured IOPS for each series of data points.
![ceph rbd write iops](../_images/storage_performance/exoscale/ceph_rbd_write_iops.png)
The results show that block storage provides:
-
6292 write IOPS on average with a standard deviation of 425 IOPS over 10 iterations without calling
fsync
during the benchmark ("no fsync"). -
225 write IOPS on average with a standard deviation of 12 IOPS over 10 benchmark iterations with an
fsync
call after each operation ("fsync=1"). -
2179 write IOPS on average with a standard deviation of 256 IOPS over 10 benchmark iterations with
fsync
calls after every 32 operations ("fsync=32"). -
3264 write IOPS on average with a standard deviation 179 IOPS over 10 benchmark iterations with
fsync
calls after every 128 operations ("fsync=128").
Write bandwidth
Block storage write bandwidth exhibits the same performance variation based on fsync
frequency as block storage write IOPS.
![ceph rbd write bandwidth](../_images/storage_performance/exoscale/ceph_rbd_write_bandwidth.png)
The results show that block storage provides a write bandwidth of:
-
310MB/s on average with a standard deviation of 63MB/s over 10 iterations without
fsync
calls during the benchmark ("no fsync"). -
23MB/s on average with a standard deviation of 1MB/s over 10 iterations with an
fsync
call after each operation ("fsync=1"). -
203MB/s on average with a standard deviation of 31MB/s over 10 iterations with
fsync
calls after every 32 operations ("fsync=32"). -
226MB/s on average with a standard deviation of 39MB/s over 10 iterations with
fsync
calls after every 128 operations ("fsync=128").
File storage performance
File storage is provided by CephFS.
Read IOPS
File storage provides 36343 read IOPS on average with a standard deviation of 1235IOPS over 10 benchmark iterations.
![cephfs read iops](../_images/storage_performance/exoscale/cephfs_read_iops.png)
Read bandwidth
File storage provides an average read bandwidth of 1.405 GB/s with a standard deviation of 92MB/s over 10 benchmark iterations.
![cephfs read bandwidth](../_images/storage_performance/exoscale/cephfs_read_bandwidth.png)
Write IOPS
File storage provides up to 6000 write IOPS on average with a standard deviation of 250 IOPS over 10 benchmark iterations.
![cephfs write iops](../_images/storage_performance/exoscale/cephfs_write_iops.png)
CephFS doesn’t provide the same guarantees for fsync
as Ceph RBD.
This difference is easily visible in the results, which show much smaller differences for different fsync
frequencies.
Additionally, there’s no clear correlation between fsync
frequencies and achieved write IOPS.
The following results have been measured for file storage:
-
6020 IOPS on average with a standard deviation of 251 IOPS over 10 iterations without
fsync
calls during the benchmark ("no fsync") -
5004 IOPS on average with a standard deviation of 152 IOPS over 10 iterations with an
fsync
call after each operation ("fsync=1") -
4659 IOPS on average with a standard deviation of 704 IOPS over 10 iterations with an
fsync
call after every 32 operations ("fsync=32") -
4202 IOPS on average with a standard deviation of 212 IOPS over 10 iterations with an
fsync
call after every 128 operations ("fsync=128")
Write bandwidth
File storage provides a write bandwidth of up to 306MB/s on average with a standard deviation of 9MB/s over 10 benchmark iterations.
![cephfs write bandwidth](../_images/storage_performance/exoscale/cephfs_write_bandwidth.png)
As already stated, CephFS doesn’t provide the same guarantees for fsync
as Ceph RBD.
Again, there’s no clear correlation between fsync
frequencies and achieved write bandwidth.
The following results have been measured for file storage write bandwidth:
-
278MB/s on average with a standard deviation of 49MB/s over 10 iterations without
fsync
calls during the benchmark ("no fsync") -
291MB/s on average with a standard deviation of 17MB/s over 10 iterations with an
fsync
call after each operation ("fsync=1") -
299MB/s on average with a standard deviation of 8MB/s over 10 iterations with an
fsync
call after every 32 operations ("fsync=32") -
305MB/s on average with a standard deviation of 8MB/s over 10 iterations with an
fsync
call after every 128 operations ("fsync=128")