Tech

OnApp vs AWS vs Azure: which cloud is fastest?

If you are an end user thinking about buying cloud services from an OnApp cloud provider – or you’re an OnApp provider wondering how your platform fares against the hyperscale clouds… this blog post is for you.

There are many reasons to buy cloud from an OnApp-based MSP, telco or host, rather than from AWS or Azure – not least of which are better support, more flexible billing and packaging, greater ease of use and more choice of locations… but what about performance?

Everyone assumes the hyperscalers will naturally outperform the competition, but how good is their service when it comes up against a well-designed OnApp Cloud? Let’s find out…

Setting up the test

Cyber Host ProI’ve wanted to do this for a while, and one of our UK-based customers, Cyber Host Pro, was good enough to allow me access to their newly-designed cloud infrastructure, running OnApp with our integrated software-defined storage solution.

I plan to put a virtual machine up against an instance in AWS and Azure with the same hardware specifications, on a comparable pricing tier, to see if OnApp cloud providers  can provide a better service for the same cost.

The OnApp platform

The test OnApp cloud is one of Cyber Host Pro’s production environments, with many workloads running.

It’s an all-SSD integrated storage setup, with Dell R740 hypervisors and a 10Gbit backend storage network. The datastore is configured with 2 copies and 2 stripes, and doesn’t have any NVMe cache configured.

The VM is placed optimally to provide a local read path, and all vdisks are in sync, so we are ready to perform the tests. I’ve provisioned a VM with 4GB RAM and 2 CPU cores on KVM. The VM is running CentOS 7 x64 with SELinux disabled, and updated to the latest release.

OnApp vs AWS vs Azure - Cyber Host Pro's OnApp cloud

The AWS setup

I’ve chosen a ‘c5.large’ instance inside AWS, with standard SSD storage, as this was the instance that would cost around the same price as the OnApp-hosted solution above. The instance has CentOS 7 x64 installed with all updates completed, and SELinux disabled. It too has 4GB RAM and 2 CPU cores available.

OnApp vs AWS vs Azure - AWS c5 instance

The Azure setup

I have taken a ‘Standard B2s’ virtual machine with CentOS7 x64, which again is around the same price as the OnApp hosted solution above. It also has 4GB RAM and 2 CPU cores available. SELinux has been disabled.

OnApp vs AWS vs Azure - Standard B2s instance

Preparation & performance testing

I will run the following on each virtual machine and then reboot to completely disable selinux.

yum -y update
sed -i -e 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
yum -y install fio git gcc screen
[reboot]

I want to run a series of tests that will best mimic a production environment, so I will be using Unix Bench to give you an overall score of how the environment performs on the underlying infrastructure. This is a good choice as it will test the hardware performance and the default template configurations provided as standard, and has a simple scoring system for comparison which we can review later.

I will also be running some random read/writes at various block sizes using fio to test the storage layer. This is a better test than dd, as in a production environment applications access data randomly across the disks in parallel, and the block sizes will vary massively depending on the workloads running inside the virtual environment. I’ll take a total number of IOPS across all 9 tests for a simple comparison at the end.

This is how I will test:

Run Unix Bench:

cd; mkdir unixbench; cd unixbench
git init; git clone https://github.com/kdlucas/byte-unixbench
cd byte-unixbench/UnixBench
screen -S benchmark
./Run

Run fio stats:

# Random read/write performance test
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=0 --name=test --filename=random.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75 | grep IOPS; rm -f random.fio
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=0 --name=test --filename=random.fio --bs=256k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75 | grep IOPS; rm -f random.fio
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=0 --name=test --filename=random.fio --bs=4m --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75 | grep IOPS; rm -f random.fio

# Random read performance test
fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=1 --size=4G --numjobs=2 --runtime=240 --filename=random.fio --group_reporting | grep IOPS; rm -f random.fio
fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=256k --direct=1 --size=4G --numjobs=2 --runtime=240 --filename=random.fio --group_reporting | grep IOPS; rm -f random.fio
fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4m --direct=1 --size=4G --numjobs=2 --runtime=240 --filename=random.fio --group_reporting | grep IOPS; rm -f random.fio

# Random write performance test
fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=1 --size=4G --numjobs=2 --runtime=240 --filename=random.fio --group_reporting | grep IOPS; rm -f random.fio
fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=256k --direct=1 --size=4G --numjobs=2 --runtime=240 --filename=random.fio --group_reporting | grep IOPS; rm -f random.fio
fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4m --direct=1 --size=4G --numjobs=2 --runtime=240 --filename=random.fio --group_reporting | grep IOPS; rm -f random.fio

Results and conclusion

I wanted to write this blog post to show what kind of results you can get from an OnApp cloud running our integrated storage system, if everything is designed and configured correctly… but I didn’t expect such a huge gap between the results.

Just look at this:

OnApp vs AWS vs Azure - IOPS comparison OnApp vs AWS vs Azure - Unix Bench comparison

I wonder how much more money you’d need to spend to get similar results with the competition?

For those of you interested, I have kept a copy of the raw output from all providers: you can find it here. Thanks Chris Danks and the Cyber Host Pro team for giving me access to your cloud for this test!

 

Optimize your OnApp cloud performance

Don’t forget, the OnApp Professional Services team offers health checks and a range of other services that allow you to get the most out of your platform – please get in touch if you’d like help improving your cloud’s performance. You can contact us on ps@onapp.com or contact me directly at paul@onapp.com.

Learn more about OnApp

If you’re new to OnApp, why not chat with our cloud specialists or take a demo? Just fill out the form and we’ll be in touch.

Thanks!