Amazon Elastic Compute Cloud EC2 – 🅰nthony on 🅰zure

Comparing Cloud Compute Services.

Jason Read at CloudHarmony does a great job sorting through all of the different cloud computer offerings by vendors and then compares their performance in a like-to-like benchmark. This is no small task and most reports I have read comparing cloud compute do not do a good enough job of comparing similar services when conducting their performance tests. You can obtain a copy of the full report from here which is 100+ pages long.

Below are some the test results along with my own comments as well.

Compute Service Provider
Web Server Comparison	Small Web Server	Medium Web Server	Large Web Server
Amazon EC2 (Instance Types Explained)	c3.large + t2.medium	c3.xlarge	c3.2xlarge
DigitalOcean (Types Explained)	4 GB / 2 Cores	8 GB / 4 Cores	16 GB / 8 Cores
Google Compute Engine (Types Explained)	n1-highcpu-2	n1-highcpu-4	n1-highcpu-8
Microsoft Azure (Tiers Explained)	Medium (A2)	Large (A3)	Extra Large (A4)
Rackspace (Types Explained)	Performance 1 2GB	Performance 1 4GB	Performance 1 8GB
SoftLayer (Types Explained)	2 GB / 2 Cores	4 GB / 4 Cores	8 GB / 8 Cores
Database Server Comparison	Small Database Server	Medium Database Server	Large Database Server
Amazon EC2	c3.xlarge	c3.2xlarge	c3.4xlarge
DigitalOcean	8 GB / 4 Cores	16 GB / 8 Cores	48 GB / 16 Cores
Google Compute Engine	n1-standard-4	n1-standard-8	n1-standard-16
Microsoft Azure	Large (A3)	Extra Large (A4)	A9
Rackspace	Performance 2 15GB	Performance 2 30GB	Performance 2 60GB
SoftLayer	8 GB / 4 Cores	16 GB / 8 Cores	32 GB / 16 Cores

Test Results

CPU Performance Results

In the web server test Amazon EC2 was a little better than the rest of the competition. And in the database server test Rackspace was slightly better until the testing of large database servers where Azure’s new A9 server won out.

Web Server CPU Performance Variability Test

Database Server CPU Performance Variability Test

Also included in this test was CPU variability. In these multi-tenant, shared resource environments the performance over time can be a risk to a customers if that is the case this test should be taken into consideration as well as the other variable tests included in the report. For this test the lower the score the better, ideally you want the same performance over the life of the service.

In both tests Amazon had the best overall score across all server types. The changing CPU types as testing across different server types should be considered when looking at these two scores as well.

Disk Performance Testing

In these tests Amazon EC2 and Rackspace were consistently faster and more reliable. Since they are SSD based storage they should be faster and more consistent than the other services. DigitalOcean is also SSD based but their performance was not on par with the other SSD based services and also had the highest rate of variability. SoftLayer is not SSD based but their overall disk performance and consistency were very good. This could be from using SSD caching but no matter how they are doing it the performance speaks for itself. Microsoft Azure and Google do not offer SSD storage which is reflected in their testing.

Web Server Tests

Small Web Server Random Reads Variability Test

Small Web Server Sequential Reads Variability Test

Small Database Server Random Read Writes Test

Small Database Server Sequential Read Writes Test

Small Database Server Random Read Writes Variability Test

Small Database Server Sequential Read Writes Variability Test

Medium Web Server Random Reads Variability Test

Medium Web Server Sequential Reads Variability Test

Medium Database Server Random Read Writes Test

Medium Database Server Sequential Read Writes Test

Medium Database Server Random Read Writes Variability Test

Medium Database Server Sequential Read Writes Variability Test

Large Web Server Random Reads Variability Test

Large Web Server Sequential Reads Variability Test

Large Database Server Random Read Writes Test

Large Database Server Sequential Read Writes Test

Large Database Server Random Read Writes Variability Test

Large Database Server Sequential Read Writes Variability Test

Memory Performance

In these test they conduct them only on the database servers and because of Azure’s older hardware based on AMD CPU and MOBO they did not perform well until the large database server test where they are using the newer Intel Sandy Bridge based platform. Amazon and Google both perform well through all testing and outperformed the others. Newer hardware is always going to win in these types of tests and that is reflected here.

External and Internal Network Performance

Network testing is always subjective and by testing cloud service providers networking throughput it just adds to the complexity of the configuration as some add additional ways to improve performance through different setups. Amazon, Google and Rackspace seem to provide higher throughput then Microsoft Azure, DigitalOcean and Softlayer. Also throughput by some vendors is limited depending on the size of the compute instance your purchase.

They also cover the value of each provider in their report which ultimately should be the deciding factor in choosing which cloud computing provider to use for your solution. After all there is no need to overspend on a service if you can accomplish everything you need with a cheaper solution.

You can download the full report here.