“Cloud Wars” bring new benefits to IoT and Big Data businesses

lock

Last week, Amazon Web Services announced the availability of larger and faster Elastic Block Storage Volumes, something we’ve been looking forward to since the original announcement at re:Invent 2014. AWS continues to add rich features to their platform and it can be difficult to stay on top of them, and understand which new capabilities are going to impact an individual business, and how.

CRN Magazine asked us for our thoughts about how this can help us in our aim to build the best, zero-administration infrastructures possible for our customers, and what it says about what AWS is trying to do now. You can read the full article here

We think this is a significant new feature as volumes of data are increasing and more businesses rely on big data or Internet of Things (IoT) information.  Analysts predict that the number of IoT units will easily hit 26 billion by 2020. Real time processing needs and data management challenges are becoming common in IoT deployments which calls for the need of efficient handling of capacity and real time query management in addition to the challenges like security and compliance.

For example, one of our customers has built their business on genomic analytics and comparison. Several vast data sets must be analyzed quickly in order to produce a solution in a timeframe that is acceptable for patients. Vast data sets here are in the order of tera or peta bytes. 50 GB is the space required to store a single person’s DNA. It’s not too long before the size of the data set reaches the order of TB. To accomplish a solution for this,  last year, we needed to have seven servers per service solely because EBS volumes weren't large enough. You can read that case study here.

The very nature of big data calls for management of humongous amounts of data. Data are of varying structures and the ability to process such large amounts at a reasonable speed is a fair expectation.  

The higher IOPS is also welcomed as it allows the use of EBS for high performance applications. This was previously possible only by using multiple smaller EBS volumes which made processing time and cost inefficient and management an overhead.

------------------------------------------------------------------------------------------------------------------------------

Flux7 CTO Ali Hussain explained to CRN that:

“The larger volumes are a much appreciated feature in the present world of growing data. Without these larger volumes we've had to setup multiple smaller volumes for customers and put them in a software RAID configuration. Increasing the complexity of the system and the setup and maintenance costs, This is especially a pain point when dealing with Big Data tools like Hadoop and Cassandra.”
 
“These larger EBS volumes also affect RDS. We have had some customers that were being forced to move out of RDS but the larger volume size will allow them to take advantage of Amazon's Relational Database Service and benefit from the reduced cost and overhead.”

------------------------------------------------------------------------------------------------------------------------------

As cloud infrastructure vendors continue to announce new capabilities in hopes of achieving feature parity with each other or creating a competitive advantage in one area or another, it’s the customers who are winning the “cloud wars.” These new features, and continued price wars, are making it more difficult than ever to put forward an argument to stay on premise. This specifically applies to businesses dealing big data, IoT or a combination of both mainly because of the nature of data handled, unforeseen spikes and lulls, and real-time needs. A traditional on-premise infrastructure could end up being a disastrous choice.  

For some businesses, the first step is to establish a simple POC where best practices architecture can be established. For others, it’s about creating a strategy to move large numbers of servers at a time with minimal effort, enabling the team they have in-house to do the heavy lifting, and not relying on third party managed services.

Wherever the challenge lies, the questions and complexity about cloud computing are being removed through platform vendor features, automation and templates, enabling businesses to achieve the agility they deserve, with the resources they have.


March 24, 2015 / Migration, Storage, IoT, Data

About the Author

Flux7 Labs
Find me on: