We have been working closely with a customer who is undergoing a business transformation. As a multimedia equipment manufacturer, the organization has a loyal following of its high quality devices. However, like many companies facing the convergence of markets and new customer demands, the company has embarked on a metamorphosis. Traditionally very focused on hardware, their software was largely ignored even though it offered customers real value. Part of the company’s transformation was a move to treat their software like a full-fledged offering, rather than a free supplement. An upcoming product release marked the first (and biggest steps), in cementing this change in company direction.
There are many reasons an organization might choose Amazon Aurora over the Amazon Relational Database Service (Amazon RDS). Superior performance, greater scalability, and the ability to restart without losing cache are just a few. However, for those organizations who are already running an important application or Website on top of the RDS managed service, it can be a challenge to migrate from it to Aurora, despite the latter’s obvious benefits. After all, you can’t just take down a service that customers expect access to 24x7.
AWS DevOps Case Study
A Fortune 500 manufacturer was using Hadoop, internal data centers, Rackspace and CenturyLink to facilitate services that connected its customers with data insights using an Internet of Things model. The overarching goal: to facilitate continuous data-driven improvement within its customers’ operations. To help achieve this goal and overcome its Hadoop scaling issues, the company engaged with Flux7, DevOps consulting group and AWS partners. Additionally, the manufacturer sought a global solution that would comply with EU data privacy laws.
Flux7 engineer Ahsan Ali and CTO Ali Hussain collaborated on this post
The rise of IoT has given rise to a new generation of needs in the world of big data processing. Now we need to handle data ingress from many sensors around the world, and make real-time decisions to be executed by these devices. As such it is no surprise we see new services to handle processing of streaming data, such as Amazon Kinesis.
Last week, Amazon Web Services announced the availability of larger and faster Elastic Block Storage Volumes, something we’ve been looking forward to since the original announcement at re:Invent 2014. AWS continues to add rich features to their platform and it can be difficult to stay on top of them, and understand which new capabilities are going to impact an individual business, and how.
Five years ago, Amazon found that every 100ms of latency cost them 1% of sales. Google discovered that a half-second increase in search latency dropped traffic by 20%.
Amazon recently introduced new types of storage-optimized instances. This new generation of instances is available within the I2 and HI1 families. All provide high storage and better IO performance compared to other instance families in AWS. Flux7 Labs decided to benchmark these new instances to better understand the tradeoffs between them that our customers face.
The Amazon I2 Instance Type
Amazon has announced immediate availability of the I2 instance type, the next generation of Amazon EC2 High I/O instance and the best solution for transactional systems and high performance NoSQL databases such as Cassandra and MongoDB. I2 instances feature the latest generation of Intel Ivy Bridge processors, the Intel Xeon E5-2670 v2. Each virtual CPU (vCPU) is a hardware hyperthread from an Intel Xeon E5-2670 v2 (Ivy Bridge) processor. Its features, price and availability can be combined to derive a performance-oriented usage and to explore new use cases.
In our previous post here, we detailed why Ganglia is a good tool for monitoring clusters. However, when monitoring a Hadoop cluster you often need more information about CPU, disk, memory, and nodal network statistics than the generic Ganglia config can provide. For those who need more finely tuned monitoring, Hadoop supports a framework for recording internal statistics and then for posting them to an external source, either to a file or to Ganglia. In fact, Hadoop now supports an implementation of the Metrics2 Framework for Ganglia. In this post we’ll discuss Hadoop Metrics2 Framework’s design and how it enables Ganglia metrics.
Recently at Flux7 Labs we developed an end-to-end Internet of Things project that received sensor data to provide reports to service-provider end users. Our client asked us to support multiple service providers for his new business venture. We knew that rearchitecting the application to incorporate major changes would prove to be both time-consuming and expensive for our client. It also would have required a far more complicated, rigid and difficult-to-maintain codebase.