AWS SQS FIFO Queues and RabbitMQ on AWS

flux7_blog_AWS-SQS-FIFO-Queues-RabbitMQ-AWS.png

High availability has become a key requirement of every layer in today’s technology stack. And, message queuing or message brokering software is no exception. In the past we’ve relied, like many of you, on RabbitMQ to create highly available message queues when FIFO (First-In, First-Out) was required. (Indeed, our RabbitMQ tutorial is one of our most-oft read blogs.) Often this is for ecommerce, financial services and other applications where it is important to strictly process messages only once and in the order they are published.

As a managed service that now offers FIFO message queues, the recent announcement of AWS SQS FIFO in the N.Virginia region is a significant update. (In November AWS made FIFO queues available in the Ohio and Oregon regions.) As an AWS managed service, SQS FIFO queues are very easy to start using, easy to provision, easy to automate and are inherently highly available and fault tolerant. Moreover, it features granular pay per use -- Amazon SQS FIFO queues are priced at $0.50 per million API requests, with the first 1 million monthly requests free -- so the cost is low.

In addition to removing one more thing from your list of things to manage from an operational and security standpoint, AWS SQS also now has several features users will appreciate:

  1. FIFO queues: Standard SQS queues do not guarantee FIFO ordering (rather, best effort ordering), nor do they guarantee that messages will be strictly processed only once. Before SQS FIFO, the application logic needed to manage ordering of incoming messages as the messages from Standard SQS were not guaranteed to reach the app in sequence. Now with SQS FIFO queues, message processing in applications will be greatly simplified. This is a boon for transactional application owners using AWS cloud computing.

  2. Supports server-side encryption: SSE lets you send sensitive data safely by encrypting  the body of a message in an Amazon SQS queue. SSE does so using keys managed in AWS Key Management Service (AWS KMS). According to Amazon, SSE encrypts messages as soon as Amazon SQS receives them; the messages are stored in encrypted form; and Amazon SQS decrypts messages only when they are sent to an authorized consumer.

  3. HIPAA and PCI compliant: SQS is PCI Level 1 certified and is part of the AWS HIPAA compliance program. Together with SSE, Amazon SQS is a strong solution for building PCI and HIPAA-compliant applications, storing messages in transit, and transmitting messages—including messages containing protected information.

SQS is a fully managed service and it is elastic (with near infinite scaling) so you need not worry about managing messaging infrastructure. This fact, coupled with these three core features, leads us to predict that AWS SQS will have a large impact on applications relying on messaging, especially those, like mobile banking, or ecommerce, where message queue order is critically important as is the protection of sensitive data. Let’s take a quick look at how the new SQS FIFO compares to SQS Standard and RabbitMQ for high availability, reliability, and more:

 

SQS Standard

SQS FIFO

RabbitMQ (AMQP)

Managed

Fully Managed Service

Fully Managed Service

Not a managed service

Reliability

A message is guaranteed to be delivered ‘AT LEAST’ once

Guarantee that a message is delivered ‘ONLY’ once

Depends on the type of RabbitMQ cluster. HA cluster guarantee that message is delivered at least once. And only once when a single queue is involved.

Ordering

Provides best-effort ordering

Provides FIFO ordering

Provides FIFO ordering

Volume

A nearly-unlimited number of transactions per second

FIFO queues are limited to 300 transactions per second (TPS)

Depends on the size of the queue and size of the cluster

Monitoring

Out of the box monitoring with CloudWatch

Out of the box monitoring with CloudWatch

Monitoring need to be implemented explicitly

Authenti-
cation

Via IAM

Via IAM

Via LDAP

Compliance

PCI DSS and HIPAA

PCI DSS and HIPAA

PCI DSS and HIPAA

(Built-In)

 

As you can see, the new SQS FIFO queue type adds a powerful reason to switch from RabbitMQ on AWS to AWS SQS, in the process eliminating the overhead associated with managing and operating dedicated messaging software and infrastructure.

Highly available, securely processed messages that are delivered in the order received -- and only once -- are imperative for digital business success. To learn more about designing and building a  highly available, secure architecture using AWS, please subscribe to our blog below.

Sign Me Up!

About the Author

Flux7 Labs
Find me on:

Join Us

Join thousands of technology enthusiasts, subscribe and get expert perspective in your inbox.

Connect With Us

Recent Posts

Categories