AMQP (Advanced Message Queuing Protocol) is an open standard protocol for message-oriented middleware. Like most standards, the direction has taken a few twists and turns during the process of standardization and is in the final stages of being ratified as AMQP 1.0 within the OASIS body. From there AMQP will move on to become an international standard through ISO/IEC.
The biggest contributor to the confusion around AMQP is due to the fact AMQP took a drastic turn in direction between 0.9.1 and 1.0.
Why does AMQP Matter?
Like any standard (remember WS-*?) the AMQP standard may not affect or matter to you but for some it will be important to understand. I’ve been informed that specific industries will require AMQP 1.0 which means if you are already or want to do business in these areas you should get informed about this standard and your available options.
Let’s create a fictional situation (I’m not proposing this is how you would solve this problem) where we want to send log messages to 2 different services. One service will receive critical error messages to dispatch to support staff while another service will receive warnings and info messages for dashboard displays. The composition of the messaging topology may look something like this.
With this diagram we can highlight the areas the different AMQP specifications take place.
Up to AMQP 0.9.1 the AMQP group was focused on standardizing client to broker wire level protocol and broker messaging capabilities.
The message broker capabilities allow composing message routing using exchanges, queues and bindings.
Exchanges receive messages from client applications via the network wire-level protocol and route messages to queues based on the routing rules defined by bindings. Routing rules using bindings are created on message properties or content.
Message queues receive and store messages from clients or exchanges. Messages are removed from a client pulling off the queue.
Bindings define the routing rules and relationship between an exchange and message queues.
This is called the “AMQ model” which attempts to provide a standard for the following:
- Guarantee interoperability between conforming implementations.
- Provide explicit control over the quality of service.
- Complete configuration of server wiring via the protocol.
The AMQP 0.9.1 specification has commands for creating and managing exchanges, queues and bindings.
Highlighted in red are the messaging capabilities (AMQP model) defined in AMQP 0.9.1.
Network wire-level protocol
This portion of the standard defined client to broker communications. The standardization of the wire-level protocol attempts to provide the following:
- Compact, using a binary encoding that packs and unpacks rapidly.
- Messages of any size without significant limit.
- Multiple channels across a single connection.
- Long-lived, with no significant in-built limitations.
- Asynchronous command pipe-lining.
Highlighted in red is the network wire-level protocol.
In summary, AMQP 0.9.1 describes the bytes going over the wire and the API’s available for configuring the message routing and quality of service of the broker.
The AMQP group decided to go down a very different direction than what had been done up to 0.9.1 before ratification and because of that AMQP 1.0 is entirely different. Some message-oriented middleware had already been introduced into the market that was built conforming to 0.9.1 and previous versions of the standard. A few examples of this are RabbitMQ, Apache Qpid, StormMQ and RedHat Enterprise MRG (based on Apache Qpid).
This change has created a pretty big divide in the community from my observations. Some people involved in earlier versions consider it a big step backwards while others consider it a simplification to build on that introduces more flexibility (more on that later).
AMQP 1.0 is a much more narrow standard than the broad standards in the previous versions. It only defines the network wire-level protocol for the exchange of messages between two endpoints. The standard supports exchanging messages in peer-to-peer or by brokered topologies.
The same protocol can be used to communicate client-to-client, client-to-broker or broker-to-broker with all of the delivery guarantees available.
AMQP 0.9.1 heavily influenced broker internals. AMQP 1.0 does not define the internals of the broker implementation and can be used with any broker. This means that existing 0.9.1 implementations (like the ones mentioned above) can add the AMQP 1.0 network wire-level protocol support. An example is an experimental plugin for RabbitMQ that provides AMQP 1.0 endpoint support even though RabbitMQ is an AMQP 0.9.1 based broker.
The AMQP 1.0 network wire-level protocol is not compatible with the AMQP 0.9.1 network wire-level protocol.
Important part lost
In AMQP 0.9.1 the protocol itself has support for creating and managing exchanges, queues and bindings. This means the broker behaviours are fixed and limited by what the standard defines.
In AMQP 1.0 the broker management is shifted from a protocol-level command to a message you transport over a standard message. This means brokers can define their own management commands that any AMQP client library can send. Broker capabilities do not have to be part of the standard that clients must implement to be compatible.
Since AMQP 1.0 defines how to communicate to an endpoint and has the concept of an address, a broker could define an XML or JSON message to send to this address for broker management commands such as creating exchanges, queues and routing behaviour.
In theory, if FooMQ was a broker that was AMQP 1.0 compliant you could send a message to amqp://hostname/queues/bar-queue that was a JSON message with the command to create a new durable queue named “bar-queue”.
There will be AMQP 1.0 related projects and products announced and in the market soon.
In my opinion, by AMQP 1.0 moving the broker specifics out from the protocol level makes it simpler to write client libraries and for these libraries to have increased longevity. Brokers also gain flexibility.
This move is similar to what we see in popular HTTP API’s today (RESTful ones included) versus WS-* of yesterday. Less standardizing of the protocols themselves and more generic communication and loose coupling. Tighter coupling won a few battles but loose coupling won the war.
A wise person once told me history has shown loose coupling usually wins.
- Trick to reduce allocations improves response latency in Haywire
- Improving the protocol parsing performance in Redis
- Mencius and Fast Mencius a high performance replicated state machine for WANs
- Tuning Paxos for high-throughput with batching and pipelining
- Scalable Eventually Consistent Counters
- Create benchmarks and results that have value
- Routing aware master elections
- My new test lab
- Responsible benchmarking
- Understanding hardware still matters in the cloud
- The “network partitions are rare” fallacy
- Messaging and event sourcing
- Further reducing memory allocations and use of string functions in Haywire
- HTTP response caching in Haywire
- Atomic sector writes and misdirected writes
- How memory mapped files, filesystems and cloud storage works
- Hello haywire
- Active Anti-Entropy
- Lightning Memory-Mapped Database
- Write amplification
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- November 2013
- October 2013
- August 2013
- July 2013
- June 2013
- May 2013
- April 2013
- March 2013
- January 2013
- October 2012
- September 2012
- August 2012
- May 2012
- April 2012
- February 2012
- January 2012
- December 2011
- September 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010