Having evaluated oss message queues for high volume web services I can see exactly how twitter might make this decision.
- activemq: high resource usage, not super stable at high loads
- rabbitmq: messages must ALL fit into memory. Your message queue should be a "rescue me" button - if all your dbs go down, just queue crap to disk. You do not want to worry about memory overflow in that situation.
- memcacheq: super stable, low resource consumption but fixed message size (padding for smaller messages).
We ended up using memcacheq but it's hardly what you would call full featured, so we'll have to do prioritization, etc in our code.
We ended up using memcacheq but it's hardly what you would call full featured, so we'll have to do prioritization, etc in our code.