Skip to content

Using Message Queues Will Improve Your Life

When you’re first frantically trying to get version 1.0 of your project out of the door, it’s easy to avoid reading about concepts that are new to you: “Naaah… it’ll be quicker for me to do it my way! is an all-too-common thought that often runs through my head.

By way of example, this is how I felt whenever I was coding a feature that involved executing a task in the future, such as scheduling a status update to be sent to Twitter. Simple enough: create a table in the database for generic tasks, and a cron job to run them at regular intervals.

And then I read this fantastic blog post about why doing the above with a database is not a good again. As has happened many times, “You’re a fool” began running around in my head instead.

For my latest project, Interactwive, I wanted to see if I could make life better. I spent 20 minutes looking for a system that’d keep my life simple: the minimum of effort on my current setup, and must support PHP. I found that activemqΒ works on Windows, so that took care of my development setup, and it implements the Stomp protocol, which makes message queuing devilishly easy with PHP.

I expected a bit of pain… but there wasn’t any. This made me suspicious, and speaking honestly, several weeks later I’m still expecting it to blow up any second now. Technology that does what it says and works straight away? Witchcraft!

Just by way of example: let’s say you have one PHP script that processes tasks. Things start to get busy, and so you want two or three scripts to processes tasks. With activemq and Stomp, this was as easy as firing up the three scripts… and that’s it. Far too easy!

Now starting to see the light, I had a spare couple of hours this weekend, so sat down with TweetingMachine, determined to switch it across to using message queues. And you know what? It took about an hour.

In total, I’ve spent about two hours learning how to use activemq and Stomp. In return I now have a vastly happier server, far more maintainable code, and lots of other nice extras, such as error notification and logging being a doddle.

It’s always irritating to look back and realise how much time one has invested in less-than-ideal solutions. I could have saved hours of frustration – and worrying about scalability – if someone had told me how easy this would be instead.

Move to message queues. You won’t regret it.

Written by Tom

Are you looking for web development or just someone who will work with your business needs and not against them? Get in touch with me here, or take a look at my consultancy’s website: Moo Unlimited. I’m confident I can help you.

Published inOpinionTips

17 Comments

  1. sam sam

    Nice, I’ve been meaning to implement a queue system in one of my projects. I think I shall do so in my next project. Have you looked into http://www.gearman.org? I’ve used it once, very cool tool.

    • Tom Tom

      Gearman looks really cool, but I couldn’t find a way to get it working on Windows that didn’t involve years of hassle with Cygwin :-(

      One of these days I’ll set up a proper local dev environment with some form of virtualization. Until then…

      • Sam Sam

        oh that’s a must! I run windows 7 unfortunately, but I have virtualbox with redhat running inside a VM for development.

    • Kristian Rickert Kristian Rickert

      I used gearman and it sucks. Crashed a system I used regularly. Consider RabbitMQ or ActiveMQ instead.

  2. andre andre

    was there a particular tutorial that you used to learn stomp and activemq? or just the documentation?

    • Tom Tom

      Just the documentation – I’m not doing anything that advanced :-)

      • andre andre

        thanks, been meaning to do one for a while

    • Tom Tom

      Honestly, I can’t tell if that’s satire or not πŸ˜‰

    • Kristian Rickert Kristian Rickert

      That’s crazy. Queue based systems are a simple pattern that have been used since the 70s. It’s nothing new. Questioning this is like questioning the validity of UDP for video or checksums for error checking.

      Queue uses are simple: if you have actions that can be processed asynchronously, use a queue system.

  3. Kaitnieks Kaitnieks

    The main problem is that by introducing another software service into your system you are limiting your deployment choices. If instead of acivemq installation there was a single php script that you copy and use queuing, that would be much better.

    • Tom Tom

      Hehe, very funny πŸ˜‰

      But surely PHP’s also problematic? A real solution would be KSH scripts! πŸ˜›

      • Kaitnieks Kaitnieks

        You said you’ve been using PHP already, which means that queuing service written in PHP would not require any additional installs.

  4. Very nice and useful post. Thanks for sharing.
    By using a queue system , it greatly improve an application’s scalability

  5. Hi, thanks for the interesting article. A couple of questions:

    * Does your queued data need to be persistent/available in the future? How would you handle that?

    * Do you think the queue would be applicable when all the data is not the same? When something draining the queue has to do so based on properties, priority, etc and a simple “pop” is insufficient?

    Thanks again.

  6. prats prats

    Sir, the article was great..
    Sir i’m trying message queuing in php. But i dont know where to start from.
    Sir please can you help me with this problem.

  7. That\’s crazy. Queue based systems are a slpime pattern that have been used since the 70s. It\’s nothing new. Questioning this is like questioning the validity of UDP for video or checksums for error checking.Queue uses are slpime: if you have actions that can be processed asynchronously, use a queue system.

Leave a Reply to XuDing Cancel reply

Your email address will not be published. Required fields are marked *