I recently changed SmallPayroll to use Beanstalkd instead of delayed_job for background processing. delayed_job is an awesome tool and makes asynchronous processing so simple. However I wanted to have multiple queues so that I could have different workers processing different queues, and have some upcoming needs to process the jobs quicker than the 5 second polling time.
After watching railscasts on beanstalkd and stalker I decided to use that. Beanstalkd is a lightweight job processor, and stalker makes it very simple to use from the client end.
I used to have an observer that said
1 2 3 4 5 6
1 2 3 4 5
delayed_job was nice in that the job would just run against the model, but now I have to process the job in config/worker.rb:
1 2 3 4 5 6 7 8 9 10 11
One thing about stalker is that it wants you to pass simple objects instead of ActiveRecord objects, so I queue the user_id instead of the user model.
The script above monitors the default tube, which I don’t use, because the nagios plugin for beanstalk expects someone to monitor it (after setting it all up, I guess I could have set it up to ignore that tube). I’m also using the munin plugin for beanstalkd to graph the activity in the daemon.
Then, script/worker uses the daemons gem to start the job and restart it if it crashes:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Finally, some capistrano magic to start the worker with the deploy inside config/deploy.rb
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
The only problem I’ve run into so far is that my HTML email seems to go out without the text/html content-type. Fixing that was a simple matter of putting
content_type 'text/html' inside my mailer, which wasn’t needed when I was using delayed_job.