26 Sep
Process Watchdog
I’m sure there’s a million variants out there, but I had a simple need to make sure a process was running and restart it if it isn’t. I’ve done this before using one off scripts, but then I figured I may as well do one that is generic:
watchdog.sh:
#!/bin/sh
# watchdog LOOKFOR SERVICE
# will restart SERVICE (/sbin/service SERVICE restart) when LOOKFOR doesn’t appear in the processlist
P=`echo $1 | sed ‘s/^\(.\)/[\1]/’` # enclose first char in brackets
ps -ef | grep -v $0 | grep -q “$P” # skip myself
if [ $? -eq 1 ]; then
/sbin/service $2 restart
fi
So,
/usr/local/sbin/watchdog.sh asterisk asterisk
will run /sbin/service asterisk if there are no asterisk processes running.
And what watches the watchdog?
/usr/local/sbin/watchdog.sh watchdog watchdog
heh.. =)
March 6th, 2007 at 4:04 pmSmartass!
Run it through inittab if you’re worried, at least if init fails you’ve got bigger things to worry about!
March 7th, 2007 at 8:20 pmSean, thanks for the simple yet effective script.
Thought I would post a warning for users, however. If you use this in a cronjob, do NOT put the name of the process you are looking for in the name of the cron. This will cause a false positive, and the script will not know that your server has gone away.
For example, I had a cron job named “watch_httpd”, and I couldnt figure out why when i would run the commands manually it worked perfect, but if I ran the cron, it wouldnt work. Well, it saw the process named “watch_httpd” and thought the server was up.
I’m ashamed to say it took me 2 minutes to implement this script, and an hour to figure out the above issue.
Thanks again!
February 9th, 2010 at 9:54 am