When I set it up a couple of years ago, I used SMS Gateway which was $10 for 100 SMSes. I was able to page with a simple curl command. However I’d get the odd page that wouldn’t go through, and despite being very responsive, the support wasn’t very reassuring.

Now that I’ve depleted my 100 pages, I figured I’d move over to Twilio because they’re pretty slick, and the reliability has to be better.

Some Nagios code, first:

define contactgroup{
        contactgroup_name       important
        alias                   Sean Buzz
        members                 sean, page
}
define contact{
        contact_name                    page
        alias                           page
        service_notification_commands   notify-by-page
        host_notification_commands      host-notify-by-page
        service_notification_period 24x7
        host_notification_period 24x7
        service_notification_options w,u,c,r
        host_notification_options d,u,r
        pager                           nobody@localhost
}
define service{
        use                             local-service         ; Name of service template to use
        host_name                       localhost
        service_description             smallpayroll.ca
        contact_groups                  important
        check_command                   check_http_string!smallpayroll.ca!Easy to use
        }

The first stanza creates a contact group called “important” that emails sean, and pages. The second stanza implements the “host-notify-by-page” and “notify-by-page” which do the actual paging. The final stanza is an example of a service that would get paged on. If the check_http_string check fails, the “important” contact group is notified.

The code to page is as follows:

define command {
        command_name    notify-by-page
        command_line    curl --data-urlencode "From=YOURTWILIONUMBER" --data-urlencode "To=YOURCELL" --data-urlencode "Body=[Nagios] $NOTIFICATIONTYPE$ $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$" https://SID:TOKEN@api.twilio.com/2010-04-01/Accounts/SID/SMS/Messages >> /tmp/sms
}
define command {
        command_name    host-notify-by-page
        command_line    curl --data-urlencode "From=YOURTWILIONUMBER" --data-urlencode "To=YOURCELL" --data-urlencode "Body=[Nagios] $HOSTSTATE$ alert for $HOSTNAME$" https://SID:TOKEN@api.twilio.com/2010-04-01/Accounts/SID/SMS/Messages >> /tmp/sms
}

To get the SID and TOKEN (note there are two instances of the SID, the second is in the URL right after accounts) go to your dashboard and look at the top:

To get a number click on Numbers then Buy a Number:

Then search for a number. It should be in the USA, as it looks like Canadian numbers don’t support SMS. You can verify this by clicking on “Buy”:

Buy the number for $1/month. You don’t have to set up any URLs with it if you’re doing outbound only.

YOURCELL should be obvious It could also be templated within Nagios.

a

[root@host tmp]# pwd
/tmp
[root@host tmp]# cd -
/auto/tmp/sean
[root@host sean]# pwd
/auto/tmp/sean
[root@host sean]# cd -
/tmp
[root@host tmp]# pwd
/tmp

What it does is cycle you between your last two directories. It also operates outside of the pushd/popd stack:

[root@host tmp]# dirs
/tmp

So you should still be able to use those!

a

git pull origin master

I move between the two environments using the host file, which sometimes means I’m not sure which environment I’m in. I put the following in functions.php to help me out:

function mysite_i_am_in_dev() {
  // Red border if we're in dev mode
  echo '<!-- dev mode --><style type="text/css"> body { border: 2px solid #FF0000; } </style>';
}

if ($_SERVER["SERVER_ADDR"] == "x.x.x.x") { // x.x.x.x is the IP address of your dev server
  add_action('wp_head', 'mysite_i_am_in_dev');
  add_action('admin_head', 'mysite_i_am_in_dev');
}

So now, anyone viewing the development server will have a small red border around the screen.

a

After watching railscasts on beanstalkd and stalker I decided to use that. Beanstalkd is a lightweight job processor, and stalker makes it very simple to use from the client end.

I used to have an observer that said

class UserObserver < ActiveRecord::Observer
  def after_create(user)
    UserMailer.send_later(:deliver_welcome_email, user)
    UserMailer.send_later(:deliver_notify_admin, user)
  end
end

This became:

class UserObserver < ActiveRecord::Observer
  def after_create(user)
    Stalker.enqueue("email.new_user", :user_id => user.id)
  end
end

delayed_job was nice in that the job would just run against the model, but now I have to process the job in config/worker.rb:

require 'stalker'
include Stalker
require File.expand_path("../environment", __FILE__)

job 'default' do |args|
  puts "I don't support the default queue"
end

job 'email.new_user' do |args|
  user = User.find(args["user_id"])
  UserMailer.deliver_welcome_email(user)
  UserMailer.deliver_notify_admin(user)
end

One thing about stalker is that it wants you to pass simple objects instead of ActiveRecord objects, so I queue the user_id instead of the user model.

The script above monitors the default tube, which I don't use, because the nagios plugin for beanstalk expects someone to monitor it (after setting it all up, I guess I could have set it up to ignore that tube). I'm also using the munin plugin for beanstalkd to graph the activity in the daemon.

Then, script/worker uses the daemons gem to start the job and restart it if it crashes:

#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'

pwd  = File.dirname(File.expand_path(__FILE__))
file = pwd + '/../config/worker.rb'

Daemons.run_proc(
  'payroll-generic-worker', # name of daemon
  :dir_mode => :normal,
  :dir => File.join(pwd, '../tmp/pids'),
  :backtrace => true,
  :monitor => true,
  :log_output => true
) do
  exec "stalk #{file}"
end

Finally, some capistrano magic to start the worker with the deploy inside config/deploy.rb

namespace :beanstalk do
  desc "Start beanstalk process"
  task :start, :roles => :app do
    run "cd #{current_path}; RAILS_ENV=production script/worker start"
  end

  desc "Stop beanstalk process"
  task :stop, :roles => :app do
    run "cd #{current_path}; RAILS_ENV=production script/worker stop"
  end

  desc "Restart beanstalk process"
  task :restart, :roles => :app do
    run "cd #{current_path}; RAILS_ENV=production script/worker restart"
  end
end

after "deploy:start", "beanstalk:start"
after "deploy:stop", "beanstalk:stop"
after "deploy:restart", "beanstalk:restart"

The only problem I've run into so far is that my HTML email seems to go out without the text/html content-type. Fixing that was a simple matter of putting content_type 'text/html' inside my mailer, which wasn't needed when I was using delayed_job.

a

Step 1 – Install the plugin

 rails plugin install https://github.com/rails/exception_notification.git

Step 2 – Edit config/application.rb:

module MyApp
  class Application < Rails::Application
    # .....
    #  somewhere in this block put the following:

    config.middleware.use "::ExceptionNotifier",
      :email_prefix => "[MyApp Error] ",
      :sender_address => %{"notifier" },
      :exception_recipients => %w{youraddress@example.com}

3. Verify:

$ rake middleware | grep ExceptionNotifier
use ExceptionNotifier

Now you’ll get any application errors emailed to the addresses in the exception_recipients array.

a

2010/11/19 09:30:08 [emerg] 27885#0: eventfd() failed (38: Function not implemented)
2010/11/19 09:30:08 [alert] 27884#0: worker process 27885 exited with fatal code 2 and can not be respawn

It turns out the new nginx rpm was compiled with –with-file-aio which uses the eventfd() call:

[root@smallpayroll ~]# nginx -V
nginx version: nginx/0.8.53
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-48)
TLS SNI support disabled
configure arguments: --user=nginx --group=nginx --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-file-aio --with-mail_ssl_module --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables' --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables'

eventfd() wasn’t available until Kernel 2.6.22. I was running 2.6.18 because that’s what my VPS offered when I started. Fortunately it was a quick option in the Linode panel, a reboot, and now I’m running 2.6.32.

a

One great feature is that git can work with an SVN repository. You use git to check out some SVN code into a local git repository, you work with the repo as if it were git, then you commit your changes back to svn.

The lowdown on this is:

# Check out the svn repo
# It may take a while because it's copying all the changes, not just a checkout
git svn init http://myrepo/myproject/trunk myproject
cd myproject
# do some work
# commit to git
git commit -a -m "did some work"
# send updates to svn
git svn dcommit

But wait, someone else has done some work and you want to pull the latest changes from svn to your git repo

git svn rebase

Nice eh?

So let’s say you’re working in your git repo, and want to branch

git branch somefeature
git checkout somefeature

You’re working along, and you want to pull changes from svn into master and your branch.

git checkout master
git svn rebase
# master is up to date
# so merge master into the branch
git checkout somefeature
git merge master

This is so easy to use! Previously, with SVN, I’d branch, work on the feature, and then merge back to trunk. Given how painful merges were, there was no hope that I’d be merging trunk changes into my feature branch.

I’m sure my tune will change the first time I blow up my repository though

a

grep foo `find . -name \*.php`

which looks for foo in all the PHP files. If the list of files gets too long for the shell, then xargs is the better option:

find . -name \*.php | xargs grep foo /dev/null

This breaks up the command into manageable chunks. The /dev/null is there in case xargs only passes one file name to the command, this ensures there will be two file names to trigger the printing of the matching file (there’s probably an option, but I like this way better)

The problem with find -print |xargs is that a file or directory with spaces causes problems. For instance “xxx yyy/blah.php” will run

grep foo /dev/null xxx yyy/blah.php

neither of which exist.

My first hack at it was to revert to programmer mode and do a loop:

find . -name \*.php | while read A; do grep foo "$A" /dev/null; done

The best solution I found so far is to use -printf to force find to quote the file when it spits out the file name and get back into the xargs/stream mode of thinking:

find . -name \*.php -printf '"%h/%f"\n' | xargs grep foo /dev/null

Maybe there’s an easier way? (besides shooting people that use spaces in directory or file names)

a

So when I read Top Five Best Database Management Tools, I wondered what the heck the value was. Some of the tools were for Microsoft SQL server, some were for MySQL, and there was no criteria for the list other than “reader favourites”.

So, in an effort to answer the “how do I manage my database” question, here’s the top 1 database management tool, and how to use it.

But first, a disclaimer. It’s only the top tool if:

• You use mySQL
• Your web server can serve PHP
• You’re me

The tool is phpMyAdmin. That should come as no surprise, because it’s on everyone else’s top N list. There may be better, but it’s easy to install and probably handles anything you can throw at it. If you’re not using MySQL, or you have something better, then go ahead and use it!

So go to the homepage and download the software. If you’re in Windows grab the .zip version, if you’re in Linux, grab the .gz version.

First, uncompress the software to somewhere outside your document root. Doing it this way makes management a bit easier in the future, but slightly complicates the installation. On my machine, I’d go

cd /var/www
tar -xzf tar -xzf /tmp/phpMyAdmin-3.2.3-all-languages.tar.gz
ln -s phpMyAdmin-3.2.3-all-languages phpMyAdmin

Line 1 gets you to the parent directory of the web stuff (at least in RedHat/CentOS). Line 2 uncompresses the software. Line 3 creates a symbolic link to the current version.

Next we’ll create an Apache configuration file to handle access to the software. You can do this in the main httpd.conf file, but most distributions have a directory where extra configurations are loaded.

/etc/httpd/conf.d/phpmyadmin.conf:

Alias /phpMyAdmin/ /var/www/phpMyAdmin/

<Location /phpMyAdmin/>
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
    Allow from 192.168.1.0/24
    Allow from 1.2.3.4
</Location>

Line 1 maps anything under /phpMyAdmin/ (note the trailing /) to /var/www/phpMyAdmin/. The latter is a symlink to the current version, so if you want to upgrade you can have both versions uncompressed and use the symlink to change the active version. You can also choose a more friendly name, if you’d like.

The rest of the code restricts access to the localhost (127.0.0.1), the local network, and a certain outside address. If you want everyone to access it, delete the Location stanza and everything inside it (leaving only the Alias)

If you go to the URL, you’ll be able to log in using your username and password.

An interesting thing about phpMyAdmin is that it can manage several servers from one web page. In this case, the login screen will allow you to choose the server. This is great if you have multiple mySQL servers, or if your web server and SQL server are on separate devices.

The easiest way to do this is to use the web based configuration:

mkdir /var/www/phpMyAdmin/config
chmod 777 /var/www/phpMyAdmin/config

Then hop over to http://127.0.0.1/phpMyAdmin/setup/ (or whatever your server is) and follow the instructions to create several servers. Unless you have a specific reason to, you can choose the defaults (except for the server name!)

Once you’re done, protect the directory from writing and copy the generated file to the root:

chmod 755 /var/www/phpMyAdmin/config
cp /var/www/phpMyAdmin/config/config.inc.php /var/www/phpMyAdmin

If you’d rather not use the web interface, copy the config.sample.inc.php to config.inc.php and go from there.

a

I’ve got some code that I’m storing in SVN. In traditional SVN form, I’ve got my repo set up as

/trunk
/branches

So my work is done out of trunk, ie

svn co http://home.ertw.com/svn/repos/payroll/trunk/ payroll

I then had some major rework I wanted to do. Rather than continuing to develop out of trunk, I wanted to keep these changes separate so that I could continue working on other things in the code base at the same time as the rework. In svn, that’s done as a branch. From the root of the checked out code:

svn copy http://home.ertw.com/svn/repos/payroll/trunk http://home.ertw.com/svn/repos/payroll/branches/engine_mark_2
svn switch http://home.ertw.com/svn/repos/payroll/branches/engine_mark_2

The first command copies all the files from the trunk to a branch called “branches/engine_mark_2″. Note that the “trunk” and “branches” are just directories, they have no meaning themselves. I could develop out of /, or even out of /fred and branch to /wilma.

The second command switches the current repository from /trunk to /branches/engine_mark_2. I had just checked in all the code, so nothing had to change, it was just a pointer to where I was storing changes.

So, I could continue working on this branch, and also check out /trunk to somewhere else to make changes.

After all the work in the branch is done and checked in, take a look at the current branch:

[sean@sergeant payroll]$ svn info
Path: .
URL: http://home.ertw.com/svn/repos/payroll/branches/engine_mark_2
Repository Root: http://home.ertw.com/svn/repos/payroll
Repository UUID: 116013b7-0df0-436d-95c2-deac7dca3705
Revision: 92
Node Kind: directory
Schedule: normal
Last Changed Author: sean
Last Changed Rev: 92
Last Changed Date: 2009-08-24 20:57:22 -0500 (Mon, 24 Aug 2009)

The revision the branch started at is 92.

I checked out a copy of /trunk to another directory:

svn co http://home.ertw.com/svn/repos/payroll/trunk payrolltrunk; cd payrolltrunk

Then merge version 92 to the head, giving the url to the branch:

svn merge -r 92:HEAD http://home.ertw.com/svn/repos/payroll/branches/engine_mark_2
--- Merging r93 through r95 into '.':
U    test/unit/accrual_test.rb
U    test/unit/employer_test.rb
U    test/factories.rb
U    test/lib/tax_calc_test.rb
U    test/lib/cpp_ei_test.rb
--- Merging r93 through r95 into 'test/lib/finalize_test.rb':
U    test/lib/finalize_test.rb
--- Merging r93 through r95 into '.':
U    test/lib/pre_flight_test.rb
A    test/lib/payroll_procedure_test.rb
U    app/models/employee.rb
U    app/models/accrual.rb
U    app/models/employer.rb
U    app/controllers/payroll_controller.rb
U    db/schema.rb
A    db/migrate/20090827031532_add_payroll_run_to_accrual.rb
A    db/migrate/20090827032200_add_status_to_employee.rb
U    lib/payroll_engine.rb

At this point you’re sitting in a copy of HEAD with the branches changes merged it. Thankfully I had no conflicts, otherwise I’d have to resolve those before continuing by looking at the files with a status of “C” and fixing the conflicts that are marked by SVN inside the file.

Finally, commit the changes to HEAD:

svn commit

Then, either wipe out all your checked out copies and check out HEAD again, or switch your working copy of the branch back to head:

 svn switch http://home.ertw.com/svn/repos/payroll/trunk

The branch can stay as is.

Edit
After doing this again I found a problem trying this again when I had added files to the repository in the branch, and was getting “Skipped missing target” errors.

svn log --verbose --stop-on-copy $BRANCH
...
------------------------------------------------------------------------
r106 | sean | 2009-09-04 07:56:23 -0500 (Fri, 04 Sep 2009) | 2 lines
Changed paths:
   A /branches/reporting (from /trunk:105)

Branching to integrate reports

# now in a checked out version of trunk
[sean@sergeant payroll]$ svn merge -r 106:HEAD $BRANCH
--- Merging r107 through r117 into '.':
A    test/unit/helpers/payroll_history_helper_test.rb
U    test/unit/accrual_test.rb
U    test/functional/payroll_controller_test.rb
A    test/functional/payroll_history_controller_test.rb
A    app/helpers/payroll_history_helper.rb
U    app/models/employee.rb
U    app/models/payroll_run.rb
U    app/models/accrual.rb
U    app/models/employer.rb
U    app/controllers/payroll_controller.rb
A    app/controllers/payroll_history_controller.rb
U    app/views/layouts/application.html.erb
U    app/views/payroll/run_final.html.erb
A    app/views/payroll_history
A    app/views/payroll_history/index.html.erb
A    app/views/payroll_history/get.html.erb
A    app/reports
A    app/reports/pay_stub_aggregator.rb
A    app/reports/pay_stub_controller.rb
U    config/environment.rb
U    db/schema.rb
U    lib/payroll_engine.rb

Second edit

So I was working in a branch (copied at revision 147) and had made some updates to HEAD that I wanted to sync up.

From my working copy of the branch:

$ svn merge --dry-run -r 147:149 http://home.ertw.com/svn/repos/payroll/trunk .
U    test/lib/cpp_ei_test.rb
U    lib/payroll_engine.rb

The –dry-run tells me which files will be updated. That was good, so I re-ran without the –dry-run.

a