As part of my work with b5 I’m wrangling with 10 servers now. When I started out it was 3, so keeping configurations in sync was pretty easy, a bit of shell scripting, a bit of rsync. But now we’ve got server roles, we’ve got config files that are slightly different per server, plus the constant need to roll out minor changes to files across different nodes.

I’ve had my eyes on cfengine for a little while. It’s a system that runs agents on all your servers, you define a policy file that ensures files are at the right version, packages are correct, and can run scripts and other such magic. I finally got around to getting it running on 3 servers to develop the policy, and finish the rest off soon. We used to have a full page of things to do on new servers, it’s now down to about half a page, and most of that has to do with editing internal scripts to recognize the new servers.

The power of cfengine is that you can assign roles to servers and have different actions based on roles. For example, I can define my nis clients and servers like this:

1
2
3

classes:
        nisservers = ( FileExists(/var/yp/nisdomain) )
        nisclients = ( any -nisservers )

And then make sure my nis clients have the right configuration:

1
2
3
4
5
6

editfiles:
       nisclients::
        { /etc/yp.conf
        AppendIfNoSuchLine "ypserver server1"
        AppendIfNoSuchLine "ypserver server2"
        }

You can even get really fancy and look for particular lines, such as changing mount options in fstab:

1
2
3
4
5
6
7
8
9
10
11
12

       nfsclients::
        { /etc/fstab
        Backup "true"
        BeginGroupIfNoLineMatching "^nfsserver:/home .*"
                Append "nfsserver:/home /home nfs rw,nosuid,soft,rsize=8192,wsize=8192,noatime,acregmin=10 0 0"
                DefineClasses mounthome
        EndGroup
        LocateLineMatching "^nfsserver:/home .*"
        # These lines had better be identical!!! probably move to a var
        ReplaceLineWith "nfsserver:/home /home nfs rw,nosuid,soft,rsize=8192,wsize=8192,noatime,acregmin=10 0 0"
        DefineClasses remounthome
        }

If either one of those succeed, they define an additional class that lets me run a shell command:

1
2
3
4
5
6
7
8

control:
  # needs to be predefined
  AddInstallable = ( mounthome remounthome )
shellcommands:
        mounthome::
                "/bin/mount /home "
        remounthome::
                "/bin/mount /home -o remount"

If cfengine has to change fstab, then home will be mounted/remounted automatically.

It’s a bit difficult to provide a full tutorial, but I’ve managed to collect some links in del.icio.us. I’d suggest reading the quick start, and getting one or two servers simply talking over the cfengine protocol. Then try to do some file copies to get the hang of that, and move on to editfiles. Once you have an idea of what you need to do, ie managing packages, managing sudo access, do google searches for those (I have some in my links too), and implement those. By the time you get that far, you should be able to use the reference guide to figure out the specific commands to do what you want.