Sean’s Obsessions

Sean Walberg’s blog

Automatic Uploading of Subid Stats for Tracking 202

Tracking202 is a great piece of web tracking software. The thing that annoys me though is that I have to upload subid stats (“Optional Info”) to load in the conversions, unless I use the postback or tracking pixel method (which I don’t always have the option to do, and I’m not a high volume guy anyway).

I wrote a quick script that uses the DT web services and updates the stats automagically. It Works For Me (TM).

Download

Usage:

  1. Put it in some directory, preferably outside your web root.
  2. Edit the script to point to your 202-config.php file, and the DT info for the companies you wish to use
  3. Call it from cron periodically (php /path/to/updatestats.php). It asks for the last 7 days of stats by default, but you can safely run this script more frequently because it will not double-count conversions.

You can also run this with the “all” paramater, ie “php updatestats.php all” and it will download and update the last 10 years of stats

How Many Mongrels or FastCGI Processes Do I Need?

I’m sitting in a course on call centre design, and yesterday there was mention of Erlangs, which are a unit of measure of the volume of traffic on a telecommunications network. These Erlangs can be used to calculate how many agents a call centre needs (or, given the number of agents, what service level can be expected)

The Erlang C model is used for this, it has the built in assumption that if agents are busy then the callers queue. This is much like a Mongrel or FastCGI “agent”, with requests coming in from a front end web server.

Using the Erlang C calculator here, I can figure out how many agents I need, given the desired service level.

  • Arrivals: 600/minute (10 req/sec)
  • Service time: 0.2s
  • Service level: 99.9% within 0.3 s

Gives 5 agents or Mongrels. From there you can work backward with existing models to figure out how many computers you need.

With the calculator you can also fix the number of agents and figure out capacity, or blockage, or whatever.

This assumes that incoming requests follow the Poisson distribution, which I have no idea if it is true. Anyone?
Edit: Sorry, I managed to lose half the post and had to redo it.

Using Rails Models From Within an External Script

For some one off scripts I wanted to make use of a couple of models I created. I had no idea how simple it would be to load the rails environment from within a script:

1
2
require 'config/environment.rb'
p MyModel.find(:first)

Sweet! Rails is so… thoughtful.

Getting Sick of Elections

(Disclaimer: I’m a conservative)

I’m getting sick of the “[insert conservative thinking party] gave tax cuts to big evil corporations and is therefore not your friend”. I think it oversimplifies the issue.

Ignore the fact that it’s a tax cut, because that’s irrelevant. It could be a suitcase of money, it’s just that tax cuts are an easier way to give money from the government to another party. No cheques need to be written.

While I’m sure there are exceptions, the idea behind giving money to a company is to entice them to do something. To open an office where it’s otherwise not economically feasible. To not close down a particular site. To get them to work in some area that furthers a government’s policy.

So from that perspective, “XXX gave money to this big faceless corporation” is only telling half the story. It’s no different than saying “I took a car from the dealership” without saying “and paid for it”.

Secondly, it can be viewed as an investment. Those jobs generate payroll taxes, income taxes, and sales taxes. It gets people off of social assistance. Even if you subscribe to the “evil faceless corporations will evade income taxes” theory, there is always some trickledown.

Sure, some more transparency would be good. Maybe a deal was made that we’ll never hear about. Maybe the deal is documented in some place, but we only hear about the tax cut side of the equation.

I don’t normally get worked up about politics. What gets me going is the use of numbers to tell a story that’s just untrue.

Testing via Screen Scraping

Trying to figure out how to write a test based on screen scraping. Easiest example is to make sure that certain URIs redirect to the login screen.

Created an “automated” dir under app/tests/cases/, with a file called permissions.test.php:


class PermissionsTestCase extends CakeWebTestCase {

var $mysite = "http://test.mysite.com";
function setUp() {
}

function tearDown() {
}

function testMe() {

$this->setMaximumRedirects(0);
$result = $this->get($this->mysite . "/");
$this->assertResponse(302);
$this->assertHeader("Location", $this->mysite . "/users/login", "Homepage redirects to login");
}
}
?>


The “setMaximumRedirects” is there because WebTestCase will, by default, follow 3 redirects. Here I tell it not to, then I get a page and check the status code and the header.

Lots of stuff you can test specific to a web page: http://simpletest.org/api/SimpleTest/WebTester/WebTestCase.html

How the Hell Do You Test a Controller?

Testing models is straightforward, see http://bakery.cakephp.org/articles/view/testing-models-with-cakephp-1-2-test-suite

Testing a controller though… Why is there nothing good out there that tells you how to test a controller, other than references to Felix’s work that doesn’t use simpletest?

That said, testing a controller should look something like

- Create controller object
- Call an action
- Poke at the controller to make sure it looks ok.

To test, I baked a controller and put in one action:


class PrintersController extends AppController {
var $name = 'Printers';
function foo() {
$this->set("something", "some value");
return 1;
}
}
?>

I then modified the baked test case (tests/cases/controllers/printers_controller.test.php)


App::import('Controller', 'Printers');

class PrintersControllerTestCase extends CakeTestCase {
var $TestObject = null;

function setUp() {
$this->TestObject = new PrintersController();
}

function tearDown() {
unset($this->TestObject);
}

function testMe() {
$result = $this->TestObject->foo();
debug($this->TestObject);
}
}
?>

After running the test, I could see the methods and variables in the controller. The stuff I could see testing in the controller is mostly the vars that get passed to the view, so after consulting the debugs I changed testMe() to

        function testMe() {
$result = $this->TestObject->foo();
$vars = $this->TestObject->viewVars;
$this->assertEqual($vars["something"],
"some value");
debug($this->TestObject);
}

$vars is an array of the stuff that’s going to be sent to the view. Logically, if my model tests are correct and I have fixtures set up, the stuff that gets generated by the controller should be predictable and therefore testable.

I also noticed some other entries in the debug output, such as pagetitle. I’m sure there’s more, but this is a good start for now. Wh

I’ll Be Speaking at Sharkfest ‘08

I’m giving 2 talks on using Wireshark to expose VoIP problems at Sharkfest ‘08 (schedule).Details are sketchy, I think one of the talks is more of a hands on lab, the other is me talking. I’ve expanded on my techniques from the Linux Journal article I wrote on the topic.

Some other fascinating topics going on at the same conference. Hope to see you there.

Frameworks Don’t Suck

I’ve often had a dim view of web frameworks. At b5media we had more than one case where we’d send out some pretty basic development to be done, and we’d get it back, done in a framework that took more time to get running than it was worth. And, from a performance perspective, all that overhead has got to slow things down.

Now that I have a tiny bit more time on my hands I’ve started to try and write some code that I’ve been meaning to. As I sat down to it I started thinking of all the overhead I’d have to write – some authentication, some mapping of URLs to functions, database stuff, and then it hit me – this is what a framework does.

Given I had no deadline to meet, I decided to try out a framework. I’m not a Python guy, so Django was out. Rails sounded sexy, but the last thing I needed was to learn a new language, especially one that looks like it was written by someone on crack and that looked more like line noise than Perl.

I’d rather have picked on in Perl, but knowing that I’d probably be putting this on a shared host at some point, PHP was probably the best bet.

Symphony caused me no end of grief at b5, so it was out. I ready a bit about Zend but decided it wasn’t for me. Code Igniter seemed like too much of a moving target, which left CakePHP. Looking further into Cake it seemed like it was well laid out, had a large community with active development, and does a lot of things Rails does. What sold it for me was that it was one of the few frameworks that supported PHP4 which is on many shared hosts still.

Over the course of an evening I went through the first 3 CakePHP tutorials at IBM. Then I picked one of my projects and started slugging through it.

After a couple of weeks I learned the following:

MVC is really good for web apps. It took me a while to put the SQL generation in the hands of the framework, but most of my queries are simple anyway. Besides, I can always do $this->Model->Query(“SELECT blah..”).
Using a framework gets you up and running fast. Most of my programming is trying something, seeing how it looks, then trying something else. The framework lets me get to that point quickly, especially with scaffolding if I still haven’t finished a particular model/table.
It’s not slow at all. I haven’t got around to profiling it, but the response time is quick, and it’s not taking up much space in my opcode cache, no more than Smarty would have
Programming’s a lot more fun when you let the framework do all the crap work

Skinny Controller, Fat Model

I understand how to work within the MVC (model, view, controller) system of development, but once I read skinny controller, fat model I realized what it’s all about. Even though it’s written for Ruby on Rails it’s easy enough to transport to your framework of choice.

I’ve heard the term “fat model” before, but really had no idea how to fatten up my model. Now I know.