After being reminded that Debugging sucks, Testing rocks I thought I’d go back to some PHP stuff I was working on and develop some unit tests. I looked around at a few testing packages including PHPUnit2 as recommended by Jack Herrington in PHP Hacks, but the overhead and complexity of creating classes to encapsulate my test was too much. Plus, I wanted to be able to test on PHP 4 and 5. Finding examples was also difficult, everyone seemed to be showing off a class that simply adds two numbers which to me is fairly useless. I wanted a real world example of someone testing code as ugly as mine.
Trying to compare different methods led me to Power PHP Testing which is a good description of testing in PHP and the various methods, and also showed me about Apache-Test.
Apache-Test is a project by the Apache foundation that will create a local instance of a web server for hosting the tests, and also uses the “Test Anything Protocol” that I was familiar with from Perl. Each test spits out something like “OK” which makes collecting and generating a simple unit test easy. The documentation on Apache-Test was quite extensive, but still it was difficult to get started with a “Hello World!” type of test to make sure everything is running. To start off I installed Apache::Test from CPAN (perl -MCPAN -e ‘install Apache::Test’)
Assume the app is in /var/www/html, and we want to test get.php which gets a web page using cURL.
function get($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
?>
The basic tests I’d like to run on this are to make sure get.php compiles, to get a simple web page, and then to introduce a few error conditions and see what happens. (Note the latter ones will probably fail because there is no error checking, defining the tests beforehand is an advantage then to ensure I don’t forget something)
To implement the testing environment:
mkdir -p t/response/TestGet
cat > Makefile.pl
# don't worry about this file - you should not
# need to edit it at all.
use 5.00501;
use ExtUtils::MakeMaker;
use lib qw(lib);
use Apache::TestMM qw(test clean);
use Apache::TestRunPHP ();
Apache::TestMM::filter_args();
Apache::TestRunPHP->generate_script();
WriteMakefile(
NAME => 'perl-php',
VERSION => 'test',
);
^D
perl Makefile.PL
Each series of tests is a directory in the t/response directory with a name starting with Test. I’ve called the tests relating to get.php “Get”, so all my tests for get.php go in t/response/TestGet.
Before getting into the tests, you have to understand the environment you’ll be testing in. The test script will be running on a web server on a non standard port (8529 by default) with a document root of t/htdocs. Each testing directory (ie TestGet) has an Alias directive so that a request to /TestGet/foo will be mapped to t/response/TestGet/foo.
So the first test to make sure get.php loads fine is
require "test-more.php";
require "../../../get.php";
plan(1);
ok(1);
?>
Note that I needed to require get.php out of a parent directory. There are ways to modify the config files to change the php include_path but this is easy enough now.
The test-more.php file comes with Apache-Test and implements some testing functions that help. I started off with plan(1) which tells the testing harness that 1 tests are expected. The ok() function is a test, if whatever is inside the parenthesis is true the tests pass.
[sean@bob html]$ make test
/usr/bin/perl -Iblib/arch -Iblib/lib \
t/TEST -clean
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl /var/www/html/t/TEST -clean
APACHE_TEST_GROUP= APACHE_TEST_HTTPD= APACHE_TEST_PORT= APACHE_TEST_USER= APACHE_TEST_APXS= \
/usr/bin/perl -Iblib/arch -Iblib/lib \
t/TEST -bugreport -verbose=0
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl /var/www/html/t/TEST -bugreport -verbose=0
/usr/sbin/httpd -d /var/www/html/t -f /var/www/html/t/conf/httpd.conf -D APACHE2 -D PERL_USEITHREADS
using Apache/2.2.3 (prefork MPM)
waiting 60 seconds for server to start: ..
waiting 60 seconds for server to start: ok (waited 1 secs)
server localhost:8529 started
t/get/0_test....ok
All tests successful.
Files=1, Tests=1, 2 wallclock secs ( 1.78 cusr + 0.32 csys = 2.10 CPU)
[warning] server localhost:8529 shutdown
I’ll create another test to get two pages, one that exists and one that doesn’t. Apache-Test places an index.html in the docroot for testing which I’ll use.
require "test-more.php";
require "../../../get.php";
plan(2);
$result = get("http://localhost:8529/index.html");
like($result, "/welcome to/i");
$result = get("http://localhost:8529/idontexist.html");
is($result, NULL);
?>
Here I used two more testing functions. like() accepts a regular expression that gets passed to preg_match. If the regexp matches the string in the first parameter, the test passes. is() compares two values. Here I’d expect that if the page wasn’t found I’d get a NULL.
Then I run it again
[sean@bob html]$ make test
/usr/bin/perl -Iblib/arch -Iblib/lib \
t/TEST -clean
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl /var/www/html/t/TEST -clean
APACHE_TEST_GROUP= APACHE_TEST_HTTPD= APACHE_TEST_PORT= APACHE_TEST_USER= APACHE_TEST_APXS= \
/usr/bin/perl -Iblib/arch -Iblib/lib \
t/TEST -bugreport -verbose=0
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl /var/www/html/t/TEST -bugreport -verbose=0
/usr/sbin/httpd -d /var/www/html/t -f /var/www/html/t/conf/httpd.conf -D APACHE2 -D PERL_USEITHREADS
using Apache/2.2.3 (prefork MPM)
waiting 60 seconds for server to start: ...
waiting 60 seconds for server to start: ok (waited 2 secs)
server localhost:8529 started
t/get/0_test....ok
t/get/1_get.....FAILED test 2
Failed 1/2 tests, 50.00% okay
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
t/get/1_get.t 2 1 2
Failed 1/2 test scripts. 1/3 subtests failed.
Files=2, Tests=3, 5 wallclock secs ( 3.75 cusr + 0.64 csys = 4.39 CPU)
Failed 1/2 test programs. 1/3 subtests failed.
[warning] server localhost:8529 shutdown
[ error] error running tests (please examine t/logs/error_log)
make: *** [run_tests] Error 1
It looks like the second test failed… I
[sean@bob html]$ make test
/usr/bin/perl -Iblib/arch -Iblib/lib \
t/TEST -clean
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl /var/www/html/t/TEST -clean
APACHE_TEST_GROUP= APACHE_TEST_HTTPD= APACHE_TEST_PORT= APACHE_TEST_USER= APACHE_TEST_APXS= \
/usr/bin/perl -Iblib/arch -Iblib/lib \
t/TEST -bugreport -verbose=0
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl /var/www/html/t/TEST -bugreport -verbose=0
/usr/sbin/httpd -d /var/www/html/t -f /var/www/html/t/conf/httpd.conf -D APACHE2 -D PERL_USEITHREADS
using Apache/2.2.3 (prefork MPM)
waiting 60 seconds for server to start: …
waiting 60 seconds for server to start: ok (waited 2 secs)
server localhost:8529 started
t/get/0_test….ok
t/get/1_get…..FAILED test 2
Failed 1/2 tests, 50.00% okay
Failed Test Stat Wstat Total Fail List of Failed
——————————————————————————-
t/get/1_get.t 2 1 2
Failed 1/2 test scripts. 1/3 subtests failed.
Files=2, Tests=3, 5 wallclock secs ( 3.75 cusr + 0.64 csys = 4.39 CPU)
Failed 1/2 test programs. 1/3 subtests failed.
[warning] server localhost:8529 shutdown
[ error] error running tests (please examine t/logs/error_log)
make: *** [run_tests] Error 1
The final test of a missing page failed. I can enable more debug output with TEST_VERBOSE=1:
make test
...
t/get/1_get.....1..2
ok 1
not ok 2
# Failed test (t/response/TestGet/1_get.php at line 12)
# got: '
404 Not Found
Not Found
The requested URL /idontexist.html was not found on this server.
'
# expected: ''
# Looks like you failed 1 tests of 2.
FAILED test 2
...
So, the script returns the contents of the 404 page rather than the NULL. After fixing that up my tests pass.
