One common thing in systems administration is “fiddling with knobs”. You know, some value that has to be changed, that affects results. One approach is to randomly try values, the other is to use a systematic approach.

I use the excellent picprog software to burn PIC microcontrollers on occasion. One day, after not using it for a while, I was dismayed to find that with one piece of hardware, it didn’t work. After lots of testing, I tracked it to picprog. The author suggested tweaking some delays in the code.

I used the value the author suggested, which caused it to work once in a while, but not reliably. What I wanted to do was find a value that worked consistently, or at least 90% of the time.

There were four instances of the following code

  delay (1000);

I wanted to try different values and see if they work.

There are two approaches to testing the values. One is to find a lower limit and an upper limit, and do a binary search. The lower limit is easy, I know 1000 doesn’t work. Since I didn’t know what the upper value was, I elected to start at 1000 and step up by 500.

So, I changed all occurences of the static value to

    delay (TEST_DELAY);

and added

#define TEST_DELAY 1000

at the top.

The next step was to write a script that changed this value, recompiled, and ran 50 tests.

val=$1

good=0

bad=0

# Change the define

perl -p -i -e "s/^#define TEST_DELAY.*/#define TEST_DELAY $val/" picport.cc

# Rebuild

make > /dev/null 2>&1

# Run the test 50 times

for i in `seq 1 50`; do

        ./picprog --erase --burn -i rs232-add.hex > /dev/null 2>&1

        if [ $? -eq 0 ]; then # return code of 0 is good

                let good=good+1;

        else

                let bad=bad+1;

        fi

done

# store in flat file

echo $val:$good:$bad >> resultfile

The code there should be self explanatory, I’m just running the same thing over 50 times after doing an in place substitution in perl (perl -p -i -e ‘s/foo/bar/’ file)

From the command line, I can then run the test over different values

for i in `seq 1000 500 10000`; do ./runtest $i; done

The seq command is a handy one to have in your toolkit.

seq start last

will count from start to last. Wrap that in a for loop, and you can run the same test a certain number of times (like I do above).

seq start increment last

will count from start to last in increments of increment.

So, in the end, I get

1000:0:50
1500:0:50
2000:0:50
2500:22:28
3000:16:34
3500:29:21
4000:32:18
4500:41:9
5000:37:13
5500:48:2
6000:46:4
6500:49:1
7000:49:1
7500:46:4
8000:46:4
8500:50:0
9000:50:0
9500:50:0
10000:50:0

Intrerestingly enough, the results aren’t completely linear. If I set it to 6500, I should be good though.

Most importantly, I have a repeatable set of tests. If a newer version of picprog comes out, I can retest the delays to see if I have to change.