One common thing in systems administration is “fiddling with knobs”. You know, some value that has to be changed, that affects results. One approach is to randomly try values, the other is to use a systematic approach.
I use the excellent picprog software to burn PIC microcontrollers on occasion. One day, after not using it for a while, I was dismayed to find that with one piece of hardware, it didn’t work. After lots of testing, I tracked it to picprog. The author suggested tweaking some delays in the code.
I used the value the author suggested, which caused it to work once in a while, but not reliably. What I wanted to do was find a value that worked consistently, or at least 90% of the time.
There were four instances of the following code
I wanted to try different values and see if they work.
There are two approaches to testing the values. One is to find a lower limit and an upper limit, and do a binary search. The lower limit is easy, I know 1000 doesn’t work. Since I didn’t know what the upper value was, I elected to start at 1000 and step up by 500.
So, I changed all occurences of the static value to
#define TEST_DELAY 1000
at the top.
The next step was to write a script that changed this value, recompiled, and ran 50 tests.
# Change the define
perl -p -i -e "s/^#define TEST_DELAY.*/#define TEST_DELAY $val/" picport.cc
make > /dev/null 2>&1
# Run the test 50 times
for i in `seq 1 50`; do
./picprog --erase --burn -i rs232-add.hex > /dev/null 2>&1
if [ $? -eq 0 ]; then # return code of 0 is good
# store in flat file
echo $val:$good:$bad >> resultfile
The code there should be self explanatory, I’m just running the same thing over 50 times after doing an in place substitution in perl (perl -p -i -e ‘s/foo/bar/’ file)
From the command line, I can then run the test over different values
for i in `seq 1000 500 10000`; do ./runtest $i; done
The seq command is a handy one to have in your toolkit.
seq start last
will count from start to last. Wrap that in a for loop, and you can run the same test a certain number of times (like I do above).
seq start increment last
will count from start to last in increments of increment.
So, in the end, I get
Intrerestingly enough, the results aren’t completely linear. If I set it to 6500, I should be good though.
Most importantly, I have a repeatable set of tests. If a newer version of picprog comes out, I can retest the delays to see if I have to change.