LINUX NEWS
RESOURCES & LINKS FROM BRAINBUZZ.COM
Thursday, August 2, 2001
Read By Over 6,000 Linux Enthusiasts Weekly!TABLE OF CONTENTS
1) Sean’s Notes
2) Linux News
Mandrake Pulls Off IPO
City of Largo Goes Live With KDE 2.1.1
Sparks Fly at Open Source Debate
KOffice Release Candidate3) Linux Resources
Programming, Graphics, and OS Tips
Sendmail
SpamMaster
Diamond Rio and Linux
The Tao of Programming4) App o’ the week
~~~~~~~~~~~~~~~~~~~~~~ ADVERTISEMENT ~~~~~~~~~~~~~~~~~~~~~~~
What can ASP training do for you? Make sure you have the knowledge you need when you need it. Train Online with FirstClass Systems and have the knowledge you need at your fingertips. Online Training is the most intelligent way to learn anything!
http://ad.brainbuzz.com/?RC06&AI345
For information on how to advertise in this newsletter
please contact mailto:adsales@BrainBuzz.com or visit
http://cramsession.brainbuzz.com/marketing/default.asp
-------------------------
1) Sean's Notes
-------------------------
One of the more common questions on the Perl board is how
to get a script to fetch a web page. Unsurprisingly, it's
a cinch to do in Perl, as long as you have the right module.
LWP is a "Library for WWW access in Perl". And who said
Unix jocks couldn't come up with good names? Actually, it's
a pretty bad name, because you can use the same set of
modules to access HTTPS, FTP, NNTP, and a few other
protocols. A fine example of one of the three virtues of a
programmer--laziness:
http://www.netropolis.org/hash/perl/virtue.html
If your system doesn't come with LWP installed, hop onto
CPAN and install it.
# perl -MCPAN -e shell
cpan> install Bundle::LWP
...
cpan>
With that, let's start our program.
---- cut here ----
#!/usr/bin/perl -w
use LWP::UserAgent;
# One user agent required per program
my $ua = LWP::UserAgent->new;
# The request object says what we want
my $boards = HTTP::Request->new(GET => "http://boards.brainbuzz.com");
# Pass the request object to the user agent to get the page
my $data = $ua->request($boards);
# $data is an HTTP::Response object, data is in content()
print $data->content();
---- cut here ----
The first line of a script always starts with "#!"
(pronounced "hash bang", or sometimes "she-bang"), which lets
the system know that the rest of the line will specify the
interpreter to run it. This time, it's /usr/bin/perl -w, the
perl interpreter.
Those who have seen me write Perl code in previous newsletters
will notice I've started using the -w flag. This turns on
warnings, which makes the interpreter point out potential
errors. For newbies and experts alike, it is very helpful for
finding bugs that lurk in your code. Next, I bring in the
LWP::UserAgent module, which gives me access to all the LWP
functions.
In LWP, a user agent is used, almost akin to the web browser
itself. If I needed to set things like proxies, I'd do it to
$ua, and the effects will trickle down to all my requests.
The new method of LWP::UserAgent just invokes an object.
Next, I create a request object, of type HTTP::Request. As
before, I invoke the new method, but this time I have to tell
it the url and the request method. Here, I'm GETting the url
for the Brainbuzz boards. Note the URL is fully qualified --
http and all. This is because behind the scenes, the system
has to figure out if it should use HTTP, FTP, or any other
method supported.
Finally, I tell the user agent to send off the request I just
made. Since it's returning an object, I use the content
method to get the data as a scalar (string).
A small sidebar:
Try leaving off the whole content part, and just printing
$data directly:
$ perl bb1.pl
HTTP::Response=HASH(0x8350c74)
That's perl telling you that you've got a data type of
HTTP::Response, and that it really doesn't know how to print
it, other than the memory location. Do a "man HTTP::Response"
to find out what your object does, and you'll find you need
the content method to get at the data.
All of this would be useless unless I were to do something.
While I've got http://boards.brainbuzz.com loaded, why don't
I print out a list of the top posters, along with their posts?
---- cut here ----
#!/usr/bin/perl -w
use LWP::UserAgent;
# One user agent required per program
my $ua = LWP::UserAgent->new;
# The request object says what we want
my $boards = HTTP::Request->new(GET => "http://boards.brainbuzz.com");
# Pass the request object to the user agent to get the page
my $data = $ua->request($boards);
# $data is an HTTP::Response object, data is in content()
my ($posters) = grep /Top Posters/, (split /\n/, $data->content());
my @table = split /<\/tr>/, $posters;
for (@table) {
printf "%20s %d\n", $1, $2 if (/vbd.asp.\*?>(\w+).\*\((\d+)/);
}
---- cut here ----
It's all the same up to the request... Rather than a lot of
temporary variables, I've taken advantage of some of Perl's
features. Start at the right side of the fifth to last line.
In the parenthesis, I'm using the split() function, which
splits a string into substrings based on a given delimiter.
In this case, I'm breaking the result of the request into
lines (\n means "newline"). The result of that is grep()'ed
(searched) for the string "Top Posters", and returned in the
$posters variable. $posters is in parenthesis because grep
is supposed to return a list, but since I only want the first
occurrence, I can use this trick. So, this line of code
returns the line containing the Top Posters phrase.
Next, I do another split, this time on the </tr> tag. Note
that I had to escape the / with a \, otherwise it would be
construed as the end of the regular expression. Now I've
got a list of strings in the @table array, each potentially
containing a poster. But there is some other junk in there
too!
The last three lines set up a loop on every element of @table.
Since I didn't specify a variable to hold the current value,
it gets stored in a special variable, $_. (Yes, that's
"dollar underscore"). The significance of that is that the
regular expression search in the next line will search $_
unless it's told otherwise. Ain't Perl great?
The regexp itself (the stuff between the /'s) can be broken
down for clarity:
/ # begin regexp
vbd.asp # look for the string "vbd.asp"
.\*?> # followed by anything (.\*) ending in a >
# adding ? to the end means to stop at the first match
(\w+) # Next will be at least one alphanumeric character
# Parenthesis mean to save it into a positional variable
# since this is the first in the regexp it is $1
.\* # skip ahead... match anything
\( # find a literal left parenthesis
(\d+) # followed by a number... save this in $2
/ # close regexp
If that is found, then print $1 and $2 (the saved username
and posts respectively) with some formatting. Finding out
what you're supposed to match on is usually an exercise in
paging through the HTML source to a web page, and slowly
building up the expression.
The output is:
$ perl bb1.pl
mrobinson52 125
Tezz 120
newd00d 119
robnhood 111
likeitontop 84
editormatt 76
mgraham44 74
fsec 67
bizzybot 61
cjaquess 49
So, as you can see, LWP is a pretty cool module that lets
you pull out the necessary data from web pages...
Poke around the man pages for LWP and LWP::UserAgent, and
find out what else they can do. Try writing a program to
calculate your stock portfolio by sucking back quotes from
the web!
The Perl board is there to discuss code like this... Post
your questions, or show us the cool things you can do with
the LWP module:
http://boards.brainbuzz.com/boards/vbt.asp?b)0
Long live the Penguin,
Sean
mailto:swalberg@brainbuzz.com
Visit the Linux News Board at
http://boards.brainbuzz.com/boards/vbt.asp?b2
-------------------------
2) Linux News
-------------------------
----------------------
Mandrake Pulls Off IPO
----------------------
Congrats to the Linux distribution based out of France for
pulling off their IPO. The stock is set to start trading
tomorrow. This news item also has relevant information for
those looking to add this stock to their portfolio, but read
the comments first, it looks like there was a typo in the
ticker symbol.
http://www.newsforge.com/article.pl?sid/07/30/2348248
--------------------------------------
City of Largo Goes Live With KDE 2.1.1
--------------------------------------
The City of Largo, Florida were looking for an upgrade to
their current system that is required to support 800 users
and 400 thin client devices. KDE to the rescue! Not only
does this announcement give some details of the system, it
has some items to look out for if you're considering such a
move.
http://eltoday.com/article.php3?ltsn 01-07-23-001-14-PS
--------------------------------
Sparks Fly at Open Source Debate
--------------------------------
What else did you think you'd get when you have Microsoft
execs speaking at an open source convention? The execs were
defending their position against Open Source (ooh! "Shared"
source!), but the audience wasn't buying it.
http://www.zdnet.com/zdnn/stories/news/0,4586,5094814,00.html?chkpt
=zdnnp1tp
01
-------------------------
KOffice Release Candidate
-------------------------
"KOffice is an integrated office suite for KDE which utilizes
open standards for component communication and component
embedding. The primary goals of this release are to provide
a preview of KOffice 1.1 and to involve users and developers
who wish to request/implement missing features or identify
problems. Code development is currently focused on
stabilizing KOffice 1.1, scheduled for final release in
mid-August, 2001."
http://www.koffice.org/announcements/announce-1.1-rc1.phtml
-------------------------
3) Linux Resources
-------------------------
----------------------------------
Programming, Graphics, and OS Tips
----------------------------------
This is one great site, containing links to various
tutorials on everything from C programming to shell
scripting, and a whole lot of tips and tricks about Linux
(and other operating systems).
http://www.osconfig.com/
--------
Sendmail
--------
This article from Server/Workstation Expert is a great
introduction to sendmail, what it is, and how it does what
it does. While you're on the site, go to the main page and
subscribe. It's free, and always has some interesting
columns. Out of all the trade rags that I receive, this is
the one I like the most.
http://swexpert.com/C2/SE.C2.JUL.01.pdf
----------
SpamMaster
----------
For those getting sick of the spam in their inbox who have
realized that conventional filters can't help, here is
spammaster. It's a custom filter that is designed to weed
out spam, and reduce the false negatives and positives
associated with other filters. Some of the ideas used
are quite interesting, not to mention effective.
http://www.lne.com/ericm/spammaster/
---------------------
Diamond Rio and Linux
---------------------
Last I checked, my Rio didn't come with Linux software, but
a gadget like that won't stay unsupported for long. The
developers managed to reverse engineer the communications
protocol, and then write the software. Great work, guys!
http://www.world.co.uk/sba/rio.htm
----------------------
The Tao of Programming
----------------------
On the lighter side of the resource section, I thought I'd
share a document that I've always found both informative,
and entertaining. The Tao of Programming explains how
programmers and management should interact and work.
http://epims.gsfc.nasa.gov/tao.html
-------------------------
4) App o' the week
-------------------------
Need some fax software for Unix that's network aware, and
has Windows, Unix, and Mac clients (don't forget the web and
email use either)? Hylafax is the answer. Do yourself a favor
and RTFM when you install this... It's not hard, but there
are several important steps that can be easily missed.
http://www.hylafax.org
-------------------------
(C) 2001 BrainBuzz.com. All Rights Reserved.
-------------------------
-------------------------
This message is from BrainBuzz.com.
You are currently subscribed to the
Hottest Linux News and Resources
as: sean@ertw.com
To un-subscribe from this newsletter by e-mail:
send a blank email message to:
mailto:leave-linuxnews-3825955Y@list.cramsession.com
-------------------------------------------------------
To Subscribe to this newsletter by e-mail:
send a blank email message to:
mailto:join-linuxnews@list.brainbuzz.com
-------------------------