Weblogs.com refresh script

Background: Weblogs.com tracks updates to people's weblogs and publishes a list of which have been updated and when. Recently this service switched from a spider-based system to a push-based system. Instead of the weblogs.com server checking each site to see whether it has been updated, it is now the responsibility of the site to let weblogs.com know.

This is great and all, except that it requires additional effort on the part of the person running the weblog. They may or may not be interested in making the effort. Most likely, they just don't care about supporting this service, which they may never have even heard of. So they never let weblogs.com know they updated, and weblogs.com is less useful because of it.

Solution: Create a script that checks a web page and lets weblogs.com know if it changes. Then schedule it as a cron job to run however often you wish.

Caveats: This script only works if the HTTP server returns a valid Last-modified: field. If it's a generated page, it probably doesn't. You can work around this by retrieving the whole page and doing an MD5 on it, but that's rather prone to abuse, since it gets the content and not just the header. In my usage, I didn't need it, so it's not in there. Also, it's not like this is a masterpiece or anything. It's really some simple perl code. But I figured there might be other people interested in doing the same thing I was. Oh, and don't screw up the cron job. Nothing other than a 0 in that first field. More then once an hour will have no effect, an it will surely make someone mad.

#!/usr/bin/perl -w

use LWP::Simple;

$sitename = "Weblog Name";
$filename = "/tmp/filename";
$url = "the url";

$sitename =~ s/ /+/;

$prev_mod = 0;
if (open FILE, $filename)
{
    $prev_mod = <FILE>;
    close FILE;
}

$modified_time =  (head($url))[2];

if ($modified_time != $prev_mod)
{
    print "$sitename\n$url\n\n";
    print "Site has been updated\n";
    print scalar(localtime(time))."\n";

    open FILE, ">$filename" or die "couldn't open file: $!";
    print FILE $modified_time;
    close FILE;

    $ping = "http://newhome.weblogs.com/pingSiteForm?" .
            "name=$sitename" .
            "&url=$url";
    get($ping);

}
 

home