PolarLava::Blog

The saga continues…

Over the past several years I've maintained a Perl module to scrape data from The Weather Channel and then later NOAA's National Weather Service.  These adventures were chronicled in The PolarLava Weather Project entry.  Today begins a departure into a new and likely more sane direction…

Last week Rob pointed out to me pxWeather, a PHP class that processes data from the NWS' XML Feed.  I was previously aware of this data feed, but the last time I looked into it, it was fairly limited.  The actual data provided was pretty minimal and the available locations were very small.  All that has appeared to have changed as the data provided is now pretty extensive and the available locations also have been greatly expanded.  For quite some time I've wanted an accurate, reliable and free XML weather feed.  And I believe I now have that.

The pxWeather class has some nice features, but when I started looking into and using it, I found some issue.  For starters there were some pretty basic coding errors in the setOption() method.  And the XML data processing was being handled by a third-party class which at one time probably was a good solution.  But with the much improved XML handling in PHP5, it seemed a bit much to me.  So what else to do than to build a better wheel?

It's certainly not feature complete, but I've now written the core portion of a weather class that is now driving my weather page.  The weather for my CycleLog is currently still using the Perl weather module, but at some point this will also be migrated to the new PHP weather class…unless I decide to reimplement it in Perl also.  That might be a possibility.

So in addition to reworking the cycling weather, the next item for the actual PHP class will probably be a caching function.  This isn't something I really need as all  of my web site uses a GZip compression data cache.  Most of my pages aren't cached, but the weather page is one of them.  This is a feature that others likely will find of use, thus I will be adding it.  One final note…this class is written for PHP5 and will not work under PHP4.  I'm sorry if you're interested but still running PHP4.  Face the reality, it's time to upgrade.  There are a number of advantages to running PHP5 and one of the most significant is XML handling.  So if you're still interested and would like to try my weather class, drop me note at kevinp AT polarlava.com.

—— 7/7/06

OK, I've completed the initial implementation of on this class.  It now supports using local images which includes downloading images the first time they are referenced.  So it will self-build a library of required images.  Additionally I've added caching of the XML data with a selectable timeout (default 1 hour).  As mentioned previously, not something I will be using but other might need it.

I've got class documentation (thanks Rob) now and will be working on putting together a page for that and making it downloadable at some point.

—— 9/3/06

I've now added functionality to retrieve observation station data.  This allows you to grab all weather stations by state/territory and then display the weather for that location on the fly.  Unlike the previous Zip Code based system, the XML feed requires that you know the four character observation station code.  Those are all listed in a giant XML file and that's what the new station fetching feature handles for you.  This new feature should be showing up on the Weather page very soon.

An afternoon of hacking at it and the CycleLog now has a quick Weather Check page.  This little project simply looks up the weather in the database for a given date —generally today or something in the past couple of weeks.  Anything older than that won't exist as it gets purged except for weather data associated with a ride.  Way too much data to keep around otherwise.  To access a particular date you pass it as a "YYYY-MM-DD", "d" or "date" GET parameter, so for example "?d=2006-06-11" (2006-06-11).

Sometimes on days I take longer rides and the weather is variable, I will look up the most appropriate (prevalent) conditions during the ride.  This differs from my standard practice of using the closest hourly weather to when I finish the ride.  And now you can forget about this because I'd guess I'm the only one who really cares.  :)

So what does the future hold for PolarBlog?  I recently was reviewing the PolarBlog Users page and checking to see who was still actually using it.  As suspected, there were a couple that have gone off-line and several that have switch to some of the larger blogging softwares.  At the time I found it rather disheartening, but in hindsight, I guess it's the reality of the situation.  PolarBlog certainly isn't for everyone and primarily was done strictly for my own edification.  I only released it publically because I knew the PWL was a dead project and this was a path forward for those who were also running it.

As for those changing to the big blog software, many left some kind comments on their blogs as they moved away from PolarBlog.  Examples would be this, this and this (down at the moment).  The simple fact of the matter is PolarBlog is a one man operation with the occasional input from others.  There is no possible way I can ever compete with the large blogging projects —some of which are built by professional software companies.  They have far greater resources than I can ever muster and will always have a much larger feature set.  My developement cycles are a lot longer and slower, but I'm trying very hard to meet the needs of both myself and those that are still using PolarBlog.  One thing I am proud of it that I have never had any reports of security issues with any of my projects.  There have been bugs, there are bugs and there will always be bugs.  It's the nature of the beast.  But to my knowledge there have been NO critical vulnerabilities in my software.

So what does the future hold?  Well…I'm still working from the PolarBlog: The Road Ahead document.  As previously noted, progress is slow these days.  I'm about 90% complete with the RBL comment integration —a feature that I don't believe any other blogging package offers, and I'm hopeful to finish that up over the next couple of months.  I'm doing what I can when I can.  I'd like to do more, but I simply can't.  I have a real job, I have a family, I have other projects, and I also have other interests.  I can only do so much.  So if there is some feature you really seek in PolarBlog, you need to either tell me or consider moving on.  I'm not quiting, but occasionally I get "when are you going too…" emails from users.  I generally try to address these requests, but my ability to respond has diminished in the past year.  Sorry.  And to those who have stuck around and intend to stick around, thank you for your support and kind words.

After much annoyance over it, I've take the hour or so required to add the bike description to the default ride information display.  So a quick jaunt over to the latest ride page should illustrate the point.  Eventually there will be a bike page and then a bike maintenance record.  But those aren't likely to happen anytime soon.

Also while adding this feature I found and fixed a bug in the search page.  If you enter a complete "Start Date" (month/day/year) and there is only one ride for that day (as is the case for most days), it should take you directly to that ride.  But there was a bug in the search page logic which would then result in an unbounded search.  This would result in the multiple results found page with all rides in the database listed as matches.  Oops!  This was due to the date being assigned (single equals) instead of evaluated (double equals) in a conditional check.  But it's all better now.  Really…you can try it here.

And this morning's ride exposed another problem…when you enter zero hours for your sleep time, it wouldn't show because heck, who would ride after being up all night?  Me apparently!  This is now fixed as you can see.  :)

Progress on the RBL module for PolarBlog can't come soon enough…

Happily until a few months ago I saw very little blog spamming here.  In the past couple of months it began to pickup with the last couple of weeks being particularly brutal.  This culminated in getting hit with 160+ messages in one blast last week.  I found a couple of patterns in who was hitting me.  Most notably if the RBL was done it would be blocking nearly all of it.  In the meantime I've hacked a quick fix in that blocks some of the common keywords I've been seeing.  If you're a PolarBlog user and want and/or need the fix, you can email me and I'll send it to you.  I'm sorry if you are enduring the new found love that these assholes have for PolarBlog.  But the reality of the situation is that I'm basically a one man operation who's real job has run his life over.  Thus I've spent very little time working on any of my personal software projects.  I just wish these assholes would realize that although inconvenient, it only takes me a minute to delete their spam.  Generally it's only going to be on my site for a matter of minutes.