How tsumea dodged the bullet that killed PALGN

PALGN, an Australian based gaming news / community website established in 2003 is apparently in the shitter. A lack of transparency, communication, and any real action from PALGN's new owner to fix the site's long standing technical problems has meant that PALGN's passionate team of volunteer editors have walked off, never to return. There will be no more articles, news updates, or reviews from them after Friday (today), and it's a damn shame because it shouldn't have ended this way.

While I've only chatted to the founder and previous owner of PALGN, James Gay, a small number of times on MSN Messenger or via email when we wanted to promote certain news items, we do share mutual admiration and respect for each other's work. I'd like to think sites like PALGN and tsumea came from the same breed, born and fostered under a particular era of the internet. In fact, on the backend side of things, PALGN and Sumea had a whole lot in common. I say that in the past tense because when I encountered the same issues that PALGN has been experiencing on Sumea, I made some pretty drastic changes to turn it all around. This journal entry is mostly about those changes and not as much about the politics behind PALGN's demise.

tsumea (or Sumea as it was known then) launched a year earlier, in 2002, and was then comprised of two parts. One part was the discussion area of Sumea, running on the free forum software called Snitz. The other half was the editorial and data part of the site which included all the other aspects including administrating news, articles, developer listings, member journals and galleries. It was pretty much a content management system, and all of it was written by myself in ASP. It was a whole lot of work.

PALGN uses PHPBB as the base of their website. PHPBB is a free open source bulletin board software, but with a whole lot of extra modifications, James extended it into a fully functioning editorial website too. From all reports, 50% of PALGN is extra code meshed in with the forum.

The private message threads between the admins and editors of PALGN have been leaked and can be read on pastebin here : http://pastebin.com/e61MdU48

It details some of the technical issues that PALGN was facing - high memory usage issues, database lock ups, and excruciatingly slow page loads. In fact, we faced those exact same issues back in 2006, and I spent a whole lot of time optimising, file caching, removing as many database connections as I could, to transfering the database to SQL server to ease some of the load. What made maintaining Sumea an absolute nightmare, however, was the damn forum software, and I know completely well why PALGN is still running on an archaic version of PHPBB.

Our forum software, Snitz, had fallen so far behind in standard forum features that I had to stitch in many modifications to add them in. Private messages, avatars, and a tonne of other basic functionality like that had to be carefully woven in through dozens and dozens of different php and include files. And when the forum had a security update or another point release, all files were replaced and every change had to be painstakenly and time consumingly re-added in. Now, considering that PALGN had a whole 50% of extra custom code meshed in with their forum, you could probably begin to understand how much of an undertaking it would be to update the forum code each time it needed to be updated. So at some point, it wasn't, and unfortunately any speed and memory optimisations and benefits you could get from new updates were missed out as well. It's also revealed in the pastebin messages that PALGN could not even update their server to PHP5 because the version of PHPBB they were using was far too old. While this meant that the site would have been running slightly less optimally than it could have, what probably compounded the issue much further was perhaps the growth and amount of people on the site.

So we were using Snitz with all these modifications, and when I figured out that the cause of many of the site lock-ups was actually due to Snitz, I began looking for better forum software. This one had to have all the required forum features already built in as well. When I found a suitable replacement called Web Wiz, I shelled out for a lifetime license to use it. I transfered the forum database and made all the necessary adjustments to get it running fine. The situation was better, but I thought I could improve the site even more by combining the two parts of Sumea - the editorial and data side with the forum. That meant extending and modifying the newly purchased Web Wiz forum and adding all this extra functionality to handle the site's editorial and other content. I delved into the Web Wiz forum's files and database to understand how things worked and began to make adjustments.

Now, in hindsight, this is exactly what PALGN did and faltered. tsumea would be in the exact same mess as PALGN, back-end and technology wise, had I chosen to continue this path. It might even be worse off since we'd still be stuck on classic ASP which isn't even supported anymore.

But as I was modifying the new forum to turn it into pretty much a content management system, it hit me. Why didn't I just use a proper content management system instead, rather than modifying and retrofitting forum software to do the exact same thing? In fact, there was a whole lot of new and exciting features being implemented on many websites at the time and I realised that I could spend the rest of my life writing in all these new capabilities by myself, or I can use a widely supported open source option which has them already available.

So 2007, I looked around for a solution and found two leading options, Joomla and Drupal. Joomla looked pretty and easy to use, but Drupal was highly configurable, extendable, and capable of so much more. You dont evertouch the core files, and you extend it with your own modules or ones implemented by the community of developers, which means it's a breeze to update. It's also used by some pretty important websites out there, including The Economist, Examiner.com, Mozilla, Ubuntu, Eclipse, The White House, The Onion, and many, many more. The downside was that the learning curve was incredibly steep. I chose Drupal and began the process of learning how to use it, then looking through the database schematics of Web Wiz and Drupal and how they each stored records to transfer five years of data over. I had to bring over the other database with all the editorial and data content as well. Think of it as combining two seperate databases into the format of another foreign database structure. After months of work and a whole lot of testing, it was finally complete. Everything on Sumea was now completely on Drupal, and I threw away all my own ASP code for good. Looking back now, while it was a lot of work, it was the best decision I've ever made and pretty much saved the site. It even took less than the eight months or so that passed since the PALGN editorial staff first brought up their site's issues until they began to leave when they realised how little progress was made to resolve the site's issues.

I won't go into the politics behind PALGN's situation other than to say I could only wish I had the same passionate team of volunteer editorial staff as PALGN had. If the fears they've expressed do come true and that the owner eventually stops paying the hosting bills and PALGN simply disappears forever, then I think it's a real damn shame and a considerable loss to have years and years of that editorial work gone just like that.

I think that data can still be salvaged and the site even brought back up to speed. All you need is a developer to do the migration work, which in PALGN's case, seems to be a lot easier said than done.