Feldon 
For months, the question in my mind has not been IF, but WHEN the SOE forums would need to be replaced. On September 18th, word broke on the Vanguard forums that SOE will finally be leaving their crufty homegrown forum solution behind in favor of new forum software.
After the jump, we look at why new forum software is absolutely imperative, and just what will happen to the millions of existing forum posts amassed over the last 5 1/2 years.
Is This Transition Necessary?
It has been impossible to ignore the increasing downtime and recent performance issues of the SOE Forums. The Platform team have thrown faster hardware at the problem with limited success. Despite their efforts, functionality has continued to fall off like parts from a derelict spacecraft.
Since the deployment of the current incarnation of SOE Forums in 2007, the team who developed it has moved on. And as with any aging software, those left behind have had to dedicate an increasing number of hours performing maintenance just to keep it running even in its current half-broken state.
Forum Search is almost useless at this point as a good portion of forum posts are not indexed. Further, users are no longer reliably e-mailed when they have received a Private Message, or a topic to which they were Subscribed received a reply. And since day one, users wanting to quote multiple other posters have been forced to wade hip-deep into editing raw HTML code. Frankly, the SOE forum software was obsolete the day it rolled out, and it was never designed to scale to the number of users and posts it’s being asked to store (over 2.5 million posts just for EQ2 at last count).
New Forum Software?
A few years ago, EQ2Wire was shown a screenshot of an in-development replacement for the SOE Forums. The new software would have been a radical departure from traditional forums, with a visual style analogous to Twitter or MySpace. Fortunately, cooler heads prevailed and the team has chosen an off-the-shelf forum product. XenForo, created by ex-developers of the industry standard vBulletin forum software, offers:
- Faster Performance
- Instant Accurate Search
- Profile Customization
- Threaded Conversations (PMs) between multiple recipients
- the ability to “Like” posts — I know you’re thinking it will be as useless as Facebook, but actually it works really well on a forum
- and more…
If you’d like to see the new forum software in action, PlanetSide 2 has been using it since late August.
Although no timeline has been announced for EverQuest II, on October 2nd, the EverQuest community got the news that their new forums would roll out on October 9th. This timing coincides with the EverQuest: Rain of Fear expansion beta so that all discussion would be on the new site.
What About Old Forum Posts?
Now for the bad news. The millions of old forum posts from the EQ, EQ2, Vanguard, and other forums are NOT being migrated to the new software.
Linda “Brasse” Carlson had this to say:
Once we put all the ancient forums into the mortuary, one by one, I am not sure how many folks will actually even try to mine them for info. More power to them that do though, as this is something we don’t have the staffing to manage on this end. Historians are always appreciated!
Further on the subject of allowing players to backup the existing forums, Brasse had this:
- We have confirmed that we are able to archive the old forums as read-only for a period of at least several months (several being more than two). This should allow everyone plenty of time to retreive what they want and need from the old forums. We are NOT going to keep them forever, but we’ll provide 30 days notice before we do take them offline.
- For those of you who want to copy the ENTIRE old forums content for perpetuity, feel free to do so. While we technically “own” the content, it is already publicly available and contains no private information.
Never Give Up, Never Surrender!
So is that it? Are the EQ2 forums just going to be blown away in a few months time? Nope.
The lack of updates on EQ2Wire over the last couple of weeks can largely be attributed to a secret project. I spent some 2 weeks writing an SOE Forum Backup Tool which crawls the SOE Forums and harvests posts, threads, and forums. I’ve been continuously running this script for several days and have already backed up approximately 15% of the EQ2 forums. According to Brasse‘s post, existing forums will not be taken down for at least 2-3 months, so there is more than enough time to make a complete backup of the EQ2 forums.
I have also provided this SOE Forum Backup Tool to folks in the EverQuest community and I know one of them has already started crawling the EQ Live forums in order to make a similar archive available.
By the way, I offered my services to SOE for free to migrate their forum data, but was politely declined.
Their new forums look very nice. It’s about time.
Thanks for your efforts to save old post. It will be appreciated 🙂
I wonder if anyone ever backed up the original Everquest forums, the ones from back before SOE threw a hissy fit and closed them all? I’d be interested to revisit The Newbie Zone from that period – it had one of the best forum communities I ever experienced anywhere.
“no longer reliably e-mailed” !?!?!? Their ‘new’ forums haven’t ever sent me an email about something i commented on.
So, it wasn’t breaking because they were disappearing posts like a North Korean truth squad? 😉
Keep On, keeping on Feldon. The EQ2 community is better off having you and even though SOE is playing the fool not recognizing your effort and commitment, we are all the better for it. At least we have one person that cares about this product around here, god knows SOE does not demonstrate the same.
Nice job Feldon on backing up the forums (again way to go SOE on not doing something they should have). As for the new forums…while the change may make it more user friendly it will still have the biggest major problem as the current one….overly heavy handed moderators that are more concerned with tagging everyone who isnt happy happy joy joy over they fluff over meat of game issues as trolling or banned…
Every gaming community needs its very own Feldon (& Dethdlr) I say. Great job on all you do for us EQ2 players.
I reckon you guys should team up with some coders and artists on some brand new MMO, maybe a kickstarter project or something like that.
It would certainly have some TLC, attention to detail, and passion injected into it f you two were involved and the community / playerbase would truly be the number one consideration, unlike most current mainstream MMO’s where at best, they only pretend the community / playerbase is at the heart of the game. I realise you can’t please all of the people all of the time and all that, but some publishers don’t even seem to try sometimes, or it’s blatantly insincere when they do.
Do you know if there will be a bit of a buffer to start with…ie: new forums going back at least a few weeks/months. Or are they just starting from the day the new forum opens with a completely blank one?
Thank you for doing this…being able to go back and view the homeshow posts is so valuable. When viewing those in the backup…will all the images in those original posts still be there/be viewable as well?
Thank you again! =)
Hopefully with new mods, because the official forums have been mod-ed to death.
Thank you Feldon for all this work. It’s really important to preserve the forums, and I am glad you decided to take the project on. SOE lost out, but what else is new.
I don’t know if they will be copying any posts over. I just can’t fathom starting the EQ2 forums from zero.
Yes.
NOTE: It will take SEVERAL WEEKS to backup the EQ2 forums. Without access to the database, I am having to “screen scrape” the forums a page at a time and it’s a slow process to say the least.
@Feldon are you grabbing the Data forums too?
I can backup one of the Data forums since it’s visible to guests. I’ve asked about the other forum being opened to the public just so it can be more easily backed up.
The cynic in mean wonders if they’re intentionally abandoning the information in the ‘old’ forums to get rid of all that ‘nasty’ criticism. If so, the moderation might be even MORE heavy handed in the new forums… to make sure every bit of ‘negativity’ is eliminated.
I’ll miss my day 1 forum account, but those forums are complete garbage and need to be replaced immediately.
Thanks a ton for what you’re doing. I was dismayed by the news of the forum demise as I searched several times for very old infos on them and was glad to find it.
You’re the best
@Feldon —
You’re doing _what??_ … Wow. You rock, and all that.
Of course, what’s also spectacular about what you’re doing is that a large commercial organization with a lot more resources devoted, ostensibly, to just this sort of work can’t be bothered to do it. There we have it.
Shouldn’t really come as much of a shock to anyone, everything SOE has done for quite some time now has been half-assed. Dungeon maker, Dungeon Finder, SOEmote, Battlegrounds, PVP revamp, itemization, re-itemization and ‘looking into’ itemization issues the list goes on.
The only reason for them not importing the data from the old forums to the new ones is nothing short of the complacency that they always have along with the regular serving of contempt towards the playerbase.
Feldon and Dethdlr you totally ROCK!
Thank you Feldon for such a wonderful public service to the EQ2 community!!
This one was all Feldon. I thought he was nuts when he described what he was going to try to do. The sheer magnitude of crawling that many pages to grab all the data, then stuffing it into actual forum software sent chills down my spine. lol
But from what I’ve seen so far, he’s pulling it off quite nicely. Major kudos to Feldon!
It can’t happen soon enough. The SOE EQ2 forums are as slow as molasses in February. I’m not surprised hardly anyone posts there any more. Half the time the stupid pages won’t even load.
It would also be nice if they were moderated by non SOE employees. It seems that every other thread is locked these days. Moderation gone mad.
At the least you might expect the stickies to be transferred to the new forums. There’s a load of useful stuff which many people have spent years of their lives producing and keeping up to date.
Oh and thanks for doing SOE’s job for them Feldon.
EQ’s transition was yesterday, looks like nothing was copied over: http://forums.station.sony.com/eq/index.php
They archived their old forums here but at the moment it won’t load for me: http://forums.station.sony.com/eqold/
EQ2Wire. We try harder.
EQ’s transition on Tuesday last less than 24 hours or so before the forums were taken down for maintenance. They remain down as at this moment. No idea why.
They were an improvement in terms of performance, but lacked full width screen support, and also had no display of members logged in so you couldn’t see if any SOE staff were active. No support for custom avatars.
Worst thing of all, the new forums maintained the chief inanity of the old ones, namely the need to log in every time you access them.
Nice work on the old forums, Feldon, thanks for that. Hopefully SOE will get the new ones back up before much longer, but the initial feedback thread was pretty negative on the whole so I’m hoping they’re making some further improvements rather than just doing maintenance – assuming the downtime isn’t because of some security flaw with the new arrangements as that seems to be a constant worry these days. I do wish companies would have separate game and forum login details (and stay away from using email addresses).
The EQ1 forums have not been down, they have been up since Tuesday for me and I have been on the forums on and off since then. Also I have been able to stay logged in at least for a few hours or till I shut down browser.
Yes, I think it’s a problem for certain people and also with certain browsers. I got straight on just now with my second machine but couldn’t get on the last couple of days on my first machine, will see later if that has changed and if not then try clearing the cache.
My crawler script has been getting errors for the last hour that I’m not seeing if I access the same URL on my browser. I’ve suspended it and will start it up again later.
301,601 out of 522,913