In the weeks or months leading up to a new Game Update or Expansion, players have come to expect the familiar call to Testers to try out new Group or Raid content. Sometimes a specific schedule is posted, with developer chaperones to monitor player reaction to encounters, zone design, and so forth. At other times, testing as at-your-own pace. Either way, Bug Reports are collected and placed in a queue to be fixed at some future date.
If any "show stopping" bugs occurred during this Testing, such as a boss that would not spawn or could not be defeated, or a door or gateway which would not activate, the testing session would end prematurely with the feeble hope that players would return the next night to make another go of it. Even if many of those players could find time in their schedule to test again, this shortcoming has led to less enthusiasm for testing and a diminishing pool of testers.
You probably recall two of the May 2012 Update Plans -- Play-As-Yourself in Dungeon Maker and Quest Targeting of NPCs. However little has been said about the third key point, a significant improvement to Beta Testing:
It’s very frustrating for designers and engineers to be testing raids and group content and have to stop due to a bug or difficulty and have to wait a day for another build. Not anymore! The engineers are already well into developing a testing tool that allows developers to make changes on the fly in real time. This is HUGE news for us and those players who help us test certainly will recognize the value.
The Sleeper's Tomb
Fast Forward to last month when Sleeper's Tomb appeared on the Beta server apparently out of nowhere and was released just one week later. I expressed my own concern that one week was insufficient to test a large new group and raid zone. Yet, the zone arrived and seemed to be relatively bug-free in both EM and HM group and raid editions. What happened?
Two things actually. First of all, I should have known that, as with past Betas, a few waves of testers had been invited to test Sleeper's Tomb before open Beta. Second, developers now had access to this shiny new beta testing tool.
When I first heard about this Beta Fix Tool, like a lot of people I assumed that it allowed developers to make real-time changes to the game code itself. Well, you know what you get for assuming. Actually, what this tool allows is for developers to make real-time changes to a specific zone. If a group of testers encounter a problem, the developer can immediately fix it within that zone and players can pick up where they left off. The end result is, players now have a good chance to test out the entire zone, even testing the same mob multiple times with subtle changes.
Such a tool allows testing to be "accomplished in a much shorter period of time and with a lot fewer testers. It also gives the designers consistency when testing to have the exact same players testing a bugfix for a specific problem."
Speaking of making testing more rewarding, the dedicated players who put the Sleeper's Tomb through its paces received a Tester's Crate of Appreciation which, among other things, contains a Title. Testing group and raid zones takes a not insubstantial amount of time away from playing on the live servers, often with little to show for it other than a warm fuzzy feeling. Providing some reward on the player's live character(s) for testing such content is something I am in favor of, when the rewards are handed out equally.
From Gninja on the EQ2 Forums:
These rewards were not just given out to everyone who tested the content. In most cases the only players who recieved these reward packs were players who dedicated a very large amount of time to testing very specific things with developers. In most cases these players gave up a very large amount of personal time testing things over and over in weird ways looking for potential issues that would be game breaking.
We are looking into other ways of rewarding players for their testing time going forward that will more accurately catch those who help in off hours but as of right now we don't have such a system in place.
I look forward to seeing how quickly EverQuest II: Chains of Eternity can be tested with the use of this new Beta Fix Tool!