How Sleeper’s Tomb was Tested and Delivered On-Time

Written by Feldon on . Posted in Commentary, Grouping, Raiding, SOE Wants Feedback

In the weeks or months leading up to a new Game Update or Expansion, players have come to expect the familiar call to Testers to try out new Group or Raid content. Sometimes a specific schedule is posted, with developer chaperones to monitor player reaction to encounters, zone design, and so forth. At other times, testing as at-your-own pace. Either way, Bug Reports are collected and placed in a queue to be fixed at some future date.

If any “show stopping” bugs occurred during this Testing, such as a boss that would not spawn or could not be defeated, or a door or gateway which would not activate, the testing session would end prematurely with the feeble hope that players would return the next night to make another go of it. Even if many of those players could find time in their schedule to test again, this shortcoming has led to less enthusiasm for testing and a diminishing pool of testers.

You probably recall two of the May 2012 Update PlansPlay-As-Yourself in Dungeon Maker and Quest Targeting of NPCs. However little has been said about the third key point, a significant improvement to Beta Testing:

It’s very frustrating for designers and engineers to be testing raids and group content and have to stop due to a bug or difficulty and have to wait a day for another build. Not anymore! The engineers are already well into developing a testing tool that allows developers to make changes on the fly in real time. This is HUGE news for us and those players who help us test certainly will recognize the value.

The Sleeper’s Tomb

Fast Forward to last month when Sleeper’s Tomb appeared on the Beta server apparently out of nowhere and was released just one  week later. I expressed my own concern that one week was insufficient to test a large new group and raid zone. Yet, the zone arrived and seemed to be relatively bug-free in both EM and HM group and raid editions. What happened?

Two things actually. First of all, I should have known that, as with past Betas, a few waves of testers had been invited to test Sleeper’s Tomb before open Beta. Second, developers now had access to this shiny new beta testing tool.

When I first heard about this Beta Fix Tool, like a lot of people I assumed that it allowed developers to make real-time changes to the game code itself. Well, you know what you get for assuming. Actually, what this tool allows is for developers to make real-time changes to a specific zone. If a group of testers encounter a problem, the developer can immediately fix it within that zone and players can pick up where they left off. The end result is, players now have a good chance to test out the entire zone, even testing the same mob multiple times with subtle changes.

Such a tool allows testing to be “accomplished in a much shorter period of time and with a lot fewer testers. It also gives the designers consistency when testing to have the exact same players testing a bugfix for a specific problem.”

Rewarding Testing

Speaking of making testing more rewarding, the dedicated players who put the Sleeper’s Tomb through its paces received a Tester’s Crate of Appreciation which, among other things, contains a Title. Testing group and raid zones takes a not insubstantial amount of time away from playing on the live servers, often with little to show for it other than a warm fuzzy feeling. Providing some reward on the player’s live character(s) for testing such content is something I am in favor of, when the rewards are handed out equally.

From Gninja on the EQ2 Forums:

These rewards were not just given out to everyone who tested the content. In most cases the only players who recieved these reward packs were players who dedicated a very large amount of time to testing very specific things with developers. In most cases these players gave up a very large amount of personal time testing things over and over in weird ways looking for potential issues that would be game breaking.

We are looking into other ways of rewarding players for their testing time going forward that will more accurately catch those who help in off hours but as of right now we don’t have such a system in place.

I look forward to seeing how quickly EverQuest II: Chains of Eternity can be tested with the use of this new Beta Fix Tool!

Trackback from your site.

Comments (28)

  • clownshoe

    |

    Let me get this straight, they implemented a tool which gives some dev the ability to reset/force spawn a trigger real-time for the sake of allowing further content to be explored by some holy tester? Then, error must be corrected at a later date prior to live?

    It all seems ‘more’ streamlined but i’m jesus-facepalming the YEARS it took for this to be realized and implemented. Yes it’s relatively bug free but ffs it IS only 1 zone (gratz?)

    Look at pathing (EQ’s trademark of fail) of drakes in underdepths. How the hell does that get through to live? Then, how less of a damn do you not give to not fix it at?

    Should I be jovial, at least, in hopes THIS time they’ll release a low-bug expansion? I’m still a sub, I enjoy this game, but when you see the competition putting out near flawless content it makes me take a step back. again…and again.

    Reply

  • Gaealiege

    |

    Yeah, but the competitions games are often fluff, sadly. Mists of Pandaria..or whatever…came out last week. Has it been beaten already? If it hasn’t it’s probably a week from being 100%’d.

    So while it’s polished, it’s like playing a console game with a few more friends. Frankly, I’ll give SOE a single clap on the back for usually making content last longer than a week with an expansion.

    Reply

  • Whilhelmina

    |

    The testing tool is kinda limited for quests though, and doesn’t persist through zone relaunch.

    We spoke about it with Cronyn on Qeynos testing as he helped a lot through this tool. He was able to temporarily fix some NPC, but the final fix could only be applied through a patch, which means that, if the server crashed or the zone resetted, the fix was lost. For example, he couldn’t fix the Antonia Bayle dialog in her Qeynos Palace instance for the whole server but only for individual players as the zone resetted every time the players left the zone.

    Reply

  • Dethdlr

    |

    It sounds like they are able to try different things to see what works without effecting the actual game code. Then, once they find what works and get it how they want it, they can go implement that in the actual game code.

    Prior to this tool, to try different things, they would have to change the game code, wait a few days for a patch, then see if it worked. If not, start that loop again.

    With the new tool, they’ve taken those days and reduced them down to minutes. Or at least that’s what it sounds like.

    Reply

  • Feldon

    |

    Originally posted by Clownshoe:

    Let me get this straight, they implemented a tool which gives some dev the ability to reset/force spawn a trigger real-time for the sake of allowing further content to be explored by some holy tester? Then, error must be corrected at a later date prior to live?

    This tool allows a developer to immediately fix BUGS within the running of an instance zone (solo, group, or raid) so that the players who bothered to show up to Test can actually TEST the content and give feedback on it, instead of being sent home with the zone half-way done. The bugs are fixed in the next morning’s beta build, but the group who encountered the bug can keep running the zone.

    I’m sorry you are so jaded and bitter that you have only jealousy and hate for something that will make the game better with less bugs.

    Originally posted by Clownshoe:
    It all seems ‘more’ streamlined but i’m jesus-facepalming the YEARS it took for this to be realized and implemented. Yes it’s relatively bug free but ffs it IS only 1 zone (gratz?)

    Holly “Windstalker” Longdale came back to SOE this spring and asked for this feature to be added. It was added sometime in August, roughly six months later. I’m not sure how that equals “YEARS”.

    The upcoming expansion will have 9 group zones and who knows how many raid zones. With this new beta test functionality, these zones can be tested and bugfixed in 1/2 or 1/3 the time they used to.

    Hey I hate the pathing and 5 meter range on mobs too, but don’t worry — even if all the content in the expansion is delivered bug free, there will still be endless bitching about something.

    Reply

  • Lunastaria

    |

    Been playing Pandaria hardcore all week long and I’m nowhere near finished with it… Seriously, you shouldn’t be playing WoW 16-18 hrs a day, and should still be having a life/sleeping while playing MMO’s…. If anyone claims they beat it, that is only cause they were hardcore playing and made level 90 by skipping all of the fun “fluff” lore and storyline and just made leveling a priority over fun and entertainment…

    Reply

  • Lunastaria

    |

    Of course, they are in dire need of content, however, I haven’t finished every single quest in WoW, so even I can’t claim to have beat it, and I have spent a lot of time on it!! All those players who “claim” they beat WoW either have no life, or simply skip all of the content because they see it as low level and not worth their time because their are no loot rewards, it isn’t always about end game max level raid gear… Whatever happened to fun, story, and just doing something even if the reward sucks?

    Reply

  • Gaealiege

    |

    My only contention with your statement is the “9 group zones” part. I don’t count EM and HM as a separate zone. I’m betting SOE does. So I’m picturing 3 zones, skyshrine style, with solo, EM, HM.

    Which is unacceptable if that’s the case, by the way.

    Reply

  • Anaogi

    |

    Er, solo != group. Even SOE isn’t that stupid.

    Reply

  • Morgan Feldon

    |

    Well, solo/duo/molo.

    Reply

  • Liftik

    |

    I agree with Anaogi… EM & HM are *NOT* seperate zones. Skyshrine is a 4 group zone IMO (if you count UD). I certainly hope SoE understands that this is not acceptable.

    Reply

  • Gaealiege

    |

    I wouldn’t be so certain.

    Reply

  • Common Sense

    |

    There are plenty of things SoE does wrong, but this sure as hell isn’t one of them. Props to the Sleepers Tomb being bug free, lets hope SoE can pass Chains of Eternity out with few bugs.

    I do agree though with Feldon on the comment of the first poster being jaded and bitter. However, I think its sort have been bred into most of EQ/EQ2′s fanbase. A decade or more of having to deal with SoE’s awful track record on how to communicate to their customers can leave many with an immediate response that is nothing but jaded and bitter. Despite that, they still play the game for some reason…

    Reply

  • Lokim

    |

    Sleeper’s Tomb is definetly not bug free…

    The most glaring bug is the Arbitor fight. Sometimes the 3 adds never stop coming the entire encounter ( I’ve had this happen 10 times in a row before, then all of a sudden it works fine for the next 10 pulls over several days). Sometimes in HM he aggro wipes every 40 seconds, sometimes he doesn’t aggro wipe at all. Sometimes you get a raptor to add the entire encounter from who knows where, most of the time you don’t. Under 20% in HM sometimes you get hardly any adds, ( I’ve had only 1 set of adds before with 1.5 mill dps group) and other times the adds come pouring out like nobody’s business ( 5 sets of adds with 1.5 mill dps group) and dps does not seem to be the issue there. Ravager hits ppl for 200k sometimes randomly, even the tank. Most of the time it goes down easy.

    Mind you, I tank him in the same exact spot every single time, group positioned exactly the same… ( Waiting for the ring event to end and Arbitor to engage on his own etc) Because, if you engage Arbitor before the ring event is over, the adds will continually spawn as well.

    Very buggy fight.

    Reply

  • Lempo of Everfrost

    |

    The raid version is far from bug free as well, in fact it has zone breaking bugs. Our current instance is borked, after it sat idle for 2 days when we went back to the zone the final mob still had not reset after it breaking during a HM pull.

    I’m sure that being able to make live changes to the zones code during testing had a very positive impact and them having this capability was long, long overdue.

    Reply

  • clownshoe

    |

    @ Feldon

    I am glad this tool is implemented; why would i not be? What i am so ‘jaded’ over Feldon, is the length of time it took for a tool like this to exist; Holly’s time-frame is irrelevant. Maybe I am jealous of other companies, who’s game(s) i don’t find as in-depth or aesthetically pleasing, be herald as superior. I want this game to succeed as much as anyone still playing and i do not plan on quitting anytime soon. You just took a ‘better here than never’ approach and caught me in the cynical ‘it’s about damn time’ light. Obviously competitors have higher expectations and their subs reflect that.

    @Gaealidge

    Creating lengthy content is simple: create near-impossible content, extremely low drop rates, faction grinding, or zones so large it takes 10+minutes to traverse (karana’s in EQ1 before mounts). EQ wrote the book. Although I agree merely jumping through a hoop in WoW rewards you for much more than this game my point still stands, regardless of length, it is probably bug free if not flawless in comparison.

    Reply

  • Stark

    |

    As I understand it the Arbiter fight in the group version have several different scripts and it choses one at random when you get him after the adds.. If you don’t like the memwipe version, reset and try for a better one.
    I may be mistaken but this is how I have had the encounter described to me and if it’s supposed to be this way then it’s just fun to not always be able to know exactly when the named uses which abilities throughout the fight.

    Anyway, the ST zones seem pretty bug free from my perspective.

    Reply

  • Daalilama

    |

    The Arbiter HM fight is as buggy as the day is long….Lokim is correct on that….damm annoying to have a solid heavy dps group implode when it bugs out with adds…

    Reply

  • Buffrat

    |

    I’m still waiting for them to finish fixing Plane of War gear.

    Reply

  • Noctew

    |

    I’ve been told that the HM Arbiter will memwipe if and only if you do not get the EM Arbiter below 50% before he switches to the HM version. And from my personal experience this seems to be true.

    Reply

  • bob0builder

    |

    They are bug free because the boss are stupid tank/spanks and their isn’t really a script running.

    - 1st mob doesnt do anything when his emote goes “stomping”
    - 2nd mob is supposed to spawn “pieces” and sometimes it doesn’t and even if the pieces do spawn they only matter if there are more than 2 up. Otherwise just kill Exoduus and don’t worry about the pieces (broken script imho)
    - 3rd mob doesn’t do anything except randomly turns invisible and ports 5 meters away (broken imho)
    - 4th mob … do the green slimes do anything… I just kill the banshee and don’t worry about the adds or green slime (ive yet to try her HM tho).
    - 5th mob if its EM Arbiter then its a stupid script (3 birds every now and then till it hits 50%)… HM is fun he just has too many hit points imho

    You can’t seriously think this zone is bug free … there isn’t anything to bug … the bosses don’t do anything.. they don’t have scripts there all tank/spanks

    Reply

  • warben

    |

    Hey feldon?

    can you post strats for st so people actually understand what is a bug and what isn’t?

    Reply

  • Stark

    |

    @bob0builder
    All those fights are scripted but you can ignore the strat if you have enough dps, like every fight… Ever. Except for some very few heavilly scripted fights you can ignore the intended strat if you just bring enough dps.

    If you are light on dps some of those scripts start being annoying, if you are light healing others start acting up.
    With a stable raidgeared group you won’t have to worry about any of it though.

    Reply

  • Lokim

    |

    I doubt the 50% HP thing anything to do with whether or not he memwipes… We can get him down to 15% or so before he switches to HM, placing him clearly under 50%. Even if you couldn’t get him down below 50% why would they penalize you for having a low dps group making the encounter even more difficult? Makes no sense at all. Having killed him well over 50 times on hardmode on different toons, it’s totally random.

    Breakdown is like this…
    70% of the time Aggro wipes throughout no adds till 20%
    10% adds throughout sometimes with raptor
    20% of the time no aggro wipe at all ever

    At least that’s been my experience. It tends to bug out in streaks though, as in, you’ll get adds throughout for multiple attempts in a row until you give up. Then it works fine for days.

    Bugged

    Reply

  • Kenn

    |

    Good work SOE!

    I’m glad to see developers pushing the envelope for the die hard gamers who are left playing eq2.

    Some of the features almost seem like an academic exercise, such as the card game, webcam facial movements, and others that are cool in technology, but seems few players use.

    What next?
    - Server unique items
    - Travel through time and actually play your eq2 toon on eq1, back and forth swapping
    - A whole new land, where you start out at level 1 with nothing, and in order to get there, you have to be lvl 92 with 300 aa
    - A server where you can only level by turning in quests

    I like the server unique items, you can only see and loot a unique item if you first destroy the one you have. Items would be cool not just illusions. For example, 1 person on the server could have 250% fly speed, another toon could have 0 broker fee, someone might loot a repair bot where they get paid when someone uses it.

    Reply

  • Gaealiege

    |

    I would say the fight is simply random. Either way you look at it the fight is pathetically easy though. If you cannot handle memwiping, you cannot handle HM fights. HM Banshee for instance also mem wipes.

    The only semi-annoying random effect are raptors…which are death effects if your group decides to ignore them once.

    Reply

  • Gaealiege

    |

    Also I have no clue what the hell bob is talking about. All those things he mentioned work.

    Reply

  • Alacus

    |

    Another possible reason for untested content is that the European realm doesn’t have many players. some players use 5-12 accounts so keep that in mind when you talk about the fact that there is xhundred player online. not much I can say about it as I am a lowly player who doesn’t play round the clock to get loaded with raid gear and pretty much a noob. but its just a small ammount of input from another player who play’s this game

    Reply

Leave a comment

- Name (required)
- Website (optional)
Please post your comments without flaming or insulting other players or personally attacking SOE employees. Comments from bogus e-mail addresses may be deleted. If you wish to have an Avatar picture, feel free to create an account on Gravatar.com. Once you sign up for a Gravatar account, then any Wordpress-powered blog you comment on will automagically show your avatar.

Powered by Warp Theme Framework