›

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links

General: MMO Underbelly: The Truth About Betas

DanaDana Member Posts: 2,415
in News & Features Discussion

In this week's edition of The MMO Underbelly, Sanya Weathers tells us the truth about how Betas work. Find out how people are selected and some other behind the scenes anecdotes.

I stuffed all three hundred mailers, took them down to the mail drop, and stuck them in the bin that says absolutely no tampering or you are doing hard time with roommates bearing poorly spelled tattoos. I got back to my desk, and started looking for that Eddie Izzard disc. You know, I’d done my work, I was ready for a reward.

I popped the disc into the computer, but instead of a cross dressing surrealist, I saw the install window for the beta I was working on.

I briefly contemplated breaking into the mailbox. Instead, I trudged to my boss’s office, confessed, apologized, waited for him to pull up Eddie Izzard on Google, apologized again, and went back to my desk. Then I held my breath for two weeks, waiting for someone on the beta list to call me in a complete tizzy.

Read it all here.

Dana Massey
Formerly of MMORPG.com
Currently Lead Designer for Bit Trap Studios

«1

Comments

  • KainisKainis Member Posts: 436

    ROFL!!! Sounds exactly like a recent discussion I've heard over lunch with a coworker. I'm starting to wonder if I may indeed have worked with Sanya in the past, and didn't realize it!

    -----------------------
    Tried- L2, Ryzom, WAR, DDO, PWI, Tab Rasa, Requiem, Champs, AA, JD, PWI, SUN, Dawntide

    Played- SWG (pre-cu), AoC, VG, WoW, LoTRO,CoX, EQ2, DAOC, GW, PotBS, Aion, MO,APB, NASA, Fallen Earth, DCUO, Rift

    Playing- EVE, Black Prophecy, TOR

    Waiting for- Tera, Jumpgate Evo, WH40K, WWE, WOD, TSW
    --
    --
    "Hey, if Activision liked it, then they should have put a ring on it," Double Fine President Tim Schafer said. "Oh great, now Beyonce is going to sue me too."

  • ArmEagleArmEagle Member Posts: 36

    Hah, that misplaced CD story made me cry a little.

    Stories like that is why I love 'dev' posts.

  • MindTriggerMindTrigger Member Posts: 2,596

    The truth about Betas in recent years is that they are early marketing gimmicks and not much more.  Having been in several myself over the years, I know from experience that many of the games GLARING problems which are discovered in Beta are never fixed before release.  How many games have released in recent years where the Beta players themselves were screaming from the hill tops and saying the game is no where near ready to go gold? Conan is the most recently famous example.  I was seeing warnings about that game from Beta players months and weeks before release, all the way up to the last day.

    I've given up on trying to Beta new games.  Most of the time they are released in an unfinshed form anyway, so why bother getting in early just to experience the same bugs I will continue to experience later?  If I sound cynical, it's because I am.  This genre is stale and in a sad state of affairs.  Our only hope will be the indy game devs who are willing to take chances on new ideas.

    A sure sign that you are in an old, dying paradigm/mindset, is when you are scared of new ideas and new technology. Don't feel bad. The world is moving on without you, and you are welcome to yell "Get Off My Lawn!" all you want while it happens. You cannot, however, stop an idea whose time has come.

  • OneEyeRedOneEyeRed Member UncommonPosts: 515

    I was a long time DAOC tester  (6 yrs.) Sanya and as always I enjoy reading what you have to say. The day you left Mythic was a sad day indeed for us players and  the Herald never was the same. Although I have moved on it's nice to see you still in the industry and working. I hope to that one day we will meet on the battlefield again. Long live Hel's Black Brigade!

    “Once the game is over, the king and the pawn go back in the same box.” ~ Italian proverb   

      

  • DaerkannonDaerkannon Member UncommonPosts: 9

    MindTrigger, you should have said "Our only hope will be the indy game

    publishers

    who are willing to take chances on new ideas."  I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"?  I'll give you three guesses and the first two don't count.

    Back on topic, this was a good article.  One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report.  Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report.  Either one will make me stop beta testing.

  • KyarraKyarra Member UncommonPosts: 789

    I was one of the lucky ones to beta test the end all expansions for DAoC, The Trials of Atlantis, I have to admit I was having fun with it, but I was on Gaheris at the time so it didn't matter that I would have to grind out that expansion to do good in pvp. I am an avid beta tester (beta testing 3 major games atm) and I am sooo careful when I fill out beta apps. Good read :)

  • MindTriggerMindTrigger Member Posts: 2,596
    Originally posted by Daerkannon


    MindTrigger, you should have said "Our only hope will be the indy game
    publishers
    who are willing to take chances on new ideas."  I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"?  I'll give you three guesses and the first two don't count.
    Back on topic, this was a good article.  One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report.  Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report.  Either one will make me stop beta testing.

     

    Well, that's their choice.  Had EVE not been self-published, it would have never happened.  A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about.  Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.

    A sure sign that you are in an old, dying paradigm/mindset, is when you are scared of new ideas and new technology. Don't feel bad. The world is moving on without you, and you are welcome to yell "Get Off My Lawn!" all you want while it happens. You cannot, however, stop an idea whose time has come.

  • megagamemegagame Member Posts: 11

    Beta are some funny things, and thos ppl that want in betas are often also.

    I have been in alphas, open/close betas, stresstest and other funny testing fases, and the is always some players that think their are smarter then the whole dev team, even if their havnt tryied the game yet. Always funny to read about how a few testers want to redesigne the whole core part of the game to their wises, when the rest of the testers are happy with the currrent form.

    I sign up for beta on games I think may be fun to play, but games I really look forward to, I say away from as much as possible, dont want to spoild the fun, but cant wait for it to come out.

    but being dev must not be easy, on one side you have your testers saying this game needs more work, on the other side you have your futere players, that keeps complaing about delays and ask why game is not out yet.

  • megagamemegagame Member Posts: 11
    Originally posted by MindTrigger

    Originally posted by Daerkannon


    MindTrigger, you should have said "Our only hope will be the indy game
    publishers
    who are willing to take chances on new ideas."  I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"?  I'll give you three guesses and the first two don't count.
    Back on topic, this was a good article.  One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report.  Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report.  Either one will make me stop beta testing.

     

    Well, that's their choice.  Had EVE not been self-published, it would have never happened.  A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about.  Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.

     

    Eve was not self-published, their hade a pulisher, but their bought back their rights to the game after a short time out.

     

  • MacAllenMacAllen Member UncommonPosts: 72

    I've been in more than 20 MMO betas, and this article is a great piece of advice for how to get in.

    • I got into the UO beta by complete random chance.
    • I got into the EQ alpha/friends and family by being positively active on the forums, and they made me found the Guide Program because of it..
    • I got into the CoH alpha by being so positively active on the forums they made me moderate them.

    Getting into an early beta is all about your presence...filling out the form won't get you in, especially these days with millions of people signing up.  Getting the early slot is all about establishing an identity to the devs.  How do you do this?  Allow me to expand aupon the OP's suggestions:

    • As the OP said, avoid drama like the plague.  Don't start it, don't join in, don't try to stop it.  Avoid it.  I don't care *WHAT* your opinion is relative to the topic on hand, everyone in that thread has a black spot on their record.  When threads like this get 4-5 pages long, the devs don't even bother reading it, they just notice who's participating.
    • Write FAQ's, helping threads, guides, suggestions, ideas.
    • Be helpful in the community.  Don't say "RTFM" to new users, welcome them...a welcome person stays and likely buys the game, an alienated one doesn't.  My honest suggestion is act like you want the community to survive and have everyone in it, not like it's exclusive and new folks are shunned.
    • Someone who's an ass on the forums will be an ass in the beta, and in the early beta lots of things are broken.  Someone who is bitching on the forums about something that doesn't exist yet will do so 10x more when they get in the game.  That kind of person may make it to the open beta, but they'll never be hand-picked to be invited in early...why court that kind of misery?
    • Don't kiss up.  Be nice, not obsequious.  Be supportive, but not a catamite. 
    • Have opinions and express them clearly and concisely.  No one reads 14 page dissertations on what you think PvP should look like, nor do they read your 12th post on how you feel the economy should be like.  Express your opinions clearly and concisely and if the devs want more info, they'll ask.  I've gotten PM's from devs of 9 MMO's, asking me for more info on something I'd posted.  1 of those ideas was actually used in some form.  The devs want early beta testers who will jump into a bottomless pit 3x just to validate the coordinates vs one that complains the first time they encounter it.
    • Don't jump on people whose ideas are different than yours.  Don't treat everyone who disagrees with you as if they're attacking you, personally, even if they actually are.  You're not a dev, they're not a dev, neither of you has a clue, neither of you is right, so the argument is not resolvable.  Be the "bigger man" and show how reasonable you are.  You obviously don't have to agree, just don't bicker, it's petty and annoying.

    This brings me to what should you do once you're in an early beta:

    • Report everything.  Not on the forums, but in /bug, in as much detail as possible.  So what if someone else has seen it?  So what if someone says they reported it.  The number of times a bug is reported is indicative of it's exposure.  A bug that's reported once is less likely to be visible than one that's reported by 50 people.
    • If you find an exploit, *DO NOT POST IT IN THE FORUMS!!!*  Do not allude to it on the forums.  Don't discuss it, don't mention it to anyone else.  /bug the heck out of it, and perhaps send a PM to the community manager about it, privately, but for pete's sake don't make it public.  People who believe "the more public it is, the quicker it will be fixed" hurt the game.  And don't keep exploiting it either.  After you've validated it a few times, move on to something else, there's no value in continuing to do it.
    • Don't bitch.  Not in game, not on forums, not on your private blog that you jot your dark soul into.  It serves no purpose and only creates negativity.  It's beta, heck it's early beta, stuff is broke.  If you can't test what you want to do due to a bug, go test something else.  Roll up a new toon.  I can't tell you how many times I've gotten a toon stuck, tried everything I could think of, then logged and made a new toon to test something else.  It was only when all 8 of my CoH toons were in some manner of bugged state that I asked the devs for help.
    • Don't act like you're entitled to anything, because you're not...early testing is it's own reward.  No, you can't keep your toon.  No, you can't get an exp or loot boost because of all your amazing effort.  No, you can't get some kind of after-release item to display your massive ePeen.  What you can do is get to know the game such that, after release, you have a "head start" play experience-wise, and that's enough of a reward.

    Good article, thanks to the OP/Correspondant for putting this out there.

     

  • SoulticeSoultice Member UncommonPosts: 112

    An excellent read and very true. 

    I got into the friends and spouse's test and later alpha testing Horizons as one of the devs was a friend of mine.  Aplha tested a few more and closed beta tested some others.

    Beta tests these days do appear to be previews. I was in Aion this last weekend.  Still a preview with some needed suggestions is all.  I will not pass judgement as it is not released yet and I really have not seen the rest of the game.  I did have fun though and the world was breath taking for what I played..

    I can rermember those long bug report forms and feedback forms for every encounter and quest.

    I do like to beta testing and hope to help bring a dev's dream to market. 

    One poster put it best and it happenend to Vanguard and Tabula Rasa is investors want to see a return on their money and will override Dev''s when they feel the game is ready or taking too long. 

    Both games had great concepts but were in development too long.  They were released before they were close to being finished.

  • MindTriggerMindTrigger Member Posts: 2,596
    Originally posted by megagame

    Originally posted by MindTrigger
    Well, that's their choice.  Had EVE not been self-published, it would have never happened.  A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about.  Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.


    Eve was not self-published, their hade a pulisher, but their bought back their rights to the game after a short time out.

    And are now free to do whatever they want with the game.  They are self published at this point.

    A sure sign that you are in an old, dying paradigm/mindset, is when you are scared of new ideas and new technology. Don't feel bad. The world is moving on without you, and you are welcome to yell "Get Off My Lawn!" all you want while it happens. You cannot, however, stop an idea whose time has come.

  • InktomiInktomi Member UncommonPosts: 663

    Great article and so true. Question: Is the star-f*cker's reference from the NIN song? If so, great song and they were in NY recently. Sadly, I couldn't get a ticket. Same analogy with getting into an exclusive beta, you need to be an active fan and know when and who to ask. I fell out of the loop on certain beta's and concerts because I just wasn't "There" with it. So I am marching my butt over to the Champ-O's forums and start...being me.

    That's gonna help.

    lol, good stuff sanya, 

    Ink

     

  • thinktank001thinktank001 Member UncommonPosts: 2,144

    This article points out everything that is wrong with betas, and it is just another reason why games release in a less than desireable state.

  • DOUBLESHOCKDOUBLESHOCK Member Posts: 5

    Nice article!

  • mOoKmOoK Member Posts: 26

    Flag.

  • RaveGodXRaveGodX Member Posts: 17

     Great read. I also tested a few alphas and betas and even some stress tests in my years of gaming. Good to see some insight from a dev. It seems like alot of betas are pre play for pre orders these days. Last alpha I was in was Rome Gods and Heros. The game had some good play to it. Shame there were so few testers to be found, when I was on. Shame it all folded up. I did Aion thx to mmorpg last weekend and I was impressed. thanks for the discussion!

  • SpittSpitt Member UncommonPosts: 26

    I would like to reiterate the obvious, which is find a bug and report it to the appropriate place.  But I also want to mention, should you find a work-around, post that as well, with the number assigned to the bug.

    Finding a bug...

    1. Play the game, try to have fun... but always be on the lookout.
    2. Try doing things when there are server shutdowns.
    3. Try doing things when the server comes back up.
    4. Try doing things when you are logging in or out.
    5. Look up information, and see what others have done.
    6. If you see something weird in the game, investigate it.
    7. If somethig weird happens, try to duplicate it.
    8. If you notice an area that has a server slow down (lag) find out why. 
    9. Don't confuse server lag with screen lag (zoom all the way in, and stare at the ground... if lag stops it is screen lag).
    10. There are new bugs introduced every patch, read the patch notes on what changed, mess with the changes.
    11. Never assume that someone else has reported a bug, if you find one.
    12. If the server or landblock shuts down when you are doing something, investigate to see if it was from your doing.
    13. Keep notes when you find something.
    14. Report bugs.
    15. Preview your post, at least 2-3 times and edit as much as possible to make the post as detailed as possible, before you submit it.

    While you might think that you can use the bug later if it makes it to gold, if you don't post it... post it anyways.  The less bugs a game has, the more likely it will become a successful and fun game.

     

    I have been in numerous beta tests from MMOs to RPGs, and the information above can help you not only make it from a preview to become a regular beta tester, but a more successful beta tester as well.

     

    By the way, Aion ships in September according to my key supplier, with pre-sales happening now.  Last weekend there were 1 million players trying the game out... but likely this is spread across all continents, as my friends in China were mentioning it as well.  So consider that probably about 150k-250k US players?

    Spitt
    Uber100.com - Add your MMO site today!

  • MustaphaMondMustaphaMond Member UncommonPosts: 341
    Originally posted by Dana

     
    The main way in is to be an active member of the message board community around your target game. There’s a lot more to being active than posting psychotic twenty paragraphs screeds on arcane design points that may not even be in the live game.
     

     

    Damn, I guess I'm S.O.L. then... (;-_-)

    And probably about half of the Star Trek Online fans (who just won't STFU about ship interiors).

  • adarshakbadarshakb Member Posts: 35

    LOL... The reciever must have been baffled but also been happy to see free movie copy!

    or worst stil is that he being a noob thought that the cd is actully a beta phase tried to decipher how to play it!

    and any way Gr8 post and GAME ON ppls 

  • brostynbrostyn Member, Newbie CommonPosts: 3,092

    Nice article. Its basically saying be a friendly person, and don't act like a moron. Not only will that help everyone in life, but it also will help you get into beta. Funny that.

     

    Personally, I beta tested one game. I hated it, and won't do it again. I found myself never logging in, and probably was taking someone's spot who would have done a much better job. I just don't like the surprise of a new game to be worn out when it releases.

     

    As always, great article Sanya. I admire your writing talents.

  • vasilchovasilcho Member Posts: 42

     


    I want 'real' opinions, real people, who may or may not agree with me but have a respectful and mature way of sharing their views,” says Katie Postma, Community Manager for Jumpgate Evolution

     

    ehhh, if only this was true at least for 10% of the games involved... seems like these days no matter how well you present your view on some problematic area, its always a purely subjective decision whether that feedback actually goes anywhere beyond the beta forums. just go say 'this is stupid, you shouldve done it in that way, because bla bla, etc etc' and if your lucky you get infraction, if not ban ) too bad companies and especially the people involved with talking to/getting feedback from the community cannot realize the simple truth - no matter how skilled your dev team is, a crowd of milion will always think of something you havent

  • todeswulftodeswulf Member Posts: 715

    My last beta test was Age Of Conan, I swore I would never pareticipate in another beta after that.

  • BhagpussBhagpuss Member Posts: 58

    A good read, as always.

    My first beta was Anarchy Online. I still have the CD in the original airmail envelope. Unfortunately, although I eventually managed to get it to install (only took about three days and large chunks of my scalp), I could never get it to run. But then, it barely ran when I got the launch version, so no surprise there.

    For a few years after that I applied for many MMO betas and got into most of them. Big ones like EQ2 and small ones like Rubies of Eventide (both of which I still play). All I ever did was fill in the application and forget about it, although occasionally I might have also been active on the forums. Mostly I'd find my invite coming in the later stages of closed beta, somewhere from two  to four months before open beta rolled around.

    For a long time i found the whole process fascinating, but eventually I realised that beta-testing a game you actually plan on playing isn't such a great idea. Several games I betaed were clearly more entertaining and enjoyable for me in beta than they were when they finally went live. There were late changes made in two that I can think of that, in my opinion, were materially significant in the problems those games had at launch, and which were reversed later. I'd actually have enjoyed both of those games more at launch had I not known how much more fun they'd been just a few weeks earlier.

    Nowadays I generally only apply for betas of games I don't intend playing when they go live. I'd rather come to the ones I am really interested fresh.

  • SuperCrapSuperCrap Member Posts: 324

    Uhhh... what about all the 14 year old mentally disabled people who have multiple beta accounts in every beta or earlier stage of development I have ever ever played in or heard of?  The way many people get into beta is simple, and it's identical to the method many people use to reach "success" in real life, but it's even easier to implement this strategy over the internets:

     

    1) LIE.

    2) Kiss ass.

     

    It's funny to read the dev comment about not kissing ass.  I got a huge laugh about that.  Then some master beta dude in this thread writes a whole long post about how to kiss ass with level 9 mastery and how it got him into 32 betas, but then tacks on the caveate, "but don't kiss ass".  LOL.  Every development community I have ever seen was quickly staffed by the #1 ass kissers on the official forums/irc [they get nominated to be moderators usually, and even admins in indy projects like Darkfall for example], and these are usually among the first of the wider community vanguard brought into "beta" after the friends and relatives stage. 

    The one and only.

Sign In or Register to comment.