Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
General: Why Youre Not a Beta Tester
SBFord
Former Associate EditorMember LegendaryPosts: 33,129
In today's Player Perspectives column, MMORPG.com's Isabelle Parsley shows her MMO age in her discussion of her history as a beta tester. Isabelle also talks about what beta testing is, or at least what it's supposed to be, as compare to how many treat it these days. Check out Isabelle Parsley's column and then, agree or disagree, tell us what you think in the comments.
They apply for the closed stages, get in, and then spend most of their time on the forums or in the global channel complaining about how a game in development still has bugs. Gee, you think? Suck it up, pull your socks up and do your job as a tester by reporting on what you find, or just shut the hell up and go elsewhere. Seriously. Oh, and while we’re at it: read the bloody patch notes, kthx.
Read more of Isabelle Parsley's Player Perspectives: Why You’re Not a Beta Tester.
)
Comments
I beta test to avoid making a bad purchase. I have found (and this is a generilization) most companys could really care less what you say.They have their own agenda and needs and that often has little to do with your input.
I agree.
If you dont pay your beta testers anything then you can't expect anything out of them. Somewhere it said something about doing your job as a tester. Sorry but I get paid to do my job, anything else is just an activity that I do because I want to and as such don't do anything I don't like to.
So if gaming companies want beta testers that actually do real testing, they should do as any other company and hire real testers. So for now I just see beta testing as a preview of a game to come and nothing more.
My gaming blog
I'm of the opinion that inadequate testing is a major flaw in the pipeline betweed design and release. With it, release faults like grand openings where the server falls flat on its face wouldn't happen. Testers, a bug tracker, and formal release candidates should be able to find and fix such flaws.
Any more I consider "open beta" to mean free-trial and server stress test. It's been turned into a marketing tool, at least that's public perception. I also tend to agree with the previous poster that by the time a game hits free-trial open beta, some devs turn a deaf ear. It's finished and the "we'll fix it in a patch" mentality seems quite common. I have to chuckle when after several patches I see comments from testers who did report significant issues with no response. Clearly someone isn't doing their job.
It's not just in MMOs. In my work I come into contact with software that's marketed as a final release version but obviously isn't. Bugs happen. Avoidable bugs that hit release because nobody bothered to test should not.
Sad.. but true.
I think Betas would be taken more seriously if more companies actually listened to the testers and 'fixed' the bugs that were reported and complained about.
Case in point - Black Prophecy.
I was one of the very first Beta testers, and it was glorious. It had some major flaws, but I reported them and went on my way like I should have. I stopped playing for a while. 5-6 months later, I rejoined the CBT, only to find those same exact game-crippling bugs still in effect. It sort of feels like you're doing all this reporting and 'helping' for nothing. I don't blame people in that scenario who complain loudly in the world chat. Its frustrating.
Most of the time it just feels like i'm playing a Beta just to help them test the server load, instead of anything to do with the game itself.
I have to disagree with this, and defend game developers.
Somehow game developers get demonized, especially when they put so much of their heart, souls and time into making a game. These guys work unbelievable hours for years straight trying to get a game shipped and somehow they end up the bad guys.
The reason why they "don't listen" to feedback is because Beta is usually so late in development and they already have a firm release date set. There isn't a single MMO developer who doesn't want more time to fix bugs before release but they're up against a financial wall.
In the case of a secondary publisher, they're told "release at X date because we're not giving you any more money." Along with contractual agreements and worrying about not being able to pay their staff, what choice do they have but to put their noses to the grindstone and get as much done as possible?
It's also a matter of many players these days being incredibly unrealistic. It's amazing how many posts I see about "You need to totally re-do this system before release or I won't buy it!" threads. Are these people serious? Game designs are locked down so long before release that it's a miracle at all anything can get tweaked between Beta and release.
I have to narrow this down to the instant self gratification that's starting to bleed into society where everyone wants it their way right now. People need to wake up to reality and know that it isn't possible. If you don't like a game, don't play it. It's that simple.
I remember a long time ago when I did a Beta test (I wish I could remember which one) where I had to sign a physical NDA and mail it back to the company before I could get an account setup. Such a massive change from the games these days.
Eh after beta testing AoC and Champions Online last; I gave up. Give them bugs and exploits, that's kind of ignored, but finish a quest and rank it 1 to 3 stars on the scale of fun.... "There's no 0 option."
The King will not hassle any more trying to help those who can't help themselves. Even when I got in to the RIFT beta it basically seemed like a "Watch us pat ourselves on the back" marathon than any sort of real testing market.
What's painful is seeing after launch ... They strive to try and barely change anything testers told them about prior.
I don't think the beta community has any blame on their heads.
It really comes down to how the company handles their beta, so in effect both opinions here are correct.
I beta'd STO and they didn't care what we said so with the exception of getting an early look at the game (and knowing not to buy it) it was just wasted time.
At the other end, I beta'd Gods and Hero's and can tell you Heatwave went out of their way to listen and make changes where they could. And if they couldn't they'd come out and tell us that. Communications in both directions; what a concept.
Another that I'd have to consider still in an extended beta would be Vendetta Online. They don't call it a beta but it pretty much still is and the dev's are willing to listen to the player base and make the needed changes, including adding in player produced material / missions.
Too often beta inviters go in with the thought of "I'm just playing to see if I should spend money on it" and that's not the how anyone should look at a closed beta. Open beta ...... well that just might be a different matter.
SWG (pre-cu) - AoC (pre-f2p) - PotBS (pre-boarder) - DDO - LotRO (pre-f2p) - STO (pre-f2p) - GnH (beta tester) - SWTOR - Neverwinter
As a sucker for immerzhuns I avoid beta's like the plague on games I look forward to play when launched.
But props to those avid beta testers who do their best to help polishing games and are forced to endure countless character wipes, pre-launch bugs, spoilers (what has been seen can not be unseen), repetitive and boring gameplay in stress tests of particular features and general incompleteness. It's a dirty job so I'm not doing it.
But *props*.
My brand new bloggity blog.
Beta testing has become nothing more than a marketing ploy, and as such, I use it to see if a game is worth buying most of the time...though I do report glaring bugs while playing if I come across any.
Fallen Earth, for example. I actually reported bugs like crazy because i wanted to help them suceed. They were doing something different with little resources. But most of the time, you aren't paying me to do this, so any kind of bug report I give is just a bonus for a developer.
Let's not kid ourselves here. Beta "testing" is just a sneak preview of a game just about ready to release nowadays. You people that go out of your way to try and break a game in testing need to find other things to occupy your time...like a job that pays actual money, for instance.
Joined 2004 - I can't believe I've been a MMORPG.com member for 20 years! Get off my lawn!
Fair enough, but then why beta test at all? Why not just call it a free preview? But then again, if your game is buggy, why would you give a free preview?
It's a proven historical fact that beer saved humankind.
The only beta I've been in that I considered "real" was my short time in Pirates of the Burning Sea.
At first I had just wanted in because I was so damn excited about the game, but it changed quickly once I saw the amount of interaction between the devs and the players. Didn't expect to see back and forth communication in the forums, irc channels, even instant messengers. It's easy to really test a game, and not just play, when you know the devs actually want your feedback and truly care.
As time went on the game evolved according to the topics that kept getting brought up, and it was pretty awesome to see. I've still not seen that amount of involvement and noticeable change by the end yet. Kudos to that dev team and the community, my favorite yet.
Another thing...I can't stand those people that are accepted into CBT phases, and demand this and this change or else they quit, like they have some kind of leverage. I understand complaining and pointing out serious, or even trivial, flaws, but don't quit because of it.
Nowadays I just feel like a number if I'm accepted. I'm here to stress test this feature or the server and no other finding I make is taken into consideration unless 50+ jump on the same bug or fault. There's just so many people now, it doesn't seem possible to keep track of each individual report.
I have only been beta testing MMOs sense 2006 ranging from my earliest that I remember, Dungeon's and Dragons to most recently Age of Conan, I felt that the Warhammer Online was the beta that took the most from their respective player's feedback and included it into the game. Even though the game turned out to be really bad, I jumped ship during the beta because of the direction they were going.
I like to beta test games mainly because I have an interest in a game, therefore would like to influence the game through giving feedback. But also, getting to play the game early is a very close second to the reason why I like to beta test. And if someone says they do not care about early gameplay, they are not being truthful.
MMOs Played: I can no longer list them all in the 500 character limit.
Ofcourse the beta community is to blame. I think all internet-based communities are filled with rotten, disgusting people. Try reading the World of Warcraft forums!
I don't think people should look at beta-testing as "free previews". Sure, you get to see how the game is going to turn out but the main focus is helping find the bugs and letting the company know how the players feel about the game in its current stage. I honestly don't believe the company would be offering beta at all if they weren't listening to the player's reports, but some people don't realize the complexity of making a game. You can't tell them to fix a bug and expect to see it fixed right away. Making and fixing such a game is an intense process.
It's sad, but the internet is becoming more and more of a horrible place each day...
The testing should be done by the alpha testers, Open beta is just a busniss move to get more players hooked.
The testing should be done by Alpha Testers, Closed Beta 1 with about 10-20 trusted, professionally-minded individuals who will actually TEST the game in all it's bug-laden glory, CB2 with 50+ WHO ACTUALLY CARE ABOUT THE GAME AS-IS AND NOT ABOUT TRYING TO INFEST THE FORUMS WITH COPY/PASTE IDEAS FROM OTHER GAMES to continue testing, CB3 to invite 100-200 for STRESS TESTING ONLY those areas that the Devs need input on and possibly deal with last minute bugs due to such tests, and THEN let in the OB 'normals', ignoring their whines and useless threats about leaving when they can't find their favorite Feature X from Generic Clone MMO #3.
Fixed.
First beta: Asheron's Call
Most Recent beta: Rift: Planes ofTelara
Not including open betas I've been in closed beta of the majority of MMO "AAA" titles (and a few others) since the late 1990s. I believe WoW was the only one I didn't participate in as I had no interest.
I do remember what you mean about being a close knit group. It even carried over into the launch phase of the game. Players seemed much more respectful, helpful, well-rounded back then. Reputation meant something because a) the community was small so everyong knew about a jerk, and b) games were made with greater emphasis on teamwork to achieve content (re: less accessible).
I agree that WoW brought in all of the people that didn't play table-top D&D every saturday night or that didn't know a hard drive from a motherboard. Whether that was a good thing or not is up to the individual. In my view it wasn't. But, it is what it is and there are, thankfully, other hobbies to spend money on.
As for beta testing, I'm not sure that I will apply for anymore. If I do it's limited to World of Darkness Online, ArchAge and World of War Planes. That said, if this genre turns back to making more AAA quality sandbox games, well, I may dive back into it. As it is now, my day riding rides at the fair is over.
"Many nights, my friend... Many nights I've put a blade to your throat while you were sleeping. Glad I never killed you, Steve. You're alright..."
Chavez y Chavez
I agree with you for the most part. I think that publishers are making a mistake inviting the masses into an open beta as a marketing mechanism. it can backfire too easily as most of the people signing up for OB have no clue what it really means to test and do QA. Also, the expectations they have, even though it is called a beta test, is that of a finished product.
The thing I difer wiht you on is the sizes of the beta tests. I think in the later betas 2-3 you need a much larger control group to not only stress test but to have a solid foundation of the group dynamic of the game and PVP featueres that might be introduced. Final closed beta should at least have enough testers to push the boundries of a server in every way, just not server load stress. In CB 3 for instance you should have enough testers to see a contigious number of people that will max server load for long periods. Today, that could be a couple thousand people depending on the configuration of the server clusters.
Just my 2 cents but I agree in general that there should not be open beta tests with these games. I applaud Bioware for making that good decision for SW: TOR.
Yes! And a little bit no. I tend to get into nearly every Beta I apply for, probably because I'm always on the prowl for new info on new games. I agree with your overall sentiment, that people don't seem to understand that a Beta exists so they can iron out as many of the kinks as possible before the public gets a hold of it.
Where I don't agree is that just as often, it's not the bugs that are complained about, but the actual core gameplay; parts of the game that can't be changed because the development is too far ahead. That's not even counting all the games that we get 'Betas' for that in the end, are only really testing their localization efforts, as the game had already existed elsewhere for a year or more (Aion).
Bugs are tiresome, but they can be worked through, and you can usually see someone bitching about them for what they are. They're the ones nerd raging on the forum incoherently. Gameplay mechanics are harder to change depending on the state of the development, and as people get into the betas and various stages of the development, you're going to get people who don't agree with the way the game is played. They'll recognise that, at this point, those mechanics aren't going to change, and that this is the game people will be playing. And they'll want to warn people not to waste their money and time on something they think is a failure. Sometimes they're right, sometimes they over react, but in the end I want to hear about it regardless. If there's some major clusterfuck happening, I'd prefer to know about t.
Honestly I'm not sure NDAs should even exist. I understand their purpose, don't get me wrong. People certainly don't want their games to be considered failures before they're even released, just because people keep bashing it in its earliest incarnation. But the NDA doesn't actually work; those people will post on forums anyway and unless they're stupid enough to use the same name, the company has no way of finding out who they are and removing them from the beta. The only ones the NDA truly prohibits are the ones who are loyal, adhere to it and most likely have something nice to say to defend it.
So really, what purpose does the NDA serve but to silent the wise and give voice to the foolish? I mean really, what are you doing to do, ban someone from playing a game they didn't like in the first place? Developers should recognise this and give up with the pretense, or at least let up a bit. Be willing to say "this is what we have so far, and it's not perfect! That's where you come in" and let players feel like they're truly a part of the creation of something. I think that, in the long run, would prove much more successful.
"Forums aren't for intelligent discussion; they're for blow-hards with unwavering opinions."
@dougmysticey;
Actually, I should have added that Open Beta would be the ultimate Server Stress Period before release, and not some marketing gimmick. I guess I was just reliving some of the Betas from 2000-2005, like Earth and Beyond, WoW, Eve, CoH, and the like, when Beta Testing actually meant something...:D
Thanks, mate..:)
Its an public request to get into TSW Beta...
Something like Sean Stelzer for president^^
"Torquemada... do not implore him for compassion. Torquemada... do not beg him for forgiveness. Torquemada... do not ask him for mercy. Let's face it, you can't Torquemada anything!"
MWO Music Video - What does the Mech say: http://www.youtube.com/watch?v=FF6HYNqCDLI
Johnny Cash - The Man Comes Around: https://www.youtube.com/watch?v=Y0x2iwK0BKM
I helped alpha test Quake 1 when all you could do was run around in an empty room, and I've done actual beta testing for half a dozen MMOs, but still the ones I really want to play never let me in until open beta at best... sigh...
still, I take the same approach as OP - I'm constantly submitting bugs in any game I play, and I'm familiar enough with what they're looking for to provide useful information. In some games, GMs seem to appreciate this and will contact me to test the problem more thoroughly. In others, they could care less...
meh... When I read the title I thought you were going to give us tips on how to get into more betas, what to include in applications, places to look for them, etc.
Unfotunately all I find is the obvious, oh look the population grew in 10 years, now there's more douchebags in it which ruin the experience/rep for the rest of us... As a result of it not being the 'good old days' anymore, beta testing sucks because it's not close-knit and that's why you're "Not a (real) Beta Tester."
Now, while I agree (can't expect anything from the instant-gratification generation), I do think there is a considerable minority which goes around policing such behavior, which posts on the forums, not to gripe, but to post guides (the very first) for classes/crafting/quests, help others, and also build those close knit communities within that larger sphere.
One last mention after reading some comments, the "Well devs don't do shit 'cause I reported something obscure way back when and it didn't get fixed," is just you feeling a little special about yourself. There is a multitude of factors that determined whether they took action or not that you aren't aware of, least of which could be your lone bug report got burried in pre-launch hubbub. If you really want to raise awareness, post on the forums, get other people talking about it, e-mail them, whatever it takes until you get a reply... You know, your job - what people did back when they actually cared about a game. If you don't care to put in the effort to fix something, why should you care to complain about it.
Developers have no one to blame about the current state of Beta testing but themselves.
Joined 2004 - I can't believe I've been a MMORPG.com member for 20 years! Get off my lawn!
I have beta tested many games going back to Anarchy Online (which just celebrated its 10th birthday). I always take beta seriously and report any bugs or potential game improvements that I find. The real pleasure is logging in after a patch and finding that the unusual bug you reported has been fixed.
Open beta is for some games a promo but for most it is there first real stress tests. I laughed when I read Saarosen's post above that said stress testing needs 100-200 people. You need 10's of thousands of people online now to do a true stress test. That is why open betas never go smoothly. A lot of silly players complain about the lag during a stress test.
I will continue to beta test games as I enjoy it and I hope my contribution helps in a very small way to make the game a success.
I have to echo a lot of the sentiment that has been expressed on this particular topic. I've beta tested a CRAP load of games. The majority of them have very little in the way of tools for beta testers to offer up their experiences in game. Some of them have a quick little multiple choice survey that you can cancel your way though. A few might require some sort of response. After awhile you realize that many of the people playing simply do not care. I'm a huge fan of helping develop a game, but: a) Give me the tools to do the job and b) Make Beta Test Reporting MANDATORY. You want to play? Then tell us what needs fixing. Beta Test != Free Trial. But if you're going to give me the tools and make me stop every five minutes to give you feed back then damn well listen to what I have to say. FAR too many games are being released broken. No wonder the players do not trust developers anymore, we've all been ripped off time and time again *cough* Warhammer *cough* Online Collectors *cough* Edition *cough*.... Do yourself a favor <insert MMO Game Company Here> LISTEN to your players. If the person is spouting like an a$$hat about how Necros should be "teh uB3r", then I can understand disregarding that person. Take their beta account away and give it to someone who does the job properly. But if the person is sending you what seems to be valid complaints and suggestions, make a beta event out of it. Get the players together (you guys run the servers, you make the rules) and discuss it. Get a concensus about the issue. It might be Bulls$#t, or it could be a game breaker. Doing so just might get you the return on your investment. Not doing so might have you hitting the unemployment line.