Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
Recent Beta Failures
docwalker
Member UncommonPosts: 12
Hey MMOers,
Recently posted as follows on Star Gate Worlds forums:
Hi SGW Fans & Devs,
I've been reading posts about game delays to do with Beta testing, I've played MMO's now for 8 years and over the years I've discovered what the Beta Testing select are made up from and it seems the way Devs choose them has fatal floors. Firstly they pick people that haven't had much MMO experience, fresh eyes you may think, but basically so in awe of what they're actually doing so less inclined to report Beta Bugs as an experienced veteran would. Veterans have had the experiences before and can get down to the job of helping to Develop the game. Secondly how can you take the judgement of a new MMOer rather than a veteran MMOer that has played all previous games. Devs arguments are that the new player would not compare to other MMOs so how can that help development, surely comparison is a good thing?! A trained eye would be a vast improvement rather than someone starting to play and Beta test for the first time who doesn't know what to do (doesn't even know they are meant to go do a mission or search for things ,explore etc).
Recently i made a few bad purchases due to the beta period of the game not fulfilling its purpose, obviously the betas i mentioned earlier. I could name thm but i wont, I'll just say rubbish planetary recruits sent through portals for war?! Legendary Barbarian nice graphics aimless?! and the latest release of the popular paper, pen and dice war game, £30 pounds a pop, not a happy player and you can't even take them back. So that's what sparked this post, a Plea to the Devs - time to start using the pool of REAL players for Betas else suffer the fate of above mentioned MMO's.
Docwalker (MMO Veteran)
Comments
Your poll should include an option for a mixture of new and veteran players as there are some flaws in your arguments.
Firstly, how do you define new players? Are we talking about people who have never played an mmo at all? If so why are they even signing up for a beta? I'm willing to bet you mean people who haven't played as many games or for as long as you have. Should anyone who hasn't played x amount of games for x number of years automatically be excluded? Surely not. Anyone who is interested in playing the game falls into the target market and therefore is permitted an opinon.
You claim vets, who are more experienced would make better beta testers because they can compare it to other games and are more likely to report bugs. Lets talk about the benefits of comparison first.
By the beta stage the game is already developed and mostly ready to go. Basically the dev's aren't looking for people to give them ideas about how things should be, but rather to let them know if things are working as intended. Would it really help them to have a flood of reports that tell them that PvP in WoW is so much better and here's why? Absolutely not, they are trying to test their game, not see what every other game does. They should already know since there is some manner of research before the game was made. Comparisons at this stage are worthless because there is no chance that they will pull the game back to make major changes. They don't want to know why others are better, they want to know how to make what they've already got work.
Secondly you said vets would be more likely to report bugs. Sorry, but that is untrue and I can prove it. Next time any game is released hop on any forum and complain about the bugs you find. You will have 2-3 people who agree with you and then a flood of veteran posters who call you all noobs because any l33t vet knows that no game is ever bug free. Sad truth is veterans have come to expect bugs in their games, hell they even defend them. So yeah they might report major ones, but there will be a ton of minor ones that they let slide by because they are too jaded and busy running around and seeing how this game compares with the 20 others they have played recently.
I'm not knocking vets, I consider myself in this group, but they are not without their flaws and elitist attitudes.
Finally you suggest paid beta testers. First of all, huge expense to pay enough people so game quality might actually suffer since fewer testers would have to be used. Also they would have to come from an independant company. Then you run into the politics of one tester working for competing companies and possibly leaking information. Plus if your being paid to play something, well it might bias your opinion a little.
I think the only choice is a healthy selection of newer and veteran players in the beta and also longer beta periods. The biggest problem I see is that devs are rushed to release games so they can start making money off of them and alot of the beta testing is sacrificed to make release scheduals.
i think it would be nice if the evaluation of beta games would come from both sides..
the noobs.. because this would determine how much the game would sell to people who are not really fond of gaming and there are more non-gaming people than gaming people so more or less, the non-gaming people are the target. the more noob-friendly a game is, the more non-gaming people could try it.
the veterans.. will tell how the game would go in the long run and if the developers are wasting time and money for investing on such a game. the developers would prefer games that would be a long-time source of income so they'd need the veterans to tell them if the game is something that would keep people hooked based from what they experienced with other games.
A lot of beta applications include areas that ask about prior testing and MMO experience, as well as your PC specs. When devs pick testers, they usually start with people they know or have tested for them before (getting invites for a new game from testing another game from the same developer, etc.)
Then they go through applications (or filter batches based on criteria) and look for people with good experience and a mix of people with less experience. They also want to make sure that they have a mix of PC specs across the board so that they can get as much functional data as possible. Yes, there are a lot of people that get into betas that should not be there, but in my experience, there is a reason why they allow the majority of testers to test.
Paid testers, if any, are usually in the pre-alpha/alpha stages just before bringing in batches of volunteer testers. Depending on funding, they are usually only to get the unpublished code up and running.
I think we should forgive those bugs or flaws in any Open Beta, for that's all Open Beta means. And there should be different people to join the test, regard of new mmos or veterans, for I think they would have different opinions to the open beta they are testing, which are both important to the devs.
Slightly off topic perhaps, but I think that one thing that stops people from bug reporting (whether they're vets or noobs) is the ease of reporting (or lack thereof).
If a player (whether in beta or otherwise) has to spend more than about 10 seconds on the administration of reporting (ie. all operations except for typing the actual issue out), you're going to turn people off and hence not get the issue reporting you want.
I've seen games where you have to type in your game location, the name of any NPCs around you, what quest(s) you were currently on etc. before you can type your issue. I've seen ones where you have to put down a lot of your machine spec. I've seen quite a few where you have to actually open up different software (such as browsers), log into/create accounts and go through a similar rigmarole. All this is admin and most of it can/should be dealt with in-game by the software. If you force people down these routes merely to report issues, they'll find that they have better things to do with their time - like hunt the next orc instead. Also, if you make people take time out and they're currently in an in-game dangerous area, they're not going to risk their character to report the issue.
I once worked as a CS director for a MMO and we had a number a mechanisms to report issues - the simplest was merely typing in /BUG <here is an issue>. We then had someone employed to look at the bug reports and channel them off to the appropriate area for resolution. Players loved it as they could contribute to the quality of the game without losing immersion.
In RL, I'm an IT Manager and often deal with similar issues on applications - the bottom line is that if you want your users to help you in this way, make it easy for them to do so!
To use a certain 4chan phrase..."Obvious troll is obvious".
Seriously, with such an elitist, arrogant tone, there's no doubt this post was meant to be inflammatory. Just take a look over at the official forums where this was crossposted; everyone there saw this was nonsense.
I also don't think I need to say how obscenely wrong the idea of vetera testers being superior. It seems to me this community already knows how blatantly false that is, and more then a few people here have said as much.
Many of the newer games got into beta way too early and should have been in alfa testing with payed professionals a lot longer. Arenanet showed the way, Guildwars had fewer buggs at the start of the beta than most games at release.
After that I want 75% veterans and 25% new players to get some fresh perspective.
But this still won't help against releasing 6 months early which is the biggest problem today.
LOLZ. What's a fatal floor? Is that like when you trip and fall, and then you die?
Playing: Various Stuff
Played: AC, AoC, ArcheAge, BDO, CoX, EQ2, Eve, GW, LotRO, Ryzom, SWG, SWtOR, Vanguard, WoW, Etc.
I agree with some but not all. I think that yes there should be a lot more experianced people in betas than there are noobs, but the simple fact is that you want those noobs in there becouse the majoirty of the peeople that are going to be playing the game are noobs. As for paid testing I believe they already have something like that, its called quality assurance if I'm not mistaken.
Cool topic!
As an indie dev, I can see both sides of this. I totally agree that a noob-ish player will never be able to spot bugs (or worse design flaws) for an MMO given a short period of time. MMO's are complex programs. A noob is still figuring out how to play the thing, and NOT doing their job of quality assurance testing.
From the other side, a developer has a responsibility to take a preassigned vision to completion. A veteran player who is totally inflexible regarding game mechanics is just about useless as a beta-tester. If "Brand-X" MMO introduced the concept that day/night is 24 hours IRL, that doesn't mean that every new MMO being created must adhere to this. Not all things reported will mesh with the planned vision, and this can upset the relationship between dev and beta-tester. NOTE: Of course there are times when dev "Just isn't listening" because of whatever reason, and concerns voiced by a beta-tester that SHOULD have been fixed are simply ignored. From what I've seen on the boards, this happens far too often.
I'll go further than the OP and touch on something I've observed:
It's in beta to look for bugs, NOT to add new gimmicky features. Adding bloat without finishing the bugs only produces a crap program when it could have been exceptional. Cool but buggy as hell... does that ring a few bells?
Ken
RealmLords MMORPG
I disagree.
You are assuming, if I'm reading you correctly, that since there are issue once a game is released that the beta players just didn't do their jobs. I can speak from first hand knowledge not only with mmo betas but single player game betas that this isn't the case.
All beta testers need to do is test. That is it. It is up to the developers to do what is needed with those results.
I remember being in the beta for Masters of Orion III. I loved "II" so I was beyond excited to be included. We tested and submitted bugs and suggestions and gripes and in the end, when a very buggy product was released, what did we see on forums? "What were the beta testers doing".
We were doing our job. We were not the developers and could not make the changes ourselves.
Developers are not stupid. they know their product and they know the issues. Especially with glaring issues, many you will find in the recently released games that had problems.
Beta testers are just one tool in the dev's arsenal. In the end it's up to the dev's.
Godfred's Tomb Trailer: https://youtu.be/-nsXGddj_4w
Original Skyrim: https://www.nexusmods.com/skyrim/mods/109547
Serph toze kindly has started a walk-through. https://youtu.be/UIelCK-lldo
The other problem Devs run into is their own deadlines.
Business types are breathing down their necks to "get it done already", and sadly some of them come to the conclusion that it's ok to release a bug ridden peice of crap, and then patch it later. They think "at least we'll get their money up front for all the hype we've generated, and we can win back the people that are pissed about the bugs by getting the patch out."
Or at least that's the impression I get sometimes..
I think Betas have become more of an advertising tool instead of what they use to be... testing the game
In a game that I'd have to pay to play, I would rather have the developers hire beta players. People who are paid to do their job will do it more willingly than any random jerk who wants to just have a good time.
Seeing as this is (apparently) concerning open beta's, that's just what it is. An open beta. People are free to join, what are you going to do... ban people who haven't found enough bugs in your game?
------------------------------
Man is the loneliest being on earth.
Veteran players for sure
I'd have to say Veteran players. You will usually see paid professionnals in huge companies (Ubisoft, EA, etc) while other company, with lower budget than these gaming monsters, prefer all players (which is a big mistake).
Am I one of those that hate new players? Not really. Company usually use them to check the server capacity/reaction. Though I do admit that I prefer Betas with pre-selected testers (selected throught interviews) as it is easier to trust the judgment of a dev more than a player. If you compare the bug forums from 2 different games using 2 different tester selection (ie Beta Key Giveaway/Free For All vs. Interviewed Testers), you can quickly see that the game using Interviewed Testers is : clean, coherent, and informative (meaning that some tester are able to easily define/locate the problem) while games using a Free For All are filled with complains, cries, etc and very few important informations (at least there's always the light somewhere that can identify the problem and possibly find a temporary solution but still) and because of this system, it is quite hard to keep track of important bugs that needs to be fixed.
I must clarify something at this point. By new player I mean: A player that hasn't tried a lot of game (different style, gameplay etc) or doesn't have a lot of knowledge in gaming. Veteran players are the complete opposite; by tasting a bit of every game there is on the market they are able to bring decent suggestions to increase the gameplay experience, know where to look for bugs and possibly bring a solution to fix them. As for paid professionals, unless they are paid by huge companies, even if they can afford it, they usually won't because it allows them to save money.
So basically, I believe Veterans should take care of the Closed Betas/Open Betas while others wait for the official release.
Note: I do not believe that Betas checked by Veterans or paid professionals are flawless, but at least there isn't as much bugs as I believe there would be.
Note2: I 'm serious, I do not hate new players. It's the opposite, I like to help them when I can and hold their hand until they grasp the game and even if some can do a good job like a Veteran it's a mere minority and that's why I believe they shouldn't be allowed to join Closed-Beta. Then again...this is just my opinion facing other players opinion so it's not worth much.
In a sense, the programmers also think that way, but different.
A programmer can cut down some features to release a bug free game now (withouth some features), or can continue programming to release a buggy program latter (with all features). So dev's will like to release now (with less features) and add then latter.
There are also something called "Developpment Hell", wen a program is not advancing forward, but backward. A program could advance to completion and less bugs, but It may also advance in other direction, to more bugs and less completion. Something as big as a MMORPG can suffer this more than other software.
Creating another MMORPG is not a good idea at all. As are *too* expensive beast. But WoW attract the money so more people will still try to become rich with it.
If there's one thing I've learned, there's always room for one more. There are literally dozens if not hundreds of MMORPG ideas out there that haven't been developed, and some of them may actually be good.
I certainly don't think my current project is that unique, but it has the potential to be. It's not going to turn out to be the next WoW, and I'm sure it wont even come close to competing with EVE, I'm not trying to.
What I'm after is something that I can call my own, make it a little different than the norm, and hopefully attract a fairly good size fan base. If I happen to make a little money at it, so much the better, but I'm not naive enough to think I'll get rich with it. I gues I'm one of those lucky few indies that develops games both for fun and for a living. I'm enjoying the process beyond words, and that keeps me motivated.
good topic, very interesting, and i a noob
My Recent Blogs
My Games List http://bestbrowsergameslist.blogspot.com/
My Game Blog http://bestbrowsergames.blogspot.com/ Welcome to comment your opinions
My Twitter http://twitter.com/leagendry
In a sense, the programmers also think that way, but different.
A programmer can cut down some features to release a bug free game now (withouth some features), or can continue programming to release a buggy program latter (with all features). So dev's will like to release now (with less features) and add then latter.
There are also something called "Developpment Hell", wen a program is not advancing forward, but backward. A program could advance to completion and less bugs, but It may also advance in other direction, to more bugs and less completion. Something as big as a MMORPG can suffer this more than other software.
Creating another MMORPG is not a good idea at all. As are *too* expensive beast. But WoW attract the money so more people will still try to become rich with it.
KISTech hit the nail right on the head. As a programmer, you have absolutely no say what features will stay or be cut, thats up to management. They decide the deadlines, no matter how ridiculous they are, because they're not the ones who will be staying until midnight.
When you see a buggy game, or a game thats been delayed over and over, and that's bad management, 100%.
Multiplayer Games!
I think veteran players have higher standards.
So maybe it depends on what your goal is, like if you just want to bring another mediocre game on the market that already exists in a dozen variations or if you want to release an inventive high quality game that is able to impress even people, who have seen it all. In the last case, you better ask some vets for their opinion already in the early stages of the game.
I'm one of the guys, who has played or tried maybe a dozen of MMORPGs and I'm not easily impressible anymore. So I'd say, if someone wants to develop something cool and inventive, I could be a good beta tester, but if the game doesn't want to set standards in gameplay, but just wants to be another clone, then I'm not a good beta tester, because after 1 or 2 weeks, I wouldn't log in anymore anyway.
I don't waste my time with testing a game that doesn't try to set standards in some areas and offers some gameplay that's still interesting for experienced people like me.
Actually I have a lot of gaming friends, who think the same. They are in a waiting stage. They don't bother to try another boring clone of game XY anymore, even if they can play that for free, they are hoping to get a beta invite for some inventive future games that might still have something to offer for the experienced player.
Vets like me only want to help improve games that have potential I guess. Something that we would like to play in the future, not something mediocre that we wouldn't subscribe to anyway.
Paid beta players, those that dedicate the most time and are the most vigorous of testing. Actual players are more likely to simply play, not test and evaluate. Checking in EVERY nook and cranny for a bug. I do believe that it should have at least one round of testing that is by players, both a mix of new and old.
____________________________
Telthalion Rohircil - Guardian - Elemandir - Lord of The Rings Online
---
== RIP == Torey - Commando - Orion - Tabula Rasa == RIP ==
---
Jordaniel Torey - Navy Megathron, Active Armor Tank - Tranquility - EVE Online
---
Torey Scott - Rifleman - Fallen Earth
____________________________
"I know not with what weapons World War III will be fought, but I know World War IV will be fought with sticks and stones." - Albert Einstein
I don't agree at all. For those with experience of how the gaming industry actually works, they know that beta testers are selected based on the dev's target audience.
They almost always have a mix (a spread) of experienced, non-experienced, free, and paid beta testers.
Now, I think that it is true that there are definitely times when the devs choose to listen to the wrong testers.
Each type of tester will generally provide a different type of feedback. Unexperienced / free testers are generally good for telling whether or not your game will fair friendly towards new players, and whether or not it is 'generally' fun. Experienced testers will generally give more indepth feedback, and let you know more about what's working / not-working, and why. Paid testers are generally good for focus testing bugs and features into the ground, and reporting what needs to be fixed.
Most games have some mix of the above spread, sometimes replacing free testers with the devs' family members. They don't always get the mix right, there is almost always a mix.