Betas WOULD be useful, if Developers would listen. Which they don't. Maybe at some Indie companies. But in the big ones, the Divas... err Developers have their "vision", prolly by overuse of certain substances. I am quite disillusioned about that part.
But for me as customer - I don't but MMos I haven't seen before anymore.
People don't ask questions to get answers - they ask questions to show how smart they are. - Dogbert
The only time I have ever felt I had an impact on a game as a tester was the very few times I have gotten into alpha stage testing. Often, alpha testers are re-invited into closed beta, but I've never seen the communication and interaction work well at that point to release. And part of the problem I have seen has mostly revolved around the tools for testers. I mean seriously, needing to be on vent, on MIRC, on the game, AND file a formal bug submission on a separate site at the same time you are reformatting jpgs into proper size and format, etc for upload is just crazy. With a small crowd in alpha, that works, but once the crowd gets larger, forget it. Give me some decent in-game tools and I will test your game to no end. Until then, it is free play time. Seriously.
But I digress. From "closed beta" onward, I feel the testing is non-existent.
I think beta testing is fine the way it is.. I know a lot of people see it as a gimmick or marketing tool.
Well, it might be but there are a lot of other things that can be tested. Most have mentioned stress testing and other hardware testing. Self explanatory I think.
The other thing that probably is done (i'm no dev so I don't know for sure) is data gathering. Every time someone does a quest, kills a mob, visits an area the team gets reports one what content is hot and what is not. This gives a last minute chance for the devs to tweak some content that may not be utilized.
Then there are the forums for beta and people are usually very vocal about what they like and don't like. Most alpha testers and closed beta testers have a lot of investment into the game - playing it regularly and is very involved in how the systems work. Open beta gives the devs a chance to see someone reaction to the game from a NEW player point of view. This way the Devs can iron out any problems with players being confused as to what to do or what something means... What comes to mind is EQ 1 --- I jumped in without reading the manual on how to play and I couldn't figure out how to get quests or sell things. It seems ridiculous now looking back but it was the first MMO I played. If players have a hard time figuring out how to do things they will not stick around in today's market.
But I think the main thing is for the investors.. they are taking the temperature on their investment. Seeing how hot the game is and to see what kind of return they'll be getting. Almost like a mid term for the dev team they hired.
Beta's are a lot of things all rolled into one. I'm not sure why anyone would be against playing a game before it's released. Try before you buy and excitement. I usually take that chance to convince my friends on why they should play the game as well. (I usually wont play a game if my core group of friends don't play) I think this is good marketing in favor of the player.
Maybe 'beta' is the wrong term to be used -- but I'm all for the open 'beta' stages.
As a long time software developer (as in Computer Programmer for 32 years), Beta are meant to final test code but sadly in the case of MANY online games that doesn't always hold true.
Personally have seen a lot of so called "Beta" games that weren't ready for "Pre-alpha testing" stage.
Then some games that are in "Beta" were so well done and well prepared that they should have already been released.
Others like Freelancer are a great game but missed the mark in some area like No Joystick Support in a Flight Simulation. It is sad too... there are SO many joystick examples that use DirectX that an oversight like that is without a decent excuse. It just doesn't take all that long to add support for a Joystick.
There is a fine line between a product that just needs some minor issues fixed and one that is just hasn't had proper pretesting.
You can't test every possible set of hardware out there... it is just not practical. So you give it your best shot andif you did your homework right it shouldn't be all that hard to fix bugs that are well documented that are reproduceable.
Betas also tend to get people excited about playing what is felt as if it is nearly "ready for prime time".
If you get chosen to play test in an open beta or closed beta, please make very good notes if you encounter bugs or strange occurances when playing the game. The more accurate the details on how the issue / bug occurred along with how the it occurred, the better the chance that the bug will be fixed.
Don't expect instant results however.
I have seen other games where WELL documented and very reproduceable errors in code are just flat ignored. That's horrible but happens a lot. Rappelz has a lot of these still with no hope for getting them fixed it seems. They just keep messing with all the skills of all character types and pet types instead of focusing on fixing glaring issue.
Any huge program has BUGS in it... Some found that can be repeated and thus be fixed... others that have yet to be discovered and those that people use as exploits because of buggy software in some particular area.
Any long time gamer will appreciate a well done game. Many that use the Unreal Engine have done very well based on a well coded and well implemented game engine. By the way, this engine has been released for people to try their hand at writing games:
Balancing is something that happens through the lifetime of an MMORPG. I dont think I have ever seen an MMORPG that did not suffer through balancing after release.
There are different kinds of re-balancing though: changing the parameters and changing the equations.
Tweaking damage output from 9 to 8 on something will happen all over the place. But I'd argue that changing the entire formula of how a skill interacts with other skills and the environment is not a re-balance, it's a redesign and with the terminology I'm used to, that would happen in alpha, not beta.
I tend to see live publishes as following the same cycle as any other product release, just on a smaller scale.
(as for the viral marketing aspect of betas ... well, it might work, but if you attract people who are there to actually sincerely test, they might get annoyed at being played as agents of infection)
Well that depends on the developer behind the beta. If the beta is run well, and the testers are given tasks... then this basicly helps prevent major bugs and issues from slipping through. I can remember my time in the early beta of Starcraft, by Blizzard. They ran a very good beta, they had target goals, and things that needed testing in mass. They would be general terms, like test the Zerg vs Human... or test this feature tell us if it works in game.
Some betas now are run like that... some features are being put out to be played with by the community on a bi-weekly basis. Bugs are found, they are then fixed. In games where I have seen a straight to public beta with no structure other than a publicity hype to try and pre-sale the game have all ended in fail. Pirates of the Burning Seas was one that pops to my mind. A great game concept. Could have been the next big SMASH hit game, something worthy of Sid Miers "Pirates!" from the old C64 and C128 days. But instead turned out to basicly be a unfinished product that they were trying to wow the masses with their pretty features.
Then there are other games that had no open beta, and a VERY short beta that have had perhaps some of the worst launches in history.... WWIIOL, Anarchy Online, Darkfall. These were calculated risks, in some cases, and just necesity in others. That really didnt pan out well for them in the short run, but for two of those on the list, have had a great run.
Beta has a use... but only if the developers use it for more than a get last minute cash gamble or a publicity stunt to get the game name out there for free.
Because there will always be the Free play beta player, who lives from open beta to open beta rarely if ever subscribing to anything. That will always end up destroying what could be a great game with stupid suggestions done in mass.
I can remember 13 years ago there was maybe a half dozen MMORPGs on the market... Ultima Online being the biggest. Now we have more MMORPGs than you can shake a stick at, and probly less than 5% of which are worth the binary code they are made out of.
Betas applied as a testing tool, like they are supposed to be, are an invaluable service to ALL gamers and the people that participate in them by submitting 100s or 1000s of bug reports and edits are argueably unsung heroes.
Betas, as they are applied today, are really just extremely long previews. Add in the fact that some companies have hit on the idea to charge for them (sneaky via lifetime registrations for other games) indirectly and you've got Betas being used to promote a game, not polish it. It really is the worst idea ever, because either A) everyone thinks its awesome and should be released already or it isn't finished and people post 100s of forum subjects (instead of bug reports) about the suckage of the game. It really is lose/lose.
I honestly think there should be NO open betas. Have 12,000 closed beta slots if you need to test servers ... but don't invite a bunch of people in and then make them watch your game crash to desktop or queue for three to four hours a night.
Betas are only useful if they are truly Betas. So many Betas now days seem to be nothing more than free trials. The company gets to stress their servers and check the code against a bunch of different PCs and the players get to take a look at the game and decide if its worth their time. No real testing is done by the majority of people under these circumstances.
A real Beta tester should be listened to by the company. They go out and see how things work and will let you know if something is wrong. They write detailed bug reports, a lot of them and they fill the forums with useful posts about game play, about the feel and the immersion.
What I see now days are kids playing with a key they got because they pre-ordered another game by the company or got a collector's edition. They whine if they lose something, cry when MOBs get their final difficulty increase, throw tantrums on the forums over trivial crap. Yeah, your servers are getting stressed but so are your GMs and Moderators...
Companies get free employees if they do things right, such as actually requiring an application to be filled out that has a lot of essay style questions that will really show what kind of Beta tester you are getting. Otherwise it just seems to be a waste of time.
I respect your right to voice your opinion and reserve the right to blow it right out of the water
I found the article to be interestingly vague, but at the same time quite self-enlightening. Let me elaborate if you will. I have taken part in a seemingly endless list of mmogs,mmorpg, mmo games. I have been doing so sing the very first Guild Wars was beta tested(which was my very first beta test). At that time I was a wide eyed, stoked wanna be beta tester wih all kinds of hopes of really helping the end product and its quality.
fast forward, 50( or there abouts) beta tests later. I still,in the deepest recesses of my mind have that child like excitement of having a real impact on a games development but just a bit tempered. I can't help but think that every single beta test I have taken part in did have its purpose, especially the 1st closed beta. everyone running around trying to find glitches, bugs, rips in the graphics, trees you could walk through, non-responsive NPCs, buildings mysteriously elevated off the ground, misspelled words or poor gramer. In all the beta tests I have taken part in, very few times have I actually found any bugs,etc. More importantly, or so i thought, was giving input on the game's mechanics, UI setup.etc. I and many others along the way offered "player feedback" on what could make it a better,user friendly , easily adaptable game. Well, alas when the final product came out, it hadn't(for the most part) changed one iota.
EXAMPE: CrimeCraft- I took part extensively in the beta testing of this game and quite frankly was quite disappointed. Throughout the testing, developers seemed to make their presence known in the forums asking for feedback. many of us gave great ideas for making the game exciting. Thousands of ideas will ( more likely than not) always bring out some great widely acceptable ideas that could actually improve different aspects of the game. Hence, we were lead to believe that our ideas meant something, were of interest or at least were heard. I can safely say .0137% of the plausable ideas were actually considered, and of that.0137%, .0035% was actually implemented(so to speak).
In conclusion, I have learned from experience that generally speaking, Beta testing is more often than not used as a marketing tool(sadly to say). Yes, many want us "beta testers" to believe out input matters, but in all actuality it really doesn't. The product has already had the ground work layed for the full mechanics, graphics,etc of the game and te developers have no interest in changing anything. So, now I use beta testing for early access to a new game. yes I still look for bugs and such, but mostly for access to a game early on. Which is what they wante all along. GEESH!! full circle.
There is a drawback to having beta testing, you get the bad apples. Often times people who are cheaters, exploiters, hackers will use a MMO Beta Test to find bugs and exploits and use them on the live servers and they find it funny to ruin the game for everyone. They think it is cool to use bugs and exploits from the beta on live to get ahead.
I'd like to relate my experience with one closed beta test in particular as this one was the only one I found to be an actual "test". Now I've participated in many so called beta tests, some labeled as "open" or "public" and some as "closed" and they all pretty much conformed to what was described as mostly a form of advertising, a "try before you buy" scheme but called a "beta test" to attract people that might not otherwise download a "trial" version" participate as a chance to be a "tester". Aion is the latest example of one of these that I've participated in. But I had been a long time addict to the original Everquest and when the chance came up to get in on the very first of the closed beta tests for EQ2 I jumped at it. Before we were able to download the software there was the usual NDA form to sign but there was also one with a description of what testing would involve and basically it said that if you were looking to simply get a chance to play the game early you would be better off waiting for when the open beta launched as that would be when the bulk of the server stress tests were to be conducted and the only requirement for those were large numbers of players and that this first series would be many hours of performing mostly boring, repetitive tasks in order to test things like game mechanics, the AI and the like. And true to their word that's what most of it was. After logging in and creating characters we would be able to begin playing as if it was just a stress test but then came a constant stream of programmers that would ask for a certain number of volunteers to test how the AI reacted to certain situations or they would ask for a guild to test a particular raid they were working on and there was complete interaction with these programmers. They weren't kidding about it being boring and repetitive but they joked around with us and did their best to make it as much fun as was possible considering the task and I have to admit that this was the most rewarding and enjoyable beta test I ever participated in. We really got the feeling that we were an important part of the game's development. As a sort of a reward for our help, when the closed testing ended and the open testing began there was a "credits" screen that came up when you logged out that listed everyone's real name and their character's name and every guild that participated in the closed test. It was discontinued when the game went live but it was a great feeling to see my name come up on that list and to this day I have never been a part of another test like that again to my great disappointment. I keep applying to many of the closed testing offers that I can in the hopes of getting involved in something like that once more. It's a shame more companies don't do it, or if they do, then I just haven't been fortunate enough to have gotten an invitation.
I think betas are useful for both the consumers and developers. For the developer side, they can use this opportunity to gauge the communities reaction of their nearly completed project, market their MMO to the masses with a taste of what is to come and attempt to find and eliminate bugs before or shortly after launch. For the consumers its an opportunity to become familiar with the game they have been waiting for, help developers locate and remove bugs and help the community grow through word of mouth (or forums such as this).
"If we don't attack them, they will attack us first. So we'd better retaliate before they have a chance to strike"
IMO they need to stop allowing Pre-Order customers into closed betas. Closed Beta testers should be the type of players that will actually test, and submit bugs, suggestions, etc. I also feel that testers who haven't logged in for a certain amount of time, or have never submitted feedback should be removed from the test. I feel it should be important to devs to weed out people who just want a preview.
Yup! It also depends on the studio. I have been in some betas that have me asking myself "what am I really doing here?" and otehrs where feedback was actively sought by the developers. If nothing else beta testing often helps me to make a final decision about a title and saved me money.
I know beta testing is a necessary evil, but recently they've become a form of RMT in my opinion. You pay extra money for the collector's edition or preorder and you get to play the game before the majority which have to wait for the release, thus gaining a slight advantage. It's not a huge deal, but it's not really fair either.
Ok, maybe you are right, some of them are just RMTs. You have every right to not like it.
But for heaven's sake it's not fair or unfair. It is what it is, you can buy it or not. Unfair would be that you MUST buy it regardless. Anybody can try to sell anything they want, that's the marketplace. Don't buy it if you don't like it.
I am really tired of hearing so many people on these forums talk about fair and unfair, and about 1 per cent of the time that's really the issue. They have every right to market junk and you have every right not to buy it.
Beta testing has become another form of P2P. In fact, MMO's are the only product I know of that people will pay to test. They charge, one way or another, for giving people the honor of testing their unfinished, buggy products and then release the game unfinished and buggy and they do this simply because they can. It will continue as long as people are willing to hand over the money.
Beta testing has become another form of P2P. In fact, MMO's are the only product I know of that people will pay to test. They charge, one way or another, for giving people the honor of testing their unfinished, buggy products and then release the game unfinished and buggy and they do this simply because they can. It will continue as long as people are willing to hand over the money.
i love where people are getting this crap from. seriously, how many games recently have even had any form of "paid" beta testing? maybe a handful, but nowhere near even half of them. so how is it that beta testing has become a form of RMT or P2P?
Would you all prefer they just cut out beta testing and just launch the games asap without any form of beta so you can all whine and cry that the game wasnt ready for launch because of bugs, server issues, etc and they should have.... zomg beta tested it first!
Beta testing has become another form of P2P. In fact, MMO's are the only product I know of that people will pay to test. They charge, one way or another, for giving people the honor of testing their unfinished, buggy products and then release the game unfinished and buggy and they do this simply because they can. It will continue as long as people are willing to hand over the money.
i love where people are getting this crap from. seriously, how many games recently have even had any form of "paid" beta testing? maybe a handful, but nowhere near even half of them. so how is it that beta testing has become a form of RMT or P2P?
Would you all prefer they just cut out beta testing and just launch the games asap without any form of beta so you can all whine and cry that the game wasnt ready for launch because of bugs, server issues, etc and they should have.... zomg beta tested it first!
How many games release in a decent form after the beta testing? I have an idea. How about actually testing the game in house and actually fixing the problems that are found? What a radical concept that is, actually selling a completed product.
Lot of excellent replies already. I'd just to like to add one additional thing. Companies need to learn about communications and expectations.
If you communicate clearly what something is about and follow through with it, you usually won't upset people. If you remain vague about it though, people might assume it to be something else. And when it doesn't turn out how they expected it to be in the end, they will most definitely be upset. It really doesn't matter if you're buying a product at a store, buying a meal at a restaurant, or doing a beta test for a game. Be as clear and concise about what you are offering and follow through in delivering what you're communicating. The closer the two match, the happier people will be.
Today people assume a beta test is to fix bugs and address gameplay issues. If you're a developer and you decide you can't address gameplay issues in a beta test (but only can address bug fixes) then make this absolutely fricken clear from the very start, so that every beta tester knows this. If gameplay issues are only addressed in alpha testing, then you better get the best focused testers you can possible get and they better represent your core audience as much as possible.
"Open betas" have become a form of free viral advertising. In the past, they were exactly what the name implys, these days they are just a chance for players to get a taste of the game, which as Tabla Rossa found out can work against you as well as for you.
For my part, it is very rare for me to volenteer to beta test anymore, I will not normally work for free. And real beta testing is mind numbingly dull. I can remember one game where I did the same quest over one thousand times, mostly to reproduce a very rare error, it turned out to be caused by one piece of armor and a weapon. You needed all three for the crash to occure. Anyways. If you want to beta test, pay your testers. If you want to give players a chance to play the game, call it what it is, and don't try to make it sound like part of the already completed development cycle.
Betas are a must wether they are open to the public or not. The primary goal in beta is to fix issues, nothing more nothing less. It shouldn't be used to form an opinion of the game or used as advertising. How can one possibly form an opinion of a game that doesn't even work right ? Thats why we have beta, to fix issues.
Betas are a must wether they are open to the public or not. The primary goal in beta is to fix issues, nothing more nothing less. It shouldn't be used to form an opinion of the game or used as advertising. How can one possibly form an opinion of a game that doesn't even work right ? Thats why we have beta, to fix issues.
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
Betas are a must wether they are open to the public or not. The primary goal in beta is to fix issues, nothing more nothing less. It shouldn't be used to form an opinion of the game or used as advertising. How can one possibly form an opinion of a game that doesn't even work right ? Thats why we have beta, to fix issues.
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
How many betas do you actually know that went extremely well ? We have failed mmos because devlopers do not fix the import issues before launch. Perhaps I look at it from a compuer standpoint because I worked in that field for quite awhile. In the real world when you work in betas/tests its to test the product and fix it. I'm not quite sure you can actually make beta into a marketing tool when only serveral few mmos have had excellent launches.
Betas are a must wether they are open to the public or not. The primary goal in beta is to fix issues, nothing more nothing less. It shouldn't be used to form an opinion of the game or used as advertising. How can one possibly form an opinion of a game that doesn't even work right ? Thats why we have beta, to fix issues.
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
How many betas do you actually know that went extremely well ? We have failed mmos because devlopers do not fix the import issues before launch. Perhaps I look at it from a compuer standpoint because I worked in that field for quite awhile. In the real world when you work in betas/tests its to test the product and fix it. I'm not quite sure you can actually make beta into a marketing tool when only serveral few mmos have had excellent launches.
To your first point about major issues not being fixed before launch --- this is most likely not a tester issue this is a issue with money and the investors wanting a return on the money. Games are getting pushed out the door before they are finished or ’polished’ because the games development costs are insanely high.
Those few games that you have mentioned had good launches also had great public beta's... so I’m not seeing the correlation between the open beta process and bad launches. WOW and LOTRO had great launches and they have very public beta programs. I'm down for getting the serious testers based of the 'on my honor' applications for previous beta work as well as getting your name known with the company as a good tester.. those are needed and should be included. But i'm not sure why it's detrimental to have fluff testers to get other feedback as well - as far as random bugs, game play, content, level progression. The feedback that these people will provide will let the developers know what a causal player thinks of the aforementioned items.
I'm not saying only rely on the public fluff testers -- companies still need a solid group of testers doing the hardcore testing. MMO's are very dynamic and need to make sure all aspects are covered. Why only focus on fixing bugs during beta when content might be sub par, advancement might be poor for leveling and geographical advancement might not be intuitive. Getting this feedback from fluff testers is going to make the devs more aware of how general players will perceive their game.
Also, games keep track of each quest that is done, how many of X item are floating around. More testers can give developers data on what’s working what is not / what is being utilized and what is not.
Betas are a must wether they are open to the public or not. The primary goal in beta is to fix issues, nothing more nothing less. It shouldn't be used to form an opinion of the game or used as advertising. How can one possibly form an opinion of a game that doesn't even work right ? Thats why we have beta, to fix issues.
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
How many betas do you actually know that went extremely well ? We have failed mmos because devlopers do not fix the import issues before launch. Perhaps I look at it from a compuer standpoint because I worked in that field for quite awhile. In the real world when you work in betas/tests its to test the product and fix it. I'm not quite sure you can actually make beta into a marketing tool when only serveral few mmos have had excellent launches.
To your first point about major issues not being fixed before launch --- this is most likely not a tester issue this is a issue with money and the investors wanting a return on the money. Games are getting pushed out the door before they are finished or ’polished’ because the games development costs are insanely high.
Those few games that you have mentioned had good launches also had great public beta's... so I’m not seeing the correlation between the open beta process and bad launches. WOW and LOTRO had great launches and they have very public beta programs. I'm down for getting the serious testers based of the 'on my honor' applications for previous beta work as well as getting your name known with the company as a good tester.. those are needed and should be included. But i'm not sure why it's detrimental to have fluff testers to get other feedback as well - as far as random bugs, game play, content, level progression. The feedback that these people will provide will let the developers know what a causal player thinks of the aforementioned items.
I'm not saying only rely on the public fluff testers -- companies still need a solid group of testers doing the hardcore testing. MMO's are very dynamic and need to make sure all aspects are covered. Why only focus on fixing bugs during beta when content might be sub par, advancement might be poor for leveling and geographical advancement might not be intuitive. Getting this feedback from fluff testers is going to make the devs more aware of how general players will perceive their game.
Also, games keep track of each quest that is done, how many of X item are floating around. More testers can give developers data on what’s working what is not / what is being utilized and what is not.
WoW had a horrible launch and yes Lotro had a pretty good one but that is a dime a dozen. You also can't use betas as a marketing tool when most mmos since WoW have been mediocre at best.
Comments
Betas WOULD be useful, if Developers would listen. Which they don't. Maybe at some Indie companies. But in the big ones, the Divas... err Developers have their "vision", prolly by overuse of certain substances. I am quite disillusioned about that part.
But for me as customer - I don't but MMos I haven't seen before anymore.
People don't ask questions to get answers - they ask questions to show how smart they are. - Dogbert
The only time I have ever felt I had an impact on a game as a tester was the very few times I have gotten into alpha stage testing. Often, alpha testers are re-invited into closed beta, but I've never seen the communication and interaction work well at that point to release. And part of the problem I have seen has mostly revolved around the tools for testers. I mean seriously, needing to be on vent, on MIRC, on the game, AND file a formal bug submission on a separate site at the same time you are reformatting jpgs into proper size and format, etc for upload is just crazy. With a small crowd in alpha, that works, but once the crowd gets larger, forget it. Give me some decent in-game tools and I will test your game to no end. Until then, it is free play time. Seriously.
But I digress. From "closed beta" onward, I feel the testing is non-existent.
I think beta testing is fine the way it is.. I know a lot of people see it as a gimmick or marketing tool.
Well, it might be but there are a lot of other things that can be tested. Most have mentioned stress testing and other hardware testing. Self explanatory I think.
The other thing that probably is done (i'm no dev so I don't know for sure) is data gathering. Every time someone does a quest, kills a mob, visits an area the team gets reports one what content is hot and what is not. This gives a last minute chance for the devs to tweak some content that may not be utilized.
Then there are the forums for beta and people are usually very vocal about what they like and don't like. Most alpha testers and closed beta testers have a lot of investment into the game - playing it regularly and is very involved in how the systems work. Open beta gives the devs a chance to see someone reaction to the game from a NEW player point of view. This way the Devs can iron out any problems with players being confused as to what to do or what something means... What comes to mind is EQ 1 --- I jumped in without reading the manual on how to play and I couldn't figure out how to get quests or sell things. It seems ridiculous now looking back but it was the first MMO I played. If players have a hard time figuring out how to do things they will not stick around in today's market.
But I think the main thing is for the investors.. they are taking the temperature on their investment. Seeing how hot the game is and to see what kind of return they'll be getting. Almost like a mid term for the dev team they hired.
Beta's are a lot of things all rolled into one. I'm not sure why anyone would be against playing a game before it's released. Try before you buy and excitement. I usually take that chance to convince my friends on why they should play the game as well. (I usually wont play a game if my core group of friends don't play) I think this is good marketing in favor of the player.
Maybe 'beta' is the wrong term to be used -- but I'm all for the open 'beta' stages.
https://www.youtube.com/user/NemsGaming
There are no real "betas" anymore. Now there are only "previews" titled beta.
As a long time software developer (as in Computer Programmer for 32 years), Beta are meant to final test code but sadly in the case of MANY online games that doesn't always hold true.
Personally have seen a lot of so called "Beta" games that weren't ready for "Pre-alpha testing" stage.
Then some games that are in "Beta" were so well done and well prepared that they should have already been released.
Others like Freelancer are a great game but missed the mark in some area like No Joystick Support in a Flight Simulation. It is sad too... there are SO many joystick examples that use DirectX that an oversight like that is without a decent excuse. It just doesn't take all that long to add support for a Joystick.
There is a fine line between a product that just needs some minor issues fixed and one that is just hasn't had proper pretesting.
You can't test every possible set of hardware out there... it is just not practical. So you give it your best shot andif you did your homework right it shouldn't be all that hard to fix bugs that are well documented that are reproduceable.
Betas also tend to get people excited about playing what is felt as if it is nearly "ready for prime time".
If you get chosen to play test in an open beta or closed beta, please make very good notes if you encounter bugs or strange occurances when playing the game. The more accurate the details on how the issue / bug occurred along with how the it occurred, the better the chance that the bug will be fixed.
Don't expect instant results however.
I have seen other games where WELL documented and very reproduceable errors in code are just flat ignored. That's horrible but happens a lot. Rappelz has a lot of these still with no hope for getting them fixed it seems. They just keep messing with all the skills of all character types and pet types instead of focusing on fixing glaring issue.
Any huge program has BUGS in it... Some found that can be repeated and thus be fixed... others that have yet to be discovered and those that people use as exploits because of buggy software in some particular area.
Any long time gamer will appreciate a well done game. Many that use the Unreal Engine have done very well based on a well coded and well implemented game engine. By the way, this engine has been released for people to try their hand at writing games:
http://udn.epicgames.com/Three/DevelopmentKitHome.html
Enjoy!
There are different kinds of re-balancing though: changing the parameters and changing the equations.
Tweaking damage output from 9 to 8 on something will happen all over the place. But I'd argue that changing the entire formula of how a skill interacts with other skills and the environment is not a re-balance, it's a redesign and with the terminology I'm used to, that would happen in alpha, not beta.
I tend to see live publishes as following the same cycle as any other product release, just on a smaller scale.
(as for the viral marketing aspect of betas ... well, it might work, but if you attract people who are there to actually sincerely test, they might get annoyed at being played as agents of infection)
How useful are betas?
Well that depends on the developer behind the beta. If the beta is run well, and the testers are given tasks... then this basicly helps prevent major bugs and issues from slipping through. I can remember my time in the early beta of Starcraft, by Blizzard. They ran a very good beta, they had target goals, and things that needed testing in mass. They would be general terms, like test the Zerg vs Human... or test this feature tell us if it works in game.
Some betas now are run like that... some features are being put out to be played with by the community on a bi-weekly basis. Bugs are found, they are then fixed. In games where I have seen a straight to public beta with no structure other than a publicity hype to try and pre-sale the game have all ended in fail. Pirates of the Burning Seas was one that pops to my mind. A great game concept. Could have been the next big SMASH hit game, something worthy of Sid Miers "Pirates!" from the old C64 and C128 days. But instead turned out to basicly be a unfinished product that they were trying to wow the masses with their pretty features.
Then there are other games that had no open beta, and a VERY short beta that have had perhaps some of the worst launches in history.... WWIIOL, Anarchy Online, Darkfall. These were calculated risks, in some cases, and just necesity in others. That really didnt pan out well for them in the short run, but for two of those on the list, have had a great run.
Beta has a use... but only if the developers use it for more than a get last minute cash gamble or a publicity stunt to get the game name out there for free.
Because there will always be the Free play beta player, who lives from open beta to open beta rarely if ever subscribing to anything. That will always end up destroying what could be a great game with stupid suggestions done in mass.
I can remember 13 years ago there was maybe a half dozen MMORPGs on the market... Ultima Online being the biggest. Now we have more MMORPGs than you can shake a stick at, and probly less than 5% of which are worth the binary code they are made out of.
So much crap, so little quality.
Betas applied as a testing tool, like they are supposed to be, are an invaluable service to ALL gamers and the people that participate in them by submitting 100s or 1000s of bug reports and edits are argueably unsung heroes.
Betas, as they are applied today, are really just extremely long previews. Add in the fact that some companies have hit on the idea to charge for them (sneaky via lifetime registrations for other games) indirectly and you've got Betas being used to promote a game, not polish it. It really is the worst idea ever, because either A) everyone thinks its awesome and should be released already or
it isn't finished and people post 100s of forum subjects (instead of bug reports) about the suckage of the game. It really is lose/lose.
I honestly think there should be NO open betas. Have 12,000 closed beta slots if you need to test servers ... but don't invite a bunch of people in and then make them watch your game crash to desktop or queue for three to four hours a night.
Betas are only useful if they are truly Betas. So many Betas now days seem to be nothing more than free trials. The company gets to stress their servers and check the code against a bunch of different PCs and the players get to take a look at the game and decide if its worth their time. No real testing is done by the majority of people under these circumstances.
A real Beta tester should be listened to by the company. They go out and see how things work and will let you know if something is wrong. They write detailed bug reports, a lot of them and they fill the forums with useful posts about game play, about the feel and the immersion.
What I see now days are kids playing with a key they got because they pre-ordered another game by the company or got a collector's edition. They whine if they lose something, cry when MOBs get their final difficulty increase, throw tantrums on the forums over trivial crap. Yeah, your servers are getting stressed but so are your GMs and Moderators...
Companies get free employees if they do things right, such as actually requiring an application to be filled out that has a lot of essay style questions that will really show what kind of Beta tester you are getting. Otherwise it just seems to be a waste of time.
I respect your right to voice your opinion and reserve the right to blow it right out of the water
dear Staff:
I found the article to be interestingly vague, but at the same time quite self-enlightening. Let me elaborate if you will. I have taken part in a seemingly endless list of mmogs,mmorpg, mmo games. I have been doing so sing the very first Guild Wars was beta tested(which was my very first beta test). At that time I was a wide eyed, stoked wanna be beta tester wih all kinds of hopes of really helping the end product and its quality.
fast forward, 50( or there abouts) beta tests later. I still,in the deepest recesses of my mind have that child like excitement of having a real impact on a games development but just a bit tempered. I can't help but think that every single beta test I have taken part in did have its purpose, especially the 1st closed beta. everyone running around trying to find glitches, bugs, rips in the graphics, trees you could walk through, non-responsive NPCs, buildings mysteriously elevated off the ground, misspelled words or poor gramer. In all the beta tests I have taken part in, very few times have I actually found any bugs,etc. More importantly, or so i thought, was giving input on the game's mechanics, UI setup.etc. I and many others along the way offered "player feedback" on what could make it a better,user friendly , easily adaptable game. Well, alas when the final product came out, it hadn't(for the most part) changed one iota.
EXAMPE: CrimeCraft- I took part extensively in the beta testing of this game and quite frankly was quite disappointed. Throughout the testing, developers seemed to make their presence known in the forums asking for feedback. many of us gave great ideas for making the game exciting. Thousands of ideas will ( more likely than not) always bring out some great widely acceptable ideas that could actually improve different aspects of the game. Hence, we were lead to believe that our ideas meant something, were of interest or at least were heard. I can safely say .0137% of the plausable ideas were actually considered, and of that.0137%, .0035% was actually implemented(so to speak).
In conclusion, I have learned from experience that generally speaking, Beta testing is more often than not used as a marketing tool(sadly to say). Yes, many want us "beta testers" to believe out input matters, but in all actuality it really doesn't. The product has already had the ground work layed for the full mechanics, graphics,etc of the game and te developers have no interest in changing anything. So, now I use beta testing for early access to a new game. yes I still look for bugs and such, but mostly for access to a game early on. Which is what they wante all along. GEESH!! full circle.
MonTe
There is a drawback to having beta testing, you get the bad apples. Often times people who are cheaters, exploiters, hackers will use a MMO Beta Test to find bugs and exploits and use them on the live servers and they find it funny to ruin the game for everyone. They think it is cool to use bugs and exploits from the beta on live to get ahead.
I'd like to relate my experience with one closed beta test in particular as this one was the only one I found to be an actual "test". Now I've participated in many so called beta tests, some labeled as "open" or "public" and some as "closed" and they all pretty much conformed to what was described as mostly a form of advertising, a "try before you buy" scheme but called a "beta test" to attract people that might not otherwise download a "trial" version" participate as a chance to be a "tester". Aion is the latest example of one of these that I've participated in. But I had been a long time addict to the original Everquest and when the chance came up to get in on the very first of the closed beta tests for EQ2 I jumped at it. Before we were able to download the software there was the usual NDA form to sign but there was also one with a description of what testing would involve and basically it said that if you were looking to simply get a chance to play the game early you would be better off waiting for when the open beta launched as that would be when the bulk of the server stress tests were to be conducted and the only requirement for those were large numbers of players and that this first series would be many hours of performing mostly boring, repetitive tasks in order to test things like game mechanics, the AI and the like. And true to their word that's what most of it was. After logging in and creating characters we would be able to begin playing as if it was just a stress test but then came a constant stream of programmers that would ask for a certain number of volunteers to test how the AI reacted to certain situations or they would ask for a guild to test a particular raid they were working on and there was complete interaction with these programmers. They weren't kidding about it being boring and repetitive but they joked around with us and did their best to make it as much fun as was possible considering the task and I have to admit that this was the most rewarding and enjoyable beta test I ever participated in. We really got the feeling that we were an important part of the game's development. As a sort of a reward for our help, when the closed testing ended and the open testing began there was a "credits" screen that came up when you logged out that listed everyone's real name and their character's name and every guild that participated in the closed test. It was discontinued when the game went live but it was a great feeling to see my name come up on that list and to this day I have never been a part of another test like that again to my great disappointment. I keep applying to many of the closed testing offers that I can in the hopes of getting involved in something like that once more. It's a shame more companies don't do it, or if they do, then I just haven't been fortunate enough to have gotten an invitation.
I think betas are useful for both the consumers and developers. For the developer side, they can use this opportunity to gauge the communities reaction of their nearly completed project, market their MMO to the masses with a taste of what is to come and attempt to find and eliminate bugs before or shortly after launch. For the consumers its an opportunity to become familiar with the game they have been waiting for, help developers locate and remove bugs and help the community grow through word of mouth (or forums such as this).
"If we don't attack them, they will attack us first. So we'd better retaliate before they have a chance to strike"
Yup! It also depends on the studio. I have been in some betas that have me asking myself "what am I really doing here?" and otehrs where feedback was actively sought by the developers. If nothing else beta testing often helps me to make a final decision about a title and saved me money.
Ok, maybe you are right, some of them are just RMTs. You have every right to not like it.
But for heaven's sake it's not fair or unfair. It is what it is, you can buy it or not. Unfair would be that you MUST buy it regardless. Anybody can try to sell anything they want, that's the marketplace. Don't buy it if you don't like it.
I am really tired of hearing so many people on these forums talk about fair and unfair, and about 1 per cent of the time that's really the issue. They have every right to market junk and you have every right not to buy it.
/rant off
---------------------------
Rose-lipped maidens,
Light-foot lads...
Beta testing has become another form of P2P. In fact, MMO's are the only product I know of that people will pay to test. They charge, one way or another, for giving people the honor of testing their unfinished, buggy products and then release the game unfinished and buggy and they do this simply because they can. It will continue as long as people are willing to hand over the money.
i love where people are getting this crap from. seriously, how many games recently have even had any form of "paid" beta testing? maybe a handful, but nowhere near even half of them. so how is it that beta testing has become a form of RMT or P2P?
Would you all prefer they just cut out beta testing and just launch the games asap without any form of beta so you can all whine and cry that the game wasnt ready for launch because of bugs, server issues, etc and they should have.... zomg beta tested it first!
i love where people are getting this crap from. seriously, how many games recently have even had any form of "paid" beta testing? maybe a handful, but nowhere near even half of them. so how is it that beta testing has become a form of RMT or P2P?
Would you all prefer they just cut out beta testing and just launch the games asap without any form of beta so you can all whine and cry that the game wasnt ready for launch because of bugs, server issues, etc and they should have.... zomg beta tested it first!
How many games release in a decent form after the beta testing? I have an idea. How about actually testing the game in house and actually fixing the problems that are found? What a radical concept that is, actually selling a completed product.
Lot of excellent replies already. I'd just to like to add one additional thing. Companies need to learn about communications and expectations.
If you communicate clearly what something is about and follow through with it, you usually won't upset people. If you remain vague about it though, people might assume it to be something else. And when it doesn't turn out how they expected it to be in the end, they will most definitely be upset. It really doesn't matter if you're buying a product at a store, buying a meal at a restaurant, or doing a beta test for a game. Be as clear and concise about what you are offering and follow through in delivering what you're communicating. The closer the two match, the happier people will be.
Today people assume a beta test is to fix bugs and address gameplay issues. If you're a developer and you decide you can't address gameplay issues in a beta test (but only can address bug fixes) then make this absolutely fricken clear from the very start, so that every beta tester knows this. If gameplay issues are only addressed in alpha testing, then you better get the best focused testers you can possible get and they better represent your core audience as much as possible.
"Open betas" have become a form of free viral advertising. In the past, they were exactly what the name implys, these days they are just a chance for players to get a taste of the game, which as Tabla Rossa found out can work against you as well as for you.
For my part, it is very rare for me to volenteer to beta test anymore, I will not normally work for free. And real beta testing is mind numbingly dull. I can remember one game where I did the same quest over one thousand times, mostly to reproduce a very rare error, it turned out to be caused by one piece of armor and a weapon. You needed all three for the crash to occure. Anyways. If you want to beta test, pay your testers. If you want to give players a chance to play the game, call it what it is, and don't try to make it sound like part of the already completed development cycle.
Betas are a must wether they are open to the public or not. The primary goal in beta is to fix issues, nothing more nothing less. It shouldn't be used to form an opinion of the game or used as advertising. How can one possibly form an opinion of a game that doesn't even work right ? Thats why we have beta, to fix issues.
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
https://www.youtube.com/user/NemsGaming
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
How many betas do you actually know that went extremely well ? We have failed mmos because devlopers do not fix the import issues before launch. Perhaps I look at it from a compuer standpoint because I worked in that field for quite awhile. In the real world when you work in betas/tests its to test the product and fix it. I'm not quite sure you can actually make beta into a marketing tool when only serveral few mmos have had excellent launches.
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
How many betas do you actually know that went extremely well ? We have failed mmos because devlopers do not fix the import issues before launch. Perhaps I look at it from a compuer standpoint because I worked in that field for quite awhile. In the real world when you work in betas/tests its to test the product and fix it. I'm not quite sure you can actually make beta into a marketing tool when only serveral few mmos have had excellent launches.
To your first point about major issues not being fixed before launch --- this is most likely not a tester issue this is a issue with money and the investors wanting a return on the money. Games are getting pushed out the door before they are finished or ’polished’ because the games development costs are insanely high.
Those few games that you have mentioned had good launches also had great public beta's... so I’m not seeing the correlation between the open beta process and bad launches. WOW and LOTRO had great launches and they have very public beta programs. I'm down for getting the serious testers based of the 'on my honor' applications for previous beta work as well as getting your name known with the company as a good tester.. those are needed and should be included. But i'm not sure why it's detrimental to have fluff testers to get other feedback as well - as far as random bugs, game play, content, level progression. The feedback that these people will provide will let the developers know what a causal player thinks of the aforementioned items.
I'm not saying only rely on the public fluff testers -- companies still need a solid group of testers doing the hardcore testing. MMO's are very dynamic and need to make sure all aspects are covered. Why only focus on fixing bugs during beta when content might be sub par, advancement might be poor for leveling and geographical advancement might not be intuitive. Getting this feedback from fluff testers is going to make the devs more aware of how general players will perceive their game.
Also, games keep track of each quest that is done, how many of X item are floating around. More testers can give developers data on what’s working what is not / what is being utilized and what is not.
https://www.youtube.com/user/NemsGaming
What about content and game play? Beta is more that 'bug' fixing especially in the open beta.
By the time the game get to open beta the majority of the bugs should be 'known' and have fixes coming at least. New bugs do pop up but are usually made known by either a bug report or someone screaming that the game is a pile of crap in the beta forms. Either way the developers get to hear about it.
Why shouldn't a solid open beta be used for word of mouth advertising? The MMO field is plagued with horrid launches. The MMO community is aware of the posibility of a buggy launch.. why so?? Why is the concept of a buggy console game not a thought. If an MMO has a solid product and goes into open beta why not get people excited enough to talk about how great the game is going to be?
There is a flip side.. if the game is in an open beta state and is still riddled with bugs...well, it shouldn't be in open beta at that point.
Closed beta needs to have a core group selected for true testing. If the company wants to add fluff testers on top of that so be it. It will result in more feedback at the least and doesn't cost the company any more than extra bandwidth at that point.
How many betas do you actually know that went extremely well ? We have failed mmos because devlopers do not fix the import issues before launch. Perhaps I look at it from a compuer standpoint because I worked in that field for quite awhile. In the real world when you work in betas/tests its to test the product and fix it. I'm not quite sure you can actually make beta into a marketing tool when only serveral few mmos have had excellent launches.
To your first point about major issues not being fixed before launch --- this is most likely not a tester issue this is a issue with money and the investors wanting a return on the money. Games are getting pushed out the door before they are finished or ’polished’ because the games development costs are insanely high.
Those few games that you have mentioned had good launches also had great public beta's... so I’m not seeing the correlation between the open beta process and bad launches. WOW and LOTRO had great launches and they have very public beta programs. I'm down for getting the serious testers based of the 'on my honor' applications for previous beta work as well as getting your name known with the company as a good tester.. those are needed and should be included. But i'm not sure why it's detrimental to have fluff testers to get other feedback as well - as far as random bugs, game play, content, level progression. The feedback that these people will provide will let the developers know what a causal player thinks of the aforementioned items.
I'm not saying only rely on the public fluff testers -- companies still need a solid group of testers doing the hardcore testing. MMO's are very dynamic and need to make sure all aspects are covered. Why only focus on fixing bugs during beta when content might be sub par, advancement might be poor for leveling and geographical advancement might not be intuitive. Getting this feedback from fluff testers is going to make the devs more aware of how general players will perceive their game.
Also, games keep track of each quest that is done, how many of X item are floating around. More testers can give developers data on what’s working what is not / what is being utilized and what is not.
WoW had a horrible launch and yes Lotro had a pretty good one but that is a dime a dozen. You also can't use betas as a marketing tool when most mmos since WoW have been mediocre at best.