Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
Champions Online: How Developers Handle Beta Feedback
Stradden
Managing EditorMember CommonPosts: 6,696
Cryptic Studios' Bill Roper recently took the time to write this developer journal about the way that Champions Online developers have been handling feedback from its testers throughout the beta process.
)
A huge part of any beta test is wading through the massive amount of feedback you get from your testers. This input can range from the ridiculous to the sublime, and we're very fortunate with Champions Online to have not just a very vocal community, but also a passionate one. Our players really care not just about the state of the game, but about how they will be interacting in a superhero-based environment and experience.
A great deal of what we have been doing over the past few weeks is in direct response to the interests and bugs reported by our testers. An excellent example of this is the creation of The Powerhouse. This is a building that exists in every major zone where heroes can try out powers before they lock them into their power framework. A "buy before you try" concept was brought up in a forum thread about training powers and the difficulties of selecting from the vast array available. It was a great concept, and we worked hard to get it into the game as quickly as possible.
Cheers,
Jon Wood
Managing Editor
MMORPG.com
Comments
Been awhile since I stated this but FIRST......heehee
I believe it is a very good thing that DEV listen to Beta testers. After all they are a big help in making the game work just as well. Only if other companies actually listen to beta testers maybe some of these games will not be such a fail.
I beta test the game myself and i love the bug report system int he game. I reported over 7 bugs big and little and each one i got a response with in a few minutes to 24 hours depending on when i submitted my complaint.
you all deserved a 10/10 on the bug report system.
Games I will be playing are: TES V, SWTOR, ME 3, TSW
Reading this makes me think that Roper may not be so bad after all.
Yea they did a pretty good job of addressing issues quickly, and did a decent job of balancing powers, I would say too good at balancing them except for a couple really OP ones.
However I don't see the game being very sucessful at launch because it simply isn't polished enough.
All the classes seem balanced so much that each one just seems like another flavor of vanila.
The FPS I was getting was lower than Crysis on 16AA16AF 1440x900 on max settings.
All this hate for Mr Roper (all over these forums) et al reminds me of an old joke.
Tourist walks into a Welsh pub and begins to get to know some of the locals, eventually he's accepted and introduced to everyone.
"This is Jones the Smith, this is Jones the Barrel Maker, this is Jones the Undertaker, this is Jones the ......" says the landlord.
"Alright, alright"; interupts the man being introduced; "For the love of god, i mean it's so unfair. I've built a hospital! But do they call me Jones the hospital builder? NOOOOO. I've built 10 schools, but do they call me Jones the educator? NOOOOOO i've built over 1000 houses but do they call me Jones the builder???? NOOOOOO"
"But you shag one goat ....."
I really enjoyed the Beta test. Yeah the game has a few problems but I was impressed with how cryptic responded to the testers. Also liked the way they kept the server open till the last minute after the official end and b4 they wiped it 4 the headstart. Miss my toon already. Its a good fun game with a few teathing issues I have no doubt will be ironed out in the long run. Good luck Cryptic hopefully be there 4 the Star Trek online beta.
I really enjoyed the Beta Test as a whole, it was probably one of the smoothest overall that I've been part of. As many people stated, reporting a bug was very easy and it felt rewarding when those bugs were fixed promptly. If the bug system is indicative of how players will be treated. This game should be enjoyable and have a great potential for a smooth launch, and if the gameplay in beta is any indication (which I'm pretty sure it is), then it should be a lot of fun too.
I believe Cryptic still has a very long ways to go in the communication department, this was an issue back with CoX and it was clear during open beta that it is still an issue. The problem is Cryptic often makes changes without informing the user base until there is an up roar or when they do inform them ahead of time they explain it in such a manner that it can be left up to interpretation. However they do tend to be quick to respond to bugs and support tickets, it would seem that Cryptic is the opposite of some MMO developers who have great PR to hype the game but fail to deliver on half their promises.
I do not know if this is something Cryptic will ever learn from, I had hoped they would have learned something from their experience with the CoX community when it came to clear communication as they did improve over the years. However it is clear they still have a very long way to go.
Outside of that I would agree with the article as it is clear to me that Cryptic (still) has a good development team and they have been quick to address some of the glaring bugs. Hopefully they will have a stable release and move on to continue to improve CO from there.
Because flying a Minmatar ship is like going down a flight of stairs on an office chair while firing an Uzi.
LOL! That's hilarious. I do feel bad for him sometimes. How would YOU like to be reminded of the biggest mistake you ever made every time you go outside?
I honestly don't know what he did that was so bad in the first place. I really LIKED HGL and played it often. I got hundreds of hours of play time out of it and got to spend time with my friends and family the whole time. Its a game that I still wish I could revisit from time to time.
Really glad to see that all of the hard work done by the DEVs and the community wasn't destroyed by marketing by reinstating the special subscription pricing.
listening my ass... (although with the community they managed to gather during Open Beta.. who could blame them for not listening)
or the wouldn't have screwed up the XP in the last few days of Open Beta, i mean you could kill Black Talon easily at level 4, but since he's dropping level 5 items, it makes it a bit weird with balance.
put me slightly off... although the items you aquire are unlocked on your account, unfortunately the weapons you unlock are ONLY unlocked on your character. yay for microtransactions which will probably sort out that little problem.
yeah thanks but no thanks, graphics aren't that good... engine is ropey... too many quests where you actually have to spawn camp (once in an area i can live with, but we're talking 6-8 times in ONE small area).
whole itemisation and crafting seems tagged on, PvP arena style makes me roll eyes... at least the jailbreak scenario made me keep playing with the idea of playing as a villian during it. But it seems far away and not much else seemed close...
a niche game for a niche market. should do better than AoC or WAR though.
"nothing actually matters, we're just slightly evolved monkeys clinging to a dying piece of rock hurtling through space waiting for our eventual death." - Frankie Boyle, Mock The Week
Bill Roper: The recent change in the game's experience curve during open beta is a great example of this. A group of very vocal and passionate players became enflamed over the changes. The experience shift was indeed dramatic, but we quickly worked to inform players that this was not the last change, and that we were going to be moving things back the other direction. The passionate discourse began, and while some people were willing to see what was going to happen with the next course correction, others would not be swayed from their belief that we had utterly destroyed their game and wouldn't be doing anything to make it better.
Describable: listening my ass... or the wouldn't have screwed up the XP in the last few days of Open Beta, i mean you could kill Black Talon easily at level 4, but since he's dropping level 5 items, it makes it a bit weird with balance.
There's irony for you...
I'm going to have to beg to differ with Mr. Roper.
I was in the Alpha and then Closed beta testing phases for this game. I was accepted into the alpha phase in Oct of 2008. One of the most griped about constants through out the whole testing process was the combat mechanic.
Granted there were many posts that had no substance other than frustration and vitriol over how much people disliked the build up to blow up combat mechanic.
However, there were more frank discussions about the ins and outs, pros and cons, ect. of COs combat mechanics than there were empty posts filled with nerd-rage. Furthermore, most of these posts went on to throw out suggestions on how the gameplay could be tuned to be more fun and not so lackluster.
These posts popped up right up until closed beta ended. They may have popped up afterwards, but I had all ready given up on this game turning a corner and being fun to play.
To further illustrate the disfunctional feedback system:
There was a forum moderator named IronAngel. I thought about saving the post, but I didn't so it's all hearsay. Anyway, myself and a few other testers were really put off when IronAngel posted saying that he or she and the rest of the moderators would be less inclined to pass on feedback that wasn't stated in a manner that they deemed appropriate.
This isn't a bad thing when taken at face value, but if you take the time to think about it you realize how counter productive that is insofar as improving on the game through tester feedback is concerned.
Who hasn't posted on an internet forum and been met with hostility because some poster or mod read aggression where there was none? Not to mention the fact that if you are a mod on forums dedicated to testing a game, isn't it your responsibility to pass on all feedback?
It really led me to believe that all of the threads discussing the combat mechanics (almost a year's worth of discussion) just got labeled as "unworthy of attention" and passed up with little to no attention.
There were a few of the developers that would browse the forums now and again and join discussions, but that seemed to become less frequent in closed beta and then petered out to almost non-existant by the end of closed beta.
I also doubt that the developers and publishers that posted announcements on the forums would have completely missed that for months there was always a long discussion about the combat system on page 1 of the general discussion forum and on the combat and gameplay forum.
I don't have anything against Mr. Roper, but let's call a spade a spade... this, my friends, is spin.
I'm sorry but holding it against the forum mods that they don't pass on every useless forum post to which you admit there were plenty of?! That's far from reasonable. The fact that devs got busier and less talkative on the forums at the end of Closed Beta when they're crunching to get the game finalized for Open Beta and release is only natural since they're working themselves to death trying to give us the best product they can in the alloted time. Even then (I only joined closed beta in the last few weeks), the dev team is the most interactive team I've seen in any beta test I've been in. Heck better than any live game that I've played! Sad fact of life, some people are just impossible to please!
I've been very impressed with how the dev team respond to bugs (sometimes within days of posting them). If they can't fix them in time, bugs are usually stickied so people know that the devs are aware of them and working on fixing them. Bug system is the best I've seen in any game, and I've never contributed so much feedback in any beta that I've tested. Yes the marketing and PR teams botched certain things up real bad, but I'm happy to see them come back and say we're sorry that you didn't like this and therefore we're doing this to fix it. What more do people want?!
Game is not perfect (all games aren't), but with all the above in mind, and the fact that it is a fun and different game from the tens of popular MMOs out there gives me hope that Cryptic will hopefully live up to the great potential that this game has.
They handled that xp change uproar nicely. If I were them I would have been calling out the idiots, but they kept their cool, didn't say a word, just made one statement saying they don't want to ruin the game or anger the players. After that, people calmed down, and now in this story we just read here they talk about it like they are grateful towards the angry mobs for being so passionate about their game. Now that I think of it, Cryptic does listen very well, just recently they brought back lifetime subscriptions after massive uproar in the forums (I remember seeing a thread about it, made only eight hours ago, with one thousand replies). Good stuff Cryptic.
Well there's a couple of things that bother me about this post. The first thing is that he says he was an alpha tester, but he was posting critiques on a forum which was then moderated. That's the first time I've heard of an alpha test that took input in this manner . At the Alpha stage you should be so open to feedback that a forum shouldn't be used. If you're using a forum and filtering the content with moderators, you've got way too many alpha testers.
The second thing is, that bothered me is that someone let this jerk into alpha and closed beta of their game. If you can't be mature enough to post in a reasoned manner when playing someone's game for free... maybe you shouldn't be a tester. You're doing a job and being compensated, act like a grownup.
The point was that they were not passing on the fact that there was non-stop comments about how bad the combat mechanics were.
That or the devs were ignoring the information.
Granted, however it also left the testers with the feeling that all of their concerns were being ignored, this was backed up by almost none of the concerns being addressed in the game.'
What betas have you been in? They were not the worst of the ones I have been in but they were very near the bottom.
Ok, this is just silly. That or for some odd reason you just managed to log in to every conversation a dev happened to reply to.
Yep, they did a great job with fixing bugs. If you listen to the people complaining about the devs it is not because they didn't fix bugs. Its because they would not listen to any criticism about the game itself.
Any comment that was not about a bug, or that did not praise the game was flat ignored by the devs. I can not think of 1 thread where people were talking about combat mechanics and suggesting changes where a dev replied. (Maybe I am just real unlucky, but I don't think so)
No argument here.
And there is the rub, for many of the beta testers the game wasn't fun. And it isn't different from the other MMOs.
The combat is massively simplistic. Press 2-1-2-2-2-2-2-3. Boring. 90% of the fights are make big attack. Make end building attack, wait and make big attack over and over. Maybe 1 fight in 10 you might use a 3rd power.
The crafting and loot are lackluster, nothing wrong with them just nothing special. What is the diff between a offensive slot item with +5 Str and +5 dam from any WoW item?
The missions structure is so WoW like it is crazy. How anyone could call them different is beyond me. The closest you get is that most CO missions hand hold you and show you where to go.
There is no reason to group. Instances are rare, and most can be soloed just fine. Bosses are also rare, but at least most of them you do need help. Leading to grouping just long enough to kill the boss... then back to soloing.
Shrug. If you are having fun. I strongly suggest you play the game. More power to you. But a HUGE chunk of their beta testers were not having fun and got frustrated when the devs ignored them and never fixed anything.
Congrats. The devs built this game for you, and not for the beta testers who wanted a more complex combat mechanic.
He's telling nothing but the truth. Hell, we didn't even open up to having more than two days a week with limited times to test until open beta.
Cryptic ran it like they had set in stone what they wanted the game to be and wouldn't budge on anything that was important. Though you should be thankful we fought so long to get the hotbar increased, otherwise you'd have a max of 7 powers available at one time.
Well in every beta you have fanbois, haters and some rational people. The issue is, to WHOM do you listen as a company in the end? My guess is most companies lean towards the fanbois, which is fatal, since they tend to overlook a lot of real issues. I have always tried to stay on the rational middle ground. For me a MMO is by and large a sort of recreation, not a religion as for some. It's fun or it's not, end of story. Then there are different tastes, some like soloing, some like grouping. How can you know to whom to listen?
Honestly, I would be surprised if CO would not sink like a lead duck after a few months, alas.
He's telling nothing but the truth. Hell, we didn't even open up to having more than two days a week with limited times to test until open beta.
Cryptic ran it like they had set in stone what they wanted the game to be and wouldn't budge on anything that was important. Though you should be thankful we fought so long to get the hotbar increased, otherwise you'd have a max of 7 powers available at one time.
Thanks to you and Slackerboy as well.
You have to love how people read what they want into what you post and then accuse you of things that you aren't guilty of.
I really liked the fact that grimfall just assumed that I posted some hostile stuff and a mod swooped in with rational thinking and quelled the flames of my ignorance. hahahaha
I didn't post in that thread until after IronAngel did, and the only reason I posted was to point out that it seemed closed-minded and self-defeating to only pass on what they thought was relevant. If I were mod and I saw multiple threads about the same issue over and over... guess what? It really doesn't matter how the players worded their discontent. The fact of the matter would be that they are not content with that aspect of the game and would therefore be a relevent issue to bring up to the developers.
It's cool that you resorted to personal attacks and aggression, grimfall... it's the last refuge of the incompetent.
i say if its in beta they changed the xp or powers looks etc. dont matter is beta .. after live and that happens then something to yell about .. i had a good time in beta and like people i meet .
He's telling nothing but the truth. Hell, we didn't even open up to having more than two days a week with limited times to test until open beta.
Cryptic ran it like they had set in stone what they wanted the game to be and wouldn't budge on anything that was important. Though you should be thankful we fought so long to get the hotbar increased, otherwise you'd have a max of 7 powers available at one time.
This pretty much sums up my experience in beta. I tried over a dozen times to provide really good constructive criticism and lots of others replied back with ideas of their own. But Cryptic was interested in making what they wanted to make. It was like taking a leak in the ocean for all I was concerned. Hopefully, this will flop hard and fast and they can learn from their mistakes while their is still time to listen to their testers for STO!
They responded to bug reports, yeah. Too bad they never responded and fixed any of the content issues, oh wait, there isn't any content. Just the same ol' kill x for y, rinse and repeat. S4 League has more appeal than CO.
I played just before the game stopped Beta and Superjump still damaged the player using the ability. Yes thats right, you jumped in the air like the hulk and then took damage when you landed due to poor implementation.
I suppose that is what you get when you sell your beta.
I'm surprised no one has seen that he was subtly jabbing at the people who were the most vocal. It was hidden, but it was there. He sort of covered it up by thanking all of the testers afterward, though.
I am not a Roper fan. If you enjoy games that he has recently helped make, then that's great, I can appreciate that and I hope you have a blast with CO. If, like me, you take what he says and read between the lines, you can see that he gives a compliment, insults people, then covers it up with another compliment.
I am neutral when it comes to Roper, but I will say that I will be passing on CO. As I said above, if you enjoy it, great. I don't find spamming 2-3 buttons fun though.