Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
Beta test like a pro
Krullen
Member Posts: 18
On the back of P-M's great thread, which sady turned into gibberish, and in the interest of having this game tested to perfection at release, I thought I'd post this up. My sister works at a big game studio and I asked her how do the pro's beta test, she replied with below which was so awesome it had to be shared.
P-M's thread. http://www.mmorpg.com/discussion2.cfm/thread/380347/Brief-Explaination-Alpha-and-Beta-Builds.html
The players that developers always like the best are not only those that find the most critical bug, but most importantly how they actually type up the report for the bug. You can imagine how badly some people report a bug half-arsed with net lingo and no professional understanding of what's going on behind the scenes..... like Collision. You already know what's "under the hood" so you'll be fine. Here's a quick guide I learnt when I first started:
When typing up the reproduction steps for a bug, remember the acronym P.A.R.
Place
Action
Result
Example:
1. Proceed to Zone 2: Forbidden Forest.
2. Perform a magical debuff whilst jumping at the same time.
3. Observe that the player character remains in mid air, about a metre above the ground plane.
NB: Reproducibility = 100%, can regain correct position after jumping again. Tested on XXXX Server, etc.... They usually have drop down menu's to collect other important data like the build, patch, server, realm, domain...
Only type a bug up in third person. Not first person. A bug is great if it has a picture, no matter how mundane. Having a screenshot of every bug occurring, is super helpful for the developer assigned to fixing it and visually knowing what's going on without having to read a ton of text.
What happens when you report a bug is that it goes to QA for proof reading. Some developers have a system where you can rate players who report bugs which internally levels up a players 'report quality' status and can get you past the initial proof read for legibility stage, and straight to sorting. After proof reading and in most cases re-typing it up, it gets sent to QA team for reproducing on current data, then if it is confirmed it gets sent straight to the department responsible for fixing. My example above would be programming. The lead programmer or an assigned Senior sends each bug individually to the person responsible for fixing it. Bugs are also rated for importance by the QA department before being assigned. This determines how quickly they need to be fixed.
Importance levels for bugs are:
Class A or 1 = Critical bugs that crash the game and prevent the user from continuing. They require the developer to drop everything and fix it. Even a bug where the player is trapped in the collision of a tree and can't move, that's a critical bug cause you have to force reset the game.
Class B or 2 = Function Bugs. When something that is supposed to occur, does not. Like a buff spell doesn't work, but you can still play the game. They are important, but the developer can finish the task they are on before they look at it. Or on the opposite end of the scale, something that occurs that should not have, like walking through the wall of a building. It breaks the illusion of the game.
Class C or 3 = Aesthetic bugs, results of artwork flickering. Missing textures. Lighting disappearing, things that spoil the look and feel but are not going to ruin your experience. Missing textures are the most dramatic class C bugs, and bad if they make it to Beta testing.
Class S, D or 4 = Suggestion bugs, which are the most common and annoying. A lot of the time a bug is downgraded to an opinion, like a class B bug where you don't think the player should be able to stand on top of another players head, Design will turn around and say that it's by design. Your bug will be delegated to an S Class which rarely get looked at. Unless they request class S bugs, which would probably be requested in a forum format like what Mark was talking about.
The bugs they like the most during Beta are Class A and B's. A's will get you instant attention daily. QA will generally do a Critical path play through to make sure the obvious ones don't exist, but Class A bugs that slip through the cracks are doing things like performing a task in an area you generally would not. Accessing a menu screen and adjusting the color balance in the middle of a major battle could cause an A class bug to occur.
Your suggestion of jumping on rocks, is another good way to find class A's. The places this occurs the most is between collision walls. Collision is invisible 3D polygons, that have no texture and allow programming to assign a barrier blocker for player characters that have their own collision box. You have to merge the seams between two planes, and sometimes this doesn't get done properly which creates a tiny gap that's micro pixels in width, but enough to allow player code through. Like a Mage performing a blink, it could allow them through the un-merged seam.
If you've ever noticed that you can see light or outside textures through the corner of a building. That's because the two walls weren't merged together making them seamless and one piece. Same thing is occurring with the invisible collision walls. So areas where a mountain meets the ground. Could be a gap that allows you to fall through the world. On top of roofs, in areas you're not supposed to be able to get to you'll find lots of gaps. A jump test is where you jump all along the walls and edges of everything. See if you can gain access to get up higher, and higher, till eventually you've jumped out of the world. These are so tricky to find in an enormous open world game as there are so many possibilities.
Another great trick is recognizing a dangerous animation. The players collision moves with the player during an animation. If you notice you take a step back or forwards during an attack, then are repositioned one step forwards at the end of the animation. Performing that near the edge of a cliff or building could reveal a bug. Dances and idle fidget animations are great for these types of bugs. Fidgets occur when you stand around doing nothing for a long time. What if you left the player near the edge of a cliff for 5 minutes, you might reveal a bug that way as well.
I think that's enough information to get you going. Main thing is to type them up correctly and in the format I outlined with P.A.R. 
DAOC - Krullen - Hero/Tristan
Comments
Think ill print that off Krullen in anticipation of Beta
Now at least I should be able to feed back the info in an intellgent form !!
This is of course what beta "testers" are supposed to be doing and traditionally this is what most of them did, but unfortunately players these days equate beta to early access or free play, very few will do any actual testing and even less will provide any useful feedback.
The problem here is MJ is asking players to back the KS to get alpha and beta access, while this will attract some players who care about the project and will use their time to aid development it'll equally attract players who simply want to play the game early. If a player pays $500 for the alpha access and wants to just play the game and provides no feedback whatsoever is this wrong? No, it's not, he has paid for the privaledge of participating, it's his/her choice how they use their time.
I was a bit concerned about the KS giving alpha access, however having participated in many recent "alphas", none of which were anything other than a pre-beta stress test im hoping the same will be the case for CU.
Most modern games are pretty much in their release state when beta begins, with a lot more players in the beta than the alpha you'd hope most will spend their time trying to break the game and report their findings rather than looking for exploitable bugs they can use on launch....which is precisely what many beta "testers" actually do.
Selling beta and alpha access is an accepted marketting ploy these days for the big boys who can pretty much absorb these gamers and not have it effect their product, but can a small indie production provide enough of it's own in-house testing to circumvent this? If so, no worries, if not it's putting a lot of trust in those with the most disposable income over those who would be best for the job and ultimately the project.
Hopefully as it's a KS more players will be willing to test properly than the usual free access beta participants, time will tell.
Expresso gave me a Hearthstone beta key.....I'm so happy
I believe this is why MJ has made a change with regards to the strict definition of the proposed 'alpha and beta' access language.
Vymm
MJ clearly said in the thread the OP pointed to than the alpha (or whatever name it has now) will be at a really early coding stage.
Thanks for the post, Krullen - very clear and incredibly helpful.
I have my fingers crossed that because a large percentage of those with interest in this project seem to be longing for a taste of truly old-school, community-based, not just "me, me, ME!" all the time gaming (realm pride, helping each other out, banding together to accomplish goals - not just to get kitted out with the latest flavor-of-the-month gear), perhaps a larger-than-usual number of those with "alpha" and beta access will understand the value of real testing feedback and put a whole bunch of non-selfish effort in because they want to contribute to a really good end product - perhaps even a *unique* end product, that they can take ownership of, and feel pride in. (And dangle their prepositions from, apparently, pfft.)
Perhaps some of the people that think a beta-access game is meant to be a finished product for their personal early-access convenience - and then proceed to piss and moan with windy self-importance all over the test forums about what they find, instead of taking the time to submit the kind of lucid bug reports detailed above - will self-select to go elsewhere. Or perhaps stealthers will gank them and use them as boat anchors - they might be handy for load testing...
Perhaps my head is full of peanut shells and I am hopelessly naive? lol
(Fingers crossed, anyhow, though.)
FYI - MJ today on http://citystateentertainment.com/camelotunchained/ :
"Finally, there has also been some concern about our “Alpha Access” tier. Unfortunately, we were thinking of Alpha Access as “our code isn’t a game yet, help us break it, beat it to a pulp” tier where the more common interpretation these days is for a tier that is more polished, feature-filled, content-filled than how I viewed alpha (old school, sorry). As such, I’ll be modifying some of the wording and access dates around to be more in line with current MMORPG standards. We will still have a very early access tier but it won’t be called Alpha Access any more."
So, we'll see.
Great post, Krullen.
Yeah, that thread was derailed slightly in order for it to be on the front page after it stopped getting responses by the community.
I'll look at editing my OP and including a link to this thread.
EDIT:
And it wasn't gibberish! It was Gaelic!
^Exactly !
And I won't derail this thread. Great post. This is exactly what modern beta testers need to read. Don't get me wrong, I love the ability to learn game mechanics as I beta. But it is the actual testing that helps a game be as polished as it can be for launch. And thats why I want to test.
I would suggest reading Plastic's post as well, linked above
It is in truth not for glory, nor riches, nor honours that we are fighting, but for freedom for that alone, which no honest man gives up but with life itself. ~Declaration of Arbroath