Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

General: Jennings: The Truth Behind Bugs

2

Comments

  • FreddyNoNoseFreddyNoNose Member Posts: 1,558
    Originally posted by erictlewis


    Very good article.  I been working in IT systems for 24 years.  I can say not only games have bugs but code for regular bussines ssystems does too. However it was my job as well as the other programmers job to go about fixing these when they did crop up.  I know that a lot of game companies dont have the staff after a huge push, but fixing bugs should be a priority when it causes a total stop to the gameplay of lots of players.
    Some bugs are easy to live with, other bugs are not.   I don't mind bugs unless they totally mess up game play and are so bad that it affecs every playrs ability to do the task.
    For instance in LOTRO you had the spiders, that went invisible at for book 7.  You also had a hole in the landscape that cause players to be stuck for hours on end in Rotten cellar. You had the entrance to Carn Dum Cassle that manullly required a gm to fix it.  Otherwise you would wait for hours.  It took months on end about 6 of them to fix most of these.  However the spiders in dolen veiw instance are still invisable, it is still a playble instance but is not that fun.  Instead of fixing these bugs, they went on a crucade to fix the bugs that were benifiting players instead of fixing the bugs that were causing players a hard time.
    Like I said I don't mind bugs its how a company goes about fixing them.  I cant only complain about LOTRO, one of the worst violators is SOE as well. I could write pages about the bugs that existed in SWG before it went to the NGE. 
     

    Look at Microsoft, a couple of decades and billions of dollars later and they have to frequently patch their systems.

     

  • LanthirLanthir Member UncommonPosts: 222
    Originally posted by Frobner

    Originally posted by eric_w66

    Originally posted by Frobner

    Originally posted by biofellis


    (sigh)

    Surgery, is complex. If you wake up from an operation with jewelry in you, it's no big deal.
    Airplane maintenance is complex. If you drop from the sky due to an oversight, it's no big deal.
    Legal practices are complex. If you're innocent, and end up in jail...

    Stuff happens. But everyday life for other professionals with way more at stake and way more (actually) uncontrollable variables runs rather smoothly for the most part. Except the Lawyers bit- I have little faith in them, but some of them try.
    Every bug can be fixed unless it is flawed due to, I dunno, the nature of math or the limits of physics. Maybe something else- but I think you see my point. Otherwise the rest are all work, or embedded in someone else's crap (library/code)- but still 'fixable', (though even more work).
    No one told them to do business with and ship a product which was beyond their ability to properly support. If you went to your favorite store in the mall, and there was gas leaking out of a pipe into the room, you'd want it fixed. "Eh, that's the landlord's to fix- besides- you know what you want, right? just shop fast!" he says from his set-up table out in the hall. You shrug, and cede to the work-around, and happily ignore the problem every time you visit.
    Right?
    The truth behind bugs is, they have your money, and as long as enough of you keep giving money, they don't care. Hell, MMOs have gotta be the only consistent 'pay for play' beta deal around! (Though the rest of the industry is catching up with the 'ship first, patch later' business model).
    And don't even talk about intentional features that 'break' gameplay that can't be called bugs, because those are non-existant.
    The developer is always right. Right?

     

    You pretty much got it right =)

    MMOs have been able to take bugs to new lvls cause aparently you can always "fix" things afterwards in MMOs...   The truth of the matter tho... is that if you didn't get it right the first time  - then you wont get it right at all.  Or rather create 10 new bugs every time you fix one.

    Look at a game like DA.  It has bugs - but at least the big picture -the entire gaming systems and fundemental performance of the game is working.  Compare that to AOC with huge number of memory leaks - billions of content bugs and worthless core systems in the game.  You payed for both - But for some reason - its ok that one is total crap for an entire year... just because its marked as MMO.  And it got 8,5 to 9.5 raitings at launch....   Ye ... right...  Funcom had their forums closed for the first week for a reason.

    Woud DA get 9.5 when if 15-25% of the players were getting memory leaks shuttdown every 5 mins ?  Cause thats what happened at AOC.  

    There are no exuses for bugs.  Never.  Do we have a bug in the banking system that is leaking 500 millions into my account?  Ofc not...  They got the basics right....  Its all in the code. 



     

    You've never made a mistake? Amazing. "There are no excuses for bugs. Never".

    Obviously written by a person who's never coded more than a "Hello, World" program, and maybe not even that.

    Heck, just NOW at my work, I'm being hit by a "bug" in software I maintain that is crashing whenever a report is being run or a query being made on the database... yet it works in hundreds if not thousands of other places just fine. Why? Because the place where the crash is happening has people who have no idea how to use or manage a computer network, and the network crashed, and they couldn't figure out how to reconnect to it on one of their machines. Is it a bug in my code? Nope, but it is my code that is crashing because of their (bad) system configuration. So I get to work on a "bug" that isn't my "bug" at all.

     

     

    Mistakes are made ofc... thats why you have tests - after tests after tests.

    I dont care what kind of crappy software you are using.  Obviously its a bad one - or your a bad coder that cant create a structured system to work with.

    Compare single player RPGs and MMOs.  Would a single player rpg that doesnt play for more than 5 mins without closing because of memory leak  be considered acceptable at release ?   No ofc not - you would ask yourself who was actually testing that game before it was released. 

    If 1000 ppl all made mistakes that leads to a bug that is causing game to crash - then something is very wrong.  It has nothing to do with mistakes.  Nothing....

    And explain to me why Games like WAR and AOC seem to have alot more of those "bugs" that are based on the user's (bad) configuration - compared to ... lets say ... WOW ? 

    You know it and I know it.  It has nothing to do with mistakes... it has to do with quality control.

    Why hasn't WOW released more battlegrounds ?  Because creating a flawless map with minimised chance of expoiding is massive task.  And they accept nothing else than "flawless".



     

    well for one thing WoW intentionaly dumb down their graphics at release to insure their game woudl play well on the greatest mumber of rigs  while other games are trying to use state of the art graphics .  Sometimes they get it wrong like vanguard did in projecting the quality of the average machine their players would be using when their game was released.  Also take into consideration that sometimes it is the OS not livign up to the standards the producing company claimed such as what happen with AOC and vista

    Magic is impressive, but now Minsc leads! Swords for everyone!

  • RealmLordsRealmLords Member Posts: 358

    Greets Scott and welcome to my list of favorite columnists!

    One part of the article really stood out for me.  Triage is incredibly difficult to do.

    One I had recently was in the process of reworking terrain and tree population for efficiency, I had a loss of resolution on how accurately trees could be placed.  This left a couple of clusters of pine trees growing at the bottom of a lake.  Hard to fix?  No, but with several zones that needed to be checked for the same problem it would take some time to do properly.  Critically important?  Nope.  Triage decision:  LOW PRIORITY, bottom of the list.

    ZOMG !!!  There's trees in the water !!!  (I get this at least twice a week)

    It's not that easy to explain that I'm not blowing off a report.  I never ignore them when they're complete enough to determine exactly what is the problem.  Just not going to happen right away.

    I can hardly wait for wide beta

     

    Ken

     

    www.ActionMMORPG.com
    One man, a small pile of money, and the screwball idea of a DIY Indie MMORPG? Yep, that's him. ~sigh~

  • SenorHappySenorHappy Member Posts: 28
    Originally posted by biofellis


    (sigh)

    Surgery, is complex. If you wake up from an operation with jewelry in you, it's no big deal.
    Airplane maintenance is complex. If you drop from the sky due to an oversight, it's no big deal.
    Legal practices are complex. If you're innocent, and end up in jail...

    Stuff happens. But everyday life for other professionals with way more at stake and way more (actually) uncontrollable variables runs rather smoothly for the most part. Except the Lawyers bit- I have little faith in them, but some of them try.
    Every bug can be fixed unless it is flawed due to, I dunno, the nature of math or the limits of physics. Maybe something else- but I think you see my point. Otherwise the rest are all work, or embedded in someone else's crap (library/code)- but still 'fixable', (though even more work).
    No one told them to do business with and ship a product which was beyond their ability to properly support. If you went to your favorite store in the mall, and there was gas leaking out of a pipe into the room, you'd want it fixed. "Eh, that's the landlord's to fix- besides- you know what you want, right? just shop fast!" he says from his set-up table out in the hall. You shrug, and cede to the work-around, and happily ignore the problem every time you visit.
    Right?
    The truth behind bugs is, they have your money, and as long as enough of you keep giving money, they don't care. Hell, MMOs have gotta be the only consistent 'pay for play' beta deal around! (Though the rest of the industry is catching up with the 'ship first, patch later' business model).
    And don't even talk about intentional features that 'break' gameplay that can't be called bugs, because those are non-existant.
    The developer is always right. Right?

     

    While you may be right, you're missing the big picture.  As you said, airplanes need to stay in the the sky, so they are maintained to very rigorous standards.  Bridges need to stay up and there are licensed professional engineers to make sure that they do with very very rare (if tragic and press-worthy) exception.

     

    Let's not pretend that there isn't software that runs to those standards just as often.  Life support systems run what is essentially bug free software.  Avionics software that is essentially bug free keeps those airplanes in the air just as much as the mechanical maintenance.

     

    So why doesn't all software run to those same standards?  Basically it's the difference between a Life-Critical system and a Mission-Critical system.  If someone is going to stop breathing because a life support machine stops, or hundreds of people will fall to their deaths, that's obviously a life-critical system.  Similarly, if the space program has to restart years of work nearly from scratch, that's very likely a life-critical system.

     

    Developers don't create bugs on purpose.  There simply have to be realistic limits that you put on the architecture an testing process to any project.  Is it realistic to expect someone building a doghouse to go through the same procedures involved in building a skyscraper that will hold thousands of people every day?  Of course not.  What it basically has to come down to is what level of validation is needed with a given piece of software.  If a kid at school falls down and scrapes his knee then the school nurse will give him a bandage.  If stitches are needed it will be handed off to a nurse practitioner.  If major surgery is needed, a specialist surgeon will be called.  That's just how things work.

     

    What is acceptable then?  Certain pieces mission-critical software are probably held to the same standard as life-critical systems (financial systems come to mind).  Should games be?  The question is what kind of validation is enough and where is the tradeoff.  If a game takes an extra two years to develop, incurring the overhead cost of the process, would you be willing to pay a premium for that extra development time if it meant a bug free game?  Should gamers be expected to at all, or should that just be considered the cost of doing business?

     

    Another factor to consider is that technology moves quickly.  If you take that extra two years, somebody who skips that step is going to release a product that looks two years newer and shinier than yours even if it's buggier.  That's the sort of thing that comes up a lot in tech business decisions.

     

    The article states it very clearly at the end.  It's a balance.  Vote with your wallet.  If a game has too many bugs then don't buy it.  Or better yet, wait until the price goes down to send the message very clearly: I like what you're selling but it isn't worth the cost you were initially asking.

     

    Or if you feel that strongly and you ever find the opportunity, feel free to shell out the extra money for a game held to the same validation standards as life support software.

  • YamotaYamota Member UncommonPosts: 6,593

     Bugs, in any software, is best found by a good and efficient testing process. A software that has many bugs, when released live, i just a sign that the testing process has not been adequate.

    That being said, a software can never be considered "bug-free", however by having extensive internal and external (beta) testing you can assure that most of the bugs are found, and hopefully fixed.

    So the excuse "every software has bugs" is not a valid excuse for releasing a game full of bugs. That just means that the devs have not invested enough into testing their software and just want to rush the game out of the door. 

  • YamotaYamota Member UncommonPosts: 6,593


    Originally posted by biofellis


    (sigh)

    Surgery, is complex. If you wake up from an operation with jewelry in you, it's no big deal.
    Airplane maintenance is complex. If you drop from the sky due to an oversight, it's no big deal.
    Legal practices are complex. If you're innocent, and end up in jail...

    Stuff happens. But everyday life for other professionals with way more at stake and way more (actually) uncontrollable variables runs rather smoothly for the most part. Except the Lawyers bit- I have little faith in them, but some of them try.
    Every bug can be fixed unless it is flawed due to, I dunno, the nature of math or the limits of physics. Maybe something else- but I think you see my point. Otherwise the rest are all work, or embedded in someone else's crap (library/code)- but still 'fixable', (though even more work).
    No one told them to do business with and ship a product which was beyond their ability to properly support. If you went to your favorite store in the mall, and there was gas leaking out of a pipe into the room, you'd want it fixed. "Eh, that's the landlord's to fix- besides- you know what you want, right? just shop fast!" he says from his set-up table out in the hall. You shrug, and cede to the work-around, and happily ignore the problem every time you visit.
    Right?
    The truth behind bugs is, they have your money, and as long as enough of you keep giving money, they don't care. Hell, MMOs have gotta be the only consistent 'pay for play' beta deal around! (Though the rest of the industry is catching up with the 'ship first, patch later' business model).
    And don't even talk about intentional features that 'break' gameplay that can't be called bugs, because those are non-existant.
    The developer is always right. Right?

     I agree with you in principle but where it is true that any bug can be fixed it is not true that any bug can be found. That is a common misstake people do to think that any bug can be found. There is no such thing as a bug free software for any software with any remote amount of complexity.

  • FrobnerFrobner Member Posts: 649

    Bugs are no diffrence than spelling, gramar and typing errors.  I do not expect a columist to write an article with such errors.  It doensnt mean it doesn't happen. 

    Again - If you are a crap speller (like myself) then you will have loads of things to fix.  And that means it takes longer to fix it.  Specially if you dont have any software to check it for you. 

    Bugs are no diffrence.  Quality counts - and if the ppl are not good at what they are doing there will be alot more to fix - And since the ppl are not good in what they are doing - fixing is actually much harder than if you had a qualitfied ppl doing the job.

  • Trident9259Trident9259 Member UncommonPosts: 860

    yes i can understand mmo companies being under staffed, on a tight schedule and/or in short budget

     

     

     

    but why do we see sometimes these companies, contrary to what people ask for, keep on adding new content for instance into the game and ignore the so called "quality of life" bugs?

     

    is it because there are two different teams working on the project? or because the producer believes they will make more money from investing the manpower into content rather than fixing existing and sometimes old bugs?

  • MrbloodworthMrbloodworth Member Posts: 5,615

     

    Check out all the experts that think finding and fixing bugs in a complicated program like a MMO is easy.

     

     

    That is quite an awesome assumption, especially given the article that came before all the posts.

     

     

     

    ----------
    "Anyone posting on this forum is not an average user, and there for any opinions about the game are going to be overly critical compared to an average users opinions." - Me

    "No, your wrong.." - Random user #123

    "Hello person posting on a site specifically for MMO's in a thread on a sub forum specifically for a particular game talking about meta features and making comparisons to other titles in the genre, and their meta features.

    How are you?" -Me

  • biofellisbiofellis Member UncommonPosts: 511

    .

  • biofellisbiofellis Member UncommonPosts: 511

    .

  • BenediktBenedikt Member UncommonPosts: 1,406

    well "best" approach to the  noncritical bugs have LOTRO (at least the european version, didnt play the US one) - to report a bug you have to first CREATE A SEPARATE ACCOUNT on codemaster's web. dont think many people go through this if the bug they want to report isn't really annoying/critical - i certainly didnt.

  • KordeshKordesh Member Posts: 1,715

     This should probably be published every other day, because forum dwellers seem to have the memory of a goldfish and keep forgetting this. 

    Bans a perma, but so are sigs in necro posts.

    EAT ME MMORPG.com!

  • shabazzstershabazzster PWI CorrespondentMember Posts: 32

     As always.... great article. Thank you Mr. Jennings for the links to the game engine companies. They will come in handy.

    Keep up the good work!

    K

  • kdkirmsekdkirmse Member Posts: 51
    Originally posted by Frobner


    Bugs are no diffrence than spelling, gramar and typing errors.  I do not expect a columist to write an article with such errors.  It doensnt mean it doesn't happen. 
    Again - If you are a crap speller (like myself) then you will have loads of things to fix.  And that means it takes longer to fix it.  Specially if you dont have any software to check it for you. 
    Bugs are no diffrence.  Quality counts - and if the ppl are not good at what they are doing there will be alot more to fix - And since the ppl are not good in what they are doing - fixing is actually much harder than if you had a qualitfied ppl doing the job.

     

    Most bugs that make it into production code have little to do with code syntax. The bugs that make it into final code will be due to logic errors, miscommunication and invalid assumptions.

    An MMO will often have a programming team of over 100 people. Getting a team this large to effectively work on a project is a major management and technical headache. There have been a number of large scale software engineering projects that have  failed due to this problem.

    Bugs due to invalid assumptions may not cause any failures during development yet cause systems to fail later in their lifetime as other fixes and changes are made. It is just not possible to exhaustively test for this kind of issues.

    Like it or not MMOs are exceedingly complex systems and the customers are simply not willing to pay for the quality they seem to expect.

     

  • tkobotkobo Member Posts: 465

    Many people have already pointed out the really relevant reasons why mmos are so buggy and poor quality.

    Im going to only focus on a key one here:Ease of reporting and exploring the bug.

    This like most mmo issues,this comes down to truly piss poor design.Devs fault entirely.

    To report a bug, i should be able to just use the games chat system.I should enter the bug channel, and be instantly fed a series of questions,one at a time, asking for specifics about said bug.

    Example:

    Kieran enters bug channel

    Bug collector: hello Kieran,what type of bug are you reporting ?

    Quest bug ?

    Character bug?

    Skill bug ?

    Map bug ?

    etc...

    Kieran: quest bug

    Bug collector;What is the name of the quest ?

    Kieran: "Beat the mmo dev with a big stick "quest.

    Bug collector: which npc awarded the quest?

    Kieran:The blue orc named giggles in generic orc town 1

    etc....

    At the end of the chat issued questionaire,the player should be asked.

    Bug collector: Would you like to repeat the quest,so as to further explore the bug,and supply further information about it?

    Kieran:Yes

    Instantly the quest is re-added to the characters quest list,with added details of how no exp ,faction, item ,etc... rewards will be given for this quest.

    The quest will be automatically instanced upon the entering,even if it wasnt meant as one originally, and the player will have certain GM like powers while inside the quest instance that allow him to pause npc and environmental action,see the structure of the terrian,travel in any direction in a flight like glide mode,etc...

    The player should at no time have to level another  character to experience a quest again,simply because hes trying to do work that should have already been accomplished by paid employees.

    The player should never have to enter a new window seperate from the game, or relog in,in his attempt to help the game get fixed.

    Anything a player does,within the rules of play,thats goals is too help discover and fix probelms,while exploring a bug in the game should be as easy as humanly possible for him to do, and praised to the high hills.Hes paying money to help the devs fix something ,they should not be putting insanely bad design issues in his way.

    And yet they do.Piss poor design after piss poor design.Its just insane.The devs have got people willing to pay them money to help fix the devs game, and yet the devs thru lack of skill to design a reasonable sytem to allow this,or simply not caring enough to do so, waste that incredibale tool.

     

  • tomacontomacon Member UncommonPosts: 18

    HAHAHAHAHAHA! I love this Article! And if you can't tell i agree. You don't go buying a NEW car , and then are told  .Well  the timing chain needs to be replaced, and the damper door doesn't close all the way, and not to mention the wiring is a little screwy. Yes us vets of MMO's are a little used to it. I dunno what game i've ever played that didn't  need a little more work.  But when you pay for the Game, and  Then a monthly. When you really  look at it your like WTF! But like everything else, nothing is perfect cause we aint.

  • mburchmburch Member Posts: 4

    Great article.  Thanks for fixing bug #308.

  • kdkirmsekdkirmse Member Posts: 51
    Originally posted by Hypnohighzer


    HAHAHAHAHAHA! I love this Article! And if you can't tell i agree. You don't go buying a NEW car , and then are told  .Well  the timing chain needs to be replaced, and the damper door doesn't close all the way, and not to mention the wiring is a little screwy. Yes us vets of MMO's are a little used to it. I dunno what game i've ever played that didn't  need a little more work.  But when you pay for the Game, and  Then a monthly. When you really  look at it your like WTF! But like everything else, nothing is perfect cause we aint.

     

    Pretty good description of the quality of automobiles in the first 60 or so years of their history. All told the MMOs have not made it out of the Model T era.

  • RealmLordsRealmLords Member Posts: 358
    Originally posted by Mrbloodworth


     
    Check out all the experts that think finding and fixing bugs in a complicated program like a MMO is easy.
    That is quite an awesome assumption, especially given the article that came before all the posts.

     

    Totally well spoken!

     

    Ken

     

    www.ActionMMORPG.com
    One man, a small pile of money, and the screwball idea of a DIY Indie MMORPG? Yep, that's him. ~sigh~

  • CaleveiraCaleveira Member Posts: 556

    Good article, a nice overview in understandable terms. Thanks Scott.

    @Biofellis; yea, some of us lawyers do try...

    Just to make things clear...
    I speak for myself and no one else, unless i state otherwise mine is just an opinion. A fact is something that can be independently verified, you may challenge such but with proof. You have every right to disagree with me through sound argument, i believe in constructive debate, but baseless aggression will warrant an unkind response.

  • CrodeCrode Member UncommonPosts: 3

    Maybe someone could help out NCSoft / Aion fix their bugs.  It has had a crash to desktop (from a crysis dll) for almost everyone that participates in fortress PvP.  Its been like that for almost a month.  very sad :(

  • MordacaiMordacai Member Posts: 309

     

     

    LOL, Wow, great article. As a lead designer/programmer on an MMORPG i'd have to agree on several fronts. I especially liked you're little tidbit of fast, cheap, good, pick 2...that fit's us to a T..we've been set on making it as cheap as possible and as good as possible and we've been sorely behind schedule from day one...fast we ain't... :)

     

    Bugs are horrible and some like the one i'm working on right now are just impossible it seems to figure out...3 weeks now i've been working on this client problem where ui portions don't show up for certain people, keeping them from launching the alpha client but yet every time I and several others have no problems whatsoever and the dxdiag files just don't explain it, nothing does...yet...we'll tackle it eventually but jeez it just drives you nuts trying to figure them out sometimes...gimme the easy stuff like fix a quest giver.. :D

  • biofellisbiofellis Member UncommonPosts: 511

    .

  • tboxtbox Member Posts: 372

     God I remember using bugs to an avantage in shadowbane.  Granted now always or by me in particular. 

     

    I think one problem with bugs in MMO's is normally if a player or group of players expoit a bug if not banished for abuse they may enjoy the benefits of that exploit for long periods of time.   One reason why I enjoyed SB a lot when a old server was wiped and the server was fresh until a new bug or exploit was abused heavily.    

Sign In or Register to comment.