Calling out MMORPG.com...

AgtSmith

I have long been annoyed by how the gaming press does reviews - rarely
do I agree with them but even mroe rarely do they even seem to
represent reality.  MMORPG.com has allways seemed to be better about this
than other sites and mags but when I read thier D&D Online review I
about puked at how thier 'score' was so at odds with thier reviewers
own words.  Is MMORPG.com paid to keep from putting out a bad number?  Hate
to put it that bluntly but when you read the reviewers words and then
look at his assigned score you are left feeling that must be the case.


Below is some text from his review and a comment or two on that:

Sound: 8
The sound is
good, but nothing special. The fact that different theme music carries
on throughout the game is nice, and the music certainly has a more
mythic-historic feel. Entering a tavern and hearing the old school
instruments is fun. Spell sounds and sword swipes are fun, but again
nothing special. I would say the sound is on par with the regular video
games today.

This really sums up my point well, the reviewers own words "I would say the sound is on par with the regular
video
games today."  So being on par, average, same but no better than most
games earns a score of 8?  Hrm, so if I rate a woman's looks as
average, on par with most women, etc - that would be an 8?  Give me a
break, given the text of the reviewers description an 8 is absolutely a
ridiculous point score.

Role Playing: 7
Dungeons
& Dragons was the Role-Playing game that started it all. However in
Dungeons & Dragons Online the game falls short. The role playing
that takes place in paper and pencil D&D is rarely captured in a
video game. You constantly have to speak and act as a character. In DDO
very little of this is encouraged. I look at role playing in video
games as a combination of character development and growing a status in
the world. With everything instanced in DDO there really is no way to
develop your character in the eyes of the community. The level and
skill progression for characters is very good and exactly like the
table top game, however, more customization for characters would lead
to better role playing. I think many MMOs suffer from this flaw and DDO
is no exception.

"...in DDO there really is no way to
develop your character in the eyes of the community." and you give it a
7 out of a possible 10 for role playing?  If there is "no way" to develop your toon in this way then
how can the role playing earn 7 points out of 10?

Value: 6This
is where Turbine has really sold its players short. Charging a monthly
fee for this game just seems to be unfair. The content is so limited
that players will not be running around for months on end trying to
explore new areas. After one month of playing I found myself repeating
several quests over and over again. The community also seems to be on
one giant loot hunt and not interested in exploring everything a
dungeon has to offer. If the pricing model for this game was different
and allowed players to pay as they play or something, I think the value
would be much higher. If you can reach level ten in one month’s time (
as some players have) the game begs the question, what’s next?

Again - given what the reviewer is saying how does the score assigned apply?  I will admit a 6 is a bad score considering the scale but when the reviewer says "Turbine has really sold its players short. Charging a monthly
fee for this game just seems to be unfair." how do you justify what amouints to a 60% for value, a 'passing' grade if you will?


I am really not trying to flame MMORPG.com, I am trying to offer legitimate criticism.  We have all seen the ridiculous reviews from sites and mags where any crap game manages to get a score or comments that it can use - seemingly so the publishers can stay in good with the developers and get the content they need for thier publication.  But it has gone too far and I for one am sick of it.  Given the reviewrs own comments about D&D Online a score of 7.3 out of ten is absolutely ridiculous, either abandon the scoring system alltogether or apply it with integrity or I for one will discontinue my patronage of MMORPG.com.

P.S.  This is not meant to be a rant against D&D Online, or the reviewers text review - for the most part I agree wth much of what he said abotu the game, it is the score assigned that begs the question about who MMORPG.com is publishing for, its readers or the development companies.

Lechuckie

omg... that is all so true...

actully i never bother to read the reviews, i look at the numbers only. couse that tells me if the game is below avrege or... yea whatever... i can see now why i only manngeged to play DDO for 4 days only before throwing it into the trashcan.

but yea... i agree with you
learn the value of numbers mmorpg.com

thumps up agtSmith

Harafnir

I have been thinking about this... I have been reading game magazines since the middle 80s, and this is pretty much how it works. Just pick up any issue of for example Computer Gaming World, or similar and you will also notice most if not ALL scores are in the top 50% of the spectrum.

Now.. I think I know why. Consider the low 50% spectrum. Its easy to say "Well, this area was bad so it should be 30%" but then you have forgotten its a computer game. 0-10% that is for example NO SOUND. No sound at all... Not a beep. 0-10% Value.. That is an empty CD. And so on and so forth... Suddenly, 10-30% is pretty DAMN bad. In sounds... it might be "Bleep... boop" at times. Sound something like an oldstyle Gameboy. In value... you got a box with screenshots of Oblivion, then an installed game of Gauntlet.

Now all of a sudden, you are up in the 30-40%+ area... and the game actually have sounds... the sounds are similar to real life sounds, but awful.  But they are real sounds with a similarity to the real ones. And so on and so forth..

I have also made the mistake of thinking most scores are too high in all kinds of reviews But I have failed to see how bad it COULD have been, in a computer game (Most of us has, at osme time in life, bought that piece of crap and know exactly how bad it really could be) And compared to that.. a pice of junk, like DDO for example, is actually pretty good and certainly in the top 60-80% frame. Because it works. it has ok graphics.. there are sounds.... and no real flaws except really boring.

AgtSmith

I think the issue has to be one of reliability - I mean in the case of DDO (a game that obviously failed to impress the reviewer on many levels) they give it a 7-something while one of thier best review games gets a 8-something, this is simply untenable and bogus.

I spent some time perusing thier reviews and found that, for the most part, they don't give bad scores which is sad.  Sure, there are a few games with bad scores from users but not many with less than a 7-something from MMORPG and what I really find disturbing is that the games from 'big' publishers seem to be allmost gaurenteed a 7 or better.

All in all I like MMORPG but given this clear lack of integrity to thier review scoring I really am feeling like the site is about as unreliable as any gaming mag that is simply repackaged press releases and advertising.

Lechuckie

i am kinda hoping that someone from mmorpg.com will read this (they uselly do course its suggesten) but this is really an issue that needs to be taken of and discussed... not only by us members but by mmorpg.com

so yea... calling out mmorpg.com come and give is a deacent repley :-P

AgtSmith

I agree and am quite disappointed that they haven't chimed in as of yet.  As I said, not trying to flame them but I think this deserves an explination.

Harafnir

So you can not see the difference between the sound in DDO for example.. and no sound at all? Ok, then you have a problem, not this site or their integrity.

I am normally the last to defend MMORPG.com, but your inability to see the difference between general bad and plain awful or nonexistant, makes this a pointless whine, not a legit complaint.

AgtSmith

If you are rating a woman's looks does a 0 mean they have no looks?  It isn't a scale of having the feature is a rating of it what it has is any good and it is supopse to be a rating of 1 to 10, 1 being the poorest with 10 being the best.  If rating a car on a similar scale are you suggesting that a 0 or 1 would only be earned for the category of handling if the car had no steering wheel or wheels? 

To say that in terms of value that charging for the game is "unfair" hardly seems to warrant 6 out of a possible ten.  Saying the sound is no better or no worse than other games hardly seems to warrant a score of 8 out of 10 and only a point or two below what would be the best.  If being no better or no worse than anything else is an eight then the scale is obviously wacked.  A no better or worse type score should reside in the middle of the scale not at the top edge.

Plain and simple the review score is obviously padded so as to not piss off the developers.  If this is obvious to you I challenge you to find a single game reviewed and scored by MMORPG.com that has a score of less than 6, or even 7.  then compare that to thier review of games scoring in the eights and tell me hos the two jive.

I have looked and what is CLEAR is that MMORPG.com is just as much a pimp to game developers as are the game mags and this DDO review (as well as others) simply make it obvious.  MMORPG.com doesn't exists to provide gamers information it exists to provide developers a platform to reach gamers and while the differance may be subtle it is important.

Stradden

Hey Guys,

Sorry for the lateness of the reply, but I hadn't had a chance to check Site Suggestions until today.

Our reviewers are free to give a game whatever scores they like and feel are appropriate. Their scores and statements are not influenced by site management. The reviews are edited for errors only, and not for content. A review is one person's opinion, which people are free to agree or disagree with.

That being said, the reviews are done on an academic scale, where 7/10 is average (othwise, a "C").

I hope that clears a few things up for you guys.

Cheers,

Jon "Stradden" Wood, Community Manager

     

AgtSmith

---

Originally posted by Stradden

Hey Guys,

Sorry for the lateness of the reply, but I hadn't had a chance to check Site Suggestions until today.

Our reviewers are free to give a game whatever scores they like and feel are appropriate. Their scores and statements are not influenced by site management. The reviews are edited for errors only, and not for content. A review is one person's opinion, which people are free to agree or disagree with.

That being said, the reviews are done on an academic scale, where 7/10 is average (othwise, a "C").

I hope that clears a few things up for you guys.

Cheers,

Jon "Stradden" Wood, Community Manager

     

---

Thanks for the reply.


But if that is the case shouldn't the scale be shown as from 6 - 10 then. 
If an A is 9+, B is 8+, C is 7+, and D is 6+ then a F would be below 6. 
And doesn't that leave allot of room for confusion?  I understand the
reason not to do letter grades as it leave less room for variance such as 7.7
or such but giving any game an automatic 6 points on a 10 point scale seems, at
best, stupid.


Even given your explanation the reviewer clear said that, in this case, DDO's
sound was "on par" with other games yet it gets scored an 8 which is
better than average.  In value the reviewer says it is seemingly
"unfair" to charge monthly for the game but gives it a passing rating
in terms of value?  Same with role playing - stating the game has,
paraphrasing, little to no way to role play in the community yet it still
scores an average 7?


While I admit I have gotten a bit worked up over this it is really just the
idea of being marketed to instead of being informed by an independent site that
upsets me.  I am trying to keep an open mind about whether MMORPG.com is marketing
partners with developers or a site meant to inform gamers but no matter how I
look at this it really seems more the former than the latter.  Alternatively, use a proper academic scale where you assign 100 point scale scores and then average based on the 4 poitn scale yielding a 0 to 4 score (this also allows a letter grade to numerical grade system such as B- being a 2.0, B being 2.5, etc..

In the interest of trying to offer a positive suggestion rather than beating you guys up I think it would be appropriate to revise your rating system to either reflect the lowest possible score as a 6 so the other values are brought into proper focus or to consider re-rating things appropriately on a true ten point scale.  The later would make far more sense as I jsut don't see a bad or barely average reviewed game like DDO being properly scored a 7ish while a well reviewd game gets only a point or so higher - it just seems meaningless at best and marketting induced misleading at worst.

Dana

AgtSmith,

We quite intentionally keep our editorial staff and business staff seperate for exactly this reason. As to the specific ratings, they are simply the ones the reviewer gave us and as a general rule we trust the opinions of our writers. In the one case with sound, you may have a point, but I cannot say more than that is what our writer opted to give it. My theory is that he meant the sound was on par with other AAA MMOs (thus a . That said, I've sent him a link to this thread so that he can respond on his own.

As to the scale, it goes to 1 because sometimes games earn a failing grade. DDO is a solidly produced, if unremarkable title in the eyes of our reviewer. Thus, it received that score. There are many games that are much worse than DDO. It doesn't lose points for being high profile.

As an alternative example, our review of RYL gave the game 4.3/10. It earned this because the game, in the eyes of the reviewer, was nearly unusable. Incidently, I do recall a good number of advertisements for that title on our site at the time, although to be honest, I have no idea (nor do I now) who advertises with us and for how much save what I can see when visiting the page.

We feel and game companies know the faults of their products as well as anyone and no professional company would be upset by a negative review. We give out scores as the reviewer sees them with no other motivation.

Finally, I thank you for your comments. You have put into perspective a problem that has been developing as the site grows. It is tough to coordinate fairly the reviews of each of our different writers. I will be working up clearer guidelines for our review criteria and giving that to the writers. I will also look into explaining some of these things on the site itself.


Thank you for your input,

AgtSmith

Again, apprecitive of the response.

I would look forward to the reviewers comments and again, I am not trying to flame anyone.  I just want a site with information for gamers and not another place marketting to me. 

Perhaps a suggestion could arise out of this that the scoring system be revised, or revamped as it does seem a problem exists in the way games allmost automatically earn 6 points (60 being passing grade) by virtue of simply functioning.  The easiest solution would be a true academic scale whereby the reviewer assigns a letter grade (A, B-, C+, F, etc) and the scores (whole and fractional) can be calculated using the 4 point system of 3+ being an A, 2+ being a B, 0 being an F.  In this system a B- would be an 2.0 while a B would earn a 2.5, etc.  You just cannot assign 1-6 as points awarded for 'working' while 7-8 is simple the differance between average and good - the scale has to be consistant to have meaning.  

It would probably be a good idea to have some standard deductions or additions at MMORPG.com as well so scores have menaing comparitively - for instance, a game that includes porper widescreen support could earn a .2 bonus, one that doesn't support the latest DX version - 1 point, etc.  This is probably another topic in and of itself but again, the idea being to make the scores meaningful instead of just marketable.

I think this would more accurately reflect a game score to its possible experience quality for a gamer.  Having an effective range of 6 to 9 under the pretense of a 1 - 10 scale simply doesn't do justice to the review or the final score.