Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
Open letter to MMORPG.com staff (specially review writers)
apertotes
Member Posts: 363
Dear MMORPG.com staff, i am a veteran MMORPG.com reader, as well as internet user, and i have to say, and probably many others will agree with me, that this site is the most important mmorpg related on the net. For many reasons, like its 570k members, the quality and quantity of news items, the prizes, the forums, the reviews, etc.
Since i started reading this site (almost 3 long years ago) i have seen many reviews. There have been great, good and average reviews. I still have to read a bad review, although many times i've seen incomplete or biased reviews. Precisely, review scores are one of the most debated topics on these forums. Everytime a new review is delivered, fanbois and haters start arguing about the scores, and no matter how accurate or un-accurate the scores were, they will never get to an agreement. Some WoW vs EVE discussions have even achieved a legendary status.
As an amateur reviewer, i understand that doing a review is a very personal task, and as such, scores will reflect nothing but the own personal opinion of the author. If the author loves RTS games, they will give a high score to Age of Empires. Meanwhile, someone who loves turn based games, but dislikes the frenzyness of RTS games will give a high score to Galactiv Civilizations, and a low score to Age of Empires or Warcraft. Probably, that is the reason for separated scores, so that even if the reviewer doesnt like FPS games, still can see that Fear has awesome looks and sound.
So, respecting personal preferences and opinions, i think that MMORPG.com staff, (specially reviewers) need to have some kind of editorial guidelines when reviewing. Well, not exactly with reviews, since those are quite subjective, but with scores. I will try to express it better with some examples, since my English is not that good.
Lets imagine that Godblesswow Hateeq is reviewing WoW. Since she is a well known WoW lover, we will expect a very positive review and high scores. She doesnt decieves us, and rates WoW with a 8'7 average score. Lets look at individual scores. For her, WoW graphics are very good, since they look great, and run smoothly even on 4 year old computers. Thus she rates graphics with a 10. Then, she rates roleplaying with a 7, cause even though she loves the game, she is a real good journalist, and she thinks that WoW offers a very poor roleplaying experience. Given that lag is almost non-existant and that the game runs well on almost any machine, she gives a score of 8 to performance.
Now, lets imagine Everocks Peeveepeer is reviewing EVE. We all know he plays EVE religiously 7 hours a day, and so we expect to read a "EVE cures cancer" review. And of course, it is, and the overall score is a 8'5. According to him, graphics are almost perfect. Spaceships look impressive, and so do stations, and the universe is quite realistic. He rates them with a 8. About roleplaying, he thinks that the open-endness, the ruthless pvp, the one-for-all server and the depth of the game makes for a very good roleplaying experience, and rates it accordingly with a 7. Since the game runs fine on any new computer and the lag is not gamebreaking except on rare ocasions, he rates performance with a 9.
Does anybody see any problem here? I couldnt care less that the first reviewer loves WoW so much that she traded her real life car for a virtual helmet. I think that every game should be reviewed for someone who likes the game, has played the game for a long time, and is still playing the game. To me that is only fair. I do not want to read a review done by someone that hates the game. Still, reviewers need to be honest, and even if it is their dream game, analyse both is good and bad features. Ok, both reviewers on the examples filled that first condition. They loved the games they were rating.
Then, where is problem? "Scores duuuude!!!!" Yeah, that's it. Scores. I find it perfectly fine that according to the first reviewer WoW has very good graphics. Even though in my mind they may not be anything more than average. It's her review, so if she thinks they are very good, THEY ARE VERY GOOD. In fact, i could even agree with the score. 10. Well, it is a very high score, but THE GRAPHICS ARE VERY GOOD. Then we read the EVE review. the reviewer thinks that EVE graphics are second to none. they are the best thing since soft toilet paper. Even though we may think that space is empty and that stations look like our small brother's legos, we accept that it is the rewievers opinion, and so THE GRAPHICS ARE IMPRESSIVE. since WoW graphics were very good and got a 10, we expect at least the same score, or even higher 
. And what do we see? a 8. Mmm, surprising.
But we say, "hey, maybe it was the translator error" and keep on reading the reviews. And so we get to the roleplaying part. According to the WoW reviewer, as much as it hurts her deep inside, roleplaying in wow is bad, almost non-existant. Well, after all, she looks like a honest journalist. "What? a 7?" Well, maybe it is too high for our taste, but hey, tetris has even less roleplaying. So we accept the grade, and move on to the next review. According to the EVE review, roleplaying is great, there are countless oportunities, staff driven events and, oh my god, PIRATES!!!! Well, that should score high. "Noway!!!!! 7 too?" how is it possible? didnt we agree that a 7 was for a poor almost non-existant roleplaying?
Well, i think that i do not need to continue with the example. it's clear enough. Writers should be given all the freedom to express and evaluate the game however they think they need to do it. Only ask them for honesty and knowledge of the game and other similar games.
But then, there needs to be some kind of guidance for ratings. A 7 can not be the score for both a poor and a very good feature on two different games. If the writer thinks that one part of the game is poor, then the score should be between 3 and 5, or 4 and 6. but never a 7 or higher. the same for good, excelent, perfect, horrible or bad features. every qualitative adjective should have a numeric value attached.
This way, when i see that a given game has a score of 9 on sound, i will know what to expect. Not like nowadays, that when i see the scores on reviews i dont know what they really mean.
Concluding, i hope you, the MMORPG.com writters dont take this as the critics of an unsattisfied customer, but rather like the suggestion of a faithful reader.
Have a nice day
Apertotes
Comments
i completely support your suggestion. i am tired of meaningless sevens, nines and tens. sometimes there seems that games like EVE are capped on their scores and cant be rated below 7 on anything. it may be a good game, but that doesnt mean everything in the game is perfect, or even good.
Ranking anything from 1 through 10 will make most people decide that 5 is the first "bad" grade instead of actually treating it as average. Another often repeated mistake by amateur reviewers is that they fail to inform their readers about what their reference is, ie they do not openly describe what their experiences with similar games have been and what they therefore are basing their scores on.
An example:
A reviewer has played Wolfenstein 3D and for some reason hasn't played FPS games since. Now the reviewer picks up F.E.A.R. and is blown away, scores of 10 all over the place. If the reviewer states that their only reference is Wolfenstein 3D then I'm right up there with the person, I can see where the scores are coming from.
More often than not amateurs are better off with more simplistic grades, let's say A, B, and C. They are easily understandable and readily available. A would be outstanding in its field within the comparison, B would be as expected, more or less equal to the reference while C would be something that is substantially worse than the reference.
Sometimes you see ridiculous sites/magazines where they use Percentile scoring. I often think, OK, this game got 59% Graphics, how would 61% then look?! It is nonsense.
Keep the categories as they are here on the site, but include a reference information, what game(s) are you comparing the current reviewed item to. And simplify the grades to something akin to A, B, C or 1 through 5 with an average that is meaningful.
It's not just MMORPG that does this, it's how 90% of every review I've ever seen works. Any time a popular game is released, it scores good on every front, not just the good parts, simply because if they rate it the right way the game will have too low of a rating and you'll have fanboys crying.
A fantastic example of a score that make no sense is WoW's burning crusade review:
And it scores a 6. Here we have a reviewer that says it himself; he has had NO contact with Customer Service. No test tickets, nothing, and bases his 6 rating only on the fact the game has 8 million subscribers and he doesn't see cheaters banned instantly.. but say, if you don't report them, which according to your review you haven't, then how do you know anyone has? No contact with customer server, no email, no ticket, no forum post for help, hey lets give it a 6. I don't care about the number, it could be a 1 or 10 for all I care, just pointing out that some grades come from absolutely nowhere.
Reviews will always be flawed this way. It's a pity but it's just the way it is I suppose.
It is good to have or two reviewers - doing review of the same game and discus it. And give their own marks.
Or make always the same person your dedicated reviewer. Like allready famed Desslock of Gamespot.
"Before this battle is over all the world will know that few...stood against many." - King Leonidas
Nice letter , however as you stated reviews are clearly subjective. When i see a well known gaming mag do an "offical review" give the game a 9 + with no stated cons i tend to ignore that and assume the author is biased. Use to be allbe to go to different areas and get a clearer picture of a game, but now fan flames my game better then yours is the norm.
One of the big rounds of fights here was eq 2 was going to bigger then wow before their release dates.........
In the end the best you can hope for in finding out information is it free for all PVP or arena style pvp or no pvp, anything beyond that and you get lucky. Once in a while you can see posts from players go "Why I will not buy this one" at times those can find a droplet of information that the others missed a chance to communicate.
Last round of releases we had lots of choices from fan sites as well for hints ,locations, helps, guides. While the newer games out do not have to seem those resources yet in.
As ar as ratings here go they are subjective as well . I do agree it would be nice to see a initial review 3 months in review and after first expansion . But since here recieves advertising dollars again if it is all "good" take it with a grain of copper...........
yes, i know. i just expected better from MMORPG.com
yes, i know. i just expected better from MMORPG.com
we all do
i would be very dissapointed if dolars influenced ratings. i mean, i understand that marketing aint evil, but press, at least reviews, should be objective
Seriously though, I think it is great to see someone actually being nice to this poor site for a change. With all the trolling and negativity it is good that someone has taken the time to point out MMORPG.com's strengths. Yeah, I've only been here for two months or so, and my postcount is not existent (I prefer to read than write), but I think that this site tried really hard to support the MMO community and it is great that other people are noticing this instead of giving their usual old criticisms.
Grreat post.
Oh before the WoW players say if I don't like it then cancel, I cancelled ages ago, I'm posting this in advance so that they don't have to bother winging.
Quoting people doesn't make you clever, in fact, it makes you all the more stupid for not bothering to read the quotes you post in the first place.
A small excerpt that pertains to this issue from an interview with Denis Dyack (Silicon Knights): link
"GS: Why do you want to see the press becoming more critical of product?
DD: I guess Im really against the whole notion of the enthusiast press. Being so enthusiastic that they want things to be good. I think if our medium is going to become mainstream, and were going to be considered an art form, we need true critics like the movie industry or even the music industry where people go up and literally critique something, and its a profession to critique it."
*snip*
"Previewing things and giving opinions on things in the way that the preview structure works I think thats broken. I think, in general, the type and analysis of how you rate a video game needs to change as well. To me, as an example, I just need to know that a games entertaining. If the technology is great, thats a bonus. If the sound is great, thats a bonus. If the story is great, thats a bonus. If the gameplay is great, thats a bonus. If the artwork is great, thats all a bonus. They all add up to: how entertaining is this package? Thats what I want to know. If was a critic, thats what I would write.
It seems you like are getting a lot more people now talking about reviewing games in that binary nature thumbs up, thumbs down and saying that way might be more relevant than a score out of ten.
Im strongly in favor of a thumbs up, thumbs down or a star rating. If you say something is five stars, youre saying its incredibly good, but youre not going to say that its a perfect game. When you say somethings a ten out of ten, its just not realistic. Its like if you give an Olympic medalist a ten out of ten across the board. Nothings ever perfect theres always ways to improve things. I dont ever care if its perfect; I just want to know whether I should play that game. Am I going to get an entertaining experience? Thats all Im looking for.
I think thats what critics need to do: they need to give a critical analysis of the game, and a yes or no. I think thats the best way to do it. I agree, a lot of magazines are going that way, and I think thats a good trend. I think people honestly want to become more critical, as well, and I think thats also a good thing. In order to be the most critical, and the most objective, we need to establish a process where you have the ability and the tools to do that.
I think, in the end, whether its exactly the way I say it or some variation, I this is going to be better for everyone."
Ah well that ran a little long for a snippet.
The rest of the interview is a good read anyway.
Its stupid because the OP wants to make a site that would be everyone's one and only stop for finding the perfect review of any mmorpg.
Yeah... NOT GOING TO HAPPEN!
I am not saying this site has bad or good reviews. I am saying that even if the score is 6/10 when it should have been 8/10, a person who searches the internet for a review will look at MORE than one site, heck more than 5 is what I tend to look for. No matter how many guidelines mmorpg.com will make, they will not agree with ALL the people who come looking for a review of the game they are thinking of buying but are not so sure about.
I love this site as it is because I have been browsing the web long enough to know that its not perfect. Neither is mmorpg.com and nor does it have to be. If you are so sure of your idea then start your own site and who knows, one day it might be as good.
I've been very disappointed in the fact that LOTRO review was pulled due to people complaining about the review. Many gaming magazines do the same thing every time and there is nothing wrong. Make a review, stick with it, then re-review later on after some patches/updates were made.
You, OP, are on a tried-this-before failed crusade. There will always be people with different opinions and no review guidelines will ever please everyone.
To MMORPG Staff, I know I been banned temporarily two times already, but I still like this site and I think many others do. So think about this before making any changes.
WTB Shadowbane 2
It seems to me that game developers are just sensitive divas. I get the feeling that a lot or reporters don't dare to do their job properly because they will be cut off from invites to demos, previews, pre-release information etc.
As if actors, directors, musicians, singers, tennis players etc enjoy hearing the bad comments. But its part of the deal.
No guts, no glory
i think you misunderstood the intention of my post. i dont want to agree with reviews. it is impossible to please everybody. i do not like Guild Wars at all, still here is one of the highest rated games. i dont get angry or mad for it. it is ok, i respect everybody's opinion, even more if he/she is an experienced review writer. what i want is consistency, i want a 2 to be a bad score, and a 7 a good score, a 4 a below average score, and a 9 a excellent score. that is all. i do not want to see a 7 given to something that the reviewer said was poor or lacking.
He's not forming a crusade at all, don't start putting words in other peoples' mouths otherwise your likely to be flamed for it, I'm not going to bother quoting you since you should know what you've put, but I personally think that the quote button should be removed because most people just quote others and then only bother posting one line afterwards which is nothing less than trolling most of the time.
As I said before though, I agree, the reviews on this site are ridiculous nowadays it's either "this g4me suxorz" or "it fcking rulezz!" or some similar variant even seen posts that apparently look well written but clearly someone has just gone and said it's good but chose to write it very badly without giving any reasons why it's good.
Quoting people doesn't make you clever, in fact, it makes you all the more stupid for not bothering to read the quotes you post in the first place.
Quote ^
One liner > I agree.
well, i hope that isnt the problem. i mean, i know every year, the new FIFA gets scores at least twice as high as it deserves, but i always wanted to believe that FIFAs, Age of Empires and Warcrafts were an isolated issue.
i still believe in critics honesty, and really hope that SOE or Blizzard dollars do not buy them one tenth of a point. that is why i am not aiming at high scores on bad games, but rather at inconsistency on scores on different games.
There should always be a constant "Ebert & Roeper" approach to game reviewing, which gives both a good conflict of interest and leaves the viewer with a healthy conscience to make changes on their opinion.
The out of control part... Say, if i decided to write a 6.6 score review for Gears of War; your already expecting negative comments immediately because theres someone out there who is totally in awe by seeing a number outside of a 9 range. A biased reviewer always has the same clinical reaction when he sees his review mocked by prepubescent and/or below average IQ children, even if it happened to be an unreliable source of critique.
I can only dream for a professional reviewer duo assigned to major sites (MMORPG.com/Gamespot.com etc) but until balance arrives, metacritic works like a charm!
The problem is that people are not realizing that not only is it a personal thing due to tastes in games, but other factors influence reviews as well. I was running on a pretty low end system when the game crossed my path it was a better than average game. If I were to take a look back at the same game now and measure it against what is out it would probably register around a 5.0 or a 4.5.
My recommendation to everyone on here looking for reviews: make sure the review is recent. Some games keep up well with the times through updates, others do not.
I don't review computer games but I have reviewed books in the past for magazines and newspapers in the UK.
Let me tell you this: your editor and sub-editor will actively discourage you from giving a book a negative review (in fact, the first book review I ever wrote for a major UK magazine was spiked because it was negative).
There are two simple reasons for this:
Firstly, all newspapers and publications do not wish to run the risk of any kind of lawsuit for defamation or loss of business;
Secondly, if a newspaper or publication does publish negative reviews then they will be cut off from the free copies, advance news, publishing parties etc etc that are such a feature of the book trade. I imagine that the computer game industry is not so different.
The only people who can publish a negative review in the UK press are other authors - ie writers or celebrities who are big enough for the newspaper to shift the onus of blame off their own shoulders and onto those of the reviewer. Otherwise, all book reviews are always 'Good' or 'Average' and that's it.