A Change To Our Review Policy - Early Access Reviews | MMORPG.com

System

A Change To Our Review Policy - Early Access Reviews | MMORPG.com

As so many live service games are released into Early Access while still asking for money from consumers, it's made us think about how we critique them. As a result, we're changing how we approach Early Access reviews here at MMORPG.com.

Read the full story here

ShankTheTank

I was the first to arrive.

Slapshot1188

Awesome and well needed!

Acornia

Something that I feel very strongly about that should be included in the ratings.

1. Is the game crowdsourced.

2. Has the makers made other crowdsourced games that was cancled in development, or canclded that game withing one year of the release.

3. Has the crowdsourced game been crowdsourcings for more that six months? (If so how long has it been crowdsourced.)

Over the years I have seen many games reviewed here at MMORPG that have meet the three items listed above.

Slapshot1188

Acornia said:

Something that I feel very strongly about that should be included in the ratings.

1. Is the game crowdsourced.

2. Has the makers made other crowdsourced games that was cancled in development, or canclded that game withing one year of the release.

3. Has the crowdsourced game been crowdsourcings for more that six months? (If so how long has it been crowdsourced.)

Over the years I have seen many games reviewed here at MMORPG that have meet the three items listed above.

I do not see how any of that merits including in a Review.  

A review should reflect the game, not anything related to funding.

Angrakhan

I would recommend preserving the EA score for games you revisit when they launch. That way you could get a sense for how much the game improved between EA and launch. Some games dramatically improve, some games barely change at all and they just launch to try to do a quick money grab on their way to shutting it down. Anyway, I'd like to see the original EA review as a historical reference as opposed to just completely updating the original review with updated scoring and information. Just my $.02.

BrotherMaynard

Good move, this will help balance things out a little. I would probably give a bit of a grace period to the games that are in an initial time-bound crowdfunding phase, but if they continue collecting money indefinitely, they should be fair game for all kinds of criticism and low scores. Don't be afraid to give very low scores if merited, especially if all players get for their money is unplayable junk.

Doing crowdfunding is fine for indie devs - if they set up a campaign to raise funds, with clearly defined objectives and deadlines and then get to work, good for them. I know a few that did it that way (e.g. Neocore and their King Arthur game that I backed) - setting up a KS campaign and once over they simply stopped the crowdfunding and worked on the game until its release. No shady business, pre-alpha packs, cosmetics for an unreleased game or anything similarly silly.

But if they continue asking for money in perpetuity in exchange for a broken mess (or no game at all) while hiding behind all kinds of nonsense "pre-alpha" arguments, have no mercy on them.

Sovrath

Acornia said:

Something that I feel very strongly about that should be included in the ratings.

1. Is the game crowdsourced.

2. Has the makers made other crowdsourced games that was cancled in development, or canclded that game withing one year of the release.

3. Has the crowdsourced game been crowdsourcings for more that six months? (If so how long has it been crowdsourced.)

Over the years I have seen many games reviewed here at MMORPG that have meet the three items listed above.

This sounds like you being angry at the developers of various crowd source games. I agree with Slapshot, it shouldn't matter.

However, there are people who get riled up because a game has taken a long time to come to launch. I suspect that when mmorpg.com revisits a game's review they could take that into account or at the very least mention it.

For example, if Pantheon or Star Citizen were to launch and if they were amazing (given what they each are trying to do) should they get a bad review for taking a looooong time? Some would say yes and some no.

lotrlore

Sovrath said:

Acornia said:

Something that I feel very strongly about that should be included in the ratings.



1. Is the game crowdsourced.



2. Has the makers made other crowdsourced games that was cancled in development, or canclded that game withing one year of the release.



3. Has the crowdsourced game been crowdsourcings for more that six months? (If so how long has it been crowdsourced.)



Over the years I have seen many games reviewed here at MMORPG that have meet the three items listed above.

This sounds like you being angry at the developers of various crowd source games. I agree with Slapshot, it shouldn't matter.

However, there are people who get riled up because a game has taken a long time to come to launch. I suspect that when mmorpg.com revisits a game's review they could take that into account or at the very least mention it.

For example, if Pantheon or Star Citizen were to launch and if they were amazing (given what they each are trying to do) should they get a bad review for taking a looooong time? Some would say yes and some no.

Yea, thoughts on crowdsourcing and whether they were successful are best left to regular opinion pieces. Sure, it's part of the game's history and path to get where it is, but at the end of the day the review should reflect the game itself, not the other stuff surrounding it.

That being said, it's not like we won't acknowledge this type of stuff in a review. There is a difference between mentioning the business model that led us to this point versus knocking a full number off the score because we don't personally like crowdfunding. The latter is unfair to both consumer and the developer - the content of the game itself and the monetization at launch should be the only things judged in a review.

Slapshot1188

Sovrath said:

Acornia said:

Something that I feel very strongly about that should be included in the ratings.

1. Is the game crowdsourced.

2. Has the makers made other crowdsourced games that was cancled in development, or canclded that game withing one year of the release.

3. Has the crowdsourced game been crowdsourcings for more that six months? (If so how long has it been crowdsourced.)

Over the years I have seen many games reviewed here at MMORPG that have meet the three items listed above.

This sounds like you being angry at the developers of various crowd source games. I agree with Slapshot, it shouldn't matter.

However, there are people who get riled up because a game has taken a long time to come to launch. I suspect that when mmorpg.com revisits a game's review they could take that into account or at the very least mention it.

For example, if Pantheon or Star Citizen were to launch and if they were amazing (given what they each are trying to do) should they get a bad review for taking a looooong time? Some would say yes and some no.

I’d say the company gets a bad review but the product gets measured on its own merits.

Otherwise It’s like reviewing a movie and taking off a star because you don’t like what an actor said.

Two different things. 

cameltosis

Happy to see this happening!

I've long felt that if they are charging people money, then they should be reviewed like every other product out there.

And if the product is in a bad state? Let us know, and how it compares with the price. I'm generally willing to forgive a low-quality early access game.....if they're only charging £10-£20. But if they're charging full price, it damn well better stand up against other full price games!

Brainy

I agree with this change.  Sick of these games sitting in early access pretending they are not released.  Technically every MMO is in early access as they are evolving.

The moment the customer is paying & playing its not early access anymore to me.

mitech616

This is a good idea. TOO MANY games are releasing unfinished and often broken "Early Access" titles that are asking for money but delivering little to nothing. It's good to have some professional evaluations of these products to let people know what they're going to get for their money.

Especially when we're starting to see more $30-40+ "Early Access" prices. A lot of the time these games wither never finish, or never deliver on their promises. Someone needs to hold them more accountable (maybe all of us).

Quizzical

I think it should depend on whether the developers are trying to get as many players as possible to pay to play now. If so, then that's a launch, so it's legitimate to review it. If they're openly telling people that it will be cheaper to play if you wait until later (e.g., $100 tier pledges can play now, but $50 tier pledges can't yet), then you should wait.

Vrika

Quizzical said:

I think it should depend on whether the developers are trying to get as many players as possible to pay to play now. If so, then that's a launch, so it's legitimate to review it. If they're openly telling people that it will be cheaper to play if you wait until later (e.g., $100 tier pledges can play now, but $50 tier pledges can't yet), then you should wait.

I disagree. Even at launch everyone knows that if you wait, within a year the game will have a sale and be cheaper to play.

Games will always capitalize on people being impatient, and that's not a reason to not review them yet.

Vrika

lotrlore said:

Sovrath said:

Acornia said:

Something that I feel very strongly about that should be included in the ratings.



1. Is the game crowdsourced.



2. Has the makers made other crowdsourced games that was cancled in development, or canclded that game withing one year of the release.



3. Has the crowdsourced game been crowdsourcings for more that six months? (If so how long has it been crowdsourced.)



Over the years I have seen many games reviewed here at MMORPG that have meet the three items listed above.

This sounds like you being angry at the developers of various crowd source games. I agree with Slapshot, it shouldn't matter.

However, there are people who get riled up because a game has taken a long time to come to launch. I suspect that when mmorpg.com revisits a game's review they could take that into account or at the very least mention it.

For example, if Pantheon or Star Citizen were to launch and if they were amazing (given what they each are trying to do) should they get a bad review for taking a looooong time? Some would say yes and some no.

Yea, thoughts on crowdsourcing and whether they were successful are best left to regular opinion pieces. Sure, it's part of the game's history and path to get where it is, but at the end of the day the review should reflect the game itself, not the other stuff surrounding it.

That being said, it's not like we won't acknowledge this type of stuff in a review. There is a difference between mentioning the business model that led us to this point versus knocking a full number off the score because we don't personally like crowdfunding. The latter is unfair to both consumer and the developer - the content of the game itself and the monetization at launch should be the only things judged in a review.

I think early access review should mostly ignore how the game got to this point, but the review and review score should try to estimate how likely the game is going to be developed until it's finished and in decent shape, and whether that looks like it's happening in reasonable amount of time.

Whether the game is crowdfunded or not should be ignored. But the reviewer should look at stuff like
 a) Is the game getting regular updates
 b) Are those updates big enough that the game will be finished in reasonable amount of time
 c) If the dev has history with previous early access titles how did those turn out

If there are any obvious warning sings that the game might not be getting properly finished in any reasonable time, then it should be given a lower review score.

Ginaz

What about games that are perpetually in "beta"? I'm thinking specifically of Warframe and 7 Days to Die, both of which have been technically in "beta" for more than 10 years.

Kyleran

Ginaz said:

What about games that are perpetually in "beta"? I'm thinking specifically of Warframe and 7 Days to Die, both of which have been technically in "beta" for more than 10 years.

Wrong sir! 7D2D is in Alpha 21, with Alpha 22 on the "near horizon."

I'll be quite surprised if TFP ever takes the game to beta, much less officially releases.

Word on the street is they are already working on a new game, one of those new fangled extraction games or something.

Scot

Of course as good as this decision is, studios are going to look to move the goal posts again. They got away with it for years with "early access" and no doubt will look for a new loophole. What about games first appearing on Steam as "First Taste" before they become "Early Access". You will need to pay for First Taste, just don't give us a review for something we are expecting you to pay for.

Having said that I do realise that was not what early access was meant to be, it was meant to provide indie developers with much needed cash to finish the game. But like some of us said from the start this will get abused to the hilt and it has been.

Talking of something that is already asking for money and on that basis could get a early access review, Star Citizen anyone?