Review Scores Must Die

Let’s get this out of the way at the top. I will be reviewing games on GameSpace. They will be scored. This categorically does not mean I cannot criticize the notion of scoring reviews. Any competent person can divorce the two. In fact, it is incumbent on media to question practices that fundamentally do not educate and inform consumers.

(Editor’s Note: As a new site, GameSpace uses a standard scoring system because without it we are unable to to obtain notice from sites like MetaCritic or GameRankings – two sites that largely determine our visibility through linkbacks. We don’t want to score games, but to grow our audience, we feel forced to.)

So for those of you who will (inevitably) falsely point out any notion of hypocrisy on my part, do feel free to go read something else that reinforces your bias. For those of you want to engage in a reasonable, thoughtful, and challenging discussion, stick around and read the article. Here’s my take.

The Set Up

Let’s first start with an example. Last summer, Jim Sterling reviewed Hellblade. His now redacted initial review was scored a 1 out of 10. Shortly afterwards, he posted a follow up video explaining why he redacted his initial score and that he’d be revisiting the review.

In short, Jim Sterling loved the game until he came across one game-breaking instance where upon he kept dying. This was troublesome for him because in Hellblade, you can only die so many times before you have a permadeath and your save file is literally deleted. He came across a bug (which I’ve seen reported elsewhere, though rarely) where he kept dying, meaning he’d eventually get the permadeath…or so it was believed.

We now know this permadeath thing isn’t real. Which makes this whole thing even more to my point.

Jim Sterling explained himself very well for why he’s revisiting the review, which he did, but again, this only furthers my point. It once again shows why review scores are useless.

Image via Hellblade

The Why

Review scores are incredibly reductive, especially in games like Hellblade where developer Ninja Theory is attempting to portray something so unbelievably nuanced and complex as mental illness through the medium of game. So reducing a game like this to a single number is not just reductive, it’s downright irresponsible and disrespectful to the consumer and to the developers. However, this same rationale can be applied to any game.

It is impossible to condense all the facets of a game to a single number, a number which in itself is meant to be an attempt at an objective representation of an inherently subjective experience. From a purely logical perspective, it makes no sense.

Scores are inconsistent across games of the same genre. For example, one can easily postulate a situation wherein the same reviewer can score two games in the same genre completely differently with seemingly no consistent means by which to measure the score.

Conversely, a reviewer can identically score two games from completely different genres.

Does a 7 out of 10 for a massive open world 200 hour RPG like Witcher 3 mean the same as a 7 out of 10 on a linear 2D sidescroller indie game which takes 6 hours to complete? Any reasonable human being would see that you cannot really compare the two, despite the “7 out of 10” score these two games share. However, in the eyes of your average gamer, they’re both 7’s.

On paper then, when judged purely by their scores (which is what many gamers will do if we’re being honest), these scores dictate that these two inherently different games from completely different genres are quite literally equal. There’s no consistency. This doesn’t help the consumer. At all.

Games Media Shares The Blame

The media no doubt plays a large role in perpetuating this issue. How can an outlet like IGN, after discussing major issues experienced by many PC gamers, score Dishonored 2 more highly than their initial review of Prey, where seemingly only they had issues? They have since updated their review score, but again, this only strengthens my point.

Image via IGN

Jim Sterling himself scored Hellblade, a game he was loving for almost his whole experience, a 1 out of 10, the same score he gave Life of Black Tiger – a completely awful game on PS4 akin to the trash you find on Steam Early Access. And he even called out this fact in his explanation video. Again, there is absolutely no consistency here. There is no baseline that your average gamer can reference to make sound purchasing decisions from.

Games media are absolutely to blame because, whether they realize it or not, they’ve trained consumers to just scroll down and look at the score instead of focusing on the content.

In other words, scores don’t do consumers any favors because they’ll just scroll down to see the number and skip the actual content of the review where all the real opinions and information are.

Circling this back to Jim Sterling, he created an issue where the point of focus became his score, not the content of his review. The whole issue was caused because he scored his Hellblade review. He then released a new review with a new score.

In other words, he fixed a problem he created – a problem that didn’t need to exist in the first place, a problem that can be so easily avoided by simply not scoring reviews.

Just look at the comments section of almost any scored review. You’ll almost always see people arguing about the score instead of the content of the review.

“But, It’s Unfair To Developers Though!”

Employee bonuses and sequels are sometimes determined by Metacritic scores. I find this completely pathetic, awful, and sad.

Again, as I spoke about in my previous article, as a discerning intelligent consumer, you must always ask yourself: What do I owe these people that I should go out of my way to defend them?

Again, the answer is: You owe them nothing. When someone asks you to spend money on a product they’re creating, the onus is on that person to ensure that said product is complete, functioning, whole, and working as advertised. Why? They’re trying to get you to spend your money.

As a consumer, the financial benefits and incentives for the person selling you that product should be furthest from your mind. If you believe that this sounds “entitled” in any way, you’re simply wrong. When you are asked to shell out your hard earned money on a product, any product, your concern – your only concern – should be whether or not that product is complete, functioning, whole, and working as advertised. That’s it. It begins and ends there.

Frankly, I don’t care about bonuses for developers, nor for anyone else for that matter. No consumer should. What I do care about is major objective business decisions, decisions which will affect creators and consumers, being made based on inherently subjective and inconsistent information. That’s asinine.

If you are somehow angered by this notion, your anger should not be directed at me. Instead of angrily typing comments or tweets at me, stop and think for a second. Actually think.

What are you really angry about? Are you angry with me? No. You’re angry about the compensation for these developers.

That compensation is determined by a model tied to review scores – an inconsistent, subjective attempt to objectively measure an inherently subjective experience. Therefore, you should really be angry with that compensation model. Instead of yelling at me, you should instead be asking the larger question: Shouldn’t this broken compensation model be changed?

The answer, of course, is yes.

Image via Kotaku

Consumers Need To Stop Being Stupid

There are, of course, ardent defenders of review scores from the consumer side of this. They are not without their share of blame.

In my experience, there are a few common “defenses” these consumers frequently regurgitate when desperately arguing in favor of review scores. Let’s examine them.

“I just want a quick score. I don’t have time to read reviews.”

No. There are plenty of gameplay videos where you can actually watch people play the game and hear their thoughts. If you want a “quick score,” you simply don’t care about how you’re spending your money. Scores are inconsistent, so a quick glance tells you nothing. This can be proven ad nauseam. Again, a quick glance at IGN’s score of Dishonored 2 would have you believe the game is fantastic and worked fine, when in reality, it very clearly didn’t.

“What if there was some constant formula across the industry?”

Even here, different individuals will interpret that formula differently, regardless of how clear the formula is. This is just human nature. This will still lead to inconsistencies. This solves nothing. It’s not even a bandaid.

Frankly, I lose respect for consumers who do place importance on review scores, and for journalists who assign them without realizing what they’re actually doing. Again, if you think there’s some deep hypocrisy here given my intro paragraph, do feel free to read something else.

By doing nothing, journalists and outlets are simply allowing this to happen, again whether they agree with scores or not.

This is why I’m writing this article. Games media, of which I’m now a part, and consumers must change their thinking. They must start thinking more rationally and objectively instead of blindly doing or saying things that inherently collapse when examined through the lens of basic common sense.

Review scores are destructive. They are reductive. They denigrate the art. They cause far more problems than they create. They drive unsustainable business practices. They insult the purchasing decision of the consumer. They rip focus away from what actually matters – the content.

Review scores must die.