Monday, 25 July 2011

Reviewing the Reviewers 4: Raters Gonna’ Rate

Now, a review might be well or poorly written. Done in a highly formal or completely informal style. But sometimes things like that won’t matter; hell, sometimes nothing that the reviewer even really said is going to matter. I’ve done it from time to time, and I’m sure you have as well. Skipping the review to simply get to the rating. Perhaps you’re in a hurry and you’ll read the review later, or some other convenient excuse. Like I said, I’ve done it too, so don’t feel bad.

The fact of that matter is that for all the talk about a game, that people crave that seemingly quantifiable snippet that summarizes everything: 4 out of 5, 7 out of 10, Buy/Rent/Skip, however you sum it up, the rating is kind of like the cherry on top of the sundae that is a review: it is, at a glance, either an endorsement or rejection of everything that a game encompasses in and of itself. That’s why the type of rating is so important.

There’s actually a few different types of rating systems, all of them with their own pros and cons. First and perhaps most rarely seen is a sort of unique rating system. Perhaps this system is sort of themed or has a gimmick. For example for this site, I could go with little blurbs for ratings like “Grahf Good” “Grahf Great” “Grahf Garbage” and so on. But I won’t, mostly because I think it sounds absolutely insipid, but also because these kinds of ratings might be the most useless overall. While any rating that you give a game is going to again be a personal reflection that cannot help but be based off your own opinions, using a personal rating system puts another level of minutia where it isn’t really necessary. With only those three ratings above for example, it’s extremely hard to tell where a game falls. Like, if a game is “Grahf Good” then is that anywhere between say 55% to 75% on a 100% scale? Or are those ratios even more skewed? Without a lot of definition it’s hard to tell.

This is why the point rating system is the industry standard. It comes in many forms: from the outright ridiculous like the 100% with decimal places, to the minimalist five point scale with no half places. This scale is ultimately appealing because it’s extremely easy to understand and employ: a one is utter garbage on a five or ten or even hundred point scale, where as on those same three, anything above a four, seven to eight, or 75 to 80 is probably going to be something worth looking into. This system is not without its faults though. For example, when breaking into a 100 digit range, as some systems that rate from either 0-100 or 0.0-10.0 do, it becomes extremely tedious to attempt to discern what exactly separates games that score only a decimal place or two apart. While this normally isn’t a huge concern, it does raise some issues if there’s a lot of competition on the market - especially between games that are from the same genre - and can lead to a larger sense of redundancy if seen a lot. The other far more prevalent problem is that when using a system like this, reviews will tend to be given an average that is weighted more towards the positive side of things.

In a lot of gaming magazines it’s not uncommon to see a seven and be able to read it as a five. Whether or not this is the fault of the reviewers or pressure from external sources is anyone’s guess, but it often comes that the average will fall more towards the middle high standard which in turn devalues the scale somewhat, because if an otherwise unremarkable title can achieve a six or seven, then getting an eight, nine or ten seems to have less of an impact. This actually was the argument that EGM used when they abandoned the numeric scale some time ago, in favour of the last kind of scale I’m talking about: the letter grade.

Like many, I was at first quite confused about the switch to letter grade that EGM employed. I remember believing that the ten point scale was more than good enough criteria to judge a game by. But in the years that have passed, I have considered the reasoning behind the change and I believe that there is some good rationality behind it. First of all, using a letter grade scale with plus/minus variance somewhat eliminates the problem of decimal places causing a general apathy in regards to scores. It also avoids a more long term problem: the perfect score dilemma. Although (it should) only happen rarely, when a game gets a 10 or 100 or what have you, it’s a big deal, especially if it’s from a publication with some clout behind it. But a 10 or 100 means that a game is, well, perfect… that is the perception at least. Where there things that these games could have done better? Of course there were, there always will be. And when the next game that gets the coveted perfect score comes along, the comparisons to past perfect games will be inevitable.

Using an A+ breaks this cycle. It denotes that game is a definitive example of how a game of that genre should be, without placing the burden of having to be “perfect” on that game’s shoulders. It means that it’s the peak of what has been accomplished to date. Those last two words are the most important part there. It means that it’s good, great, fantastic even, but that there is and always will be room for improvement.

Letter grades also have the added benefit of supplementing a written review well, and making the reader curious about the content of said review. For some reason a letter grade makes us ponder just why the recipient was judged so, more than a number grade tends to at least.

Of course, again, no system is perfect. But I believe that as with all things, that the systems we use can make strides forward much like the games that they are reviewing, and while things might never get perfect, perhaps they can always get better.

1 comment:

  1. You know there's a vid i'd like you to watch, its by GrapplingIgnorance on Youtube, and its his comment on why games deserve to be respected as an artform. http://www.youtube.com/watch?v=cCR69vm4V0M

    ReplyDelete

Note: only a member of this blog may post a comment.