This post had been a long time coming, where I discuss grading media with a score, and specifically how I do it, particularly with regards to anime on this blog, but it’s transferable.
Now, each reviewer and sometimes each review is different, there are multiple grading systems: You can have each show start at 10, and then deduct points for anything that bothers you, and then you might not know what you liked, but you certainly know what stopped a show from being perfect.
You can do the opposite, start from 0 and add points for each thing the show does well. Such reviews often have categories, such as “Production”, “Plot”, etc. and either average all of them or you can earn up to a certain mark in each score and then you add up.
I don’t engage in these forms of scoring. I also want to stress again, that the number is the least important and least interesting part of any review, especially mine – I mean, I use so many words to describe my thoughts regarding the media, so that the final few digits shouldn’t carry so much weight.
I also don’t rate all shows using the same measuring system – I have the “How much do I enjoy the show?” meter, and “How good do I think the show is?” meter (often has a lot to do with aesthetics, but not visual only, but also how I judge the plot to be). I can really love a show that isn’t that good and give it a good score, or not that good of a score. I can think a show is “objectively great” (those are scare-quotes) but not connect to it emotionally, and it might get a good grade or an average one. There’s a sort of averaging going on here, but it’s weighted by how much weight I give to the meter with regards to the particular show – that is, whether how much I like it outweighs how not-great it is or vice versa.
There’s one more thing that is important to remember here before I get to a particular case – I can think a show is an “8 show” (“Very Good”) but reduce it a point or two based on say, “intrusive fan-service” whereas if I graded by categories I couldn’t deduct more points than each category would warrant. I can think a show’s production is so sublime or so God-awful that if I had a “Production” quality that could only move the show by 0-2 points, wouldn’t allow me to deduct or add 3 points due to production values, when I think a show merits that.
Finally, let’s talk a bit about an example – Gin no Saji, my latest review post, which again requires me to mention how I don’t grade all shows equally, and again reiterate how scores are quite arbitrary and don’t mean much (I have shows I rate 9-10 I don’t actually like, and shows rated 7 which I suggest to everyone) – I don’t rate all genres the same. A slice of life getting an 8 is often at the peak of what it can get to. A comedy that doesn’t make me laugh non-stop is unlikely to get over 7, but will get a 9-10 if I laugh non-stop.
Gin no Saji made me smile, it warmed the cold cockles of my heart – but there were ways in which it fell short for me: It didn’t really make me tear up enough, they aborted the “feels” moments too soon whenever they reared their head, they had what I thought was a god-awful episode, which out of 11 episodes is quite a bit (referenced in post), they engaged too much in telling us how Hachiken is growing rather than showing us (referenced in post), I find the way the teachers talked about Hachiken’s role in his peer-group to be another act of telling us, and non-believable at that (both the teachers talking like that and that it all went so well – gross sugar-coating here, it usually works the exact opposite). So, it’s not like I thought the show was perfect, and my post, albeit briefly, touched on these points.
I also don’t begin from the assumption a show is “perfect” and then need to find things to mark it down for (which is but one grading system). I either just watch the show and then search my soul (not very hard, the numbers are arbitrary, decided by me, after all) for the number I think it deserves, or (it’s actually and) I begin from the assumption shows belong in the 7-8 territory, and shows that end up at 6.9 or lower have clear things for which I deducted points, and those at 8.1 and higher (such as this show!) are the shows I consider really worth one’s time. So in a way, I do engage in the “add/subtract” mentality – I begin everything somewhere between 7-8, and then you have to fail, or earn extra merits to break from that range. This show just hadn’t done enough to merit the 10, or the 9.
What should it have done to earn that perfect 10? I’m not sure. I find the question “What should the show have done to merit a 10?” quite weird, because even if I give you a list of things I think it should’ve done better, in the end that’s all that it is – a list, and should they add all those things, the end result might still not be perfect, which is often a show giving you more than what you expect, or something that is hard to quantify. The difference between 9 and 10 is often one of “love”, and had it been so simple to make such a list, many more shows would get a 10.
As a result of the above, here are some possible objections and questions someone had raised:
I don’t understand this. Even if it was a masterpiece of its genre, if you didn’t enjoy it, for whatever reason, you shouldn’t be rating it a 10. If you are rating based on a combination of genre + personal enjoyment + aesthetic quality, then stick with that for every show.
I also don’t like the 10 grade system much personally, being too granular – I can think a show is 93/100, but it’s a “Masterpiece” so will get 10/10, or a show is 86/100, but is an “8″ show so receives 8/10.
Thing is this – some shows I like enough that I really love them, so issues of quality aren’t as important. On the other hand, I can have a show I don’t really “enjoy” (I’ll elaborate on this later) but which I think is at the peak of plot, character and production values, so I think this show should just be judged completely on that scale. Or to put it another way, each of the two meters is out of 100, but the decision on how much I should value each is what shifts. I mean, if I think music is what a show/movie/video game should be judged based on, and I gave the music 80, and the rest of the game 100, the game’s score can be anywhere between 80 and 100. There’s a reason IGN said their scores weren’t an average back when they listed scores for each component.
Judging how important each aspect of the show is, and thus how much weight to give that aspect in the scoring is a part of the reviewing process, and definitely part of the scoring process. All reviews which don’t have a strict “sub-category” system where you add up the points in each section and total them engages in this. I’m just describing it. It’s nothing new, and almost every single review score you come across is created in this manner. And they don’t even tell you that they review each work on its own.
Now, I promised to return to this, about shows and enjoyment. I recognize enjoyment to be something that changes with rewatching, and also changes based on my mood as I watch the show, or what I watched prior. I also can “appreciate” a show for what it does, or even tell myself “This sequence is really quite sad”, but I don’t enjoy it viscerally – I can’t feel the sadness. I can on the reverse tell you a segment is incredibly emotionally manipulative and ham-handed, but I shed tears watching it. “Bias” is writ large over everything. And one day I might decide that I will give the emotional connection more weight, and another day I’ll give more weight to how I judge it to be written as an artistic endeavour. Trying to claim “objectivity” here is a fool’s errand, as what we identify as critical or superfluous is already mediated by our biases, both the transient and the stable ones.
Finally, if anyone is truly bothered by my ratings, then they can read all the hundreds of words that come before and after the score, which in the example review’s case are 4/1493, or 0.27%. As for me, I’m going to keep track of my numbers, because I see a point in it – but only when it comes along with the words.
Bonus Questions: How do you rate shows? A holistic approach, do you begin with 10 and deduct, begin with 10 and add categories? Do you have different rating systems for different media, such as 7 being a good movie but a barely playable video game? What do you think of this whole issue?
Filed under: Editorial Tagged: Rating, review 
