Forum

This poll is about what kind of approach one thinks best for the scores that accompany anime reviews.

The question of the worth of scores has been raised before and disputing scores often becomes a significant, even dominant theme of review discussion threads. So I’d like to know how people feel about the use of scores and the kind of scores used. I think it might be interesting to see whether people object to scores in general, or think that the system used is more significant.

I’ve tried to avoid an obvious ‘middle choice’ because I think that too many people would just flee to the center rather than make a choice. I thought about a poll with a while range of possible systems, but I felt that would just make the results hard to interpret.

For convenience, I’ll define the terms as I imagine them, but if a different meaning emerges from use, I can’t complain, only reïnterpret.

Maximally Granular: Systems that have the most possible scores short of absurdity with small distances between them. (E.g. Percentage scales; 10 point scales with decimals)

More Granular: Systems with a large numbers of scores, but well short of the maximum. (E.g. ANN’s present 15-score F- to A+ scale; 10 point scales without decimals)

Less Granular: Systems with relatively few possible scores, but enough with give the impression of some specificity. (E.g. 4-Star and 5-Star scales without half stars)

Minimally Granular: Systems with very few, very broad categories. (E.g. Shelf Life system; thumbs up/thumbs down)

No Scores: The absence of any scoring system, but perhaps with rare special distinctions for extraordinarily good and bad things.

I don’t like the use of scores for reviews. I well recognize that they won’t be abandoned, but that doesn’t mean that I have to like it. I believe that scores are ultimately unhelpful to any of the ostensible purposes of a review, viȝ. to inform opinion and provoke discussion, and can in fact distract from or corrupt discussion.

The first, most obvious problem is that scores cannot adequately represent a full opinion. A single letter, perhaps with a mathematical operator appended, cannot stand in for twelve hundred words, or even a sentence with more than one clause. If we end up discussing a score, it’s not really a discussion of the review or even opinions on the subject of the review anymore. This is exacerbated by the illusion of an absolute, common value that scores can create. The application of scores varies not just from reviewer to reviewer, but from review to review and the interpretation of them varies from person to person as well as over time. People aren’t perfectly, or even highly consistent, but by specific values assigned as judgements on something can create an illusion that there is some absolute standard. The scores are applied subjectively, but masquerade as an objective measure. Two people may find themselves squabbling over the score that something was given because while the score appears as though it somehow represents a common value, in fact the people each assign different values to it, so they will argue under a misleading assumption that conceals the actual substance of their opinions.

The problems with scores become worse because it’s very easy for them to overtake discussion. One of the purposes of a score is to provide a simplified representation of opinion, so as people seek the path of least resistance, they can easily be drawn to fixating on the score because by its nature a score is easier to grasp and deal with than a fully expressed aesthetic judgement. It’s easier to argue about something like a score that looks as though it can be discussed abstracted from context than passages of text that are obviously a part of a larger, interconnected whole. I think that there’s also a problem because of the adolescent male status competition that runs through the internet (not necessarily practiced only by adolescent males). There’s a certain desire to ‘win’ and scores provide an artifice for racking points up. It’s hard to win on the basis of the best expressed and most thoughtful opinion, but easy with 7.5 versus a D-. That’s bookkeeping, not opinion, but it’s easy to tally.

With the anonymity, lack of personal connections and invisibility of body language or other features of personal interaction, it’s easier to value the particular opinions or perspectives of individuals. As such, people can tend to look toward scores as something that, again, seem independent of subjective judgements. Within the somewhat perverted social structure of the internet, this makes it easy to integrate the scores that appear to correlate to one’s opinions into one’s sense of identity. In lieu of a complex construct of past experiences, beliefs, intelligence, knowledge and perspectives, we can be tempted to put numerical values that can be simply ratcheted up and down. The internet is by its nature an inhuman place to interact, so there is a temptation to dehumanize ourselves within it, which I believe the use of scores can enable. This is a worse case and I don’t think that it really exists in its pure, horrible form, not even within the score-obsessed stygian bowels of contemporary video game fandom, but I nevertheless worry that scores encourage the lesser angels of our nature.

I will concede, however, that I see a value for scores in limited contexts. They can be useful as a categorization mechanism. If a scoring system is not very granular, thus conveying a relatively vague implied judgement it can rest between deceptive certainty and irrelevance and be useful as a guidepost for finding things that one might enjoy. Such vague scores don’t supply enough information to make one feel in any way informed about the quality of something, but can create a category of things that might warrant consideration and closer examination. I think that the Shelf Life system is a fair example of this. It has only three values, each with a broad reach. I can only have a vague sense that things deemed ‘Shelf Worthy’ tend to be good, so it seems foolish to choose something just because it attained that, but it creates a useful category of things to look into, usefully sorting a lot of the things likely not to be worthwhile out. Even more granular systems can be adapted to this with some creativity if one doesn’t fret over things like the difference between B+ and A- and just treats the whole B- to A+ range as a general category. The special recognition of the A+ might have some merit, but I suspect that something like an ‘editor’s choice’ award or other special commendation that doesn’t appear integrated with a value system would serve that purpose better.

If we are to have scores, I think that the less granular they are the better. As suggested, I prefer systems with relatively broad categories that don’t appear to systematic, such as the Shelf Life system or [i}Siskel & Ebert[/i] thumbs up/thumbs down system. These provide a potentially useful tool for guiding the audience without appearing systematic or illusorily rigorous enough to enable the worst features of more granular systems. The most granular system that I like is a five-star system with no half stars. (i.e. Scores of *, **, ***, **** or ***** only) Lowering the number of possible scores widens the breadth of each, thus reducing the variability so while three stars still won’t be used consistently and be interpreted the same way by all, there is likely to be less disagreement. It can also function as just an extended three-value scale with most things falling between two and four stars while the one and five star scores serve as rare, special distinctions for works that are extraordinary, one way or another. (This, incidentally, is how I handle things for MyAnime and MAL; they are coded for, respectively, eleven and ten point scales, but I assign only scores of 2, 4, 6, 8 or 10, each being equal, in my mind, to twice what I would assign on a strict five-star scale)

I voted for minimal. I use a three point system for my own purposes.

Good (entertaining, worth watching etc.) this would cover the bulk of what I have.

Very Good (stuff I really enjoyed) a handful of titles

Dispose of (bad purchase, get this out ot the house, etc.) Assigned only until the show is, well, disposed of. Since I started following ANN and with the advent of streaming this catagory has been small, but I do still get caught by other's enthusiasm on occasion.

I think anything more five catagories is too many. Even if the person who assigns a rating on a ten point scale understands the distinctions he has made and is consistant, it won't translate to someone else.

I like the rating system on My Anime here, because it seems more subjective than other scales. "Very Good" can mean different things to two different people, as can 8/10 or B+ etc etc. Yet somehow, people seem to get far more defensive about letter/number grades than they do about the My Anime scale. I think the closest to that on your poll would be Less Granular, so that's what I picked.

Thinking about it, for a single review no scores at all would probably be best - ANN's reviews, for example, could replace most of the scores with short comments, even if they're just summaries of the points stated in the main body.

However, for mass-review systems, a scoring system definitely becomes necessary. A ten-step scale is almost certainly the most they'd ever need, though.

I rather like something along the lines of the Shelf Life rating system. You'll never have any uniformity with a 10 point scale or even a A-F scale, but the simplicity of the pretty good/fair/don't bother scale can actually say a lot more. It's succinct and to the point.

I've come to realize that I don't need differing rankings for bad anime. 0-4 on a 10-point scale is pretty much worthless. Shitty McShitterfadden Shit anime is Shitty McShitterfadden Shit. And I rarely ever operate in that area, because I tend to avoid anime that I'm pretty certain won't have many qualities to like about it.

A normal bell curve occurs between 5 and 10. So all I really need is a 0, 5, 6, 7, 8, 9, 10. A five star scale comes close enough to this, and I think I could somehow figure out a way to categorize them.

I only care about chocolate granular,

MOAR GRANULAR

I prefer minimal. Sometimes I just need a straight up "skip it", I don't need to know to which degree it is terrible. If it's rent or buy it, then I prefer to ask other people whose opinions I trust if it's worth my time or not. I feel like actually reading is the best way to go if I have even a passing interest and buy, rent, or skip is a good, simple way to more broadly categorize them. I mostly want to know if something is good or not, if things are hyped up too much and don't live up to them, then I find the rating system worthless. Likewise, if they're slammed but they're actually quite good shows, it's also worthless. I feel like three categories, good, bad, and average, means I can find out for myself how good without as much hype as a perfect score might give. But since I want some sort of bottom line, no score isn't quite helpful either, so I want minimalist scores.

Forum - View topic
Scoring Systems for Reviews

GoHands Reveals Original Anime Momentary Lily for January 2025

SAND LAND Video Game Review

This Week in Games - Atlus With the Old, In With the New

The Story of Suikoden

This Week in Anime - What's Weekly Shonen Jump's Next Big Manga?

Voice Actor Takaaki Seki Dies at 54

The ANN Aftershow - Will You Watch Solo Leveling Season 2?

Dark Moon: The Blood Altar Volumes 1-2 Manhwa Review

Interview: Voice Actors Rika Hayashi and Hiroki Tōchi Talk About Their New Golf Anime, Tonbo!

This Week in Anime - So, Is Sound Euphonium Queerbaiting or Not?

Wonderful Precure! Episodes 1-12 Anime Review

	Anime News Network Forum Index -> General -> Polls	All times are GMT - 5 Hours
Page 1 of 1

Forum - View topicScoring Systems for Reviews

Forum

Forum - View topic
Scoring Systems for Reviews