Forum - View topic
Why not get rid of letter grades?


Goto page Previous  1, 2, 3  Next

Anime News Network Forum Index -> Site-related -> ANN Feedback
View previous topic :: View next topic  
Author Message
dtm42



Joined: 05 Feb 2008
Posts: 12502
Location: NZL

PostPosted: Tue Nov 15, 2011 12:44 am Reply with quote
Dan42 wrote:
I find it ironic (or gratifying?) that people are now saying that we need descriptions because the meaning of letter grades is too unclear. That was the reason I went with descriptions instead of numbers or letters for the ratings in MyAnime; the meaning is built right into the rating at the moment you choose it.


While I do like that there are descriptions, I have always been annoyed at "Not Really Good", not only because the name sounds odd but also because the description makes it sound higher than it actually is. There have been quite a few times that I've been troubled over whether to rate something as Not Really Good or So-so, simply because of the descriptions, which sound very similar. I would actually swap the descriptions of both; the Not Really Good description comes across as the one for So-so.

And I have never rated anything Worst Ever because that implies that only a single title can hold that 'title', if you know what I mean. I think it should instead be called Horrendous or Irredeemable or something like that.

At least with numbers there is no question about whether five (So-so) is better than four (Not Really Good). Language's greatest strength is also its greatest weakness, as people have different ideas about what 'Good' or 'Weak' or even 'Masterpiece' means.
Back to top
View user's profile Send private message My Anime
ikillchicken
Space CowboySpace Cowboy


Joined: 12 Feb 2007
Posts: 6908
Location: Vancouver

PostPosted: Tue Nov 15, 2011 1:53 am Reply with quote
dtm42 wrote:
While I do like that there are descriptions, I have always been annoyed at "Not Really Good", not only because the name sounds odd but also because the description makes it sound higher than it actually is. There have been quite a few times that I've been troubled over whether to rate something as Not Really Good or So-so, simply because of the descriptions, which sound very similar. I would actually swap the descriptions of both; the Not Really Good description comes across as the one for So-so.


Really? To me, "Not really good" is certainly worse as it implies an actual dislike. If I say it's "So-so", that implies mere indifference. But by specifically saying that it's not a good show I'm clearly implying that it's a bad show.

I suppose it is open to interpretation though. If you read it "[not really] a [good] show" then it sounds more negative but if you read it as "[not] a [really good] show" then it could be interpreted more just as not being especially good. So yeah, I wouldn't mind the "really" being dropped.

Quote:
Irredeemable or something like that.


This I don't mind. I don't specifically have an issue with "Worst Ever" (I tend to think that there's an implied "one of" in front of it) but Irredeemable would make sense.

Quote:
At least with numbers there is no question about whether five (So-so) is better than four (Not Really Good). Language's greatest strength is also its greatest weakness, as people have different ideas about what 'Good' or 'Weak' or even 'Masterpiece' means.


Eh, I don't know. You can argue that Not Really Good is a bad choice but aside from Not really good is there any other term on that list that you could interpret as out of order? I don't think so.
Back to top
View user's profile Send private message My Anime
dtm42



Joined: 05 Feb 2008
Posts: 12502
Location: NZL

PostPosted: Tue Nov 15, 2011 3:06 am Reply with quote
Well I think Weak -> Bad -> Awful is fine, both in terms of the actual names and the order of the names, and the same goes for Masterpiece -> Excellent -> Very Good -> Good. Decent is kind of redundant because it is a colloquial synonym of Good, at least that's how I see it. But, I don't think I would ever consider Decent to be better than Good if both are used in the same scale, so it isn't too much of a problem.

I too think dropping the 'really' would be a good move. Not Good does not necessarily mean Not Bad, but it does imply it. I don't have the same problem with Not Good as I do with the incumbent Not Really Good.

I think a language-based scale is better than a numeric one, and I don't have a problem with most of the current scale. I definitely don't think we should ditch it. But I just wanted to point out a problem or two (or three) that I have with the current scale, since we're kind of on the topic of ratings and grades and whatnot anyway.
Back to top
View user's profile Send private message My Anime
PetrifiedJello



Joined: 11 Mar 2009
Posts: 3782

PostPosted: Tue Nov 15, 2011 7:55 am Reply with quote
dtm42 wrote:
I have always been annoyed at "Not Really Good", not only because the name sounds odd but also because the description makes it sound higher than it actually is.

One of the reasons I don't rate titles is due to the number of options. Even with descriptions, I feel there are just too many. I'd use only 5: Masterpiece (A), Good (B), Decent (C), Bad (D), and Awful (F) of the 11 total.

I read the scale looks to see if there's gaming attempts and will ignore ratings by users if it's determined. It never made me feel comfortable using only five of the options could be considered gaming, so I opted to choose none of them. There would be six unused ratings by me, which is more than sufficient to determine a gaming attempt, in my opinion.

I'm not sure if others do the same, but I immediately dismiss any rating of "Masterpiece" and "Worst Ever" because they're often unjustly used (especially the Masterpiece rating, which then prompts excessive "Worst Ever" to counter) and don't give me an accurate reflection of the series. I tend to stick to the "in between" ratings to get a more accurate description of the series.

It would be nice to see a score reflected which doesn't include Masterpiece/Worst Ever options in the calculations. I think the Arithmetic votes do this, but not sure.

Though I don't participate in the rating, I do appreciate what it does deliver. It's helped take some blind buys from being "on the fence" to "in my collection" just because people rated the more So-So/Bad than Worst Ever. Green Green, according to those who warned me, should have more "Worst Ever" votes but didn't.
If you buy this, for the love of anime, watch it subbed. Trust me on this.

My two cents. Spend it wisely.
Back to top
View user's profile Send private message
Keonyn
Moderator


Joined: 25 May 2005
Posts: 5554
Location: Coon Rapids, MN

PostPosted: Tue Nov 15, 2011 12:59 pm Reply with quote
I'm a fan of the simple 0 through 10 kind of system. I find that 1 through 5, like Amazon uses, isn't quite enough for me to give what I feel are accurate scores. I run in to this all the time when I'm rating on Amazon and I feel that I really like something but 5 stars just seems too high and 4 isn't quite good enough. It would be nice if they at least allowed half stars as their overall ratings show for averages, but unfortunately they don't.

Generally though when I rate titles I simply go by 0 through 10 and ignore the descriptions. They provide a good guideline about where ones opinion might lie in the number scheme, but some of the descriptions are almost too specific.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
EireformContinent



Joined: 30 May 2009
Posts: 977
Location: Łódź/Poland (The Promised Land)

PostPosted: Tue Nov 15, 2011 1:28 pm Reply with quote
Are the numbers accurate for you to tell how you feel about the shows at all?I've never found any ratings reliable, because they tell us nothing besides number of the users who clicked on the random number. It won't tell what can we expect, nothing about pros and cons of particular show. Not to mention that in such ratings not so popular positions are usually not treated well.
Back to top
View user's profile Send private message Visit poster's website
Keonyn
Moderator


Joined: 25 May 2005
Posts: 5554
Location: Coon Rapids, MN

PostPosted: Tue Nov 15, 2011 5:22 pm Reply with quote
EireformContinent wrote:
Are the numbers accurate for you to tell how you feel about the shows at all?I've never found any ratings reliable, because they tell us nothing besides number of the users who clicked on the random number. It won't tell what can we expect, nothing about pros and cons of particular show. Not to mention that in such ratings not so popular positions are usually not treated well.


Accurate enough for what a rating can provide. I mean, ratings alone are unreliable by their very nature. For one thing, everyone has different criteria about what qualifies as a 10 or 1 or anything in between. Some people might only give a select few a rating of 10, while others might rate anything a 10 that they particularly enjoyed. There is always going to be a significant margin of error in place.

On top of that, without notes or a review from each voter a rating is also fairly unreliable in that respect as well. I see someone give something a rating of 3, but without more than that I have no idea why. Maybe they rated it poorly because there were strong elements they didn't like, or the genre itself was simply one they aren't a fan of. Well that rating of 3 is meaningless for anyone who likes those elements or that genre, but without such a note attached or a review it's impossible for anyone viewing the rating to make that distinction.

That's where the averages come in to play, as it helps offset the unreliability a little bit since you can generally assume that if 90% of the users gave something a rating of 2 or below then it's probably a bad title. In the end though a rating score is just a baseline and nothing more, it is far from reliable because of their disconnected nature. It's a bit pointless to analyze ratings to any great degree because they are a very basic baseline and nothing greater, so they never will be reliable.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
dtm42



Joined: 05 Feb 2008
Posts: 12502
Location: NZL

PostPosted: Fri Nov 18, 2011 6:38 pm Reply with quote
This is something that I've known for a while, but nevertheless it has only just really hit me; why does the ANN Encyclopedia have so many failing grades compared to the reviews?

To me, Decent is a C+ and So-so is a C. Not Really Good is D+, Weak is D, Bad is F+ (if such a rating even exists), Awful is a straight F and Worst Ever is a nightmare. Which means five out of eleven grades are failing. That's half. In the reviews there are eight passing grades and what, three failing grades? Maybe four?

Seems weird, that's all I'm saying.
Back to top
View user's profile Send private message My Anime
dormcat
Encyclopedia Editor


Joined: 08 Dec 2003
Posts: 9531
Location: New Taipei City, Taiwan, ROC

PostPosted: Fri Nov 18, 2011 6:48 pm Reply with quote
dtm42 wrote:
To me, Decent is a C+ and So-so is a C. Not Really Good is D+, Weak is D, Bad is F+ (if such a rating even exists), Awful is a straight F and Worst Ever is a nightmare. Which means five out of eleven grades are failing. That's half. In the reviews there are eight passing grades and what, three failing grades? Maybe four?

Seems weird, that's all I'm saying.

Ask yourself: do most New Zealand schools give letter grades or centigrade (out of 100) to their students? The former is A to D plus F for Fail; anything below 60 in the latter system is a fail.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
egoist
Pirate KingPirate King


Joined: 20 Jun 2008
Posts: 7669

PostPosted: Fri Nov 18, 2011 6:59 pm Reply with quote
School grades = objective.
Review ratings = subjective.

As technical as you wanna make it, I don't think it'll work out to compare or connect both.

Doesn't really matter that much to me, but that Fate Zero review was clearly too biased. Which doesn't annoy me either, since I know that reviewer tends to do that sort of stuff. If I wanted a fair review I'd either give it to Bamboo or Theron. Carl isn't bad, but I'm sure that's because his taste is similar to mine.

With that said, ANN's ratings are fine, even though shit happens. Change the system and it'll simply happen again, but with that system instead.
Back to top
View user's profile Send private message My Anime My Manga
dormcat
Encyclopedia Editor


Joined: 08 Dec 2003
Posts: 9531
Location: New Taipei City, Taiwan, ROC

PostPosted: Fri Nov 18, 2011 7:25 pm Reply with quote
egoist wrote:
School grades = objective.

What was your last time writing an essay for homework or during an exam? Is multiple choice the only type of test you've ever taken?

For the record, I'm for centigrade, or an integer out of 10. Numbers are universal; letters are not.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
dtm42



Joined: 05 Feb 2008
Posts: 12502
Location: NZL

PostPosted: Fri Nov 18, 2011 7:39 pm Reply with quote
This is not about what system the New Zealand or American school systems use.* This is just about two parts of ANN - the Encyclopedia and reviews - using different systems to rate Anime. It means that we have to use some sort of conversion method (which differs from person to person) when comparing the two.

Here is one I made earlier:

9.5-10/10___A+___Masterpiece______________4.0/4
9.0/10______A_____Excellent_________________3.5/4
8.0/10______A-____Excellent or Very Good___3.5/4
7.0/10______B+___Very Good________________3.0/4
6.5/10______B_____Good____________________3.0/4
6.0/10______B-____Good or Decent__________2.5/4
5.5/10______C+___Decent___________________2.5/4
5.0/10______C_____So-so____________________2.5/4
4.0/10______D+___Not Really Good__________2.0/4
3.0/10______D____Weak_____________________1.5/4
2.0/10______F+___Bad______________________1.0/4
1.0/10______F____Awful_____________________1.0/4
0.0/10______OH GOD, THE HORROR (A.K.A. Worst Ever and 0.0/10)

It is hardly scientific, but it suffices for an approximation.



*
In New Zealand schools do not use A to F, but rather 0 to 100. Well, the socialists brought in E to N (don't ask), but no-one pays much attention to NCEA except to say how bad it is.

However, our universities use both A to F and 0 to 100; 51% is a straight C and a passing grade, whereas 85% is an A+. At least, that's how it was at my university.
Back to top
View user's profile Send private message My Anime
PetrifiedJello



Joined: 11 Mar 2009
Posts: 3782

PostPosted: Sat Nov 19, 2011 10:31 am Reply with quote
Here's the ultimate solution:
-No grades
-No numbers

This way, some can think "Yes! Fate/Zero was trashed! A+ review!" while others can think "Damn you, Carlos. F- review!" and the comments may actually focus on the opinions, not the grade.
Example only: not to imply the review was either.

I believe the grades are simply there for the tl;dr crowd, and nothing more.

It's a shame so many people feel the grade is the final say. When someone says "But what the review said doesn't match the grade!", many of us are thinking, "No, what the reviewer said didn't match your grade!"
Colorful adjectives optional.

Then again, without the drama, would anyone post comments?
Back to top
View user's profile Send private message
dormcat
Encyclopedia Editor


Joined: 08 Dec 2003
Posts: 9531
Location: New Taipei City, Taiwan, ROC

PostPosted: Sat Nov 19, 2011 12:10 pm Reply with quote
dtm42 wrote:
This is just about two parts of ANN - the Encyclopedia and reviews - using different systems to rate Anime.

Not just two, but at least three:

Encyclopedia: 0-10 (integer only, no decimals) out of 10
Reviews (standard): letter grades from A+ to F
Seasonal previews: 1-5 out of five, with half points

Columns: case by case; Right Turns Only uses letter grades, while Shelf Life has its own proprietary system (shelf worthy, rental worthy, perishable/flushable). Bamboo's new column, The Stream, has just introduced a completely new system.

dtm42 wrote:
However, our universities use both A to F and 0 to 100; 51% is a straight C and a passing grade, whereas 85% is an A+. At least, that's how it was at my university.

We also use the centigrade system, but the passing grade is 60.


PetrifiedJello wrote:
Here's the ultimate solution:
-No grades
-No numbers

You've answered the question with the last sentence of your own post. Wink
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
V1046-R



Joined: 02 Dec 2011
Posts: 172

PostPosted: Fri Apr 06, 2012 5:11 pm Reply with quote
Whatever system you use people will complain. That is the nature of the Internet.

I wonder why people get mad over review ratings. If they are getting mad, it must mean they have already seen the show & disagree. If they have already seen the show, why are they reading a review?

Oh yeah, I remember why. Because people go out of their way to find things to complain about on the Internet. Laughing
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Anime News Network Forum Index -> Site-related -> ANN Feedback All times are GMT - 5 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 


Powered by phpBB © 2001, 2005 phpBB Group