If this is your first visit, be sure to check out the Forum Rules. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.
Welcome to RPGamer's new forums running under Vanilla Forums! If you're run into any odd or strange issues after our software migration please see this thread for details

Why We Should Drop Review Scores - Editorial

InstaTrentInstaTrent OpinionatorRPGamer Staff
edited September 2012 in Latest Updates
Many aspects of the video game industry are predicated upon review scores, but is this reliance doing more harm than good? I think it's high time we took a step away from review scores and let the reviews speak for themselves:

http://www.rpgamer.com/editor/2012/091212ts.html
"To tell you the truth, I like drinking tea and eating fresh vegetables, but that doesn't fit with my super-cool attitude. I guess I have to accept this about myself."
«13

Comments

  • DarkRPGMasterDarkRPGMaster A Witness to Destruction Moderators
    edited September 2012
    I have to agree. The most you need to know is whether it's worthy of buying or renting.
    User Database Check
    Pages cleared: 98
    Users cleared: 2940
    Current Year: 2010
    Spammer-Banning Battle Theme: You Will Know Our Names
    Number of Spammers banned: At least 2000 at this point.
  • CofLSilkCofLSilk RPGamer Staff RPGamer Staff
    edited September 2012
    I'm an analytical person. I like seeing numbers. The fact that review scores are heavily opinionated should be taken as a given. I usually try to find multiple scores and compare them (adjusting for rating schemes when necessary) or find reviews by people that I've found to be close to my own tastes. I honestly don't like just seeing a solid "buy" or "don't buy" when I'm looking at a review. I want to see the conclusion first. I want to see how the different aspects of the game broke down for the reviewer. Something as definitive as "buy" doesn't tell me how strongly the reviewer felt about the game. How do you quantify a polar conclusion? Could you imagine the total mess a binary system would make of Metacritic? How am I supposed to know if it is a "buy right now" or a "buy in the bargain bin?" I should be able to get all of this information without having to read a lengthy review...give me your conclusion and then I'll decide from there if the game is even worth my time to read a review about.
  • Severin MiraSeverin Mira News Director/Reviewer RPGamer Staff
    edited September 2012
    Going on a bit of a tangent because I'm deliberately ignoring its effect on stuff like Metacritic/Game Rankings, which seems the biggest issue for the Editorial, a "Buy/Don't Buy" type thing on it's own is still just as bad as a review score. If you don't have the context for why it's given that then you're quite probably going off even less information on how good the reviewer thought it was. The way around that of course is to have a "Buy/Don't Buy If..." list, which starts to get into the area of why not just read the review and figure it out yourself (assuming it's been written clearly enough to do that).

    Also being an analytical/mathsy person, my personal opinion is that review scores (including minor scores and +/- points) are an interesting metric that I generally quite like to see - as with Silk it helps me decide if I might be interested in reading the review further. But it shouldn't be taken as the be-all and end-all of everything (especially when comparing across sites/reviewers), which is as much an attitude thing for readers as anything else. Why should I be punished because others can't take them with a grain of salt? :P
    "Time is an illusion. Lunchtime doubly so."
    Twitter: severinmira | Xbox Live: Severin Mira | PSN: severinmira (EU) | NNID: severinmira
    Final Fantasy XIV: Sevvi Taubemira (Leviathan)
  • Rya_ReisenderRya_Reisender Solipsist Snowflake Full Members
    edited September 2012
    The fact that you can't really filter games on RPGamer depending on their score, makes a number completely pointless.

    In any way, I can really only agree. Back when I was still in that high-class gamer community it was completely forbidden to give reviews a score. It was also forbidden to use smileys of any kinds on forums for example. That would have lead to an instant ban. Reason behind this was basically than every good journalist should be able (or should LEARN to be able) to express all his thoughts in such a way he can convert the quality of a game without adding a number to it. The smiley rule was basically to learn to express feelings in words. It helps writing better reviews.

    The only time where I really EVER made use of scores was when I'm browsing RPG Maker game lists. But there the score isn't from one person but rather an average from hundreds of people. It helps a lot there because you will have 100000 crappy RPG Maker games and are looking for the 100 gems inside of them. In such a case a score is nice. But in a review? I don't really see the purpose.

    But then again, I also dislike stand alone-reviews. I hardly read reviews, because I think without a discussion, there is not much information you can draw from it (at least not more information than you can get by simply watching a gameplay video). In a discussion this is much better. You can talk about details there and go in depth how that can be good or how they can be bad and what kind of gamer you need to be to enjoy it.
    That's why I hardly read reviews (I really only read the + and - points, I don't read the text nor do I care about the score, RPGamer scores are total non-sense anyway, at least for my taste), but am pretty active in forum discussions about games and game design.
  • MacstormMacstorm Ysy St. Administrators
    edited September 2012
    Big difference between nonsense and something you can't comprehend.
    "The universe is already mad. Anything else would be redundant."
    Twitter @FinalMacstorm
  • tilinelson2tilinelson2 New Member Full Members
    edited September 2012
    The problem with reviews, with score or no score, is that they are nothing else than the personal opinion of the reviewer. The benefits of a well-written review come from the fact that the better a reviewer can put transform his own impressions in words, the more the reader can relate to his experience understanding why he liked or not a certain aspect of the game.

    A "professional review" is no better than an advice from a friend when it comes down to deciding whether a game is worth buying or not. It all depends on how much the recommendation or advice weighs on the decision of buying a game. I would say trusting just in one opinion is not very recommended, no matter who shared his/hers opinion, or how detailed was the opinion. However, depending on the affinity of the person with who provided the opinion, it may be enough. Someone can argue that even just the score is enough for him/her, and if he/she is happy with that, I see no problems.

    In the end, the only thing that matters is that be it just one number, or scores for a few categories, or a detailed review made by someone who played 5,000 games in his entire life, or just a "cool, bro" from a random person, everything is just a opinion and it is up to the buyer to decide if he/she will take it into account. Then, if the person follows blindly the average Metacritic score or a RPG Gamer staff review to buy a game and repents it, this person has no right of complaining about Metacritic or the reviewer because it was his/hers own decision to trust the number or the words and make it the only parameter in his/hers decision of buying the game.
  • TG BarighmTG Barighm Member Full Members
    edited September 2012
    I have to agree. The most you need to know is whether it's worthy of buying or renting.
    Pretty much this. If I did reviews my ratings would be in the realm of "Day 1 Buy, Buy New, Buy On Sale, Bargain Bin, Rental".

    Reviewers have to provide some kind of easy to print score though, so there isn't much point in talking about this.
  • MacstormMacstorm Ysy St. Administrators
    edited September 2012
    tilinelson2 said:
    if the person follows blindly the average Metacritic score or a RPG Gamer staff review to buy a game and repents it, this person has no right of complaining about Metacritic or the reviewer because it was his/hers own decision to trust the number or the words and make it the only parameter in his/hers decision of buying the game.
    Agreed. We give words and scores here at RPGamer to offer up as many parameters as possible for helping gamers understand how we felt about a game. I would never recommend someone using any review anywhere as the sole basis for purchase making. The words within can help to shape said purchase, but in the end, it's all on the buyer. We're just here to inform.
    "The universe is already mad. Anything else would be redundant."
    Twitter @FinalMacstorm
  • Rya_ReisenderRya_Reisender Solipsist Snowflake Full Members
    edited September 2012
    tilinelson2 said:
    A "professional review" is no better than an advice from a friend when it comes down to deciding whether a game is worth buying or not.
    Actually I disagree on this one. A really good reviewer is able to understand what is his opinion and what are relevant facts and can actually write a review that is completely without any personal impressions. If you ask a friend, he will just give you his opinion. That's a big difference.
  • omegabyteomegabyte He's just this guy, you know? RPGamer Staff
    edited September 2012
    Rya.Reisender said:
    Actually I disagree on this one. A really good reviewer is able to understand what is his opinion and what are relevant facts and can actually write a review that is completely without any personal impressions. If you ask a friend, he will just give you his opinion. That's a big difference.
    Yeah, that wouldn't be a review. It would be a game manual. You can't have a review that isn't at least partially subjective. It's impossible.
    "It's okay to fail as long as you learn that you failed!" - Neptune, Hyperdimension Neptunia Victory
    Follow me on Twitter
    Read my serialized, comedy-fantasy web novel, The Almanac of All Things - http://www.thealmanac.ca - Read All of Part One Now!
  • riulynriulyn Member Full Members
    edited September 2012
    I am guilty of looking at review scores to judge whether or not I'm going to buy a game, but only because I wasn't planning to buy the game in the first place. If the review scores look decent enough, I'll read the reviews and then decide. If the game is in a series I'm into, I usually just look at previews before buying the games.

    Without review scores, it'd probably take me longer to search through lists of games to find things to buy when I feel the urge to get more games, but that's probably the only big difference it would make for me.
  • tilinelson2tilinelson2 New Member Full Members
    edited September 2012
    Rya.Reisender said:
    Actually I disagree on this one. A really good reviewer is able to understand what is his opinion and what are relevant facts and can actually write a review that is completely without any personal impressions. If you ask a friend, he will just give you his opinion. That's a big difference.
    Everything that falls under the arts and entertainment umbrella can't be said to be objective. The only "objective" thing you could say about a game is whether it has bugs that render it unplayable or not. All the rest is a matter of taste. Even sentences like "it uses the traditional turn-based battle system", or "it offers world exploration" may be misleading. Besides, what are the "relevant facts" on a game? You can't explain that objectively.
  • tilinelson2tilinelson2 New Member Full Members
    edited September 2012
    Macstorm said:
    Agreed. We give words and scores here at RPGamer to offer up as many parameters as possible for helping gamers understand how we felt about a game. I would never recommend someone using any review anywhere as the sole basis for purchase making. The words within can help to shape said purchase, but in the end, it's all on the buyer. We're just here to inform.
    Yep, that is pretty much what reviews are worthy for. I'm not against reviews, and in fact I read lots of them and sometimes use their info to make a decision. Sometimes I opted for games that every reviewer seemed to dislike because the information provided was enough for me to understand I would like the game (and one of those games is one of my favorite RPGs).

    I'm more worried about the use people make of reviews, scores, top lists, etc, than the reviews, scores, lists themselves. Unless the person who writes the review, gives the score or compiles the lists lie on purpose to mislead the readers to fullfil a secret agenda, a review or a score is just another piece of information.
  • PawsPaws Purr RPGamer Staff
    edited September 2012
    *twitch* *twitch*
  • lolwhoopslolwhoops Member HalifaxFull Members
    edited September 2012
    Well, I have a blog and I game a lot so I decided at some point I'll review a game or two, or more maybe I dunno, just whatever I beat along the way and I decided that having a number as a score would be unnecessary for me. For a non professional review done on something like a forum post or a blog where you have all the time and space you want to discuss the game at length I figure it is easier for the people reading to be given an opinion such as Must Play, Great Game, Average/Pretty Good, Below Average/For Fans Only, and the ever popular Bad, Just Plain Bad.

    But for an actual review site, magazine or anything trying to be professional there should be:
    -a score
    -rating breakdown of game elements (such as graphics, story, battle system, ect)
    -pluses and minuses
    -a concise and accurate description of the high and low points

    I, as just some guy on the internet, want to just talk about a game whether I like it or not, a reviewer should be trying to inform and for that I think the score is an important part. I particularly like RPGamer's review style so I stick around here.
    lolwhoops: a Gamer's Blog
    ^^is my blog! Updates whenever I feel like it! :D which happens a lot more often now!
    this is my twitter! come twit with me
  • Rya_ReisenderRya_Reisender Solipsist Snowflake Full Members
    edited September 2012
    omegabyte said:
    Yeah, that wouldn't be a review. It would be a game manual. You can't have a review that isn't at least partially subjective. It's impossible.
    Maybe for you.
    Everything that falls under the arts and entertainment umbrella can't be said to be objective. The only "objective" thing you could say about a game is whether it has bugs that render it unplayable or not. All the rest is a matter of taste. Even sentences like "it uses the traditional turn-based battle system", or "it offers world exploration" may be misleading. Besides, what are the "relevant facts" on a game? You can't explain that objectively.
    That facts are misleading or forgotten doesn't really make it more subjective, it just makes it worse. A good reviewer should be able to tell what relevant facts are.

    The examples you gave already work pretty well:
    "It uses the traditional turn-based battle system." is a good objective fact. The readers know themselves if they like this kind of battle system or not. Just from the fact, they can tell it. However a sentence like "The battle system feels slow and boring." doesn't help the readers much as this is subjective. Just because the reviewer didn't like it, it doesn't mean that they won't like it.
  • QuinQuin ne cede malis RPGamer Staff
    edited September 2012
    Rya.Reisender said:
    "It uses the traditional turn-based battle system." is a good objective fact.
    No it isn't. "Traditional" is a subjective term.
    [SIGPIC][/SIGPIC]
    befriend (v.): to use mecha-class beam weaponry to inflict grievous bodily harm on a target in the process of proving the validity of your belief system.
  • TokyoOctopusTokyoOctopus New Member Full Members
    edited September 2012
    They dropped the numbered scores in Play magazine a few years ago... I guess it didnt really work out very well for them considering they are no longer around... I loved that magazine too.
  • NekobasuNekobasu RPGamer Staff RPGamer Staff
    edited September 2012
    Rya.Reisender said:

    "It uses the traditional turn-based battle system." is a good objective fact. The readers know themselves if they like this kind of battle system or not. Just from the fact, they can tell it. However a sentence like "The battle system feels slow and boring." doesn't help the readers much as this is subjective. Just because the reviewer didn't like it, it doesn't mean that they won't like it.
    What if the reviewer offers context for why battles feel slow and boring? The supporting evidence won't appeal to every reader, but it situates the opinion among a continuum of less hotly debated facts.
  • retrodragonretrodragon Member Full Members
    edited September 2012
    I've been wrestling with this issue for a long time. So many games receive ambiguous scores that sometimes mean its a good game and sometimes mean its total crap. When rpgamer reviews hit the 2.5-3.0 range I basically avoid them. (Incidentally, I trust your reviews more than almost anywhere else because most general gaming sites think jrpg's are going the way of the dinosaur. A theme for a different post.) But what's even worse are the ambiguous grades that games receive on a full 10 point scale. What exactly is the difference between a 6 and a 7.5? Seriously? These numbers typically include ALL KINDS of mediocre crap, as well as niche titles that certain gamers will absolutely adore.

    Bottom line is I loved the post, and I personally check numerous sites reviews, as well as make my buying decisions based on whether I am a fan of the series etc.

    When I write my own retro reviews I have chosen to avoid a numeric scale, and simply end the review by pointing out the redeeming qualities to look for when replaying an old fan favorite. The only thing I think we all really want to know in the end is if the guy or gal reviewing the game recommends it or not. Maybe in our attempts to be generous we have clouded what should be black and white, i. e. this game is good or this game is crap, with a huge range of grey.

    While I am on my soapbox, let me also point out that if you walk through gamestop (yes, I do go there sometimes on my lunch break gasp!) you will see literally HUNDREDS of games that are all a grey shade of crap. A huge portion of games being developed are done so because they could make a buck, not to share something new or artistic. So maybe we need to stop rewarding crappy games with mediocre reviews, and instead call them out as the consumer fodder that they are? My point being that sometimes numeric final scores tend to hide the crappy games behind middle scores, which serve the industry but not the individual consumer. I would rather the total number of games were less but the quality was higher. I suspect I am not alone in this...
    Playing: Wild Arms 3, Kingdoms of Amalur Reckoning, Star Ocean First Departure
    www.retrodragon.wordpress.com
  • flamethrowerflamethrower Member Full Members
    edited September 2012
    I agree with Rya. In the world of board games, different games are labeled by their mechanics (or game mechanics). "Turn-based battles" is an RPG game mechanic. There's no reason the reviewer can't mention that a game has turn-based battles and in practice, reviewers often do mention the type of battle system.

    Board games have mechanics like "cooperative" (where the players work together to beat the game), "hidden traitor" (where one or more players are secretly working against the others), "hand management" (where the player must decide whether and which cards to discard for an advantage), "card drafting" (where players must choose one card from a pack of several and pass remaining cards to the next player), et cetera. You get the idea. It's not wrong to mention which mechanics a game has. This will help gamers looking to try something new and gamers looking to avoid mechanics they don't like.
  • PawsPaws Purr RPGamer Staff
    edited September 2012
    I think we can all agree terminology is not in question. It is generally understood when someone says "grid based combat" or "level-based progression" what that entails, as it is self-descriptive. It is not useful, however, to presume readers know a fluid label like "traditional."
  • lolwhoopslolwhoops Member HalifaxFull Members
    edited September 2012
    if your only foray into the RPG genre is Mass Effect and Fallout 3 traditional is gonna mean nothing. Turn based combat in which you select actions for all your party and then the turn plays out with your party and the monsters exchanging blows is a little easier to understand.
    lolwhoops: a Gamer's Blog
    ^^is my blog! Updates whenever I feel like it! :D which happens a lot more often now!
    this is my twitter! come twit with me
  • LOLOttertardLOLOttertard Member Full Members
    edited September 2012
    I don't really trust scores since I've played plenty games that have gotten low scores that I liked, and plenty games that have gotten wonderful scores that I actually disliked.

    I more have a problem with the reviews themselves and their writers, since more often than not I disagree with the logic (and in many instances illogic) behind review scores, and each review only represents one person's biased opinion.
  • ascii256ascii256 Member Full Members
    edited September 2012
    I like the whole thumbs down thumbs up thing. With Siskel & Ebert you basically ended up with a 3 point scale based on the opinions of two guys. You had to listen to their review to determine whether or not it was an "enthusiastic" thumbs up or not.

    The problem with a 10 point scale is basically a 1-5 all mean the same thing.
  • daveyddaveyd Member Full Members
    edited September 2012
    I'd definitely like to see review sites completely do away with numerial review scores. Another issue with them that hasn't been mentioned is that many professional review sites like RPGamer has a set formula that the reviewer must abide by such as (a) score for Graphics, score for sound, score for gameplay, etc. and all of those categories must be weighted equally as they are averaged to yield the official 1-10 score. IMO gameplay is far more important than graphics / sound quality so it is very irritating to me when a the score for game that a reviewer clearly loved is downgraded because the graphics are dated or the music is generic.

    If people are just looking at the final score or a site like Metacritic, they aren't seeing the whole picture. I guess you could argue that's their problem, but it also hurts companies that release great games, and ulimately the gaming community at large. If a really fun, little known RPG receives mediocre review scores, then it may not sell well. If it doesn't sell well, then we don't get a sequel. If doing away with numbers will force more people to actually read the reviews, then I think that is a win-win (Gaming journalists will actually have their work read, and gamers may end up making better decisions.
    Currently playing: The Shadow Sun (Android), Shadowrun: Dragonfall Director's Cut (PC), Pillars of Eternity (PC)





  • Strawberry EggsStrawberry Eggs The Bemused Administrators
    edited September 2012
    I'm fine with review scores, though I almost always read the review in question to see why that score was given. I never go by one review either whenever I'm uncertain about a game to purchase. Reviews are subjective by nature, and I check multiple opinions to see if what all types of people think about a particular game. While my experience will likely be different from everyone else who has played it, I can still get a general idea of how much, or if I'll enjoy a game.
    daveyd said:
    I'd definitely like to see review sites completely do away with numerial review scores. Another issue with them that hasn't been mentioned is that many professional review sites like RPGamer has a set formula that the reviewer must abide by such as (a) score for Graphics, score for sound, score for gameplay, etc. and all of those categories must be weighted equally as they are averaged to yield the official 1-10 score. IMO gameplay is far more important than graphics / sound quality so it is very irritating to me when a the score for game that a reviewer clearly loved is downgraded because the graphics are dated or the music is generic.
    The Overall score is not an average of the scores of game's components here. to quote the review score definitions: "This is an overall assessment of the game as a whole. It is not an average."
    Reincarnation, realization
  • PawsPaws Purr RPGamer Staff
    edited September 2012
    Our overall score isn't an average, and never has been. In fact, that sort of weighting has been lifting from most sites. I'd be curious to find one who still uses it.
  • ChickenGodChickenGod Overdosing Heavenly Bliss Moderators
    edited September 2012
    Gonna avoid getting into the whole, "What is a fact and what is an opinion?" philosophical discussion here. To be honest, like Strawberry, I kind of like seeing the number that is assigned to games. Typically when I look at a review, I'm more interested in how that reviewer's judgement of a games matches my own. Sometimes I also read them for games I know I'll never buy just to see. My decision to buy a game has almost never been based on a review or a number of reviews, but rather my own research into its mechanics by watching videos and reading specific information such as "How long did it take you to finish" as an example. It is quite sad that the industry has placed so much importance upon the number that I look at as just being there for fun to act as a tl;dr or a way to rank games on a list compared to one another. The number itself is not at fault here, its people's perception and use of it, at least in my book.

    That said, I'm not sure that I like the idea of reviews changing numerical score to "Buy or Skip". More goes into a decision to purchase a game than a simple judgement of its overall quality, and I think most reviews miss or skim over key aspects as to whether someone should buy them. Take my example before, "How long does it take to finish?". All too often, this information isn't even provided to the reader. One's interpretation could also vary wildly as to what the word "finish" means, such as the fact that collecting every meaningless thing on every difficulty level. Certainly less savory reviewers would use this method to try and get games that are otherwise very good, such as Uncharted 2, to be on the "Buy" list. Would the Uncharted's be as successful as they are if every reviewer said they were a "Rent" instead of a "Buy"? The fact is that even though the Uncharteds are relatively quality titles, I would never even dream of doing anything but renting them. Then there is also the issue of replay-ability. Personally, almost no game offers any sort of incentive for me to replay them unless they are multiplayer games or I am in a very peculiar mood. Outside of MGS and Vanquish from recent memory, most single player games I only replay years down the road. It'd be a disappointment for me to see something like, "This game is about 25 hours long with 2 branching paths to take" only to find out that each individual path is only about 12 hours long, and the split comes halfway into the game, where even then the new route isn't completely different content than the path you took before. Of course, there is also the issue of rarity, as well as the MvC3 scenario where an enhanced edition might be released several months down the road. In the end, I suppose it is possible to make this system work, but I'm still more interested in the game's overall quality than a review trying to justify whether it should be bought, rented, or skipped entirely.
    "Looks like Teach just got tenure!" - Teach
  • MasterChiefMasterChief I didn't learn anything! Full Members
    edited September 2012
    Reviews are of course opinion pieces. They can only ever be the opinion of the reviewer based on his or her experiences. It's actually part of why I dislike review scores. Quality isn't exactly something you can assign an empirical number to. It requires more nuance than a simple number or a set of stars, or what have you.

    I laugh at the notion of "objective reviews," because opinions are subjective by their very nature. They're based on tastes and preferences and feelings of comfort versus discomfort. Also, a list of facts is basically an eShop press release, and serves no purpose whatsoever. Opinions are subjective, bar none. All I seek is that reviews are fair, in that the reviewer gives the game a chance. This is why it's important to have reviewers that have diverse tastes. No one cares if a dude who dislikes RPGs gives an RPG an unfavorable review, because that reviewer wasn't going to like the RPG regardless of how good it was for its genre.
    [SIGPIC][/SIGPIC]
    "What the f--- is a Shakespeare?"
    -Rico Valasquez, showing off why no one likes him.
Sign In or Register to comment.