User rating and comment

Post Reply
User avatar
Timo
Posts: 418
Joined: Sun Oct 29, 2006 5:11 pm

User rating and comment

Post by Timo » Tue Apr 15, 2014 12:23 pm

One of the common misunderstandings of a novice user and site owner is to confuse user ratings and comments. The comments are visible but ratings are partly hidden (the selected category is visible). People make assumptions of users’ ratings based on categories but it's not as accurate as one might think. The correlation between category and rating is not that straight-forward, e.g. rating for "other/opinion/religion" -categories can be anything from 0 to 100. This unfortunately causes misunderstandings. One way to avoid the misunderstanding is that sometimes our active members tell how they have rated the site on site review request. 
 
From the above: why don't we show individual user’s rating near his comment. This would allow others to get an even better idea how the individual user truly felt about the site. We already show the category and the comment so why hide the rating in such case. We can think that the user has already made a decision to share his opinion with public when he/she posted the comment.
 
I believe that increasing transparency like this would help us to decrease misunderstandings and some of the critique that we are facing. Maybe even help site owner to understand what is really going on.
 
What do you think about this? If we do this, should we show the exact rating or just dark red, red, yellow. right green, dark green? 
 
PS. Of course ratings without comment would be private and no-one can know them except the user who gave them.

User avatar
NotBuyingIt
Posts: 3313
Joined: Fri Mar 11, 2011 6:21 pm

RE: User rating and comment

Post by NotBuyingIt » Tue Apr 15, 2014 2:44 pm

My initial thoughts: I suggest a second webpage of detailed statistics for scorecards, while continuing to keep individual users' ratings confidential.

1. Currently, WOT promises that a person's ratings will be held in confidence. Therefore, I strongly warn against disclosing any ratings which have been submitted under that promise.

2. Users who leave comments are already unjustly blamed for a site's ratings. We should expect even more intensely unfair blame will be heaped upon those users whose ratings become visible. That isn't transparency and it won't be any clearer because the preponderance of the ratings would still be hidden in more cases than not. We've seen site owners demand perfect scores of 100, even when a site was rated around 92 for example; a few may have even threatened legal action.

3. To be transparent and clearer, every single rating would need to be displayed (anonymously) with the associated weight which WOT assigned to each one. Alternatively, more statistics could be displayed about the rating, say a bar graph that represents the relative number of ratings in each rating range (e.g., 5 point increments) and somehow also indicates the relative weights. Various statistical charts would take a second webpage for a scorecard, but it would very seldom need to be generated.

4. A user (or at least I) cannot precisely set the rating slide bar to assign a specific score (e.g., 87) but can only approximate. Therefore, it doesn't make much sense to display the specific rating score for that user.

5. Nothing prevents an active member from voluntarily disclosing his/her (approximate) ratings now in any evaluation request.

6. WOT has just taken two months to correct a bug that kept users from even reviewing most of their ratings. It is easy to imagine that future bugs would misrepresent a user's ratings on a scorecard, causing the user prolonged embarrassment.

User avatar
Timo
Posts: 418
Joined: Sun Oct 29, 2006 5:11 pm

RE: User rating and comment

Post by Timo » Tue Apr 15, 2014 4:48 pm

<quote user="notbuyingit">
My initial thoughts: I suggest a second webpage of detailed statistics for scorecards, while continuing to keep individual users' ratings confidential.
[/quote]

Sure this is something that we could do. However, showing histograms of users ratings with their relative weights won't work. It's because then users would be able to see their actual reputation and users would immediately start gaming with the system. Also it could create bad blood between users e.g. why I'm less trusted than xxx, I'm clearly better than him etc. In cases where there are equal amount of ratings and comments it would be clear who rated and how they rated. Then those people would get accused even more about the bad rating.

<quote user="notbuyingit">
1. Currently, WOT promises that a person's ratings will be held in confidence. Therefore, I strongly warn against disclosing any ratings which have been submitted under that promise.
[/quote]

Thanks for a candid opinion. This is exactly the reason why I wanted to discuss with active members before planning or implementing anything.

<quote user="notbuyingit">
2. Users who leave comments are already unjustly blamed for a site's ratings. We should expect even more intensely unfair blame will be heaped upon those users whose ratings become visible.
[/quote]
That is true. This was actually one of the reasons why I think this would be helpful. When rating is visible then there won't be any false claims from site owner. Surely it's still unfair to blame individual user of the whole rating.

<quote user="notbuyingit">
That isn't transparency and it won't be any clearer because the preponderance of the ratings would still be hidden in more cases than not.
[/quote]
Again true. I don't feel comfortable to even suggest a change that would remove the option to rate anonymously.

<quote user="notbuyingit">
3. To be transparent and clearer, every single rating would need to be displayed (anonymously) with the associated weight which WOT assigned to each one. Alternatively, more statistics could be displayed about the rating, say a bar graph that represents the relative number of ratings in each rating range (e.g., 5 point increments) and somehow also indicates the relative weights. Various statistical charts would take a second webpage for a scorecard, but it would very seldom need to be generated.
[/quote]
See my first comment.

<quote user="notbuyingit">
4. A user (or at least I) cannot precisely set the rating slide bar to assign a specific score (e.g., 87) but can only approximate. Therefore, it doesn't make much sense to display the specific rating score for that user.
[/quote]
I agree. Showing the exact rating would be pointless.

<quote user="notbuyingit">
6. WOT has just taken two months to correct a bug that kept users from even reviewing most of their ratings. It is easy to imagine that future bugs would misrepresent a user's ratings on a scorecard, causing the user prolonged embarrassment.
[/quote]

I'm sorry that it took so long to fix that bug. We have been busy doing Groups and other changes in the company.

User avatar
Myxt
Posts: 2089
Joined: Sat Mar 05, 2011 6:18 am

RE: User rating and comment

Post by Myxt » Tue Apr 15, 2014 11:38 pm

Let's say silver user "NeutronStar" carries a weight of "100", while platinum user "CrackPot" has a weight of "2". When they rate the trust component of dystopia.com, NeutronStar assigns 0, and CrackPot assigns 100 for his pals. Anonymous detail displays a smoothed, compressed histogram: from 100 points approaching zero to 2 points approaching 100.
________

The scorecard disclosure:
"Comments provide additional information for other users, but do not affect the reputation. Read more"
should be updated. The terms "comment", "rating", and "reputation" confuse; and the medium green "Read more" is not easy to distinguish from the remainder of the medium grey text.

"Reputation" is a non-specific, broad-brush social term. A business owner can easily think he does not need to "Read more" because, as a maker of quick decisions, he is certain that bad comments make a bad reputation. The scorecard disclosure should say something like:
"Hidden ratings determine reputation. Comments provide additional information, but do not affect ratings. [green]Read more[/green]"

User avatar
Satchman
Posts: 691
Joined: Mon Dec 28, 2009 1:08 pm

RE: User rating and comment

Post by Satchman » Wed Apr 16, 2014 12:12 am

Dear WOT Community,

I believe that individual ratings should remain confidential for security reasons and to prevent an "Us verses "Them" type of scenario were the site owners could just accuse the raters of lying. In most cases, I provide comments with my ratings, and often will say why I rated a site in in the "Suspicious or Poor category." But I also go a steps further most of the time, and note what the site owner should do to improve the rating for his/her site. (i.e Remove the malware, get rid of this unwanted toolbar, use secure encryption.) When I rate on My WOT and comment, I attempt to do two things:

1.) Provide an accurate assessment of a site's Trustworthiness and Child Safety.

2.) Explain the reason for my rating in a comment, and often what I think the site owner or webmaster can do to improve their site's experience.

Additionally, if a site is Excellent to Good, I will explain the reasoning behind that as well.

Satch

User avatar
alphacentauri
Posts: 1670
Joined: Mon Nov 02, 2009 12:52 pm

RE: User rating and comment

Post by alphacentauri » Wed Apr 16, 2014 3:52 am

I don't see any point in changing it. If you can only rate anonymously by not commenting, I would be loathe to comment. ... which means more headaches for website owners trying to clean up after their site was compromised and had its scorecard rating tank.

Right now, comments are their only clue website owners have of who to contact to request re-evaluation of their sites. But personally, I really don't want to get into an argument with someone who thinks their site should have gotten 100 instead of 92, or who wants to know why I rated two points lower than someone else.

I can think of a lot more things I would change about WOT before I would try to disrupt its most basic features.

User avatar
jpzip
Posts: 13
Joined: Wed Apr 16, 2014 6:48 am

RE: User rating and comment

Post by jpzip » Wed Apr 16, 2014 6:48 am

Isn't it so that - with this change or without this change - comments play a significant role in answering "Why?". There are very few systems in the world that can get away with not answering that question. Knowing that the meaning of life is 42 just don't cut it for most people and they start seeking for a better answer.

And, hence, in our case, as most WOT-rated Web sites do have at least one comment, the comments provide the de facto explanation, whether we mean it that way or not. And, as AlphaCentauri points out, those commentators do sometimes get unjust blame for completing this "explanation paperwork".

Maybe I am overexposed to the other review sites, but I am tempted to think that quite many WOT users - and I am now not talking about the special interest group of the particular web site owner and his/her buddies - actually read the scorecard the following way:

- look at overall reputation ("topline summary")
- read one comment that gives the supporting evidence
- if this "feels right" accept and move on
- if this "feels not quite right", read one opposite comment to establish the both ends of the opinion spectrum
- read more to form own opinion or alternatively conclude that the review is likely all over the place and hence useless.

If it was so, wouldn't it make sense to show the comment with the actual rating too? Then readers could very easily map the particular comment within the spectrum of given reputations (maybe we could even change the user interface accordingly).

So the point would not be to show how exactly the reputation is calculated or to expose people who opt to review non-publicly. But this way, we would be able to give more understandable samples about how different people felt about the site.

p.s. Let me quickly introduce myself. I am the new guy heading WOT

User avatar
NotBuyingIt
Posts: 3313
Joined: Fri Mar 11, 2011 6:21 pm

RE: User rating and comment

Post by NotBuyingIt » Wed Apr 16, 2014 11:17 am

@jpzip

Thanks for introducing yourself. As you may have read in these forums, community members hope to learn more about the changes at your company. (Perhaps a blog entry would be helpful.)

User avatar
jpzip
Posts: 13
Joined: Wed Apr 16, 2014 6:48 am

RE: User rating and comment

Post by jpzip » Wed Apr 16, 2014 11:59 am

@NotBuyingIt

Yes, I read and follow the threads quite keenly. So I am aware of the need for more context and will follow through with a blog post. Though I must say I am slightly wary of grandiose "new guy" declarations that might make things that ultimately are quite normal evolution of start-up/growth companies to sound more dramatic than they really are. So I want to first make sure I am addressing the right issues the right way. If you have anything specific that you'd thing I'd need to address in that blog post (or in this forum), don't hesitate to share.

(and yes, do continue talking about this linkage between comment and associated rating. I do think that is an important issue in making the insightful contributions of WOT community more explainable, and hence hopefully more understandable, to broader audiences)

User avatar
NotBuyingIt
Posts: 3313
Joined: Fri Mar 11, 2011 6:21 pm

RE: User rating and comment

Post by NotBuyingIt » Wed Apr 16, 2014 1:15 pm

[cite]What do you think about this? If we do this, should we show the exact rating or just dark red, red, yellow. right green, dark green? "[/cite]

Would it kill you every once in a while to add a little splash of fuchsia, puce, mauve or teal?


Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest