Page 3 of 9

RE: User rating and comment

Posted: Tue Apr 22, 2014 9:17 am
by jpzip
if you truly expect each member to leave a comment after rating, you are in for a big surprise, though not a pleasant one
That's not a consideration at all here. Mandating a comment every time would be quite counter-productive. Rather this discussion is about the inevitability of some people leaving comments, and hence those comments ending up being interpreted as the justification for the rating.

RE: User rating and comment

Posted: Tue Apr 22, 2014 11:28 am
by Timo
<quote user="alphacentauri">
If you want to clarify the full-scale part, change the scale from all positive to a negative to zero to positive scale. It's easier to conceptualize that one's site starts with a rating of average (zero) and has to be pulled up by good ratings.
[/quote]
I don't think its fair for site to get good or bad rating by default. If we don't have any information it's rating should be unknown like we currently have. Maybe I misunderstood something here.

<quote user="alphacentauri">
I didn't understand what they were talking about in that link as far as what a microformat is or is meant to accomplish. Is there a tl;dr for non-programmer folks?
[/quote]

Microformats gives search engines structured data about the content of the page. In this case search engine could display comments on search result page and show how many start user has given to this particular website, e.g. https://www.google.fi/search?q=good+restaurant+helsinki&oq=good+restaurant+helsinki (check star ratings).




RE: User rating and comment

Posted: Thu Apr 24, 2014 3:50 am
by alphacentauri
<quote user="timo">
I don't think its fair for site to get good or bad rating by default. If we don't have any information it's rating should be unknown like we currently have. Maybe I misunderstood something here.[/quote]

Ok, suppose a site has never had anyone rate. Now suppose a user with a strong reputation gives it 100% positive ratings. Without revealing details of your algorithm, is it the case that the site would have a 100% green rating on its score card? My impression is that it won't, that it will be about 65-70% green. On the other hand, if that rater gives a site a 0% rating, it's going to be pretty deeply red after a single rating.

And that's completely appropriate. A site can prove itself dangerous at first glance, but a good reputation requires not only a good first impression but also consistent good performance over time. A ten year old site with good ratings is simply more reliable than a two week old site with good ratings.

The problem is that the owners of those unproven good sites don't understand that. They come to the forums all bent out of shape that their site has a bad rating because it's less than 100%. The problem is not that the ratings aren't appropriate, but that the expectations are not appropriate. They think they started with a rating of 100% and someone gave them a bad rating that pulled it down.

However, if instead of a scale of 0-100 the scale was -50 to +50, it is easier for people to grasp that positive is all good, and that it takes a lot of good ratings to get into higher scores.

It's not about any particular default value, but about making the scale easier to understand.

RE: User rating and comment

Posted: Thu Apr 24, 2014 5:03 am
by c۞g
<quote user="jpzip">Rather this discussion is about the inevitability of some people leaving comments, and hence those comments ending up being interpreted as the justification for the rating.[/quote]
There are several hundred discussions in these forums about just that.

Though [url=https://www.mywot.com/wiki/FAQ#Ratings_vs._comments t=_self]comments are not ratings[/url] it has been argued many times that comments do have an affect when a scorecard is viewed. And when people who are confused we repeatedly explain offering a link to the Wiki FAQ. Of course, as the new CEO, I would expect you to know how WOT works. :)

Many comments are spamvertised, others are purchased, and some are simply BS. But many are genuine thoughts and opinions from the users that posted them. All must be taken "with a grain of salt."

Ratings however are private, especially those offered by users who do not have a forum registration or who are not logged in and synchronized. I would certainly hope that WOT doesn't change by forcing ratings disclosed when a comment is offered, OR when one or more rating tag is selected. This is not [url=http://www.angieslist.com/ t=_self]Angie's List[/url] Anonymity aids, though it does not ensure, protection of WOT's users. Some of us have been included in the lawsuit brought up against WOT and we do not desire to relive that experience.

IMHO, the best thing to do would be to eliminate comments altogether, return WOT to it's [url=https://www.mywot.com/blog/against-textual-comments t=_self]origin[/url] this would eliminate fraud, spam, etc. as well as reduce server bandwidth. The rating categories and reputation should be enough to "see why" a domain is trustworthy or not.

RE: User rating and comment

Posted: Thu Apr 24, 2014 7:50 am
by jpzip
Though comments are not ratings it has been argued many times that comments do have an affect when a scorecard is viewed. And when people who are confused we repeatedly explain offering a link to the Wiki FAQ. Of course, as the new CEO, I would expect you to know how WOT works. :)
I don't think educating me, you or any other people who actually are registered to this Forum is the point. The point is educating the rest of 2.4 Billion Internet users. I would argue that the discussion rising up time and time again and the answer basically being "Read the Manual" should also be considered as the proof of the current service design maybe not being the most intuitive one.
Anonymity aids, though it does not ensure, protection of WOT's users
No one disputes that. The open question is that if one decides to post open-ended comments what other things need to be disclosed so that the comment can be interpreted understandably in conjunction with the rating.
IMHO, the best thing to do would be to eliminate comments altogether, return WOT to it's origin this would eliminate fraud, spam, etc. as well as reduce server bandwidth. The rating categories and reputation should be enough to "see why" a domain is trustworthy or not.
All must be taken "with a grain of salt."
Turning back the clock is always one option, but one should bear in mind that the "web literacy/savviness" (in the lack of a better word) has increased greatly since 2006 when the core of WOT was created. The increasing need for "grain of salt" is equally applicable to people reading WOT ratings. At least I am under the impression that the general public - and again I am not narrowing down only to just raters vs web site owners - want more justification, as opposed to less justification.

RE: User rating and comment

Posted: Thu Apr 24, 2014 5:05 pm
by NotBuyingIt
Besides the WOT forums, the following comment could apply even more pointedly to the Mozilla FireFox addon reviews for WOT.

I would argue that the discussion rising up time and time again and the answer basically being "Read the Manual" should also be considered as the proof of the current service design maybe not being the most intuitive one.

Well then, what would "the rest of 2.4 Billion Internet users" consider intuitive? This is obviously an important matter for the human interface design of websites. Yet speculation about the psychology and expectations of billions of internet users is grossly error prone. That won't keep any of us from speculating of course.

When internet users are looking for an explanation of a website's reputation, displaying an unrepresentative set of ratings is unhelpful and worsens misunderstandings.

If a scorecard displays the individual ratings in its comments, webpage visitors will most often expect the reputation to be an arithmetic mean of the listed ratings. Since more often than not (I'm given to understand), the reputation is markedly different from that calculation, they will suspect the WOT system of some sort of blunder or dishonesty. As previously discussed, this is mostly how current misunderstandings arise when the consensus of the comments apparently doesn't match a site's reputation. The addition of these unrepresentative ratings should be expected to amplify and reinforce the current confusion and misunderstandings, not lessen them.

Beyond that, I don't see the attraction in having scorecard comments resemble the judging of Europe's Got Talent. Perhaps a mockup of the proposed scorecard would help to demonstrate how it would lessen misunderstanding,

RE: User rating and comment

Posted: Thu Apr 24, 2014 8:11 pm
by Guest
<quote user="jpzip">
The point is educating the rest of 2.4 Billion Internet users.
[/quote]

ehr and the teachers are supposed to be?
Just curious


RE: User rating and comment

Posted: Fri Apr 25, 2014 5:59 am
by c۞g
<quote user="jpzip">The open question is that if one decides to post open-ended comments what other things need to be disclosed so that the comment can be interpreted understandably in conjunction with the rating.[/quote]

You do not need to rate the domain to select rating tag(s) and leave a comment; neither in the old WOT 1.0 or the new WOT 2.0
example, a fictitious domain: https://www.mywot.com/scorecard/fsdfdskjfsdfkjsdf.biz/comment-78771753
I never rated it
  • [url=http://i.imgur.com/Ttgvtye.png t=_blank][img]http://i.imgur.com/Ttgvtye.png[/img][/url]
Now that's [url=https://www.mywot.com/blog/447-the-new-wot-improves-the-transparency-in-user-ratings t=_self]transparency[/url]...
[url=http://en.wikipedia.org/wiki/Transparency_%28behavior%29 t=_self]Wikipedia: Transparency (behavior)[/url]
Transparency, as used in science, engineering, business, the humanities and in a social context more generally, implies openness, communication, and accountability. Transparency is operating in such a way that it is easy for others to see what actions are performed. It has been defined simply as "the perceived quality of intentionally shared information from a sender".

I've argued for years that comments without rating are simply scorecard spam; WOT is a domain trustworthiness reputation system, comments alone are not trustworthy.

Comments are not necessarily an option.
Take for example, ratings posted with the MRT.
Staff decided that when using this mass-rating tool, mass-comments were a requirement.
This took the ensured privacy away from the user submitting ratings, a bit contradictory to ensuring ratings remain private, since people viewing a scorecard cannot "see" ratings, they can see comments associated with those ratings and determine (assume) how the referenced commentator rated. This is how certain users were identified and included into WOT's lawsuit; every individual user included in that fiasco was referenced through MRT comments.

<quote user="jpzip">Turning back the clock is always one option, but one should bear in mind that the "web literacy/savviness" (in the lack of a better word) has increased greatly since 2006 when the core of WOT was created. The increasing need for "grain of salt" is equally applicable to people reading WOT ratings. At least I am under the impression that the general public - and again I am not narrowing down only to just raters vs web site owners - want more justification, as opposed to less justification.[/quote]

WOT 2.0 introduced rating categories and their associated / grouped selection of [url=https://www.mywot.com/blog/454-new-category-selector t=_self]rating tags[/url]
Removing comments altogether would not be "turning back the clock"
It would simply define reputation by the choices offered in the category sections and streamline WOT's rating system.

Quoting the OP

Posted: Fri Apr 25, 2014 4:38 pm
by c۞g
One of the common misunderstandings of a novice user and site owner is to confuse user ratings and comments. The comments are visible but ratings are partly hidden (the selected category is visible). People make assumptions of users’ ratings based on categories but it's not as accurate as one might think.
That is because all categories and their associated selectable rating tags are displayed for all forms of ratings.
Why would "Malware" be available for a rating above dark red | red?
Why is "Good site" available for anything less than green?
WOT allows a user to select any tag no matter how they rated.
Tags should be controlled in conjunction with ratings which would offer more validity to "how" the reputation was computed.
I remember this was discussed in the closed (non-public) WOT 2.0 beta thread.
There is no such thing as a "neutral rating" I never understood having a "neutral" rating category.
The positive category only has 1 tag; "Good site" which is already infered with the green reputation.
C'mon Staff reinstate the Useful | Informative | Entertaining as separate positive-group tags and maybe create a few more which define "why" a site might be "good"
The correlation between category and rating is not that straight-forward, e.g. rating for "other/opinion/religion" -categories can be anything from 0 to 100. This unfortunately causes misunderstandings. One way to avoid the misunderstanding is that sometimes our active members tell how they have rated the site on site review request.
Forum intervention does not tell "how a user rated"
It merely informs the domain owner what problems may exist and advice which may aid to increase trustworthiness.
From the above: why don't we show individual user’s rating near his comment. This would allow others to get an even better idea how the individual user truly felt about the site. We already show the category and the comment so why hide the rating in such case. We can think that the user has already made a decision to share his opinion with public when he/she posted the comment.
Ratings are weighted.
The weight fades away over time.
Comments have no "weight" - they exist forever unless the commentator edits or deletes them.
Here's an example:
domain: anti-spy.info
scorecard: https://www.mywot.com/scorecard/anti-spy.info
There are comments from over 5 years ago, one by a user who has not deleted their account but who no longer uses WOT either.
Comments like these are confusing because people do not understand that the ratings made by the poster are no longer valid and have little if any influence with the reputation.
I proposed a method for handling this type of thing over a year ago and it was "shot down" by Staff.
re: https://www.mywot.com/forum/32236-registered-account-proposal
Now I see Staff is concerned about these comments but displaying old innefective ratings is not going to solve anything.
I believe that increasing transparency like this would help us to decrease misunderstandings and some of the critique that we are facing. Maybe even help site owner to understand what is really going on.

What do you think about this? If we do this, should we show the exact rating or just dark red, red, yellow. right green, dark green?
Since users do not have the same rating reliability, and their ratings have different affect towards any overall reputation displaying the number or color would not be an accurate display, especially since ratings fade with time, would the number or color lose focus as it's weight is reduced? I don't think so. Ratings should be treated equally whether or not a rating tag(s) are selected and/or a comment offered - they should remain private.

RE: User rating and comment

Posted: Sat Apr 26, 2014 6:42 pm
by alphacentauri
@ jpzip
You are getting a lot of pushback because these proposed innovations are at odds with the intrinsic nature of what WOT has been up until now. We accept that our own reliability score is secret, but we are assured that our ratings themselves are secret.

If there is a problem with users leaving lame comments -- and my personal feeling is that highly skilled raters have taken shortcuts and left vague, one-size-fits-all comments that fail to reflect their true expertise, and that copy-cat raters have followed them around duplicating their ratings and comments in hopes of boosting their own weight, but that's just my opinion -- then contact those users to request more useful comments. Lame comments are often used to allow use of the MRT to copy and paste unsorted groups of domains from other people's lists, and people not capable of creating their own lists don't need the MRT, IMHO.

I get that you have concerns about sites with controversial ratings. When people's pet causes have technically safe sites that are rated red, it hurts WOT's value in their eyes. So I would again suggest a new color scale for those sites, such as blue for sites with highly bimodal distributions of ratings.