Mass Rating Tool

NoScams
Posts: 384
Joined: Tue Jun 22, 2010 3:43 am
Contact:

RE: Mass Rating Tool

Post by NoScams » Tue Jan 22, 2013 2:14 am

<quote user="armored">....granted access to the MRT: .....It would be up to Admin ....[/quote]

I think it's difficult to have an automated procedure for that, and doing it manually might not be possible considering the number of mywot contributors.

<quote user="scientific frontline">There should always have a thread to any MRT rating[/quote]

No. The MRT works only with an comment in it. Put what you want to say into that comment section. To open a forum topic to every site comment is totally unrealistic and will make the forum unusable, and then what's the use of writing a comment anyway?

I saw posts which have a link to the forum - but the reported site ain't mentioned in the forum. That is counterproductive since any person using that link will look for exactly that site.

NotBuyingIt
Posts: 6548
Joined: Fri Mar 11, 2011 6:21 pm

RE: Mass Rating Tool

Post by NotBuyingIt » Tue Jan 22, 2013 3:56 am

I think that I use the MRT responsibly and that my work may be of some help to WOT users, evidently excluding the OP. I choose how much or how little that I wish to say. Typical details which I include have been influenced by discussions that I have read in the WOT Forums. In scorecard comments, made with or without the MRT, I often include links to forum threads or to better informed outside sources, but I decline to participate in obligatory threads.

c۞g
Posts: 21225
Joined: Mon Jan 05, 2009 4:02 am

RE: Mass Rating Tool

Post by c۞g » Tue Jan 22, 2013 11:50 am

<quote user="scientific frontline">
The very first changes I would like to see is admin add a field in the MRT that would link to the forum post that discussed the reason for the rating. An additional comment would be optional.

Since almost all MRT ratings do have a related forum thread, this would be more informative and may cause less disputes and prevent new threads regarding a domain.

There should always have a thread to any MRT rating, without a URL the MRT would reject the rating.

Forum threads are more informative thenn a simple comment.
[/quote]

Not all MRT rated domains have a supporting myWOT forum reference.
On a personal note: I do not always create public topics for every MRT session.
Also, not every MRT Submit has multiple domains, I use MRT frequently for single domain entry; it saves time with comment creation since I incorporate two DNS reference links.

I agree that MRT comments should have a link referenced as evidence to validate the comment category / comment text.

However, a "link input text box" would only support 1 "reference" where there may be many.
It would break the comment by placing reference "outside" the comment text.
This is why the !domain was created; an example:

MRT domain list
example.com
example.net
example.org

MRT comment category
Malicious content / viruses
MRT comment

Code: Select all

Rogue Java Update
remote code execution on a target system without user authentication
ref: http<em>:</em>//www<em>.</em>gfi.com/blog/malware-masks-as-latest-java-update/
see also: http<em>:</em>//www<em>.</em>esecurityplanet.com/malware/fake-java-update-delivers-malware.html

Trojan:Backdoor::Keylogger
java.update.<strong>!domain</strong>/java/update/JSE7_update.exe

Google's SafeBrowsing Diagnostic's page
http<em>:</em>//www<em>.</em>google.com/safebrowsing/diagnostic?site=<strong>!domain</strong>

DNS: http<em>:</em>//dns.robtex.com/<strong>!domain</strong>.html
DNS: http<em>:</em>//bgp.he.net/dns/<strong>!domain</strong>
[/i]

Comments in general should have more information other than copying the comment category; ex:
Good site
Good site

Armored
Posts: 626
Joined: Tue Jul 06, 2010 10:38 am

RE: Mass Rating Tool

Post by Armored » Tue Jan 22, 2013 8:27 pm

<quote user="noscams">
I think it's difficult...
[/quote] IMO It would be worth the effort if it means only members with reasonable [url=http://www.mywot.com/wiki/Rating_reliability t=_self] rating reliability[/url] have access to the MRT.

c۞g
Posts: 21225
Joined: Mon Jan 05, 2009 4:02 am

RE: Mass Rating Tool

Post by c۞g » Tue Jan 22, 2013 9:33 pm

<quote user="armored">
IMO It would be worth the effort if it means only members with reasonable [url=http://www.mywot.com/wiki/Rating_reliability t=_self] rating reliability[/url] have access to the MRT.
[/quote]
From your link: Most users whose ratings are considered highly reliable are not very active.
So you're suggesting the MRT should be [url=http://www.mywot.com/wiki/Activity_scores t=_self]awarded[/url] at Bronze level rather than Platinum?

Armored
Posts: 626
Joined: Tue Jul 06, 2010 10:38 am

RE: Mass Rating Tool

Post by Armored » Thu Jan 24, 2013 10:24 am

<quote user="c۞g">
So you're suggesting the MRT should be [url=http://www.mywot.com/wiki/Activity_scores t=_self]awarded[/url] at Bronze level rather than Platinum?
[/quote] I was suggesting that "level" should be irrelevant to somebody having access to the tool. The 2 "conditions" I spoke of were intended to allow access to the tool only to people who have proven to be reliable over a long period of time. Off-course my suggestion would need tweaking but the principle of reliable over time is in my opinion important. So if that means it's possible for a reliable "bronze" member to have the tool rather than an unreliable "platinum" member. .YES.

It would be better than the current system based solely on "activity" which makes it possible(although unlikely) to gain access to the MRT without the need to rate ONE site. Now that is scary.

Also be careful with how you interpret Sami's words they are general and open to interpretation:
Most users whose ratings are considered highly reliable are not very active. You won't meet them at the forum and they don't rate unusually many sites. Unless Sami specifies what "not very active" & "unusually many sites" means I wouldn't be presuming that most bronze members have higher reliability than the average platinum member. Somebody who has rated 2000 websites may be considered "not very active" especially when compared to a user who has rated 1 million, and it's impossible for somebody who has rated 2000 websites to be "bronze". I'm just mentioning this because we don't have any detailed or specific information about who is reliable and who isn't. Offcourse Sami can post on this thread and tell me how wrong I am:) Until that happens we don't really know.






Jazspeak
Posts: 7295
Joined: Fri Oct 17, 2008 4:20 pm

RE: Mass Rating Tool

Post by Jazspeak » Thu Jan 24, 2013 1:00 pm

<quote user="armored">
"...be careful with how you interpret..."
[/quote]

Well said. A substantial drawback to the direction that this thread has taken is that none of us, except perhaps Sami and Co, has access to sufficient information to be able to define the best way forward for awarding and using the MRT.

For instance, without the Reliability Scores being made available to everyone, we can't definitively say whether that would be a good criterium for awarding the MRT. We can each get a rough idea about our own Reliability Score by accurately, and dispassionately, rating little known but not necessarily bad sites that have not previously been rated, and wait to see how many of the Confidence Indicators (the little men on the scorecards) darken in response to the WOT system's assessment of your own rating reliability. However, that method is far too inaccurate to decide whether the number of Confidence Indicators, and hence Rating Reliability, should be a criterium for awarding the MRT.

Perhaps another possibility would be to enforce a forum posting for every use of the MRT so that the URLs, domain names, and so on, being submitted via the MRT are also listed on the forum where they can be double-checked by anyone, and not just WOT Admin. It would quickly become apparent when someone is using the MRT inappropriately, which seems to be the main complaint at the moment. So the MRT can continue to be awarded at Platinum level of activity but its misuse can be quickly challenged if necessary.

Guest

RE: Mass Rating Tool

Post by Guest » Thu Jan 24, 2013 1:10 pm

<quote user="jazspeak">
Perhaps another possibility would be to enforce a forum posting for every use of the MRT so that the URLs, domain names, and so on, being submitted via the MRT are also listed on the forum where they can be double-checked by anyone, and not just WOT Admin. It would quickly become apparent when someone is using the MRT inappropriately, which seems to be the main complaint at the moment. So the MRT can continue to be awarded at Platinum level of activity but its misuse can be quickly challenged if necessary.
[/quote]

Thought I said that ~smiles~
Great minds think alike.

Guest

RE: Mass Rating Tool

Post by Guest » Thu Jan 24, 2013 1:27 pm

<quote user="armored">Most users whose ratings are considered highly reliable are not very active. You won't meet them at the forum and they don't rate unusually many sites. Unless Sami specifies what "not very active" & "unusually many sites" means I wouldn't be presuming that most bronze members have higher reliability than the average platinum member. Somebody who has rated 2000 websites may be considered "not very active" especially when compared to a user who has rated 1 million, and it's impossible for somebody who has rated 2000 websites to be "bronze". I'm just mentioning this because we don't have any detailed or specific information about who is reliable and who isn't. Offcourse Sami can post on this thread and tell me how wrong I am:) Until that happens we don't really know.
[/quote]

A good point there Armored. Reliability, what exactly is that and how accurate is the system in determining this, as so many factors can produce different results! Consider this scenario using two gold members rating say 2000 fake pharmacy sites:

1. Member A works from red lists contained in the forum posted by other members.
2. Member B works on previously undetected (grey) sites.

Who of the two would have a higher reliability out of them, according to WOT? Probably member A BUT does this mean member B is any less reliable! No, member Bs reliability would catch up and match Member A at some point as the grey sites get rated by more people. Thus, it seems logical to me that a persons reliability is determined by how other members rate the same scorecards.

It stands to reason that the more sites you rate, your own reliability will flutuate between a certain percentage, based on the algorithm used within WOT. Most Platinum users will predominately have a lower reliability than Bronze, Silver, or Gold members but does this mean their ratings are any less accurate! No.

One reason I pay little attention to reliability as I rate according to my findings and know that I have helped the community in some way.


c۞g
Posts: 21225
Joined: Mon Jan 05, 2009 4:02 am

off topic: reliability

Post by c۞g » Thu Jan 24, 2013 5:38 pm

Interesting
while making this reply, I came across: [url=http://www.newagepublishers.com/servlet/nagetbiblio?bno=001739 t=_self]Reliability and Quality Management
<a href="http://www.newagepublishers.com/samplechapter/001739.pdf">PDF sample chapter[/url]
The word reliability is associated with the civilization of mankind to compare one item/person
with another. Trustworthy, dependable and consistent are the words, which can be used to
give an indication of why the characteristic of reliability is valued. Reliability cannot be
precisely measured with respect to human behavior but can give an indication that a
particular person is more reliable than the other. The judgement can be easier when it is
associated with human behavior or function.


Determining the reliability of anyone takes time; it's a learning process - whether it be a new person introduced to a group, or interpretation through an algorithm. People are human so they will make mistakes from time to time, also people have differing opinions so they'll offer testimony based upon their current opinion and emotional state; though their opinions (and emotions) may change because their perception changes. For those interested in the "hardcore"
PDF (1.2MB) [url=http://www.everyspec.com/NUREG/download.php?spec=NUREG_CR6883_2005.029226.pdf t=_self]The SPAR-H Human Reliability Analysis Method[/url]

Rating reliability in WOT is merely a confidence indicator of a single user compared to a group of users; it is a dynamic "score" which changes with time. Think of it as different circles which intersect each other, but the circles are moving in and out and the points of intersection change - your "reliability" is the center "point" of these intertwining circles (a root of unity). There's an old blog post which speaks of [url=https://www.mywot.com/blog/differing-opinions t=_self]personal reputation[/url] but WOT has evolved well beyond the initial idea of a "circle of friends"

Since a user's reliability changes and assuming the MRT is to be awarded at a certain level or factored percentage as suggested, then it would be logical to remove the MRT when that reliability drops below the set threshold. One day you may have the MRT tab in your profile and another day you may not. This is why the MRT is normally awarded when reaching a specified activity level. The MRT is a tool to serve the purpose of a community and everyone within that community should be eligible to access it assuming they meet the requirements; playing "favorites" would be... prejudicial.

Ratings have a confidence level.
This confidence determines their weight, or their ability to affect change with the overall reputation.
If an MRT user has a very low confidence level (rating reliability), they can rate thousands of sites but those ratings are quickly changed when new testimony is received. This is how WOT reduces abuse within it's reputation rating schema.

The reason the MRT was removed from Gold level activity to Platinum level, was to ensure people gained more experience at ratings, overall. Back then there were little, if any, "MRT lists" on the forums, so it took people longer to identify and rate sites. While activity is a combined score of ratings and comments, albeit ratings have a higher value over comments. But with the forums becoming more active as well as the introduction of a Site Evaluations forum, many domains are referenced so it's easy to find sites and reduces the time for a Rookie to reach Platinum, which is why it may be time to increase the activity level to become Platinum.

Disclaimer
this post is information based upon my personal understanding of WOT and it's functionality,
there may be inaccuracies due to my misunderstanding or lack of information.
please don't quote me ...


Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests