Criticism thread for my ratings methodology

Supported
Posts: 10
Joined: Fri Aug 19, 2016 9:56 am

Criticism thread for my ratings methodology

Post by Supported » Thu Sep 29, 2016 3:48 am

Hello myWOT community, hope you all are doing well and that this is the right forum. :)

I've been a member for a little over a month, and I suppose we're approaching that time where I either will soon be or already am being evaluated for the next step.

As noted on my status, since the initial phase of reviews is more or less finished, I am attaching a copy of my statement on methodology here, as well as a link to My Scorecard Comments, to be open for review by anyone who may so be interested. I will be responding to replies as best as I can.

In truth, I can see a number of potential weaknesses in my general guidelines.

As I am still relatively new to this forum, I would indeed greatly appreciate any suggestions and/or constructive criticism that you may have to offer.

Respect statement: No matter what the nature of the replies are, I will do my best to treat them with the utmost respect and consideration. In my book, you all are the most prestigious members of this sector of the World Wide Web.

P.S.: Unfortunately, it seems that the scope I have listed is fixed; with the amount of work already invested, there doesn't seem to be much that I am allowed to do that can expand upon it. Sometimes I regret this choice, and wish I had ventured into a 'Scamsite' related category instead, where so many of this community's most valuable members seem to operate. At this point, I would hopefully like to know how I can focus on improving the quality of my future comments in this area, while keeping the information presented relevant. Additionally, there doesn't seem to exist a very large number of options under Positive Reasons; the only relevant ones I can find are 'Good site' and 'Other'. Nevertheless, I look forward to the upcoming castration (or was it castigation) from you all. Thanks!

P.P.S. I have already received criticism from the myWOT team, found here for privileged members at hxxps://www.mywot.com/user/7764612/board#comment-2240303, although I'm not sure if in writing it the information I had presented has been read through fully.

The following is a relatively lean description of my methodology, to be amended constructively in the future through, hopefully, the aid of your suggestions.

==================================================================

Short Foreword on Scope:

To best avoid the risk of mis-rating, I have temporarily restricted my scope to the sites of public universities and community colleges, art academies, music conservatories, professional schools (currently medicine, dental, pharmacy, law, architecture, business, veterinary, library, public policy, and nursing), and widely-recognized private post-secondary educational institutions. From experience, I've found them relatively easy to comprehend, whereas constituents of other industries prove less reliable in this regard (e.g. in the financial services sector, organizations like Point72 Asset Management and Icahn Enterprises L.P. seem harder to evaluate for factors such as general trustworthiness). Still, I recognize it is possible to objectively address the cyber-safety of these websites using third-party tools, and may eventually do so. For now though, I'm sticking with schools.

==================================================================

My Comment Methodology:

When evaluating a site for comments, I generally look at four parameters, which are expressed respectively on the scorecard using four keywords.

Official [əˈfiSHəl] - The site actually represents the institution it claims to represent. Supporting evidence for this is compiled using, where applicable, program directories published at specialized accreditation agencies recognized by the U.S. Department of Education (for art academies and most professional schools), Wikipedia article or Google Search rankings heuristics, and publicly-available WHOIS information provided by the Internet Corporation for Assigned Names and Numbers (ICANN), a non-profit organization operating under functional contract with the U.S. Department of Commerce. This parameter contributes significantly toward determining the site's 'Trustworthiness' score under My Ratings. Any part or component of a rated site that does not fall under this definition is not covered in the evaluation.

Public [ˈpəblik] - The site's content is publicly accessible. This is as opposed to material locked behind membership areas, single sign-on systems, password-protected directories, etc. Any part or component of a rated site that does not fall under this definition is not covered in the evaluation.

Website [ˈwebsīt] - The site is a group of World Wide Web pages usually containing hyperlinks to each other and made available online by the institution (Merriam-Webster). This is as opposed to directory browsing, downloadable files (such as portable document formats or media documents), landing pages, web applications, and 3xx redirected addresses. Generally, more than one web page must be hosted on the domain. Any part or component of a rated site that does not fall under this definition is not covered in the evaluation.

Trustworthy or Reasonably Trustworthy [ˈtrəs(t)ˌwərT͟Hē] - The institution faithfully provides the services it claims to provide, in both nature and quantity. Supporting evidence for this relies on, where applicable, my scope limitations, which partially restrict the amount of risk faced in this area, registration under the EDUCAUSE global-level top domain, which itself limits eligibility to webmasters operating under the express permission of "U.S. postsecondary institutions that are institutionally accredited" by "agencies on the U.S. Department of Education’s list of Nationally Recognized Accrediting Agencies", for public schools the underlying assumption that state and federal funding come with regular methodical checks and inspections, fifty million dollar endowment minimum for private educational institutions (pursuant to the philosophy that non-credible organizations generally fail to secure significant donations), availability of information describing academic programs or disciplines offered, availability of instruction regarding the admissions or application process, recognition from independent rankings systems such as the U.S. News and World Report, mention of notable staff or faculty, and for professional schools accreditation status with the Liaison Committee on Medical Education, Commission on Dental Accreditation, Accreditation Council for Pharmacy Education, American Bar Association, National Architectural Accrediting Board, Association to Advance Collegiate Schools of Business, American Veterinary Medical Association, American Library Association, Network of Schools of Public Policy, Affairs, and Administration, or the Commission on Collegiate Nursing Education. This parameter contributes significantly toward determining the site's 'Trustworthiness' score under My Ratings.

The site receives a scorecard comment only if all of the above parameters have been successfully verified; the use of keywords serve as my recognition of their validity.

==================================================================

My Ratings Methodology:

On Web of Trust, ratings consist typically of three parts.

Trustworthiness - As mentioned before, my Trustworthiness rating is heavily influenced by two parameters, those being my Scorecard Comment opinion of whether the site is a) 'Official' and b) 'Trustworthy or Reasonably Trustworthy'. If both are deemed valid, the site receives a score of 'Good' (light-green on the bar) or higher. So far, failed candidates have not received a rating for this part.

Child Safety - To be forthright, it is my opinion that the nature of this category is more subjective than the rest, since each parent has his or her own idiosyncratic preferences for what they are willing to expose to his or her respective child. These discussions are often controversial and heatedly debated within social groups and among family members. As I am neither a minor nor a parent, this rating is even less tangible and even more difficult for me to precisely evaluate. The risk of controversy here is significantly limited by my scope, as I personally find it highly unlikely that the websites of post-secondary educational institutions are in fact non child-friendly. As such, most sites that are rated receive a score of 'Good' (light-green) or higher. Thus far, sites that would score lower have not received a rating for this part.

Reason - All sites that receive: a) a score of 'Good' or higher on Trustworthiness and b) either a non-existent score or that of 'Good' or higher on Child Safety shall receive for this part the Positive/Neutral reason of 'Good site'. Others may be applied, to be included as additional reasons only, if deemed necessary.

My ratings are not currently publicly disclosed, but those with privileged access will find that they have in fact adhered very closely to the methodology outlined here.

==================================================================

What My Methodology Does Not Currently Consider:

Cyber-safety issues are not presently audited and there is no expectation for this parameter to bear any influence on my evaluations in the foreseeable future. With website hacks happening in greater frequency presently than ever before, I have found this metric to be uncomfortably transient, and personally believe that there are a plethora of other available resources for this sort of information, including McAfee SiteAdvisor, Norton Safe Web, Sucuri SiteCheck, Google Transparency Report Safe Browsing Site Status, and more.

==================================================================

Updated Library of Sources:

Please note that while these are the sources I have referred to during the information-gathering and cross-checking processes, they are not the exclusive determinants of my final opinions.

U.S. Department of Education Nationally Recognized Accrediting Agencies - hxxp://www2.ed.gov/admins/finaid/accred/accreditation_pg5.html#NationallyRecognized

EDUCAUSE Eligibility Requirements - hxxps://net.educause.edu/edudomain/show_faq.asp?code=EDUELIGIBILITY

Internet Corporation for Assigned Names and Numbers WHOIS Lookup - hxxps://whois.icann.org/en

Liaison Committee on Medical Education Accredited Membership - hxxp://lcme.org/directory/accredited-u-s-programs/

Commission on Dental Accreditation Accredited Membership - hxxp://www.ada.org/en/coda/find-a-program/

Accreditation Council for Pharmacy Education Accredited Membership - hxxps://www.acpe-accredit.org/shared_info/programsSecure.asp?sortby=status#Accredited

American Bar Association Accredited Membership - hxxp://www.americanbar.org/groups/legal_education/resources/aba_approved_law_schools/official-guide-to-aba-approved-law-schools.html

National Architectural Accrediting Board Accredited Membership - hxxp://www.naab.org/school-search-results/?f1_6472=

Association to Advance Collegiate Schools of Business Accredited Membership - hxxp://www.aacsb.edu/accreditation/accredited-members/global-listing

American Veterinary Medical Association Accredited Membership - hxxps://www.avma.org/ProfessionalDevelopment/Education/Accreditation/Colleges/Pages/colleges-accredited_results.aspx

American Library Association Accredited Membership - hxxp://www.ala.org/cfapps/lisdir/lisdir_search.cfm

Network of Schools of Public Policy, Affairs, and Administration Accredited Membership - hxxps://accreditation.naspaa.org/resources/roster-of-accredited-programs/

Commission on Collegiate Nursing Education Accredited Membership - hxxp://directory.ccnecommunity.org/reports/rptAccreditedPrograms_New.asp?sort=institution

U.S. News and World Report, Education - hxxp://www.usnews.com/education

StateUniversity.com, U.S. University Directory for Private Colleges and Universities - hxxp://www.stateuniversity.com/rank/score_rank_by_privc.html

Wikipedia - hxxps://en.wikipedia.org/

Google - hxxps://www.google.com/

==================================================================

My Scorecard Comments:

hxxps://www.mywot.com/user/7764612/comments

destinationtruth
Posts: 806
Joined: Tue May 12, 2015 7:47 pm
Location: Cherokee Nation

RE: Criticism thread for my ratings methodology

Post by destinationtruth » Fri Sep 30, 2016 10:08 pm

We have completed your evaluation, and sadly have to inform you that we have no other option then to lower your ranking back to "Rookie."

Kraftwerk
Posts: 7981
Joined: Thu Nov 05, 2009 12:30 pm

RE: Criticism thread for my ratings methodology

Post by Kraftwerk » Sat Oct 01, 2016 8:09 am

<quote user="destinationtruth">
We have completed your evaluation, and sadly have to inform you that we have no other option then to lower your ranking back to "Rookie."
[/quote]

Heidi, that's maybe one of the best replies i read on this forum for a long time:D

procyon
Posts: 162
Joined: Sat Oct 01, 2016 8:22 am

RE: Criticism thread for my ratings methodology

Post by procyon » Sat Oct 01, 2016 8:22 am

removed

A440
Posts: 4675
Joined: Sat Nov 20, 2010 1:56 am

RE: Criticism thread for my ratings methodology

Post by A440 » Sat Oct 01, 2016 4:38 pm

These are not the ratings we are looking for . . .

Supported
Posts: 10
Joined: Fri Aug 19, 2016 9:56 am

RE: Criticism thread for my ratings methodology

Post by Supported » Sat Oct 01, 2016 10:48 pm

<quote user="procyon">
So your methodology could be summarized by:

Comments:
For every official public websites, I copy/past the same sentence:
"Official public website of ......... Trustworthy."

Ratings:
I assess them in green because they are official and I don't care about security, others addons are here to do this job.
[/quote]

1: If it is the Official Publicly-available Website, each term being particularly defined and having its assigned subset of requirements, of the organization, and if that organization meets the definite criteria listed pertaining to Scope and Trustworthiness, I may have little other choice than to oblige with a similar statement. I try to avoid using any extraneous words which may bear the risk of throwing confusion into the specific meaning of my comment.

Not "every official public website" of every post-secondary educational institution is reviewed; again, those that don't meet the requirements specified are not assessed. For instance, organizations such as Trump University, Strayer University, DeVry University, American Intercontinental University, and North Central University have not currently received ratings.

2: There are indeed other overlapping add-ons and web tools that already screen for technical cyber-security issues, including Comodo Web Inspector, AVG Threat Labs, Kaspersky URL Advisor, McAfee SiteAdvisor, Norton Safe Web, Sucuri SiteCheck, Google Transparency Report Safe Browsing Site Status, Opera Fraud Protection, Netcraft Site Report, Dr.Web Link Checker, and many more.

In addition to that, there are a variety of cyber-security add-on compilation services such as VirusTotal, Scanurl.net, URLVoid, urlQuery.net, and Webutation.

In addition to that, there are a few reasonably prolific, well-reputed members in the myWOT community that already facilitate ratings and comments based on these results, including Shazza (hxxps://www.mywot.com/en/user/108271), drsumit (hxxps://www.mywot.com/en/user/5655802), and pavelskov (hxxps://www.mywot.com/en/user/7673216). I realize their comments may not cover the sites that I reviewed, but perhaps their ratings do (it's impossible to know because the information is hidden for most users).

In view of this, I feel as though the accommodation of this specific criteria may be redundant. However, if the importance of including it becomes stressed further, I will be appreciably more likely to seriously consider attaching reports on cyber-security to future scorecard comments.

In closing:
Thank you very much for your reply, and any others that have been posted so far. As I am still relatively new to this forum, I would indeed greatly appreciate any suggestions and/or constructive criticism that you all may have to offer, and will be reading everything carefully and communicating as best I can.

drsumit
Posts: 1584
Joined: Sun Jan 05, 2014 5:15 pm

RE: Criticism thread for my ratings methodology

Post by drsumit » Sun Oct 02, 2016 12:25 pm

[*]Just to clarify the user "pavelskov" you are talking about is a rookie member, since June 2016 with 0 activity score

https://www.mywot.com/en/user/7673216

I agree with Procyon - even if they are official sites there can be security issues and often many of these sites
are compromised due to outdated software versions or other vulnerabilities. Its better to mention the current state
of affairs of that site even if the organization behind that site is a trusted entity. We all are entitled to our opinions
but its up to us to be truthful in our rating

NotBuyingIt
Posts: 6576
Joined: Fri Mar 11, 2011 6:21 pm

RE: Criticism thread for my ratings methodology

Post by NotBuyingIt » Sun Oct 02, 2016 3:17 pm

At least two other WOT users have made enormous efforts to identify "official" sites in the educational, charitable/non-commercial and government domains. Their scorecard comments don't disclose whether or not they consider the sites' safely or truthfulness.

Typically, the best people to rate a site are its frequent visitors or customers. I'd like to discourage people from rating sites about which they know next to nothing. Other very knowledgeable community members insist that comments should only be posted in order to explain and justify a user's ratings — which is the original purpose of scorecard comments. I don't object to scorecard comments without ratings that only indicate sites' authentic ownership. That sort of taxonomy seems innocuous and may be helpful to a few interested WOT users once in a while, although most of those comments will remain unread. However, DMOZ.org ([url=https://www.mywot.com/scorecard/dmoz.org t=_self]scorecard[/url]) would seem to be a much more useful place to contribute. When they submit ratings without doing additional research, WOT users who rely almost entirely upon DMOZ.org, similar directories, blacklists or whitelists are expected to identify their sources in their scorecard comments.

The WOT reputation system is about reputation, not about taxonomy. Submitting a predetermined rating only because a site is "official" and ignoring everything else about it is an act of rating out of ignorance. Please don't do that.

Guest

RE: Criticism thread for my ratings methodology

Post by Guest » Sun Oct 02, 2016 8:55 pm

<quote user="drsumit">
[*]Just to clarify the user "pavelskov" you are talking about is a rookie member, since June 2016 with 0 activity score

https://www.mywot.com/en/user/7673216

I agree with Procyon - even if they are official sites there can be security issues and often many of these sites
are compromised due to outdated software versions or other vulnerabilities. Its better to mention the current state
of affairs of that site even if the organization behind that site is a trusted entity. We all are entitled to our opinions
but its up to us to be truthful in our rating
[/quote]

And I agree with you =
Four years ago, we had a situation, threads in which "original sites" were posted, many agreed with this idea because it appeared to be the right thing to do, and to a great extent it was, but scorecards evolved while those comments and ratings remained

The idea was well intentioned, but the fact is that it did not address what could happen in the long- term, I strongly believe that a comment should be the responsibility of that member and her or his free opinion of that site as in my case, I will never rate as trustworthy a social media site, or the latest craze [Pokemon go] that is leading people to walk the streets in the middle of the night, w/o looking or paying attention to their surroundings, is my personal opinion and is respected by WOT

@Supported =

I invite you to ask others as I did when I first joined WOT, the only privilege they have is their hard work and willingness to help anyone in need

There are guidelines that will help you too, like this one

https://www.mywot.com/en/guidelines

My advice to you is to be truthful or sincere in your ratings and comments

My best regards

destinationtruth
Posts: 806
Joined: Tue May 12, 2015 7:47 pm
Location: Cherokee Nation

RE: Criticism thread for my ratings methodology

Post by destinationtruth » Mon Oct 03, 2016 4:57 pm

You do realize that just because it's an official site does not mean it is safe; for example look at hxxps://msu.edu/
This university site is currently blocked and found to be malicious by Emsisoft. https://www.virustotal.com/en/url/00a2fbe3f2a9e139e5a83d6b31e76dc166abe162f02489096828bf42435acbc2/analysis/1475525621/

Now most likely a FP (and I have submitted for evaluation to Emsisoft), yet still needs to be investigated before writing a blank check for "Trustworthy." as you did.

You have lot to learn grasshopper before you can walk across the rice paper.

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests