On Statistical Significance
By Beracah Yankama
Director, StudentsReviewWhen students and parents are reading reviews, often-times the question will come up, "how statistically
significant is this data?". How can only a few reviews be representative of several thousand
students' opinions?
Few people really understand how statistical significance works or what it really means. While
statistical significance itself has a meaning rooted in replicability of the data, most people
interpret it as, "is this enough data to trust?"
Consumers tend to interpret statistical significance as the
quantity needed to dismiss exceptions
It is often
used as a marketing term by pharmaceuticals. Consider this:
1/1000 experienced side effects when taking our drug
'Statistical
significance' as many people understand it, will lead you to believe
that the 1 person was some kind of an exception, a chance occurrance,
the side effects are not likely to happen to you.
Suppose, however, that the drug in question was given to 999
adults, and 1 child. The child experienced the side effects. Now is
it statistically significant?
Invariably about 66% of people are satisfied with their college. 33% however, are not, and wished they had gone
elsewhere. The larger percentage would convince you that the 66%, the majority, is all you need—"most people worked
out ok". Because that lets the blame fall on the consumer, not the school.
Lets consider one last example. Suppose that a school consists of 95 males and 5 females. The 95 males say the safety of the school is
great, then 4 of the females say it is "ok". The idea of Stastical
significance might have you (the generic, genderless consumer)
believe that the safety is great. You the parent, you the prospective
female student, should be rightly curious of any exception.
What you
really want to hear is from the 1 girl who might have
been attacked at night, who tells her story. Does the fact that 4
out of 5 women said that safety is "ok" matter? Does the opinion of
the 95 other men matter?
As the parent, the prospective student, you want to know about
what happened to that 1 girl, because that tells you that 'your
little girl' might not be safe on the campus. You don't want to know
"the probability of attack", you want to know a clear-cut, answer
to the question, "does it happen, or not?"
After all, most people don't die when they cross the street—but does
that mean you shouldn't look both ways?
StudentsReview is intended to catch the
exceptions, the people who were misled by marketing, by well publicized
majority opinion, into making an incorrect decision. Read the opinions knowing that they are experiences. They
did happen. They didn't have some "probability of happening", and weren't some kind of "just a complainer"
event. Where something happens once, it happens twice, so read, learn, and make an informed decision. Don't let
others dupe you into dismissing evidence so that you make the same uninformed decision that they did. Many people
simply do not want to face the fact that they may have, or are making a mistake.
StudentsReview exposes the experience behind the student, so that the reader can determine
what questions they should be asking for themselves. As always, caution is the best advice, and we've provided
a mechanism for students to be cautioned.
