When students and parents are reading reviews, often-times the question will come up, “how statistically significant is this data?”.  How can only a few reviews be representative of several thousand students' opinions? 

Few people really understand how statistical significance works or what it really means.  While statistical significance itself has a meaning rooted in replicability of the data, most people interpret it as, “is this enough data to trust?”. 

Consumers tend to interpret statistical significance as the “quantity needed to dismiss exceptions”.  It is often used as a marketing term by pharmaceuticals.  Consider this: “1/1000 experienced side effects when taking our drug”.  'Statistical significance' as many people understand it, will lead you to believe that the 1 person was some kind of an exception, a chance occurrance, the side effects are not likely to happen to you.

Suppose, however, that the drug in question was given to 999 adults, and 1 child.  The child experienced the side effects.  Now is it statistically significant?

Invariably about 66% of people are satisfied with their college.  33% however, are not, and wished they had gone elsewhere.  The larger percentage would convince you that the 66%, the majority, is all you need — “most people worked out ok”.  Because that lets the blame fall on the consumer, not the school.

Lets consider one last example.  Suppose that a school consists of 95 males and 5 females.  The 95 males say the safety of the school is great, then 4 of the females say it is “ok”.  The idea of Stastical significance might have you (the generic, genderless consumer) believe that the safety is great.  You the parent, you the prospective female student, should be rightly curious of any exception.

What you really want to hear is from the 1 girl who might have been attacked at night, who tells her story.  Does the fact that 4 out of 5 women said that safety is “ok” matter?  Does the opinion of the 95 other men matter?

As the parent, the prospective student, you want to know about what happened to that 1 girl, because that tells you that 'your little girl' might not be safe on the campus.  You don't want to know “the probability of attack”, you want to know a clear-cut, answer to the question, “does it happen, or not?"

After all, most people don't die when they cross the street — but does that mean you shouldn't look both ways?

StudentsReview is intended to catch the *exceptions*, the people who were misled by marketing, by well publicized majority opinion, into making an incorrect decision.  Read the opinions knowing that they are experiences.  They *did* happen.  They didn't have some “probability of happening”, and weren't some kind of “just a complainer” event.  Where something happens once, it happens twice, so read, learn, and make an informed decision.  Don't let others dupe you into dismissing evidence so that you make the same uninformed decision that they did.  Many people simply do not want to face the fact that they may have, or are making a mistake.

StudentsReview exposes the experience behind the student, so that the reader can determine what questions they should be asking for themselves.  As always, caution is the best advice, and we've provided a mechanism for students to be cautioned.

Author:
Beracah Yankama
Director, StudentsReview

Link me! Link to page from your webpage or MySpace account: Just copy and paste! <a href='http://www2.studentsreview.com/articles/on-statistical-significance.php3'> On Statistical Significance </a>

Forward me!