The headline at the Social Weather Stations website announcing the results of its Third Quarter National Survey is not supported by the statistical data collected in the survey:
NET SATISFACTION WITH PGMA INCHES UP TO -11
As the SWS article points out, GMA's Net Satisfaction Rating (NSR) was calculated to be -13% in the Second Quarter. Now it is calculated to be -11%. The change in the NSR is therefore plus 2 percentage points, correctly rounded.
So did GMA's NSR "inch up"?
NO! It was UNCHANGED.
Why? Because the standard Margin of Error in the RAW STATISTICS of each SWS Survey, as SWS well knows, is about plus or minus 3% at the 95% confidence level or two standard deviation confidence interval. On this account alone, the headline is utterly false and misleading already. But it's actually worse than that. As a COMPUTED STATISTIC, being the difference between two separate raw statistics -- those who are satisfied with GMA minus those dissatisfied -- the Net Satisfaction Rating actually has a statistical Margin of Error twice as big, that is, plus or minus 6%!
Now if you take the Third Quarter NSR and subtract from it the Second Quarter NSR, and then observe that the result if plus 2%, the technically correct way to report this is that the CHANGE in the NSR was plus 2% PLUS OR MINUS 12%, since each of the NSRs themselves contain a statistical Margin of Error of plus or minus 6%.
So let me summarize the important points to remember as an intelligent reader of public opinion surveys:
(1) Any statistic that comes directly from a tally of the responses to a question on the survey questionnaire has a built in Margin of Error of plus or minus 3% (actually plus or minus 2.89%, which is the number you get when you compute the reciprocal of the square root of 1200.)
(2) Any statistic that is computed from any two statistics that fall in category (1) above, such as the Net Satisfaction Rating which is a difference of two such statistics, will have a built in Margin of Error of plus or minus 6%.
(3) The CHANGE from one survey to another of a computed statistic like those in category (2) above, will have a built in Margin of Error of plus or minus 12%.
One thing is clear, the so-called Net Satisfaction Rating is a REALLY LOUSY STATISTIC when your random sample is only 1200 respondents.
No wonder they say that Statistics is the Science of proving any damn thing you want to!
The headline should have read:
GMA's Net Satisfaction Rating Was Unchanged
Any apparent arithmetic change is NOT statistically significant. Not by a LONG SHOT! Thus the use of this headline is just as UNETHICAL as if a broadsheet were to publish some bald lie. Except this isn't so bald, i.e., obvious, which is what makes it despicable to me: the misuse of science and mathematics.
Public Opinion Polling Is a Genre of Journalism
How the Surveys Have Lost Their Sting
Fallacy of the Leading Question
Checking If A Coin Is Fair