Thursday, October 15, 2009

What's Wrong With That SWS Survey?

MATHEMATICAL THEOREM: Anyone can get 100% mentions and TOP the SWS Presidential Survey ("Choose-3") yet get ZERO votes in the election. (Proof: Imagine a candidate named as the 2nd choice of 50% of the respondents and 3rd choice of the other 50%. Quod erap demonstrandum!).

I think it would be interesting if SWS would reveal HOW Noynoy Aquino got 60% of the mentions in its 3rd Quarter survey: in other words what percentage mentioned him as first, second or third on their list of names in answer to the main question posed by this survey:

"Under the present Constitution, the term of Pres. Arroyo is up to 2010 only, and there will be an election for a new President in May 2010. Who do you think are good leaders who should succeed Pres. Arroyo as President? You may give up to three names." 

This of course assumes that it would be the natural tendency of respondents to list down first on the list their actual preferred choice for president if the election were to be held then.  In my earlier post on this topic, I was thinking about the second and third place mentions of respondents in order to explain why the percentage reported do not add up to 300% as they should: it is because SWS does not report names mentioned by less than 0.14 percent of the respondents.  Now it turns out this is nearly half of the data in each quarterly survey, judging from the fact that for example this most recent survey adds up to 153% or so!

This does bring up another important point.  In their Table of Results, SWS places dashes ("-") in the first nine runnings of the poll in the row for NOYNOY AQUINO.  I think it would be interesting for SWS to reveal how Noynoy did in those previous polls, or was he NEVER mentioned in any of the polls since 2007.  That hardly seems likely.  But even if his numbers were less than 0.14 percent, they should report it, even if it reveals how sensitive to current events their poll actually is, like the funerals of Democracy's Icons.

I personally do not believe that Social Weather Stations is making any of this data up.  However, the Public and even the Blogosphere has not yet wrapped their minds around this SWS survey. It's a tricky little gem of public opinion polling that seems to me built to generate headlines.

Statistics truly is the Science of Innuendo. Unless we can get "under the hood" and see what is really going on, there is plenty of mischief possible because public opinion polling has become a lucrative business, a genre of propaganda.


Jesusa Bernardo said...

And Noynoy got two consecutive flat no-butal ratings. 50% last time and 60% now. How statistically coincidental.

Not to mention that Noynoy's figures are sooo high, so good to be ... Are his provincial sorties reflecting such high figures? I mean how great are people's reception to him?

Dean Jorge Bocobo said...

But SWS is definitely measuring something of a social phenomenon here. It is however a statistical transient. Quite apart from that however is the bigger issue of how the public really needs to understand this survey, its mechanics, what is reacts too and how that determines the results and the spin.

Anonymous said...

it should only add up to 300% if and only if all three names should be listed down. but as the question stands, may leaves bigger room as an option, as in i may or may not list 3 names at all.

the weakness in this survey is the validity of the question. is the respondent required to order the rank of his/her choices? (in which case, you cannot just assume that the first name is first in rank). or if i am a fanatic follower of a certain candidate, will sws invalidate my response if i list all three with the same name?

vague question leaves open this survey to question of validity. what exactly was the instruction indicated in the survey?

inodoro ni emilie

Dean Jorge Bocobo said...

A very valid observation INE. about the vague question I mean. But in regards to blanks, please note that there is an entry for NONE or DONT KNOW which SWS includes. That is why all the data does ADD Up to 300% but only if you throw back in some assumed percentage below .14%.

Jesusa Bernardo said...

Jose Zaide in MB asks:

"Dagdag-bawas in poll surveys?," adding that "163%" responded in the surveys.

With all such questions in the blogosphere, I'm sure that Mangahas is preparing or will prepare a response in his Inquirer column soon.

GabbyD said...

isnt the interpretation simple?

60% of the 1,200 people mentioned noynoy as one of the best candidates for president.

its evidence of how its widely accepted that he's a good presidential contender.

its not evidence that people, 60% of them, will vote for him.

Jesusa Bernardo said...

@ Gabby D, is it really simple? I'm actually in sort of a lost.

This post from The Daily Tribune asks questions similar to DJB's:

The question is, does Aquino really have 60 percent of the vote, given the fact that respondents can name three choices? Definitely not, since the respondents could have also voted for, say, Roxas and Escudero, or Villar, which makes the vote percentages meaningless, as the vote is not an accurate gauge of the voters’ pulse...

Since this is not an accurate gauge of the voting public’s pulse, why does SWS adopt this method for its quarterly surveys, when a one respondent one vote method would prove to be a more accurate snapshot of the public’s preference at the time the survey was conducted?

Dean Jorge Bocobo said...

i think they chose this form because it IS sensitive to breaking developments like noynoy, or the publicity blitz of Villar. It is not an invalid strategy for catching trending candidates, but most people really were not aware of the 300% thingy, nor have most wrapped their minds around the survey to understand that it is not the same, as you say, getting a snapshot of actual preference. After all the question, does not ask who the respondent would vote for if the elections were held today.

Maybe they kept getting a lot of I don't knows so they traded it in for that chooose 3 with a 3rd choice of throwaway names (the ones under .14 percent, which i think includes noynoy but SWS has not revealed that fact!)

It's the survey question that people should focus on before they look at the results or the spin headline. Usually it's backward and they never get to the question because the spin is faster than the mind.

Jesusa Bernardo said...
This comment has been removed by the author.
Jesusa Bernardo said...

You're right DJB, SWS should reveal those under .14 in the past surveys.

I'm really thinking more in the line of the SWS controversy in 2004. Supposedly, the Arroyo camp was able to influence the surveys that showed her prevailing over FPJ in the weeks leading to the elections. The camp of the Illegitimate, it is said, was able to get the schedule of the poll workers and did pa-pogi acts of goods' distribution, etc. so their survey answers will favor Arroyo.

I remember that either Mangahas or another SWS official denied any deliberate foul play on TV but when confronted with the issue (though in subdued wordings) did say that perhaps the methodology/confidentiality or something can be improved somehow.

I'm 99% sure Noynoy wouldn't do or order any such thing I imply here; however, the 'civil society' who backs him is another matter. The integrity of that markedly hybrid part of Philippine society should also be questioned as loudly as question question the politicians'.

For what it's worth, here's the intro of Maceda's recent column post:

Several people called us yesterday from as far as Cebu and Cagayan de Oro to tell us that they don’t believe the recently released Social Weather Stations (SWS) survey putting Sen. Noynoy Aquino in first place with a super typhoon rating of 60 percent, the highest in Philippine political history for a presidential candidate.

My good friend Carmelo Santiago, who is a veteran political leader, pointed out that SWS top honcho, Mahar Mangahas went on TV on election day 2004 to project that GMA would win over FPJ in Metro Manila with a margin of plus 14, according to his exit poll.

The actual result: FPJ won in all the cities and towns of Metro Manila except Las Piñas. SWS suffered a total loss of credibility then...."

GabbyD said...

de quiros says there is another question, where the respondent should only put 1 name. noy noy still tops