Pulse Asia SONA Survey Was Propaganda, Just Like the SONA

After looking at the survey questions, the methodology and the purported analytical results of the July 2008 Pulse Asia Survey poll on President Arroyo's State of the National Address, I am forced to the inexorable conclusion that it was basically propaganda, not a valid scientific public opinion poll. The main numerical data are contained in Tables 1, 2, 3, and 4.

To begin with, the pollster admits that only 60% of the 1200 respondents to the survey were qualified to participate and be tallied because only that fraction of them claimed to have "heard or read about any previous SONA." Thus the proper random sample size is 720 and the one-sided statistical error is closer to 4% than 3%. But this is hardly its biggest or most egregious flaw.

If you look under a magnifying glass at the bottom of Table 4 you will discover that the first question asked of each respondent was whether they had heard or read about any previous SONA ("Kayo ba ay nakarinig na, or nakabasa na, sa anumang nakaraang Ulat Sa Bayan o SONA ni Pangulong Arroyo?"). To those who answered YES, two further questions were asked soliciting an opinion about the truthfulness of the 2007 and the likely truthfulness of the 2008 SONAs. There is a further microscopic notation that "truthful" means "mostly or completely truthful" while "untruthful" means "mostly or completely untruthful."

This is a badly flawed question because the 2008 SONA, for example, has over 4600 words, and 500 sentences spread out over many dozens if not hundreds of possibly truthful or untruthful individual assertions by the President. But the follow-on question forces the respondent to decide whether he or she thinks they were (in a previous SONA) or would be (in 2008) "mostly or completely true or untrue." Well, no wonder 46% of the 720 qualified respondents said they were UNDECIDED about the likelihood of the 2008 SONA's truthfulness, while 52% were UNDECIDED about the actual truthfulness of the 2007 SONA.

It is a rule of thumb, at least among scientific, professional pollsters that when a question elicits such large percentage of UNDECIDED responses, there is something badly wrong about the question itself, either its design, content or meaning for the respondents. Among physical scientists this is equivalent to having half of one's data set being corrupted, unreadable or unusable because we are measuring the wrong parameter or using the wrong proble. We usually throw away such tests and their resulting data sets as being unreliable or positively useless.

What the analysis keys on, as presented here and by Ana Marie Tabunda on ANC last evening, was the large percentage (40% of 720 respondents) opining that the 2008 SONA would likely be untruthful while only 14% of 720 respondents opined it would likely be truthful. It is of course not surprising that 46% thought the 2007 SONA was untruthful while 13% thought it was truthful.

With respect to the 2008 SONA, this is numerically gussied up ASTROLOGY and not statistical opinion polling. With respect to "any previous SONA", I doubt very much that any of the respondents could've made a fair mental evaluation of the totality of the President's assertions. Rather, human nature being reflexively suspicious and skeptical towards politicians (especially one as opaque and plastique as Gloria Macapagal Arroyo, who truly deserves it, in my own opinion), there is a natural tendency to remember some unbelievable or untruthful statement and ascribe dishonesty to the entire speech, rather than to mentally maintain an inventory of a large number of possibly true statements. After all, when a person tells even one lie in such a forum, it's easy to believe such a person is telling many other lies.

Since the nature of the basic question asked of the respondents, requires of them to make a global judgment ("mostly truthful" or "mostly untruthful" but nothing in between) of some past or future SONA, and given that she has most verifiably lied on a number of very important and public occasions in her checkered political career, the results might've been predicted without all the trouble of a full blown statistical survey. After all, the respondents were qualified to answer these questions if they had merely read or heard about some SONA or other -- most likely from an habitually skeptical or even hostile media.

Nevertheless, two wrongs just don't make a right. This Pulse Asia Survey was what we call a setup or a "gimme" for anti-Arroyo forces. It is another example of the cynical use of science for propaganda purposes.

Deliciously malicious propaganda at that, since of course propaganda can itself be truthful or not, just like the SONAs themselves!

Tee-hee, but shame on you Pulse Asia! I bet if somebody did a survey on the surveys, the results would run against the pollsters with similar percentages being cynical and skeptical about them as propagandists.

---

UPDATES: Cogent discussion of the brewing bribery brouhaha in the Court of Appeals is to be found at La Vida Lawyer.

---