Why Self-Rated Poverty and Hunger Statistics Are Scientifically Meaningless

pinion pollster Social Weather Stations needs to find something better to do in between elections. Their quarterly reporting of SELF-RATED poverty and hunger "statistics" seems designed to ensure that they will have some kind of feel-bad news for the headline writers of the scoop-addicted and sensation-dependent Press to report. The general public, unfortunately, isn't savvy enough to see through the fog of dire-sounding results when even the normally savvy scientists who work at SWS are complicit in the fallacious tsk-tsking.

Now whenever I see a headline like "52 Percent Feel Poor" or "20 million Pinoys say they are going hungry" I am not really impressed. To see why neither should you, substitute some negative self-describing adjective for "poor" and it would make sense but not be sensational. For example, consider a headline like "52 Percent Think They Are Short."

In fact, if one considers any "normal distribution (bell shaped curve) that represents some kind of social good or human preference, like wealth, access to food, health care, insurance, good looks, political influence, etc. it is unavoidable that half of the population will be below the Median Level and half above the Median Level. And so it is always possible to measure a "feel bad" and therefore "newsworthy" statistic merely by reporting the percentage of the population that describes itself as being one, two or three standard deviations below the Mean of that distribution. In the case of wealth for example, there are always "the poor" and "the rich" at either end of the distribution, even though the poor in one country may be far better off than even the rich in another country in terms of all the primary social goods that are actually available to them.

In the self-rated poverty and hunger surveys recently reported, SWS and its client news outlets like PDI, try very hard to make it seem deeply profound that,

"Household heads' ratings of general poverty, food poverty and experience of hunger were "internally consistent," SWS noted. Nationwide, the proportion of households who experienced hunger during the past three months was at 31 percent among those who rated themselves "food poor" and 28 percent among the self-rated poor..."

Considering that food, shelter and clothing are basic necessities and primary social goods, I wouldn't be surprised either if they next report that among the self-rated poor, a near-equal percentage would rate themselves as not having enough clothes to wear or good enough homes to live in.

The gentle reader will please notice that I am not questioning the scientific integrity of the data on "self-rated" statistics, or the accuracy of its measurement, merely on its real significance.

And why does SWS have to trot out an utterly meaningless and tendentious idea like "internal consistency?"

Well, because unlike the voter preference surveys that have made SWS deservedly famous, there is simply no independent, objective event or process by which their findings on self-rated anything can be falsified or verified. And so they make self-rated hunger stats and self-rated poverty stats, each other's verification. Because of that, self-rated statistical polls cannot be considered to be scientifically meaningful polls in the same way that voter preference polls are scientifically meaningful, which are falsified or verified by the results of the elections. The potential for egg on their face keeps SWS honest. But in the case of the self-rated stats, there is no such verification or falsification, and so merely reporting them allows people to draw their own conclusion, to which they are often led by the nose by the pollsters when "record levels" of hunger are noted and highlighted. I have criticized these results in the past because of seasonality and periodicity in the data indicates there may be nothing more than statistical variation to such "record levels."

The only reason SWS can get away with these numerically meaningless polls about hunger and poverty, is that most people cannot deny that there must be SOME level of poverty and hunger in the land, just like there have to be some short people if there are tall people and a bell shaped curve or normal distribution of the height attribute to begin with.

Because of the essential use of rigorous statistical and mathematical techniques to produce their raw data, SWS and other public opinion pollsters like Pulse Asia, Inc. have a just claim to being "scientific organizations" in much the same way as the weather bureau Pagasa.

But this can be very deceptive. Just as most newspapers and broadcasters have fairly rigorous news gathering components, with ethical standards that call for verification and substantiation of what gets reported as "news" -- which are the equivalent of the data gathered by public opinion surveys--the pollsters also offer "opinion". By this I mean that SWS and Pulse offer interpretations and explanations of why their data comes out a certain way, what factors they think account for the scientifically gathered results. Here things are not so scientific and often reflect the pollsters biases and ideology. In contrast the weather bureau does not offer personal opinions about WHY the weather is the way that it is, why a typhoon has suddenly appeared on the horizon.

Public opinion polling is unavoidably a genre of journalism. The data reported are the equivalent of news stories, but the attempt to explain the factors that led to the statistics are more like editorials and opinion columns, which may or may not be correct. In the case of the self-rated hunger and poverty stats, there is not really even the opportunity to check what the objective reality is relative to the numbers reported. And so they are really the equivalent of "letters to the editor"!

And so, as with the mass media, the advice is also: caveat emptor!