Search P-Shock

Outside the margin of journalistic error

posted 2007.06.06 Wednesday

M. E. Sprengelmeyer and Alan Gathright of the Rocky Mountain News write:

Tancredo has been polling below the margin of error in polls around the country.

The "margin of error" we often hear quoted for polls is the range of sampling error for answer options that one expects about 50% of those surveyed to choose, e.g., as in a question on the general election preference between the nominees of the two major parties.  In a poll with 500 respondents, at the 95% confidence level, a question where we expect a 50%-50% response split has a +/- 4.4% margin of error.  However, if to the same question there is a third response we expect to be chosen 1% of the time, the margin of error on the frequency of that response would be +/- 0.9%.  In other words, say Giuliani is polling at 50%; assuming a random sample, the "true" figure in the population is probably between 45% and 55%.  If in the same question Tancredo receives 1%, the "true" figure is likely no higher than 2%.

It may not be worth it to a political reporter to gain the thorough understanding of survey research that would enable him or her to get such things correct; oddly, it is still his or her job to write about such things.

AddThis Social Bookmark Button




1. M.E. left...
2007.06.07 Thursday 12:02 am :: http://blogs.rockymountainnews.com/denve

Point taken.

Since you know your numbers and also appear to have a way with words, how would you describe the candidate's typical showing in these polls? We could attach a number range to his typical showing (ie., "less than two percentage points in most national polls"). I gather you think it's bogus to compare the showing to the reported sampling error. So is there another way?


2. Jason Briggeman left...
2007.06.07 Thursday 12:48 am

I think the "less than 2%" approach is right for a candidate who polls 1% - that perfectly incorporates the lack of precision, i.e., sampling error. If the latest poll result were 2%, then the range is essentially 1% to 3%; of course, if you have twenty recent polls in which he never actually registers the 3% but often registers 1%, you would probably want to say 1% to 2%.

Honestly, I didn't find it bogus, just rationally ignorant (and this is not a criticism; "rational ignorance" is economics jargon). Sure, it was scientifically imprecise, but neither you nor 99% of your audience has any reason to care; you put the general point across fine. Being a generalist writing for a general audience means that you will almost never get specifics exactly right and that extremely few of your readers will know any better; that's just the nature of mass journalism. By contrast, my niche audience is academic economists and economists in training, and this is the sort of thing we are supposed to understand thoroughly; the lesson is for them, not really for you. I suppose it is admirable that you are trying to understand these semi-arcane details of sampling error, but if you tried to attain that deep an understanding of every subject matter that impinges on the stories you are supposed to cover, you would probably never get around to writing an article for all the time spent studying.


3. M.E. left...
2007.06.07 Thursday 7:25 pm :: http://blogs.rockymountainnews.com/denve

Thanks, Jason. Best of luck to you and all the number crunchers here.

Pay us a visit at the "Back Roads..." sometime.

http://blogs.rockymountainnews.com/denver/sprengelmeyer/