There has been a little bit of a to-do in light of the shocking admission by the progressive blog Daily Kos that the last year and a half of polls they have released were, you know, fraudulent . Conservative blogs predictably gloated and analysis abounds as to what went wrong and what this all means. To my mind, here is the question that I think everyone should be considering: Is it time the media stops treating these ‘horserace” poll results as actual news? I believe it is. Further, I believe our addiction to the horserace distracts from what should really matter in an election and gives legitimate research a bad name.
Maybe this story resonates with me because of how strongly I believe in solid research. Polling is a vital strategic tool when utilized correctly, and I am certainly not against polling political races. My issue is with these snapshot in time polls published in the paper, sometimes with super-sketchy sample sizes, complete with conclusive sounding statements from the pollster, which are then reported mindlessly by the media as fact.
Do these polls get it right from time to time? Sure. After all, a stopped watch is right twice a day. Is there a high level of accuracy when all of these polls are averaged and outliers are weaned out, ala Nate Silver? You betcha.
But I believe it is no less than harmful to our democracy for reporters to mindlessly report the results of any given poll as an absolute fact, and then use those polls as the basis for actual hard news stories. Every poll taken has a margin of error, usually in the 4% to 5% range. Plus, you can easily twist the result you get depending on how you ask the question, or even how you qualify the sample of people you’re polling. (Are you polling likely voters or everyone? And how do you determine who a likely voter is? Are you using real people or machines to make your calls? Are you calling cell phones? Are you using bilingual surveys?)
The majority of media outlets do not weigh these variables; they simply report the results they get as fact. After all, this is math people! And that provides the valuable service of letting us unwashed masses know who is (maybe) winning and losing at that moment, which is important to know because…?
Here at R&R Partners, we have a first-class research team, led by our Vice President of Research Todd Gillins. After bouncing this blog-post off of some folks to ask them if it was too much of a rant and too light on thought-leadership, they pretty much universally replied “yes”…on both counts. But they were only really concerned about the thought-leadership part. So I cornered Todd, whose knowledge of this stuff dwarfs my own. I asked him if he could provide a little context. Todd?:
Even though many attempt to develop, conduct and analyze research, it is a specialty that not everyone is qualified to do. Maybe at first glance, most think they can ask a series of questions and then report the findings… seems simple enough. But understanding the science and art behind the numbers is not that straightforward. I would venture to say that most who may exhibit the symptoms of heart disease would not self-diagnose this serious health challenge, but rather they’d seek the expertise and advice of a qualified health professional. While definitely not as serious as heart disease, the ability to conduct surveys should be left to those who have immersed themselves in research techniques and understand the nuances and implications associated with studying a sample of the population. Journalists and the media are better suited at reporting the facts. And qualified research professionals are better suited to provide the insights that will assist the writer in developing a story. As with any industry there are those who are striving to provide the best work possible and those whose primary goal is only to increase their wealth. The old consumer cliché “buyer beware” should also be applied to those seeking to understand consumer opinions through research—find a reputable and credible research partner that has a proven history of providing solid and accurate research findings.Of course, he’s right. And as we see from what happened to the Daily Kos and subsequent stories that were based at least in part on their supposedly fraudulent polling, from time to time there is a complete abdication of journalistic responsibility. It is my humble opinion that we would all be better served if the Fourth Estate would focus on the actual issues that surround any given campaign, and not on the horserace.
