Red flags: Maybe the Star Tribune CD8 poll is a joke
Earlier today, the Minneapolis Star Tribune published their poll of CD8 showing Rick Nolan ahead of Congressman Chip Cravaack by 7 points. Later, David Brauer of MinnPost.com explained what “red flags” went up for him with the methodology used by Pulse Opinion Research, the polling firm commissioned by the Strib.
First, POR polls on a single day — Tuesday, Oct. 16 in the Nolan-Cravaack case — compared to three for SUSA and four for Public Policy Polling (which also has a good Minnesota record).
Though pollsters refer to their results as a “snapshot in time,” single-day polling is not a best practice. It is too susceptible to a one day’s news or advertising. It doesn’t allow for callbacks, which some pollsters attempt to mitigate low response rates, perhaps the biggest trap door in this year’s numbers. If POR’s methodology box is gospel, they reached all 1,000 voters in just four hours.
Lesson learned: Polls, like baseball playoff series, should not last one day.
Second, POR only calls land lines, even though a quarter of Minnesotans are cell-phone-only, according to Census Bureau estimates. POR says it augments landline respondents with cell phone panels, but Star Tribune digital editor Dennis McGrath says “that proved difficult at the [Congressional District] level, so it’s land lines only.”
Though there’s some evidence Democrats are underrepresented when cell phones are excluded, that might not be the case in the 8th. This is a split district, with a more DFL north (Duluth, Iron Range) and a GOP south (what I call the Emmer Exurbs). Cell phone coverage is spottier in the more remote north, so POR might have missed a lot of young Republicans.
Lesson learned: Mobile phones exist on the Iron Range.
Then there’s using POR in the first place. Fivethirtyeight.com polling analyst Nate Silver gave Pulse Opinion Research remarkably low marks for the 2010 cycle, noting they “missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters.”
Lesson learned: If a lefty stat nerd can’t trust your polling firm, why should we.
The Star Tribune already has a terrible track record with their polls; you’d think they’d want to be accurate one of these elections.