Four polls focusing on Maine’s Senate race are conducted within two weeks.

Three have Angus King at 43 to 44 percent. One has him at 50.

Charlie Summers’ support ranges from 28 percent to 35 percent.

King’s lead is as big as 22 percentage points and as small as 8.

So, what’s wrong with this picture?

Actually, nothing, according to Charles Franklin, founder of PollsAndVotes.com and a nationally known polling analyst.

The Portland Press Herald asked Franklin to analyze three independent statewide polls conducted in mid-September. One was a poll by Critical Insights commissioned by the Press Herald. A fourth poll released late last week by Rasmussen Reports was not part of Franklin’s review.

We asked Franklin if something was wrong with our poll, or the others, and why there seemed to be so much variation.

“I’ve spent the last eight years actively comparing polls to each other,” Franklin said after looking at the methods, questions and results of each poll. “Based on that experience, it doesn’t surprise or bother me when I see this level of variation, and it doesn’t mean that any one of these polls is particularly out of line with the others.”

Franklin said the Maine polls all appeared to be valid, and the variation was generally within or very near the margins of error. He also identified basic differences between the polls that can be expected to lead to variations in the results.

Because all polls are estimates, pollsters typically say they are 95 percent confident in the numbers, give or take a few percentage points.

In the Maine polls, the margins of error were 3 percent to 4.5 percent. That means that the 43 percent to 50 percent range for King is at the combined margin of error in the two polls. The same is true for Summers’ range — 28 percent to 35 percent.

Franklin also said the variation is to be expected given differences in the polls themselves.

The Critical Insights poll, the one that has the largest lead for King, was the only poll to conduct live interviews, include cellphone users, in its sample and ask undecided voters if they are leaning toward any particular candidate. The other polls used automated phone calls to landlines only — automated polls are not allowed to call cellphones — and did not separate leaners from undecided voters.

Those so-called “robopolls” are far less expensive and can produce basic horse-race numbers much faster than live interview polls such as the Critical Insights survey, which asked deeper questions to allow the newspaper to explore voter sentiment and political trends.

Different pollsters can make valid cases for their methodologies, Franklin said, but it’s no surprise that they get variation in the numbers.

For example, if leaners are removed from the Critical Insights poll and counted as undecided the way they are in other polls, King’s support drops to 47 percent and his lead shrinks from 22 points to 19 points.

The impact of other differences is harder to measure.

“Clearly, your cellphone sample is going to pull in more young voters,” Franklin said. “I believe that adding cellphones probably helps King a little bit.”

The polls show King has especially strong support among the youngest Maine voters, so it makes sense that the poll including cellphone users would have higher numbers for King. The cellphone bump for King was probably not more than a percent or two, he said.

Nationally, cellphone users make up as much as 30 percent of political poll samples and can have a big impact on results. However, Critical Insights uses cellphones for only 10 to 12 percent of its polling sample, because that is the estimated percentage of Mainers who are now cellphone-only users. The state’s spotty cell coverage is likely the biggest reason for lower rates of cell use here.

Cellphones also may have contributed to stronger support for the same-sex marriage referendum in the Critical Insights poll. Young voters, the ones most likely to have cellphones and no landlines, are strongly behind the legalization effort.

The use of live interviewers instead of automated voice recordings might also have contributed to more same-sex marriage support in the Critical Insights poll.

Pollsters have clearly found that voters are more likely to say they support gay marriage than to actually vote that way in the privacy of a voting booth. It may be that voters opposed to same-sex marriage are more willing to say so with computerized calls than live interviewers, Franklin said.

“I think that there could be some effect here.” But, he said, “the bottom line for me is that you have three polls here that are all pointing in the same direction.”

One other difference is worth considering, he said.

Critical Insights is the only one of the three polls Franklin reviewed that mathematically adjusted the results to match the party affiliations of Maine’s voting population.

While pollsters all tend to “weight” their samples by age and gender, they are split on whether to statistically adjust by party affiliation.

The polls by Public Policy Polling and Maine People’s Resource Center did not weight by party, although the surveys included a higher percentage of Democrats and a lower percentage of independent voters than the state population in general.

Both methods are valid and neither is perfect, Franklin said. “Different pollsters might make good principled arguments both ways.”

Party affiliations may have had some effect on the polls, but it doesn’t appear to be a large one.

Critical Insights’ unweighted data shows its adjustment for party affiliation slightly increased King’s lead but had almost no effect on the same-sex marriage question.

Meanwhile, the Rasmussen Reports poll released Thursday also was weighted by party affiliation, and its results were closely in line with the polls by Maine People’s Resource Center and Public Policy Polling.

In the end, Franklin said, all four polls are estimates, and they’re telling the same story.

“We certainly shouldn’t expect strong agreement on all of these numbers,” he said, “but my sense is they line up pretty good.”

Publishing poll results two weeks after a survey is conducted — as the Press Herald did today — is unusual for horse-race polls and potentially risky because events could change the race dynamics in the meantime, Franklin said. It also opens a poll to more criticism than usual from campaigns that don’t like the numbers, he said.

But such a delay in publishing results is not unethical if disclosed, and it is less unusual in the case of value-added polls used to explore issues and trends, Franklin said. 

Staff Writer John Richardson can be contacted at 791-6324 or at:

jrichardson@pressherald.com

Copy the Story Link