Lessons From Polling In Hawaii - Honolulu Civil Beat


About the Authors

Matthew Fitch

Matthew Fitch is the executive director of Merriman River Group, an election management firm with offices in Hamden, Connecticut and Los Angeles, California. He is also a partner in MRG Research, the firm that provides polling services to Civil Beat.

Seth Rosenthal

Seth Rosenthal is a research methodology consultant to Merriman River Group. He received a Ph.D. in experimental psychology from Harvard University, and studied polling methods during a Fellowship at the Center for Public Leadership at Harvard's Kennedy School of Government.

Luke Henrici

Luke Henrici is a partner in MRG Research, authors of the Civil Beat/ Hawaii News Now polls.

Sara Elliott

Sara Elliott is a partner in MRG Research, authors of the Civil Beat/ Hawaii News Now polls.

Hawaii has long had a reputation as a difficult state in which to conduct an accurate poll. This year, our pre-primary and pre-election polls, conducted in partnership with Civil Beat and Hawaii News Now, proved to be very accurate.

In particular, our October poll was, top to bottom, one of the most accurate state-level polls in the country.

While we are always pleased to get it right, we have had our occasional misses, too. When we’ve missed in Hawaii, we have written about it and learned from it, adjusted our methodology accordingly, and invited conversation about it after the election.

You learn more from your mistakes than from your successes. This year, with significant errors in many other state polls throughout the country, we wanted to offer our experience from polling in 2020; the things that went right and the challenges we and our colleagues face going forward.

And, as our sister company Merriman River Group focuses on election administration, we want to start a conversation about the increasingly diverse ways people are able to cast a vote and what that means. While we don’t want to speculate too much about polls we did not conduct, we did notice response rates dropping, especially in battleground states.

Our conclusion is that part of it was an “over-fishing” phenomenon, where the sheer volume of campaigning and polling may have promoted a form of political fatigue where it became increasingly difficult to get people to participate in polls.

Very frequent voters are always more likely to take polls, making lower-proclivity voters (those who vote sometimes but not always) that much harder to reach. And data has suggested that President Donald Trump has been especially good at motivating those lower-proclivity voters to come out and support him.

While participation rates in polling dropped in many states, voter participation at the polls actually soared.

Despite his second defeat in the popular vote, President Trump actually exceeded his 2016 national vote total by over eight million votes, despite a perceived erosion among more frequent voters (indeed, President-elect Joe Biden received over ten million more votes than Hillary Clinton’s 2016 total).

It is likely that Trump’s delivery of these lower proclivity voters had a large impact on Senate races, where many Republicans over-performed public polls by large margins. And of course, expanded early voting and mail-in voting that was allowed in response to the pandemic further drove turnout. (Hawaii approved a full vote-by-mail program before the pandemic.)

Voters continue waiting in long lines to cast their ballots at the Honolulu Hale after dark on Election Day Tuesday, November 3, 2020. (Ronen Zilberman photo Civil Beat)
Voters waiting to cast their ballots at Honolulu Hale on Election Day. Ronen Zilberman/Civil Beat/2020

In Hawaii, we thankfully didn’t experience a drop in response rates to our polls, and our new, upgraded method of reaching cellphone-only voters actually improved our response rate with this growing sector of the electorate. Using our sample of registered voters, we now send a text message with a link to our online survey to a random selection of voters.

To reach landlines, we use IVR calling, in which a recorded voice asks questions and survey participants respond by pushing buttons on their phones. And once the cellphone and text data are collected, we weight the results by age, gender, race, location and educational level.

This double refinement of contacting a representative sample of voters and further refining the results by demographic weighting has worked well for us in the past and will remain our methodologic approach in the future, even as we always seek to improve the way we reach voters.

Some polling aggregators believe that polls conducted by live-agent callers are superior, but we disagree, especially in places like Hawaii. We believe a good IVR poll greatly reduces bias for a number of reasons.

First, using one professional recorded voice ensures that everyone has the same experience. It dramatically reduces mispronunciations, it eliminates data entry errors, and we believe people are more truthful and honest talking to a friendly and neutral voice than to another person who may be judging their responses.

Some short-cut IVR polls use a computer-generated voice. We strongly believe our use of an actual voice makes for a more motivating experience. We’re proud that our polling in Hawaii has consistently been the most accurate polling in Hawaii for several years now and hope to build on this in the future.

Finally, while voting in a pandemic raised obvious challenges, our democracy largely rose to the occasion. New ways to vote were offered to voters, and voters responded with the highest total turnout ever and the highest turnout by percentage in 120 years, two decades before women were allowed to vote.

Obviously, this was not universal. Some states did as little as possible to make voting accessible (hello, Texas). Other states, Pennsylvania most notably, expanded mail-in voting but forced election officials to wait until Election Day before touching the crush of incoming mail.

Hawaii is a clear success story. By expanding mail-in voting, turnout soared, and by allowing election officials to process that mail as it came in, Hawaii results were available very quickly. Ironically the slight delay in Hawaii reporting was related to last minute in-person voters. Hopefully, more states will follow Hawaii’s lead in the future.

Access to voting should not be a political issue. While mail-in voting greatly favored Democrats this year, it does not necessarily follow that it will do so in the future. Indeed, in many states with a history of mail-in voting, results typically are fairly evenly split.

This year, one party urged voters to vote by mail and the other party urged voters to vote in person. Mail-in balloting and early voting are reliable partners to in-person voting and should be expanded nationally in the years ahead.

Community Voices aims to encourage broad discussion on many topics of community interest. It’s kind of a cross between Letters to the Editor and op-eds. This is your space to talk about important issues or interesting people who are making a difference in our world. Column lengths should be no more than 800 words and we need a photo of the author and a bio. We welcome video commentary and other multimedia formats. Send to news@civilbeat.org. The opinions and information expressed in Community Voices are solely those of the authors and not Civil Beat.


Read this next:

Denby Fawcett: This Diamond Head Mom Found An Unusual Way To Fight City Hall


Local reporting when you need it most

Support timely, accurate, independent journalism.

Honolulu Civil Beat is a nonprofit organization, and your donation helps us produce local reporting that serves all of Hawaii.

Contribute

About the Authors

Matthew Fitch

Matthew Fitch is the executive director of Merriman River Group, an election management firm with offices in Hamden, Connecticut and Los Angeles, California. He is also a partner in MRG Research, the firm that provides polling services to Civil Beat.

Seth Rosenthal

Seth Rosenthal is a research methodology consultant to Merriman River Group. He received a Ph.D. in experimental psychology from Harvard University, and studied polling methods during a Fellowship at the Center for Public Leadership at Harvard's Kennedy School of Government.

Luke Henrici

Luke Henrici is a partner in MRG Research, authors of the Civil Beat/ Hawaii News Now polls.

Sara Elliott

Sara Elliott is a partner in MRG Research, authors of the Civil Beat/ Hawaii News Now polls.


Latest Comments (0)

State level polls? That's nice, I suppose.Anyone with common sense should now be well aware to never trust national polls and to just go vote.  I hope the polling industry significantly decreases.

pueobeach · 2 years ago

What I find most amazing is that Joe Biden's vote totals, 76,314,648  far exceeded the two most popular democratic candidates in recent history, Barrack Obama and Hillary Clinton.  In 2012, Obama's vote total was 65,915,795.  Hillary's 2016 total was 65,844,954.  Even more remarkable was the attendance at Biden's campaign rallies in comparison to Trump's.  This is a most incredible achievement by Biden and his organization and will go down in American politics as a truly astounding accomplishment!  Congratulations, Joe Biden.

incredibles2 · 2 years ago

I saw an article that looked at how predictive mask wearing correlated to the presidential vote which demonstrated equally predictive reliability. I also showed a pattern among responders Trump voters had no response to mask use by approximately 10%. Biden voters always responded to the mask use question.  This could be in part due to the shy Trump supporter effect which if Hawaii went 50/50 could account for a possible up to 5% error.  IVR may have been effective in obtaining an honest response from the otherwise shy Trump voter. 

Willyee · 2 years ago

Join the conversation

About IDEAS

IDEAS is the place you'll find essays, analysis and opinion on every aspect of life and public affairs in Hawaii. We want to showcase smart ideas about the future of Hawaii, from the state's sharpest thinkers, to stretch our collective thinking about a problem or an issue. Email news@civilbeat.org to submit an idea.

Mahalo!

You're officially signed up for our daily newsletter, the Morning Beat. A confirmation email will arrive shortly.

In the meantime, we have other newsletters that you might enjoy. Check the boxes for emails you'd like to receive.

  • What's this? Be the first to hear about important news stories with these occasional emails.
  • What's this? You'll hear from us whenever Civil Beat publishes a major project or investigation.
  • What's this? Get our latest environmental news on a monthly basis, including updates on Nathan Eagle's 'Hawaii 2040' series.
  • What's this? Get occasional emails highlighting essays, analysis and opinion from IDEAS, Civil Beat's commentary section.

Inbox overcrowded? Don't worry, you can unsubscribe
or update your preferences at any time.