The Definitive SourceIndustry Insights
Learning from voter survey experiments
Leading up to the November election, AP worked with GfK Custom Research and the National Opinion Research Center at the University of Chicago to examine new ways to survey voters.
One study, designed by NORC and funded by the Knight Foundation, explored the opportunities and challenges of using smartphones to conduct exit polls of early voters and Election Day voters.
“The study was designed to use geolocation technology in smartphones to verify that participants had voted, either at Election Day polling places or early voting centers. Then, they would be sent exit poll questionnaires to complete on their smartphones,” explained David Pace, AP’s news editor for race calls and special projects.
He continued:
NORC first identified 576 panel members who had smartphones and agreed to take a consent survey. Sixty percent agreed to participate in the study. Email addresses were provided by 300 of those, who also agreed to download a smartphone app needed for the study. Younger people and those with more education were more likely to agree to participate.
By the time polls closed on Election Day, only 119 of the 300 people had downloaded the app, despite twice-weekly email reminders from September through Election Day. A follow-up survey found that nearly two-thirds of those who did not download the app had privacy concerns about allowing researchers to track their locations.
The app verified voting by just 22 of the 119 people, and 17 of those ended up completing the exit poll questionnaire. A follow-up survey of those who downloaded the app but did not receive a questionnaire found that 18 percent did not vote and 17 percent voted absentee by mail. Another 44 percent said they had voted early and 25 percent said they had voted on Election Day. Most of those who said they voted reported either that they deleted the app or did not activate it.
NORC’s recommendations on how to increase participation in such a survey are included in its full report on the study.
Separately, GfK Custom Research and NORC each conducted experiments using their probability-based online panels to survey self-identified voters about whom they supported in the presidential race and why.
Those online panel results then were compared with estimates from the traditional national exit poll of Election Day voters combined with telephone surveys of those who voted before Election Day. Both the online and traditional surveys overstated support for Hillary Clinton and understated support for President Donald Trump.
Pace said:
They GfK experiment showed Clinton winning 49 percent to 40 percent; the NORC experiment had Clinton winning 50 percent to 36 percent. The exit poll/telephone poll combination gave Clinton a 49 percent to 43 percent edge. While Trump won an electoral vote majority, Clinton won the national popular vote 48.2 percent to 46.1 percent.
Exit polls are adjusted to make each candidate’s share of the various demographic and attitudinal subgroups match the final vote totals at both the regional and national level. After GfK and NORC made similar adjustments to their online surveys, question-by-question comparisons showed only minor differences with the exit poll on the vote breakdowns in most subgroups.
In the NORC survey, the biggest differences with the exit poll were in the age, religion, education and income subgroups, and on two attitudinal questions — the timing of vote decisions and way the government is working. The most significant differences between the GfK survey and the exit poll were on black and Hispanic subgroups and on the timing of vote decisions.
Both online surveys were funded with a grant from the Knight Foundation. The full report on the NORC survey can be found here and here. A spreadsheet with question-by-question comparisons between the GfK survey and the exit poll can be found here.