Polling: What Went Wrong?

People who want to know the future have many tools.  There are crystal balls, fortune cookies, horoscopes, and just a bit more credible, public opinion polls.  The 2020 election featured a spectacular failure of the polling industry.

RealClearPolitics catalogues all major (non-candidate) polls.  They also average the results of all recent polls, which effectively increases the sample size and reduces the margin of error.  Their national polling average was Biden +7.2, while the popular vote was Biden +4.5, an error of 2.7%.  The results in many state polls were even worse.  In Ohio, the error favored Biden by 7.2%.  In Iowa, it was 6.2%.  In Wisconsin, it was 6%.  Overall, Trump outperformed the polls in 33 states.  Curiously, Biden did overperform the polls in a few states, including Minnesota.

A review of how polling works is in order.  It is neither possible nor practical to ask everyone in a population their opinion.  Thus a pollster seeks a sample of the population and tries to infer the views of the whole population based on the sample.  For this to work, the sample should be randomly selected, that is, every possible sample should be equally likely to be picked.  A randomly selected sample is unlikely to match the population exactly, but there are well-understood mathematical laws that describe how far from correct the results are likely to be.  This kind of error is known as sampling error–error caused by a sample not matching a population.  The margin of error is the margin on either side of the estimate that we can be 95% confident contains the true value.

In practice, however, the sample a pollster obtains is not random.  Nobody can be forced to participate in a poll, and if even one person declines, the sample is not random.  It used to be the case that most people answered the phone and talked to pollsters.  But over time, response rates declined due to telemarketers, robocalls, answering machines, caller ID, and cell phones.  Now response rates typically range between 1% and 5% of people called.  The sample a pollster obtains is usually wildly unrepresentative of the population.

How do pollsters deal with this?  They ask respondents various demographic information (race, sex, political party, education).  Then they weight the results so they match the presumed demographic breakdown of the electorate.  But they don’t actually know this breakdown.  Pollsters make an educated guess based on demographics of past elections (which can be estimated, but not known exactly) and their beliefs about what the electorate will look like.

Essentially this makes the poll itself an educated guess.  Educated guesses are often close to accurate, and they are more accurate than the sort of wishful thinking that predominates among political ideologues.  But educated guesses can be wrong, sometimes wildly so.

This cycle, it appears that many Trump supporters didn’t answer their phones, or refused to participate in polls.  This skewed the samples, even with the adjustments that pollsters made.  But why did Trump supporters refuse to talk to pollsters?  Some may simply hate the media.  Others may be concerned about admitting their views, even in a supposedly anonymous poll.

This phenomenon is known as social desirability bias.  One previous example of this is the Bradley effect, in which voters were supposedly more likely to say they would vote for a black candidate than to actually do so.  This effect remains controversial, however.

Researchers try to account for social desirability bias in various ways.  One way is to ask people what their friends or neighbors think about the election.  This is the methodology used by Trafalgar, a pollster who found much better results for President Trump than other pollsters.  The problem of social desirability bias applies to issue polls, as well.

Polls can be useful when appropriate precautions are taken, but other indicators of public sentiment should not be ignored.

You Betcha! (4)Nuh Uh.(7)

  9 comments for “Polling: What Went Wrong?

  1. SUPER-SPREADER Corinthian Scales
    December 8, 2020 at 10:11 pm

    Why is this asshole still here. Not a question.

    You Betcha! (5)Nuh Uh.(0)
  2. Jason
    December 8, 2020 at 10:47 pm

    "This cycle, it appears that many Trump supporters didn’t answer their phones, or refused to participate in polls. This skewed the samples, even with the adjustments that pollsters made. "

    Bull.

    The #narrativebrokers did everything they could do to suppress the vote. There. That is "What went wrong" We have idiots out there who don't pay attention long enough to fully understand what is being done to them.

    Just like the FRAUD in the election.

    So you want to argue with the science? Maybe science says Biden should have played the lottery.

    "The probability of former Vice President Biden winning the popular vote in the four Defendant States—Georgia, Michigan, Pennsylvania, and Wisconsin—independently given President Trump’s early lead in those States as of 3 a.m.on November 4, 2020, is less than one in a quadrillion, or 1 in 1,000,000,000,000,000. For former Vice President Biden to win these four States collectively, the odds of that event happening decrease to less than one in a quadrillion to the fourth power (i.e., 1 in 1,000,000,000,000,0004). SeeDecl. of Charles J. Cicchetti, Ph.D. (“Cicchetti Decl.”) at ¶¶ 14-21, 30-31 (App.4a-7a, 9a)."

    God Bless Texas.

    https://www.texasattorneygeneral.gov/sites/default/files/images/admin/2020/Press/SCOTUSFiling.pdf

    What went wrong? Geez

    You Betcha! (2)Nuh Uh.(0)
    • Conservative First
      December 14, 2020 at 2:01 am

      While it is conceivable that pollsters could skew their polls to support their preferred outcome, there are a couple problems with this theory. One is that they may also discourage their own side from voting due to complacency. This may have happened in 2016, as many D-leaning voters assumed that Trump couldn't win and didn't vote.
      The second problem is that as far as I can tell, R pollsters were finding similar results as public pollsters. Granted, the evidence is not conclusive here, but campaigns will often leak internal polls when public polls are off. That rarely happened this time. Also, R committees spent a lot of money in districts that R candidates ended up winning comfortably, indicating that their polls were showing them competitive.
      I do believe that pollsters often skew issue polls, typically by manipulating the wording of the question to support their side.

      You Betcha! (0)Nuh Uh.(0)
    • Conservative First
      December 14, 2020 at 2:14 am

      The analysis by Cicchetti makes an incredibly basic logical error. It assumes that each ballot counted has the same probability of being for Trump. But this is wrong. Election day voters favored Trump, and absentee voters favored Biden. In Michigan and many other states, election day votes were counted first and absentees were counted later. That is why Trump led early and trailed late. Note that Ohio counted absentees first, and Trump trailed early there before winning.
      https://www.nationalreview.com/corner/the-dumb-statistical-argument-in-texass-election-lawsuit/

      You Betcha! (0)Nuh Uh.(0)
  3. Sue Schwatrz
    December 9, 2020 at 12:11 pm

    Obviously, CF, you didn't flunk statistics like the rest of us. And, speaking of statistics, vote totals being expressed in 7.2%, 6.3%, etc., should alarm any 7th grader's math sense. Translated, 7.3 means, an obiden vote was worth 1.3, and a Trump vote was worth .7. There has been testimony of this, IN MICHIGAN.

    Ya heard about the bank teller who embezzled 1Cent from every account, every day and became a millionaire. This went on for years as an account missing $3,65 a year, was hardly noticed, or if it was noticed, too minor a figure to dispute. Well same principal.is being employed by duh minion. I suppose you don't want to hear that your bank is using the same technology employed by duh minion. You've been warned.

    You Betcha! (0)Nuh Uh.(0)
    • Conservative First
      December 14, 2020 at 2:25 am

      If the vote counting machines were rigged, why didn't the Trump campaign ask for a hand recount of the paper ballots? I can only conclude that don't believe this theory. There was a hand recount in Georgia, which revealed some human error, but no skewing of the machine count.
      https://www.nationalreview.com/2020/11/how-we-know-dominion-voting-machines-didnt-affect-the-election-outcome/?taid=5fb91e1d6dc962000115a032&utm_campaign=trueanthem&utm_medium=social&utm_source=twitter

      You Betcha! (0)Nuh Uh.(0)
  4. KG One
    December 9, 2020 at 12:17 pm

    "This cycle, it appears that many Trump supporters didn’t answer their phones, or refused to participate in polls. This skewed the samples, even with the adjustments that pollsters made. But why did Trump supporters refuse to talk to pollsters? Some may simply hate the media. Others may be concerned about admitting their views, even in a supposedly anonymous poll."

    Now, why would we not trust the media???

    When I was in college (very long ago), I remember getting chastised when I tried to change the data in class to fit the formulas I was taught. EVERY instructor ended that chastisement by "reminding" me that manipulating the data IS NOT how real science works.

    The researchers are missing the forest for the trees regarding why the 2020 Election played out the way it did.

    Pure, unadulterated election fraud, period.

    You Betcha! (0)Nuh Uh.(0)
    • Conservative First
      December 14, 2020 at 2:40 am

      There is good reason not to trust the media! I wouldn't respond to a pollster these days.
      There are many reasons why the election was unfair-media bias, big tech bias, children of illegal aliens voting, etc. I don't believe that voter fraud swayed the election, however. I have tried to check many claims of fraud, and none that I have checked stand up to scrutiny. I haven't gone as deep into the Detroit allegations, since they are hand to verify. The problem with relying on Detroit is that Biden got 1000 fewer votes in Detroit than Hillary did, which puts a limit on how much fraud there could plausibly be.

      You Betcha! (0)Nuh Uh.(0)
  5. Stuart
    December 9, 2020 at 2:05 pm

    What went wrong? seriously? Not a damn thing went wrong. Polling statistics are not meant to inform - they are meant to influence. They are pure propaganda. Moron

    You Betcha! (2)Nuh Uh.(0)

Leave a Reply

Your email address will not be published. Required fields are marked *