US elections and danger of social science methodology

The world peers wittily at television sets and computer screens as the live votes pour in from the United States election. While the election battle seems largely settled, counting continues for thousands of mailed in absentee ballots that must be tallied by hand instead of the standard vote tabulation machines in key states.

One salient takeaway that leaves us flabbergasted here in Kenya: the pre-election poll predictions were dramatically wrong….again. CNN, the Economist, NBC, Quinnipiac, Ipsos, Reuters, Fox News, Wall Street Journal, among many others, except notably the Trafalgar Group, leave the election with the proverbial egg on their faces for a second time in a row from faulty polls or defective probability predictions.

So, let the happenings on the other side of the world stand as a lesson to all of us involved in social science research. Data collection methodology matters deeply and bias errors can create mass distortions.

The effects of misrepresentations are not only theoretical, but intensely practical. Investors, governments, families, and entrepreneurs all make decisions based on predictions generated from various social science research forecasts and estimates, like polling.

Geoff Payne and Judy Payne delineate in their social science research guides that bias occurs when systematic errors happen in data collection or analysis. Such errors can originate from derisory technical procedures that can include sampling population mistakes, data distortion, interviewing mishaps, coding data improperly, intentional favouritism, or analytic flaws.

Unlike other social science research, polling analysis typically does not pass through the rigourous process of peer review from scientists. Inasmuch, methodology errors could oversample from internet polls which are less likely to include older citizens, utilise push polls that make respondents feel embarrassed like they must answer a certain way, or take samples from only narrow segments of the population.

Take USIU-Africa as an obvious example. Celebrating its American and African heritage as an institution, USIU-Africa hosts an open symposium that coincides with the United States presidential election every four years.

In this American election cycle, the colloquium was held last week and was open to Business Daily readers. Over 100 students and community members participated in the live Zoom session. Following the panelists' lectures, participants went through an actual American absentee ballot.

In the mock vote through Zoom's polling feature, Democratic Party presidential candidate Joe Biden and his running mate vice presidential candidate Kamala Harris won 76 per cent of the Kenyan vote with Republican Party presidential candidate Donald Trump and his running mate vice presidential candidate Mike Pence garnering the remaining 24 per cent. Such results clearly did not also occur in the United States election.

Why? The sample of Kenyans attending the colloquium were not similar to the average population of American voters.

Beyond polling, other business-oriented social science research can get distorted. Take the entrepreneurship discipline as a key example. One must remember that people generally try to please others.

So, if a normal polite survey respondent or interview participant gets asked an entrepreneurship product development question such as “you really like and appreciate this product feature, don’t you?”, which is biased towards a favourable response, then they will answer affirmatively to be polite even if in their hearts they feel very differently. Many suspect that push surveys and push interviews were one of the reasons for Yahoo’s early chaotic landing page search design that left the door open for Google to dominate the internet search market.

Another pitfall that entrepreneurs who are not trained researchers fall into, they sample the opinions of those not in their target markets. Entrepreneurs often turn to their social circle and families for opinions but not take the extra effort and initiative to go find dozens of respondents that would actually be actual users of their product or service.

So, should we give up on social science predictions born out of research? Absolutely not. We must take this unique opportunity to refine models, update rigourous peer review structures, and not just blindly believe everything we read without first examining its methodology.

Dr. Scott may be reached on [email protected] or on Twitter: @ScottProfessor