Iran: measuring attitudes, busting myths

The findings from the international polling organisations were widely reported as fact in Western media and foreign policy. However, the research was based on telephone or in-person interviews; modes that do not guarantee anonymity. In a surveillance state where human rights are routinely violated, such modes make truthful responses extremely unlikely.

In 2020, Dr. Ammar Maleki, Assistant Professor of Comparative Politics at Tilburg University, and  Dr. Pooyan Tamimi Arab, Assistant Professor of Religious Studies at Utrecht University, developed a new methodology to understand the ‘nonpublic’ opinion that Iranians fear to reveal, usually at risk of imprisonment and torture.

The findings show increased secularisation of  the population with 47% of respondents stating that in their lifetime they had changed from being religious to non-religious, and only 22% support the idea of an Islamic Republic. GAMAAN’s surveys and these findings repeatedly went viral, and in response the Islamic Republic blocked the Survey Monkey App, and in 2022 the supreme leader Khamenei himself discredited the research. Despite data gathering and security challenges, GAMAAN’s findings continue to debunk myths about Iranian society and contribute to a clearer understanding of how Iranians feel and why protests and civil unrest is sweeping across the country. 

Can you briefly summarise how your methodology differs from that used  by other organisations to understand opinion in Iran?

DR. AMMAR MALEKI (founder and director, GAMAAN): Our surveys are conducted online. They are anonymous and give respondents a greater feeling of being safe. To generate reliable results, we mass disseminate the survey links through the most used social media platforms  in Iran, used by about 85% of the population (see f ig below). We spread the links through major digital channels as well as the specific pages and channels of groups that are harder to reach such as women, people who live in rural areas, and proponents of the regime. Since 2021 we also collaborate with Psiphon, which is among the most used VPN applications in Iran and they distribute the survey link to hundreds of thousands of people who connect to the Internet. Our survey sample sizes are large, for example in the last survey conducted in 2022 about 20,000. We then employ weighting and matching methods to generate a representative sample of the target population. 

DR. POOYAN TAMIMI ARAB (secretary, GAMAAN): It is perhaps also worth mentioning that our approach to surveying is interdisciplinary. For example, to design questions about religion we combined insights from disciplines such as anthropology and sociology with political science and public opinion research methods. We worked with Ladan Boroumand, who received the Lech Walesa Prize for her work to promote democratization in Iran, someone  with deep knowledge of Iranian history.  Or when we did a survey on the death penalty,  we consulted a legal expert and human rights defender such as Shirin Ebadi, a Nobel Peace Laureate. By combining disciplines and relying also on the advice of experts inside and outside Iran, we make sure that our approach is contextsensitive. Sensitivity to context appears to  have been lacking in some of the major survey institutes that conducted research in Iran, producing results that appear absurd to Iran analysts of different political persuasions. 

How do you ensure the research is representative? Don’t social media echo chambers just reflect the more extreme positions of opinion?

MALEKI: Conducting successful so-called nonprobability surveys is no sinecure. But our innovative data gathering methods – our access  to the population, being able to repeatedly make surveys go viral and with the cooperation of a major VPN platform – are not ordinary and have not been done in the same way in other countries. We systematically target all major groups to make sure that we can gather data from all walks of life and regions. We check our sample quality in realtime and adjust our targeting strategy accordingly. 

To make sure our results are representative, we have successfully replicated external data  such as on employment rates, household income distribution levels, and the languages people speak at home. We have also successfully replicated politically non-sensitive results of the World Values Survey’s last research in Iran, conducted in 2020  by face-to-face interviews.

TAMIMI ARAB: Moreover, there are other external data that match our results. For example, when studying Iranians’ secular shift, away from institutionalized religion, it is noteworthy that  the Iranian state’s own figures on topics such  as baby names show a sharp decline in popular Muslim names. And, of course, the current uprising, which has already been called a revolution, shows that results such as that a majority oppose the compulsory hijab match field experiences. In fact, no one specialized in Iran was surprised to find a secular shift. The only question was to which extent this process had already developed and whether interesting demographic patterns could be detected.

Are you concerned about government intervention and attempts to skewer your results, for example using fake accounts?

MALEKI: Yes, this is a challenge that we are always alert about. We monitor the sample in real-time. We also use digital mechanisms such  as to avoid multiple responses per device and bot attacks on the survey. We also test respondents’ honesty, using controlling questions.

Is there still fear among the Iranian population that even online responses are monitored and not anonymous, potentially leading to reprisals?

TAMIMI ARAB: Fear is not a static phenomenon, even though it certainly plays a role in any Iranian interactions that may be charged politically. While it is a challenge to get pro-regime individuals to participate in our survey, it is also  a challenge – even online – to convince a fearful population to participate and answer honestly. Our assistants showed us various instances of respondents expressing their concerns, even  if the survey was conducted anonymously by an institute based in the Netherlands.

MALEKI: We should also consider that Iranians know how to be present on social media with pseudonyms, which is a daily routine for them. The context-specific ways in which Iranians use the internet are crucial to understand why this research is feasible in Iran. On the other hand, many people are eager to express their opinions on socio-political issues if they feel they would leave no evident footprint. So we try to provide this safe mode to respondents.

How do you ensure participation from those who don’t have access to the internet? Are there particular groups that remain hard to reach?

MALEKI: We always mention in our reports and media appearances that our target population are “literate adults” as for answering an online survey, a respondent should be able to read and use mobile phones or a computer. We know that about 85 to 90 percent of Iranians are literate. On the other hand, other research and surveys showed that  the opinions of the illiterate population are not very different from those with lower education. So, the results of our survey with a specific margin of error can be generalized only to the target population. Right now, we are working on developing a mixed methods approach that will allow us in the near future to collect samples from those groups that are illiterate and have no access to the internet.

What communications tactics do you use to appeal to conservative, as well  as liberal views?

MALEKI: Other than using different images, we usually have various promotional texts for different groups of people. For instance, we understood that if we use some religious elements or a quotation from the supreme leaders of the Islamic Regime in our promotion, the regime supporters and conservatives may participate more. On the other hand, the survey itself  matters a great deal of course. We strive to design professional and impartial questions and answer items, which are crucial to convincing people to participate. People with different socio-political orientations should find their desirable answers and positions on each question. We design our questions given these standards so that all respondents feel that their preferred options are not excluded.

Beyond some of the headline findings regarding secularism, what other findings really surprised you?

MALEKI: One of the first surveys we  did was replicating the referendum on the Islamic Republic on the fortieth anniversary of that referendum in 1979. That survey showed that  86% of those who voted for Rouhani in the 2017 presidential election, i.e. for a so-called reformist or moderate candidate, would vote “No” to the Islamic republic in a free referendum. This was an eye-opening finding as for many years the regime has propagated that people voting in elections is an indicator of its legitimacy and public support for the regime. I was very happy that, for the  f irst time, a systematic study could open this black box.

TAMIMI ARAB: In another survey we understood that Iranians are extremely disappointed in the United States’ hesitations to promote democracy in Iran and in the region. Think, for example,  of the disastrous retreat from Afghanistan.  A result that will baffle many in Europe is that,  as a consequence, Trump, who is perceived to at least have bullied the ayatollahs, has been much more popular than Biden. On the other hand,  the United States is viewed favorably by many Iranians: in a 2021 survey, 20% thought “very favorably” and 32% “somewhat favorably”  about the U.S.

Does the current situation mean authorities will crack down further on national internet access – how would GAMAAN deal with such challenges?

MALEKI: Filtering and blocking the internet is an ongoing challenge for us, but the regime cannot fully shut down the internet. Right now we are dealing with a new situation because popular social media like Instagram and WhatsApp have been recently filtered in Iran. In response we are seeking to find new methods to reach people like using satellite TVs that are widely watched in Iran. On the other hand, new restrictions make VPN platforms like Psiphon more popular and used. All in all, we continue finding and utilising innovative approaches to contact people and collect their opinions.

Do you think the larger research organisations will adopt your approaches in Iran and/or other countries where freedom of  expression is restricted?

TAMIMI ARAB: I hope, especially, that larger research organisations start by giving up on  the idea that it is possible to do phone surveys  in authoritarian countries for measuring  sensitive questions. Even if they do not adopt  our approaches, which is also not that easy  and requires a unique team and access, they should at least stop producing numbers such  as that 95-99% of a country are X or Y. These totalising numbers only benefit the regimes and their apologists.

MALEKI: Our approach in GAMAAN has attracted the attention of scholars and other reputable pollsters like Pew and Gallup. We believe the issue of anonymity sampling is not only crucial for authoritarian contexts but also important in some cases in free societies. We know that even in free societies, some conventional polls could not measure people’s real attitudes. The US 2016 election was a case in point: a remarkable share  of voters did not reveal their honest opinions  or did not participate in some phone surveys. Given these challenges, we hope to develop  our methodology further to suggest alternative methods for measuring sensitive questions.

Given the current protests and government interventions what is the most useful role your organisation  can play, and in particular what areas  of research will you be focusing on  in the next 12 months?

MALEKI: We are going to conduct a new survey  in the coming month to measure what Iranians think about the ongoing protests, which can be said to be a revolution. What do Iranians expect from the various groups that support the revolution? What do they expect from the international community? In 2023, we will also conduct the second wave of our religion survey, f irst conducted in 2020. From the outset, our mission was to measure the real opinions  of Iranians and reveal them to both society and  the world. This will help to defeat the regime’s propaganda about what Iranians believe and demand, and it will help all actors to have a  more realistic picture of Iran and its societal developments.

You can find out more about GAMAAN here.

You can see all the winners and finalists of  the MRS Awards 2022 here.