Citizens’ opinions are crucial for action on climate change, but are, owing to the complexity of the issue, diverse and potentially unformed1. We contribute to the understanding of public views on climate change and to knowledge needed by decision-makers by using a new approach to analyse answers to the open survey question ‘what comes to mind when you hear the words ‘climate change’?’. We apply automated text analysis, specifically structural topic modelling2, which induces distinct topics based on the relative frequencies of the words used in 2,115 responses. From these data, originating from the new, nationally representative Norwegian Citizen Panel, four distinct topics emerge: Weather/Ice, Future/Impact, Money/Consumption and Attribution. We find that Norwegians emphasize societal aspects of climate change more than do respondents in previous US and UK studies3, 4, 5, 6. Furthermore, variables that explain variation in closed questions, such as gender and education, yield different and surprising results when employed to explain variation in what respondents emphasize. Finally, the sharp distinction between scepticism and acceptance of conventional climate science, often seen in previous studies, blurs in many textual responses as scepticism frequently turns into ambivalence.
Numerous studies of public opinion about climate change show that agreement with the scientific consensus, concern about consequences and support for mitigation policies vary with age, gender, income and education7, 8, 9, 10, 11, 12. However, fewer studies address differences in how climate change is interpreted and what associations are made by individuals. In this study, we make three contributions. First, we examine Norwegians’ conceptions of what type of problem climate change is, and contrast this with previous studies of climate change imagery in the US3, 4, 5 and UK6. Second, we test whether the structurally stable variables that have explained differences in degree on indicators such as concern or trust in science also explain differences in kind, that is, what type of association individuals choose when asked to write about climate change in their own words. Third, analysing the most representative answers of each topic, we often find emotional or affective expressions.
The overwhelming number of sub-topics that link to climate change makes it difficult to condense this issue into a few salient dimensions. The Intergovernmental Panel on Climate Change (IPCC) divides the area into three sub-fields: the physical science basis, impacts and mitigation13. One study1 suggests six distinct frames: scientific uncertainty, national security, polar bears, money, catastrophe and justice/equity. Analysis of blogs shows how visions of negative impacts compete with more positive perspectives in the climate change debate14, 15.
Open-ended survey questions that permit respondents to use their own frame of reference, ‘even if this might seem inappropriate or ’irrational’ to the survey designer or analyst’16, thus add great value to the study of public perceptions of climate change. US respondents emphasize ice melt, heat, ‘alarmist’ and ‘naysayer’ topics when asked to associate a word or phrase with ‘global warming’ in four studies from 2003 to 2010 (refs 3, 4, 5). Overall, physical images (ice melt, heat, nature, flood/sea level, weather) dominate, whereas ‘naysayer’ views increase over time. A study using open-ended questions to elicit reasons for supporting or opposing mitigation measures in two US states17 finds four main categories of answers: economic, moral, political and technological. Men were more likely to bring up political rationales; women and young people more likely to bring up costs to self. Education, perhaps surprisingly, played no significant role predicting topic choice.
The main explanation for the relatively low number of studies of this kind has traditionally been cost, both to interviewers transcribing textual responses and to scholars analysing and categorizing the output. Recently, online survey methods and quantitative text analysis have brought those costs down, but to our knowledge this combination has not yet been exploited to shed light on climate change opinion. This study breaks new ground by including a greater number of responses, longer responses, a new country context (Norway) and crucially by employing automated techniques to induce a set of key topics based on mutual exclusivity and internal cohesion.
We aim to explore how diverse climate change discourses may influence and be reproduced by members of the public in their own words. Mental images of a phenomenon arguably precede cognition and thus serve as priors in decision-making, influencing how new information is processed4, 5. Discourse creates, reproduces, challenges and excludes different representations of the world, thus forming the basis of decisions and actions. From this perspective, the present study permits us to uncover some of the fundamental constraints on and opportunities of climate action. Specifically, the degree to which citizens cast climate change as personal and immediate rather than distant may influence the extent to which policymakers perceive support for controversial mitigation and adaptation measures18.
The basic components of natural language—the words—typically have many meanings19. Through the textual answers, we see that ‘climate change’ is associated with many different phenomena, some related to physical reality and others to people’s subjective attitudes, their beliefs, values and interests20.
Among the textual responses, the median response length was four words and the mean length was 10.1 words (62.7 characters); the longest response had 310 words. The total corpus contained 21,470 words (110,247 characters not including spaces). Of these, the most frequent words were ‘extreme weather’ (one word in Norwegian, used 142 times), ‘weather’ (131), ‘warmer’ (94), ‘natural disaster(s)’ (78) and ‘human-made’ (77).
Through manual analysis of a range of alternative model specifications, we found that running a structural topic model with four topics yielded the most semantically coherent and distinct topics, compared with specifications with more or fewer topics (see Supplementary Methods). The selected model is shown in Table 1. We propose the following labels for the four topics.
Table 1: Most discriminating words by induced topic, with suggested topic labels.
STM generates mixed membership results where each textual response is assigned proportions of each topic. Choices such as model specification, number of topics and interpretation of topic cohesion and exclusivity are made qualitatively by the authors.
The topic proportions are based on word frequencies. Topical prevalence is a vector that sums up to one for each individual text or response: for example, in a three-topic model, one response may be deemed by the model to belong 70% to Topic 1 and 15% to each of Topics 2 and 3. Similarly, aggregate topic prevalence is based on overall frequencies of words associated differentially with each topic. Earlier studies using open-ended questions in the context of climate change3, 4, 5 register counts of one or more topics per respondent, making aggregate topic proportions exceed 100%; these proportions are thus not directly comparable with the current study.
Responses with a high prevalence of words strongly linked to one topic are said to be representative of that topic, and close reading of representative responses is key to evaluating a model’s quality. When choosing the most representative responses for close reading, we select the 100 responses with the highest values on that topic’s prevalence vector, as generated by the model.
As STM is a multimodal estimator, there is a risk that initial model conditions produce unrepresentative results. To counter this problem, we initiated over 500 potential models from randomly generated starting values, including different numbers of topics, automatically selecting the 20% with the highest expectation-maximization values for full convergence runs using the selectModel function21 The resulting 100+ different model runs were then analysed qualitatively to arrive at our preferred model shown in the paper. The qualitative analysis was based on the authors’ readings of both the most discriminating terms by topic (Table 1) and most representative responses by topic (Table 2).
Our selected model contains four topics as qualitative analysis showed this number to yield the greatest semantic coherence within topics in combination with exclusivity between topics. This model reflects the general tendencies seen in the more than 100 model runs mentioned above. Specifically, most models tend to distinguish between the overall topics of impact/future/consequence, attribution/scepticism and weather/ice/sea-level rise. Supplementary Tables 1–3 show sample results of alternative model specifications containing five, six and eight topics. When greater numbers of topics were requested, a clear tendency was that ice melt and sea-level rise separated from weather events. Ice remained separate and coherent, whereas the other topics increasingly mixed at five or more topics.
Data were collected as part of the online Norwegian Citizen Panel24, based on postal recruitment of 25,000 individuals running from 6 November 2013 until 5 January 2014. Gender, age and education biases in the response rate were low25. The first wave garnered 4,905 survey subjects, amounting to a response rate of 20.1%. Of these, a randomly drawn sub-sample of 2,297 responses was used in the current study. The open-ended question had the wording: ‘What comes to mind when you hear the words ‘climate change’?’ This produced responses from 2,115 individuals or 92%. The question appeared at the end of the study to minimize loss of respondents.
Seven other questions related to climate change were posed early in the study, in part to correlate with the open-ended answers. These questions asked about opinions on oil production in Norway, personal concern about and causes of climate change, threat assessment, ease/difficulty of mitigation and moral obligation to reduce emissions. To minimize context effects26, the climate questions were asked early in the survey, with 41 questions (82 if counting individual battery items) about immigration, domestic terrorism, individual work situation and other demographic matters serving as a buffer. This strategy satisfies recommendations of a distance of at least six irrelevant items between related questions27. The fact that the questions are formally different (closed versus open), have dissimilar wordings and measure different constructs27, 28 also serves to avoid contamination effects. Close reading of the open answers finds no evidence of context effects, for example, as ‘oil’ (appearing early in the survey) is mentioned by only 26 respondents (1.2%) in the open answers. In contrast, ‘weather’ and ‘ice’, mentioned by 449 and 468 respondents, respectively (over 20%), were not brought up by earlier survey questions.
Concern was measured with the question ‘How concerned are you about climate change?’ with a five-point answer scale ranging from ‘Not at all concerned’ to ‘Very concerned’. Views on climate science were measured with an instrument used in previous studies29 asking respondents whether climate change was human-induced, natural, or not happening, with ‘don’t know’ a fourth option. The textual responses were stemmed and stop words and punctuation removed using the SnowballC package. Subsequently, 371 terms remained. For the most frequent terms, varieties of the written standards nynorsk and bokmål were harmonized.
Figures 1 and 2 were generated on the basis of linear regression models with topic proportions for each of the four topics as dependent variables and age, gender, education and concern about climate change as explanatory variables. Detailed regression results for each model are given in Supplementary Table 4. Significance testing was performed using T-statistics based on standard errors that incorporate both estimation uncertainty from the topic induction process21, 22 and estimation uncertainty from the regression models. The figures thus show the most conservative uncertainty estimates available.
Data deposition.
A replication data set with R code has been deposited at the Harvard Dataverse Network: E.T.; K.F., 2015, ‘Replication data for: Explaining topic prevalence in answers to open-ended survey questions about climate change’, http://dx.doi.org/10.7910/DVN/28689 Harvard Dataverse Network [Distributor] V2 [Version].
Smith, N. & Leiserowitz, A.The rise of global warming skepticism: Exploring affective image associations in the United States over time. Risk Anal.32, 1021–1032 (2012).
Endre Tvinnereim. Explaining topic prevalence in answers to open-ended survey questions about climate change[J]. Nature Climate Change,2015-06-01,Volume:5:Pages:744;747 (2015).