Volume 30, Issue 5 p. 990-999
Review
Open Access

Learning and the transformative potential of citizen science

Györgyi Bela

Corresponding Author

Györgyi Bela

Environmental Social Science Research Group, H-1024 Rómer Flóris Street 38, Budapest, H-1024 Hungary

email [email protected]Search for more papers by this author
Taru Peltola

Taru Peltola

Université Grenoble Alpes, Irstea, UR DTGR, 2 rue de la Papeterie-BP 76, St-Martin-d'Hères, F-38402 France

Finnish Environment Institute, Yliopistokatu 7-PL 111, Joensuu, 80101 Finland

Search for more papers by this author
Juliette C. Young

Juliette C. Young

NERC Centre for Ecology and Hydrology, Bush Estate, Edinburgh, EH26 0QB U.K.

Search for more papers by this author
Bálint Balázs

Bálint Balázs

Environmental Social Science Research Group, Rómer Flóris Street 38, Budapest, H-1024 Hungary

Search for more papers by this author
Isabelle Arpin

Isabelle Arpin

Université Grenoble Alpes, Irstea, UR DTGR, 2 rue de la Papeterie-BP 76, St-Martin-d'Hères, F-38402 France

Search for more papers by this author
György Pataki

György Pataki

Environmental Social Science Research Group, Rómer Flóris Street 38, Budapest, H-1024 Hungary

Department of Decision Science, Corvinus University of Budapest, Fővám tér 8, Budapest, H-1093 Hungary

Search for more papers by this author
Jennifer Hauck

Jennifer Hauck

Department Ecosystem Services, UFZ- Helmholtz Center for Environmental Research, Permoserstraße 15, Leipzig, 04318 Germany

German Center for integrative Bioidversity Research (iDiv) Halle-Jena-Leipzig, Deutscher Platz 5e, Leipzig, 04103 Germany

Search for more papers by this author
Eszter Kelemen

Eszter Kelemen

Environmental Social Science Research Group, Rómer Flóris Street 38, Budapest, H-1024 Hungary

Search for more papers by this author
Leena Kopperoinen

Leena Kopperoinen

Mechelininkatu 34a, P.O. Box 140, 00251 Helsinki, Finland

Search for more papers by this author
Ann Van Herzele

Ann Van Herzele

Research Institute for Nature and Forest, Brussels, Kliniekstraat 25, 1070 Anderlecht, Brussels, Belgium

Search for more papers by this author
Hans Keune

Hans Keune

Research Institute for Nature and Forest, Brussels, Kliniekstraat 25, 1070 Anderlecht, Brussels, Belgium

Search for more papers by this author
Susanne Hecker

Susanne Hecker

Department Ecosystem Services, UFZ- Helmholtz Center for Environmental Research, Permoserstraße 15, Leipzig, 04318 Germany

German Center for integrative Bioidversity Research (iDiv) Halle-Jena-Leipzig, Deutscher Platz 5e, Leipzig, 04103 Germany

Search for more papers by this author
Monika Suškevičs

Monika Suškevičs

Estonian University of Life Sciences, Friedrich Reinhold Kreutzwaldi 1, Tartu, 51014 Estonia

Search for more papers by this author
Helen E. Roy

Helen E. Roy

NERC Centre for Ecology and Hydrology, Wallingford, OX10 8BB U.K.

Search for more papers by this author
Pekka Itkonen

Pekka Itkonen

Mechelininkatu 34a, P.O. Box 140, 00251 Helsinki, Finland

Search for more papers by this author
Mart Külvik

Mart Külvik

Estonian University of Life Sciences, Friedrich Reinhold Kreutzwaldi 1, Tartu, 51014 Estonia

Search for more papers by this author
Miklós László

Miklós László

Technical University in Zvolen/Institute of Landscape Ecology of SAS, Stefanikova 3, P.O. Box 254, Bratislava, 814 99 Slovak Republic

Search for more papers by this author
Corina Basnou

Corina Basnou

CREAF, Cerdanyola del Vallès, Spain

Search for more papers by this author
Joan Pino

Joan Pino

Univ Autònoma Barcelonas, Cerdanyola del Vallès, 08193 Spain

CREAF, Cerdanyola del Vallès, Spain

Search for more papers by this author
Aletta Bonn

Aletta Bonn

Department Ecosystem Services, UFZ- Helmholtz Center for Environmental Research, Permoserstraße 15, Leipzig, 04318 Germany

German Center for integrative Bioidversity Research (iDiv) Halle-Jena-Leipzig, Deutscher Platz 5e, Leipzig, 04103 Germany

Institute of Ecology, Friedrich Schiller University Jena, Dornburger-Strasse 159, Jena, 07743 Germany

Search for more papers by this author
First published: 17 May 2016
Citations: 146

Abstract

en

The number of collaborative initiatives between scientists and volunteers (i.e., citizen science) is increasing across many research fields. The promise of societal transformation together with scientific breakthroughs contributes to the current popularity of citizen science (CS) in the policy domain. We examined the transformative capacity of citizen science in particular learning through environmental CS as conservation tool. We reviewed the CS and social-learning literature and examined 14 conservation projects across Europe that involved collaborative CS. We also developed a template that can be used to explore learning arrangements (i.e., learning events and materials) in CS projects and to explain how the desired outcomes can be achieved through CS learning. We found that recent studies aiming to define CS for analytical purposes often fail to improve the conceptual clarity of CS; CS programs may have transformative potential, especially for the development of individual skills, but such transformation is not necessarily occurring at the organizational and institutional levels; empirical evidence on simple learning outcomes, but the assertion of transformative effects of CS learning is often based on assumptions rather than empirical observation; and it is unanimous that learning in CS is considered important, but in practice it often goes unreported or unevaluated. In conclusion, we point to the need for reliable and transparent measurement of transformative effects for democratization of knowledge production.

Abstract

es

El Aprendizaje y el Potencial Transformador de la Ciencia Ciudadana

Resumen

El número de iniciativas colaborativas entre los científicos y los voluntarios (es decir, ciencia ciudadana) está incrementando en muchas áreas de investigación. La promesa de una transformación social junto con avances científicos contribuye a la popularidad actual de la ciencia ciudadana (CC) en el dominio político. Examinamos la capacidad transformativa de la ciencia ciudadana, en particular del aprendizaje por medio de CC ambiental como herramienta de conservación. Revisamos la literatura sobre CC y aprendizaje social y examinamos 14 proyectos de conservación en Europa que involucraban CC colaborativa. También desarrollamos un patrón que puede usarse para explorar los arreglos de aprendizaje (es decir, los materiales y eventos de aprendizaje) en los proyectos de CC y para explicar cómo los desarrollos deseados pueden obtenerse mediante el aprendizaje de CC. Encontramos que los estudios recientes que buscan definir a la CC por propósitos analíticos fallan continuamente en la mejora de la claridad conceptual de la CC; que los programas de CC pueden tener potencial transformativo, especialmente para el desarrollo de las habilidades individuales, pero dicha transformación no está ocurriendo necesariamente en los niveles institucionales y de organización; que existe evidencia empírica de los resultados simples de aprendizaje, pero la aseveración de los efectos transformativos del aprendizaje de CC está basada continuamente en suposiciones en lugar de observaciones empíricas; y que es unánime que el aprendizaje en la CC está considerado como importante, pero en la práctica continuamente sigue sin ser reportado o evaluado. En conclusión, señalamos la necesidad de una medida confiable y transparente de los efectos transformadores para la democratización de la producción del conocimiento.

Introduction

The development of collaborative initiatives between scientists and volunteers, who often have limited formal training in science, is increasing across many research fields (e.g., Freitag & Pfeffer 2013; Bonney et al. 2014). Much of current understanding of nature and biodiversity is derived from data that have been collected, transcribed, or processed by volunteers (Bonney et al. 2014). Only recently has the term citizen science (CS) been coined to describe the involvement of volunteers in the scientific process. CS may be thought of in terms of specific arrangements that enable scientists to collaborate with volunteers to generate new and legitimate knowledge about nature and to devise new approaches and methods while empowering volunteer citizens and enhancing their scientific literacy. Some authors even argue that CS results in improved science–society–policy interactions and leads to a more democratic approach to research that is based on evidence and informed decision making (Serrano Sanz et al. 2014).

Environmental CS holds promises to improve the knowledge base of conservation science and increase society's support for conservation. Making room for CS in conservation programs may improve the visibility and acceptability of conservation science's research findings in the policy field and may contribute to the transformation of conservation science into a more transparent, open, democratic, and socially relevant endeavor (Conrad & Hilchey 2011; Cosquer et al. 2012). The possibility of combined societal transformations (through transparency, public participation, local knowledge, science–society–policy interrelations, decentralization, innovation, and democracy) and scientific breakthroughs is often mentioned in discussion of CS and contributes to its current popularity in the policy domain (e.g., Serrano Sanz et al. 2014). Despite this emphasis on the social effects of CS, there is little systematic evidence on the transformative capacity of CS. CS projects are, in fact, frequently described as merely a source of scientific data because biodiversity research is often driven by the need to gain more scientific information for research, management, monitoring, and planning processes (Rotman et al. 2012). Focus on acquiring data and information may, however, neglect the broader transformative potential as explicitly emphasized in research on resilience (Berkes et al. 2003), risk perception (Wynne 1992), and adaptive and coadaptive management (Pahl-Wostl et al. 2007; Armitage et al. 2008; Cowling et al. 2008). Although the quality and validity of CS data, and the costs involved, can be evaluated and measured relatively easily, the organizational, social, and political impacts are much more difficult to evaluate (Bull et al. 2008). This means there is an urgent need to account for the multidimensional and dynamic character of CS and to assess its outcomes (Overdevest et al. 2004; Jordan et al. 2012).

We sought to critically analyze the transformative capacity that some claim environmental CS possesses. We focused on learning, which can be seen as a primary (although not the only) mechanism through which transformative effects are produced. Learning in environmental science and resource management has recently received much attention (Armitage et al. 2008; Rodela 2011) as a major way of raising ecological consciousness and overcoming the current ecological crisis (O'Sullivan & Taylor 2004). Many learning outcomes, such as increased environmental awareness, place-based and global stewardship, enhanced trust, improved management of social–ecological systems, and closer connections to others may affect conservation initiatives (Gruenewald & Smith 2008; Pahl-Wostl et al. 2008; Reed et al. 2010).

The key question we addressed was how can one best shape CS processes to enable and support transformative learning for the benefit of conservation? Transformative learning is defined as a deep, structural shift in awareness that alters one's way of being in the world and how one views interconnectedness among the universe, the natural environment, one's personal world, and the human community (O'Sullivan et al. 2003). Although many social benefits are associated with learning processes inherent to CS, evidence about them is often fragmentary and anecdotal. Modalities and outcomes of learning arrangements in CS may be linked to motivations to participate in CS, whereas disappointment concerning learning processes and outcomes of CS may lead to enduring skepticism and even withdrawal from CS approaches. It is therefore essential to enhance understanding of who learns what in CS and how this learning happens.

To critically evaluate the transformative potential of CS, we invited researchers involved in CS projects in the field of conservation to engage in a collaborative assessment and writing process. First, we reviewed the current literature on learning related to natural resource management and CS that shows learning processes are complex and polymorphic and that learning can be facilitated by various CS approaches across the spectrum from contributory to collaborative (Lawrence 2006; Rotman et al. 2012). We then devised a template for the identification of various crucial elements of learning in CS and used the template to assess learning processes in 14 CS case studies from across Europe. Finally, we considered our results as they relate to designing learning arrangements for CS projects that have the potential to increase collective reflection about the outcomes of such initiatives and to provide more meaningful arenas for public participation in science.

Methods

Case Studies

Using a narrative review method (Machi & McEvoy 2012), a 20-person collaborative assessment team with different disciplinary backgrounds and experiences in CS and from different institutes in A Long-Term Biodiversity, Ecosystem and Awareness Research Network (ALTER-Net) conducted the literature review using a narrative review method (Machi & McEvoy 2012). They focused on literature reporting on the learning outcomes of CS in relation to biodiversity and ecosystem services and on literature reporting empirical evidence on learning. Articles were selected by keyword searches for CS, civic science, crowd science, and public participation in scientific research in Web of Science.

To facilitate reflexive inquiry into the learning aspects of CS by researchers involved in CS projects, we applied a collaborative assessment approach. Such an approach can be considered a knowledge coproduction process in which participants are the key informants (Lowry et al. 2004). The assessment team collaborated in an iterative and transparent way that included several virtual meetings and a 2-day face-to-face workshop in 2015. The online meetings served mainly to clarify the aims of the joint work and to share and discuss findings from the existing literature, whereas at the workshop we developed the structure and content of the template. An independent facilitator helped workshop participants share and reflect on their personal experiences in CS projects.

Participants selected key questions and developed a unified template for the case study review. We used the template to collect secondary data from 14 environmental CS projects covering a broad geographical range across Europe (Supporting Information). The key criteria for case selection were the quantity and depth of available information and a balanced geographical representation across Europe. Because learning aspects of CS projects are rarely explicit in publications, we selected projects undertaken by the team members’ affiliated organizations or projects for which a priori data could be easily gathered (e.g., through key-informant interviews). We used completed templates to conduct an ex post case study review by applying the method of interpretive synthesis of qualitative case research (Weed 2005). The assessment and interpretation of results was done at the case-study level, and the interpretations of the case-study data were the raw material of the review.

Template for Understanding Learning in CS

There are numerous learning aspects discussed in the literature on CS (Overdevest et al. 2004; Evans et al. 2005; Fernandez-Gimenez et al. 2008; O'Farrell & Anderson 2010; Jordan et al. 2012), including substance of learning in CS, nature of the learning processes in CS, identity of learners and level or scale of learning, and participatory arrangements of CS designed to facilitate intended learning.

We synthesized the discussion of these aspects into a template (Supporting Information) we used to identify the key elements of learning and to analyze how CS arrangements in the 14 case studies facilitated learning. Key elements of the template were the substance of learning (knowledge gained, skills and capacities developed by different types of participants, reflection); the nature of learning (nature and level of collaboration and the nature of the knowledge generation process); the distribution of learning effects among scientists and citizens; and the design of learning arrangements (how and through what kind of arrangements—training events, materials, information and communication technologies, etc.—learning is intended to happen).

Citizens participating as volunteers may be expected to learn about the natural object they monitor, increase their ecological literacy (e.g., Evans et al. 2005), hone their observation skills, or learn to use new instruments. Professional scientists involved in CS may improve their knowledge and understanding of biodiversity or ecosystem services by analyzing the often large data sets collected by citizens (Rotman et al. 2012). However, learning can be understood in broader terms, such as understanding the nature of scientific work and the roles and identities of the participants. Volunteers may learn how scientific knowledge is produced (Devictor et al. 2010) and to understand the challenges of scientific research (Fernandez-Gimenez et al. 2008). Scientists may learn to see the role of the citizens in new ways and to treat them not as anonymous, replaceable, and exchangeable members of a crowd (which is most frequently the case [Buytaert et al. 2014]) but as holders of specific knowledge. This learning experience may lead to transforming practices: citizens may be invited to articulate research questions and hypotheses or even to design research methods and contribute to the interpretation of results, as in the case of “extreme CS” (Rowland 2012). Such outcomes depend on the level and modality of information given to participants about the results of the CS programs and their potential inclusion in decision making and management.

Although learning in CS has commonly been analyzed in terms of learning outcomes (e.g., scientific literacy), various theoretical perspectives—from anthropology, organizational studies, management sciences, educational psychology, cognitive science, cybernetics, and computer sciences—suggest widening the perspective to include the nature of learning processes, distinguishing between individual and social, exploitative and explorative, single- and double-loop, and incremental and transformative learning (Mezirow 2003, 2006; Illeris 2009). From an individual perspective, learning is seen as the acquisition of explicit, codified, and stored knowledge. The social view emphasizes learning as a process in which knowledge—which may be implicit and not easily codified or inseparable from practice—is socially constructed and distributed. According to behavioral theories (Levitt & March 1988), learning can be understood as an adaptive process taking place through the best use of what is known; this type of learning is called exploitation. Exploitative learning accumulates a knowledge base incrementally and refines and extends existing competencies, technologies, and paradigms. In contrast, learning can also be understood as exploration (i.e., a process in which new alternatives are experimented with). Explorative learning introduces variation into the system; hence, its returns are uncertain, distant, and often unintended. Learning through exploration is a transformative form of learning because it relies on a particularly complex approach involving cognitive, social, and emotional aspects that enables people to adopt frames of reference that are more inclusive, self-reflexive, and integrative of experience (Mezirow 2006; Illeris 2009).

The separation of single- and double-loop learning constitutes a similar distinction (Argyris & Schön 1978). Learning is single-loop when one follows routines, detecting and correcting errors without questioning the underlying assumptions, implicit knowledge claims, or paradigms. Double-loop learning entails a critical reflection upon, and a consequent change in, underlying norms and principles guiding performance. Double-loop learning occurs in CS projects associated with unintended but significant personal, social, and political transformations that go well beyond the acquisition of planned and expected data that takes place in incremental learning. For example, CS can trigger new policy discourses and concerns regardless of the policy goals at which it was initially directed (Ellis & Waterton 2005; Wilson 2011; Peltola 2015).

Despite these typologies, types of learning processes are not mutually exclusive and are often combined unevenly in CS programs. Hence, there is a need to examine empirically the nature of learning in CS projects.

It is important to consider who participates in CS programs—who learns and who is supposed to learn. This may have broader social implications. If participants belong to the educated part of society, as often seems the case (e.g. Evans et al. 2005), CS may reinforce social inequalities. CS programs enrolling participants from less-educated backgrounds may help address inequalities (e.g., Gura 2013).

Learning, and subsequent transformative potential, can be analyzed at different scales or levels: individuals, knowledge communities, and organizations and institutions. Another important aspect of learning concerns its distribution among participants, particularly between professional scientists and volunteers. Lawrence (2006) refers to the multiple ways in which data collection contributes to the relationship between person and place or person and natural objects and may affect the values of citizen participants. While the primary goal may be a scientific one, engagement in CS could lead to skills in forming and maintaining partnerships. At the organizational level, routines and norms of conducting scientific inquiry may change and lead to new ways of dealing with the science-society interface or to new identities for professional scientific organizations. In the process of CS, science as an institution may also be questioned or reflected upon critically, leading to changes at a systemic level. Because science can be considered a significant institution in contemporary societies, institutional learning may increase the resilience of socioecological systems (i.e., their capacity to absorb shocks yet maintain function) (Fernandez-Gimenez et al. 2008; Jordan et al. 2012). This could take place as the format and processes of scientific knowledge generation are broadened and adapted to the needs of society in general and to the needs of specific social groups or contexts. For example, volunteer participation could lead to new ways of identifying, documenting, and reacting to changes—ways that scientists alone could not have anticipated.

There are specific learning arrangements in CS programs, and these arrangements affect learning in different cultures (Bonney et al. 2009). CS initiatives offer a range of learning opportunities, including allowing participants to ask research questions relevant to them and training on data gathering and analysis to answer these research questions. Several issues arise regarding these learning arrangements. What is the ideal organization for volunteer–scientist collaboration that supports the coconstruction of knowledge? How can CS arrange and promote better learning or aspire to single- and double-loop learning, exploration, and exploitation? How can different types of learning be combined in CS to facilitate learning, and which types of social interactions between scientists and volunteers are critical? Even technical designs can make a difference in this respect. Similarly to other types of citizen participatory arrangements, CS forums may attract and motivate different groups and offer different ways to act depending on the type of arrangement (Jupp 2008).

Techniques for Evaluating Learning in CS

The multiple dimensions of learning are not easy to evaluate. Exchange of factual biodiversity knowledge can be measured through pre- and postproject surveys of CS participants, through document analysis of the contents participants created, or through interview and focus-group techniques (Bonney et al. 2009). However, evaluating learning in CS programs requires extending the evaluation of factual outcomes at the individual level to the evaluation of the various forms of learning taking place at different social and temporal scales, using a range of methods capable of accounting for these dimensions of learning.

Some studies aimed to assess whether volunteer participants improved their ecological literacy or understanding of scientific processes. Brossard et al. (2005), for example, measured the impact of CS on participants’ attitudes toward science and the environment, on their knowledge of bird biology, and on their understanding of the scientific process by applying a rigorous standardized survey method. Crall et al. (2013) identified improvements in participants’ scientific literacy in a training program on invasive species by using a survey method, whereas Cragg et al. (2001) developed quantitative methods to measure various aspects of cognitive, affective, or behavioral change. An exploratory study combined multiple methods (survey, focus group, and interviews) to study how CS contributes to increased environmental awareness among the general public (McKenzie et al. 2014). Another study (Price & Lee 2013) used pre- and postproject surveys and interviews with participants to investigate whether any change in attitudes toward CS projects was observable. They state that CS may result in “the application of scientific thinking to everyday life.” Shwartz et al. (2012) used quantitative and qualitative methods and found that short conservation-education programs can enhance experiences with nature and increase knowledge and awareness of, interest in, and concern for nature.

Yet these studies address only learning outcomes. Assessment of the nature of the learning processes in CS initiatives constitutes another important but challenging question. In particular, identification of transformative learning processes is rather limited in existing literature. Assessing the nature of learning could be achieved using qualitative strategies. A learning activities survey, for example, could be used to assess the design and delivery of programs from a transformative perspective (King 2009). However, there is a critical lack of methods for and examples of studies analyzing learning processes, the type and level of participation, and the implications of learning arrangements. This explains why these aspects of learning are rarely discussed within the CS literature and implies that such methods are needed. Our template approach is an attempt to address the multiple dimensions of learning in CS.

Assessment of Case Studies

The assessment and interpretive synthesis of the 14 case studies (details in Supporting Information) demonstrates the variety of ways in which learning takes place in CS. All the case studies were biodiversity related and focused on surveying, measuring, monitoring, or observing the natural world. Some projects concentrated on certain groups of species. Other projects collected data on the flora and fauna of a given area or explored the cultural ecosystem services of a landscape. Results for each case study relevant to level of citizen engagement, program objectives, learning outcomes for scientists, and learning outcomes for citizen participants are available as Appendix S2.

Results

Case Studies

Nearly all 14 programs examined had the general aim of contributing knowledge to support decision making and clear scientific ecological interests such as obtaining distribution data and detecting threats to species. Other scientific interests included the development and testing of new methods and social-scientific objectives such as understanding citizens’ perceptions and values of nature and citizens’ relation to nature. The objectives and learning outcomes for the 14 case studies are in Supporting Information.

Half the reviewed cases focused explicitly on environmental education, awareness raising, and increasing interest in conservation issues. Empowerment of citizens or engaging them in conservation activities were also mentioned as objectives. Six projects aimed to connect people with nature and engage them in scientific monitoring (Vadonleső in Hungary [Vadonleső]; Big Bumblebee Discovery in the United Kingdom [Bumblebee]; Open Farm Sunday Pollinator Survey in the United Kingdom [Open Farm]; Catalan Butterfly Monitoring System in Spain [Butterfly]; BioBlitz in Barcelona Spain [Bioblitz]; Nature Observations’ Database in Estonia [Observation]) and 3 aimed to create a network among involved actors (Porot Reindeer in Finland [Reindeer]; Amphibians and Reptiles from BirdLife Hungary [Amphibians]; Virtual Biodiversity in Spain [Virtual Biodiversity]). Two projects focused on increasing citizen participation in planning processes (Helsinki-Uusimaa Regional Plan 4 in Finland [UUSIMAA]; Sibbesborg local master plan in Finland [Sibbesborg]). In the latter case, the activities were partly driven by the planners themselves and scientists were only the facilitators.

In all reviewed CS initiatives, learning could be considered a 2-way process that enabled both volunteers and professional researchers to learn; albeit, they learned different things. Not all case studies reported learning by citizens as a stated objective.

In all the CS programs, learning by scientists occurred. In some cases, scientists’ methodological knowledge was increased and publishable ecological knowledge and understanding were enhanced. Scientists’ factual knowledge was also enhanced through the CS projects because large data sets became available for further analysis and because locally relevant, place-specific knowledge was revealed as previously scattered pieces of knowledge were put together to form verified local knowledge.

In some of the reviewed CS initiatives, the case-specific application of the scientific method was codeveloped by scientists and citizens. This can be regarded as learning to carry out science collaboratively. Such codesign of the research methods can be a source of transformative learning. Scientists learned how their universal methods could be flexibly adapted to case-specific circumstances. Codesigning the methodology also necessitates the development of a common language, combining scientific terminology with the language of citizens and practitioners (e.g., in the case of Sibbesborg). Use of this common language was also an instance of transformative learning. Transformative-learning potential also included improved knowledge and understanding about citizens’ perceptions and values. This was related to, for example, how local communities think about biodiversity issues (i.e., volunteers as the antennae of society) in the Marten Network in Belgium and Bioblitz cases. In one case, the unintended lessons about challenges of CS (estimation of wolf population, Finland [Wolf]) became evident, and in 5 other cases the skills that scientists had developed to facilitate such processes were mentioned (Reindeer, Observation, Open Farm, and Vadonleső, Amphibians). Procedural knowledge was gained by scientists through experimenting with new technologies and through facilitation and management citizens’ involvement.

In most cases it was assumed that the citizens involved had gained some ecological knowledge and understanding, and in some cases it was proven that citizens learned how to use scientific methods. The ecological literacy of citizens was perceived to have increased because of learning about certain species or ecological conditions through data collection and because of developing an overall picture of species and ecosystems by seeing larger data sets or maps. Learning how science is approached was more difficult to detect, but some case studies indicated citizens also gained knowledge about this, for example, by exploring new technologies (i.e., GIS-based observations and generation of data sets in the UUSIMAA and Sibbesborg cases), by talking with scientists, or by adopting a scientific method and using it to challenge the primacy of scientific knowledge production (Wolf case).

In 4 cases, the knowledge created was reported to have raised awareness of conservation issues. In 2 of these (Ladybird Survey in the United Kingdom [Ladybird] and Vadonleső) increased awareness led to spin-off projects. In the other 2, citizen action emerged (Observation and Amphibians).

Empowerment of citizens was mentioned as a goal in half the cases, including through problem-solving skills gained through the use of the scientific method. In the Reindeer case, for instance, increased understanding of other social groups was one of the major outcomes of the project. As a consequence, the reindeer herders were empowered because they learned to use scientific data to discuss their issues with other land-user groups and to find common management strategies.

Learning arrangements can enhance not only citizens’ learning but their commitment and motivation. Such transformative outcomes were actively sought after by scientists in the Marten Network (by means of newsletters, personal emails, and “autopsy days”). The effect of CS projects on the level of trust between citizens and scientists (and other stakeholders such as planners or decision makers) was mixed. Some cases demonstrated an increase in trust throughout the project due to colearning and reflection (e.g., Observation), but others shed light on unintended negative effects on trust between citizens and scientists (e.g., Wolf). The changing level of trust can be mixed even within cases. In the Amphibians case, for example, data providers were trusted by facilitators and scientists but abundance maps of some rare and little-loved species (e.g., vipers) were not made publicly available because the organizers feared average citizens would harm these species if they knew their location.

Design and Engagement

Training and guidelines or protocols for data collection were usually provided in our CS case studies. Introductory reading material that had been distributed was perceived by the collaborative assessment team as enhancing learning. However, it was not always possible to assess how thoroughly the citizens had acquainted themselves with the introductory material. Internet communication for consultation and exchange were considered by the collaborative assessment team to be as important as face-to-face interactions during participatory workshops.

Social media and traditional media such as magazines and press releases were used to reach citizens and to invite them to contribute or to provide results. In some cases, organizations such as museums, existing administrative structures, or events including exhibitions were utilized for outreach. In other cases, Facebook sites were established as a place for virtual learning and used to reach a diverse audience and to promote science literacy. In the Vadonleső case, scientists maintained and improved social interaction between citizens and professionals by creating a Facebook page to promote awareness of nature conservation, to provide people with resources and updates related to the CS program, and to encourage critical thinking. In the Wolf project, however, the broad media coverage of the issue sparked debate in social media, and citizens beyond the actual participants discussed the legitimacy of CS activities. The development of maps presenting project results was considered a success in 3 case studies. The GIS tools allowed for analysis and further refining of the gathered information and other spatial data. Maps produced by GIS analysis were seen as powerful tools for visualizing information in a way that could be readily used to meet the needs of land-use planning.

In most cases, individual data collection was combined with events such as guided walks, bird observation campaigns, or land-use planning exhibitions. These events served not only to collect data but also to share scientific knowledge or skills. In projects applying a more interactive approach with a higher degree of engagement, these events were used to facilitate mutual exchange of knowledge. In the cases in which mutual knowledge exchange was facilitated, the classical roles of scientists as knowledge holders and citizens or decision makers as knowledge users became blurred. In one of the cases, scientists even mediated interests between different interest groups and observers (Reindeer).

In 3 cases (Sibbesborg, Open Farm, Amphibians), data analysis was, to a varying extent, conducted together with citizens. Among these the first 2 cases applied a more traditional design, where scientists saw their role as mentors or coordinators whose task was to ensure data quality. Data processing in these cases was seen as a task for professional scientists only. The third project (Amphibians) used CS as a tool to develop partnerships within the civil community and to recruit new members to the nongovernmental organization.

Contributory projects are generally designed by scientists, and members of the public primarily contribute data (Shirk et al. 2012). Collaborative projects are generally designed by scientists, but members of the public contribute data and help refine project design, analyze data, and disseminate findings (Shirk et al. 2012)

Nature and Level of Collaboration

In addition to the learning by individual professional scientists and volunteers, some kind of organizational learning was reported in 6 of the cases. For example, lessons related to information exchange were mentioned. Many data sets were made available from databases or were fed into larger databases, where they were available for use in decision making or where they could be used to comply with the reporting requirements of the Convention on Biological Diversity (Amphibians, Butterfly, and Observation). Some of the projects became institutionalized via public administration (Bioblitz and Butterfly) or as a semiformal component of a governmentally managed database (Observation). In some cases, changes in the institutional practices of scientific research occurred (e.g., by developing a new, universal language combining scientific terminology with citizens’ common language and practitioners’ jargon).

Furthermore, transformative effects at the institutional level occurred (e.g., integrating the results of CS activities into decision-making processes [Observation] and empowering citizens or local communities to engage in land-use planning [UUSIMAA]). Results of CS activities were not often used beyond the scientific sphere, but there were some convincing examples of how the activities supported initiatives for habitat improvement (Vadonleső and Marten Network cases) and of how the process of knowledge creation fed into administrative use (Wolf). In most cases, the newly generated data positively contributed to the political process, although the extent of the impact was difficult to estimate (e.g., Vadonleső and Marten Network). However, in the case of estimating the wolf population, the data did not help resolve the issue; rather, it sparked a new controversy.

Although two-thirds of the reviewed CS programs had taken a top-down approach in which the design was done solely by scientists and the role of the citizens was limited to data collection, the remaining one-third had varying degrees of engagement of citizens in the scientific process. In 2 projects (UUSIMAA; Butterfly), research setting and methodology were mainly designed by scientists, but an expert group of stakeholders and land-use planners contributed. In nearly half of the 14 case studies, participation had been broadened to all citizens. However, in some cases, participation was only possible through internet access. Two CS activities involved mainly volunteers who had a direct stake in the issue (Wolf and certain tasks in the Amphibians case). Two projects had a mixed approach, involving both the general public and a specific stakeholder group. For Sibbesborg a case-study advisory board was established that was composed of scientists and local practitioners to which land-use planners from the municipality and local stakeholders could contribute. In UUSIMAA an expert stakeholder group was organized, consisting of municipal land-use planners, representatives of interest groups, and environmental organizations, among others.

Discussion

We aimed to offer a critical evaluation of the transformative potential of CS and the complexity of learning in the CS context. Based on our review of the literature and collaborative assessment of 14 case studies of CS initiatives, we arrived at 4 key discussion points.

First, although a number of challenges exist in assessing the learning aspects of CS, reflecting on the learning outcomes can benefit both citizens and scientists, whether or not these were intended or defined as goals of the CS initiative. Recent studies aiming to define CS for analytical purposes contend there are no internationally standardized definitions of CS and that several practices could be interpreted as CS. Yet CS is perceived as having the potential to offer immediate value to the delivery of EU biodiversity policy and to assessments being carried out in the recently created Intergovernmental Platform on Biodiversity and Ecosystem Services. Reflecting on the learning pathways and outcomes will help achieve this potential. Second, CS programs may have transformative learning potential, but such transformation does not always occur. This type of learning may contribute to the development of individual-level skills, resulting in radical changes in awareness and behavior related to environmental issues. However, transformative-learning outcomes should not be associated with only the individual level. Learning outcomes at organizational and institutional levels may contribute to a more effective and efficient practice of scientific research as well as science–society–policy interactions. The case studies provide evidence that both citizens and scientists learn and that learning outcomes and processes should be a focus of assessments of CS programs. Scientists generally acquire and improve their skills for collaboration and participation and they may, over time, change their awareness and expectations with regard to the institutionalized ways of conducting research. A community of practice around CS may gradually build up and have a transformative potential at higher levels, including both individual research organizations and science as an institutional field in modern societies.

Third, simple and visible learning outcomes that are easy to assess (i.e., factual and instrumental learning) are reviewed most frequently in the literature, whereas the more complex and multifaceted aspects of individual and collective learning are rarely evaluated in a systematic way. Moreover, assessments of transformative effects of learning are often based on assumptions rather than empirical observations. The lack of systematic evaluation strengthens the overly optimistic view of the positive effects of CS. Based on our literature review and case-study assessment, we suggest a detailed common-template approach can improve the assessment of various aspects of learning in CS projects. Key dimensions to be included in the template are the substance of learning, the nature of learning, the distribution of learning effects among scientists and citizens, and the design of learning arrangements.

Fourth, learning in CS is considered an important feature, but it is often unintended, indirect, unguided, and not reflected upon. Learning to use scientific methods (scientific literacy) can lead to desired social empowerment and enhanced bottom-up decision making. By offering citizens a methodologically guided, scientifically sound approach to increase understanding of the nature of scientific inquiry, CS can foster a structured understanding of the complexity of learning. Professional scientists may also benefit by looking at the world, together with nonscientists, in new ways and creating a shared understanding of what science means in society. Through such processes, CS projects that intentionally incorporate learning arrangements, objectives, and expected outcomes may result in far more effective, diverse, and long-lasting learning outcomes.

Our assessment and review suggested that, despite the methodological guidance that is available to evaluate the outcomes and effectiveness of learning processes, most environmental CS projects do not report their learning achievements and those that do focus mainly on factual learning results. This is especially critical because CS holds out the promise of having transformative effects on both researchers and participants, but transformative learning is often not articulated as a goal and not evaluated in a transparent way by the projects. Because of this lack of evaluation of transformative effects, we could not judge which participatory arrangements and contextual factors contributed to the realization of transformative effects; thus, it is not clear how to increase the transformative potential in these projects.

We took a preliminary step toward filling in this evaluation gap by showing that learning aspects of CS can be re-assessed, self-reported, and explicated through collaborative assessment and writing techniques. The collaborative assessment and writing approach we applied used the active participation of experts involved with CS to carry out the ex post evaluation of CS projects by creating and filling in a standardized template. Generating and discussing the templates in a small group setting supported self-reflection and informed debate over what can be considered a transformative effect and what cannot. Nevertheless, the learning effects reported for our case studies still mirrors the perceptions of the scientists only and not those of the practitioners or citizens who participated in the project.

To arrive at a more reliable and transparent measurement of transformative effects, evaluation should be built into environmental CS projects as an inherent step in the process. Because learning often happens in an unintended way and induces transformations at various levels (including changes in the values, beliefs, emotions, and actions of learners), citizens and scientists should be equally involved in the process of evaluation through self-assessment and reflection. Assessing the transformative potential of CS projects at an institutional level also requires a balanced view of how the institutional system of science, policy, and practice might change. A more critical view on the democratization of knowledge production and use should be applied.

Acknowledgments

The research for this article was initiated and financially supported by ALTER-Net, (http://www.alter-net.info/), Europe's ecosystem research network of excellence, within its AHIA program. In developing the cases presented here, we received helpful input from O.Váczi of Vadonleső, B. Halpern of Amphibians and Reptiles –BirdLife Hungary, S. Kana of the Estonian Naturalists’ Society, R. Roasto of Estonian Environmental Agency. M.L. received support from the Scientific Grant Agency of Slovak Republic VEGA project 1/0828/15. Contributions by S.H., J.H., and A.B. were supported by funding of the German ministry of research and education through the GEWISS Citizens Create Knowledge project (reference. 01508444). We thank D. S. Rogers for language editing.