Broader impacts in conservation research

This article explores the “broader impacts” of research related to biodiversity conservation. We analyze a sample of abstracts of proposals funded by the United States National Science Foundation's (NSF) Biology (BIO) and Social, Behavioral, and Economic Sciences (SBE) directorates. The analysis, based on the NSF conceptualization of broader impacts, identified 21 important types of broader impacts mentioned in the abstracts. Our results show that the vast majority of NSF grant recipients propose a small range of broader impacts, and that predictable differences exist in the types of broader impacts that are more and less common for conservation‐related work in the biological and social sciences. BIO abstracts contained proportionally fewer mentions of equity, case studies, international links, and workshops for conservation practitioners. SBE abstracts contained proportionally fewer mentions of dissemination to local communities, data sharing, websites, and curriculum. We discuss multiple forms of broader impact (e.g., process‐based impacts vs. sharing of results), and then, like the abstracts we analyzed, focus on communication and engagement mechanisms. In order to aid future efforts, we provide examples of unusual and particularly creative approaches that conservation scientists can use to enhance the broader impacts of their work.


| INTRODUCTION
The question of how science interfaces with the rest of society has been a topic of conversation for as long as science (and its predecessors) has existed. A number of questions characterize these discussions: Why are we doing science? What topics should we study? What should scholars spend their time on? How and by whom should findings be shared? These questions undergird the analyses in this article.
The first two questions address fundamental issues of how research topics are decided. They relate to the complex concept of co-production, which emerged, and which has slightly different meanings, in multiple intellectual fields. All of these meanings, however, share a core similarity: they recognize that society and science are or should be intricately intertwined, and that consequently "knowledge and action are interdependent" (Miller & Wyborn, 2018, p. 1). In Science and Technology Studies, co-production refers to the social embeddedness of science-the fact that science cannot exist independent of a cultural and societal framework, and constantly both impacts and is impacted by that framework (Jasanoff, 2004). Sustainability science, on the other hand, "makes co-production into a normative aspiration; science should be co-produced with its users" (Miller & Wyborn, 2018, p. 3).
The last two questions involve a debate that has not two camps, but a spectrum of opinion, running roughly from the "researchers should only research" standpoint, to the "researchers should be deeply engaged in sharing their work and discussing its implications" standpoint. Most scientists fall somewhere between these two extremes. Conservation scientists, on average, likely fall nearer the engagementwith-societal-implications end of the spectrum, given the applied motivation for the field-yet the appropriate degree of engagement with decision-making is a perennial focus of debate (Chan, 2008;Soulé, 1985).
The U.S. National Science Foundation's (NSF) Broader Impacts (BI) criterion is one of the most well-known modern manifestations of the enduring set of questions in the first paragraph. Since the inception of BI, scholars have discussed them at length-in a variety of fields, using a variety of methods, and with a variety of conclusions (Watts, George, & Levey, 2015; see their Table 1 for a helpful summary of past research on BI). Many scholars make conceptual arguments that apply across fields and disciplines. Most empirical studies, however-those that analyze how the BI criterion is operationalized-focus on a particular discipline, or often even a particular program (e.g., Nadkarni & Stasch, 2013). We took a different tack and focused on a topic that spans NSF jurisdictions: ecosystem conservation. We analyzed proposals that address ecosystem conservation, regardless of the program or particular call to which they correspond.
Our study focuses on NSF's BI criterion and adds to the body of literature on BI in two ways: we explore disciplinebased differences in BI, and we seek out particularly unique or creative examples of BI-that is, those that differ in thought-provoking ways from the vast majority of BI we encountered. First, we build on research and practice that emphasizes the necessity of interdisciplinary approaches in conservation (Leslie, 2017;Sievanen, Campbell, & Leslie, 2012). This work acknowledges that conversation challenges, like many other challenges in today's world, are complex mixtures of social and ecological issues, and thus that addressing them requires a complex mix of research approaches. Yet despite widespread recognition of the importance of interdisciplinary approaches, bridging disciplinary divides is often challenging for many welldocumented reasons (Lélé & Kurien, 2011;Moon & Blackman, 2014;Sievanen et al., 2012). Disciplinary differences in approaches to BI, however, have received little attention. At NSF, directorates provide a convenient way of separating natural and social science-which is by far the most common divide discussed in work on interdisciplinary conservation. Directorates are organizational units that oversee scores of programs, and each program addresses some subset of the directorate's area. To explore potential differences in BI between natural-and social-science proposals, we compare the BI in abstracts of NSF-funded proposals from (a) the Biological Sciences (BIO) directorate, which includes programs such as Environmental Biology and the National Ecological Observatory Network, and (b) the Social, Behavioral, and Economic Sciences (SBE) directorate, which includes programs such as Social Psychology and Political Science.
Second, we explicitly explore creative approaches to BI. This aligns well with the intention of the criterion, as NSF intends the BI criterion to be inclusive and expansive. In the foundation's words, "NSF's mission is to fund innovative science, and so the foundation does not want to be prescriptive about what qualifies as 'broader impacts'" (NSF, 2015, p. 3). NSF also states that "targets for broader impacts are purposefully not prescribed, but left open to innovation from the field" (NSF, 2015, p. 3). As we systematically coded abstracts, we noted creative or novel BI approaches to explore to what extent that intended innovation was happening. Frodeman et al. (2013, p. 153) describe our perspective: Broader Impacts 2.0 exhorts us to go beyond our traditional notions of supporting graduate students or disseminating project results through publicly available Web pages to actively promoting the contributions of our scientific knowledge to policy, our economy, our culture, and pressing societal needs.
In this article, we explore the BI from publicly available abstracts of funded proposals related to ecosystem conservation. We combine quantitative and qualitative analysis to address two research questions: Are there substantial differences in the types of BI between directorates? And, what are some of the most creative, promising, and idea-inspiring BI in NSF-funded conservation research? In addressing these questions, we aim to elucidate strengths and gaps in the types of BI addressed, and also to provide a series of diverse examples of atypical BI to inspire researchers' creativity. We hope that this article incites researchers to build on and modify the ideas in these exemplary BI.

| Motivation
We were inspired to pursue this question as a result of our own effort to creatively interpret and share research results. This experience involved collaborating with community members in our study site on Hawai'i Island to create a hula show that conveyed core ideas and findings from research on cultural ecosystem services (Gould et al., 2014; see www.researchspeaks.org for description and video of the event). We found this experience deeply rewarding and thought-provoking, and wondered how much it differed from the norm of sharing intellectual work. The experience also led us to explore trends in conservation scholarship related to forms of collaboration and engagement embedded within the research process.

| Sample of abstracts
The NSF publishes abstracts for all active and expired grants. We analyzed grants awarded between 1997 and 2012. We downloaded abstracts that fit the following criteria: their title or abstract contains the word "conservation"; the grant was made between January 1, 1997 and December 31, 2012; and the grant was made through one of two directorates: Biological Sciences or Social, Behavioral, and Economic Sciences. While this decision may miss a minimal number of grants in, for instance, the Engineering or Education and Human Resources directorates, the term 'conservation' has diverse meanings in these fields, and often did not refer to biodiversity conservation in our explorations.
These criteria resulted in 4,366 abstracts (3,003 in the Biological Sciences and 785 in the Social, Behavioral, and Economic Sciences). We randomly selected half of these abstracts for analysis, removed duplicate abstracts, and then manually removed abstracts that did not relate to biodiversity conservation. We defined biodiversity conservation as an effort to maintain or reduce degradation of non-human animals or plants, excluding crop plants. Our resulting sample size was 1,451 abstracts. All identifying information, such as host institution and investigator names, was omitted from the coding process.

| Data analysis
To address our research questions, we needed to categorize how each abstract we analyzed dealt with BI. For our a priori coding categories-that is, categories decided upon before we began coding abstracts-we drew on NSF's materials on BI to create coding themes in two categories: "research process" and "dissemination." Though not all definitions of research impact consider process-based features (e.g., international partnership) or simple dissemination products (e.g., a website) to be "impact" (Reed, 2016), the NSF clearly includes such features in its conceptualization of BI (NSF, 2015).
A team of four researchers qualitatively coded the "Broader Impacts" sections of the selected abstracts for statements. We combined the a priori coding described above with open coding: coding based on themes we found in the data (Patton, 2002). We used 25 abstracts as "training abstracts," and all coders analyzed this same set. We coded both for our a priori list of themes and also for relevant ideas in the abstracts not captured by the a priori list. After this first round of coding, the four researchers compared analyses. We discussed the few discrepancies in a priori coding and refined our definitions of these categories. We also discussed emergent themes and decided how to combine and split these emergent ideas to create additional coding categories. After adding emergent codes, we had 21 BIs, divided into the two broad categories of research process and dissemination (Table 1). After this initial process, each researcher proceeded to code a distinct set of abstracts individually. We remained in constant contact throughout the process, and when we encountered content in an abstract for which the coding was unclear based on our definitions, we discussed this issue with all coders and revised our definitions to incorporate the new situation. We thus developed a comprehensive "codebook" that included guidance on how to code all relevant content.
We coded all 1,451 abstracts in the sample for presence or absence of our 21 themes. Important to note is that, because we had only abstracts, our coding characterizes authors' intentions to act, not their reported activities. We can only analyze what people proposed in their abstracts; interim and final reports to NSF, which might include information on actual activities, are not publicly available.
As we coded abstracts, we also kept lists of BI that we found particularly innovative, outside-the-box, or unexpected-"novel or creative BI." These BI also often fell into multiple of the 21 pre-determined categories, and our identification of them as "novel or creative" was subjective. The research team, however, engaged in repeated discussion about the type of BI appropriate for this designation, thus ensuring a high level of consistency in our selection of these distinctive examples.
To explore whether different disciplines might approach BI differently, we compared abstracts from different directorates. Since two directorates, BIO and SBE, made up 98% of our sample (1,418 out of a total of 1,451 abstracts) we focused our comparative analysis on those two directorates. We ran Chi-square tests to determine whether inclusion of particular BI varied between the two directorates.

| RESULTS
Abstracts in our sample addressed all 21 of our BI, with varying frequencies (Figure 1). The average number of BI in an abstract was 2.0. About a quarter of abstracts (24.9%) mentioned only BI from the four most common categories (training young researchers; minority inclusion; international links; create/strengthen partnerships). In addition, 20.0% of abstracts mentioned just one BI, and over half of these (10.1% of total abstracts) mentioned only training young researchers. Our analysis of statistical differences in the presence and absence of particular codes between directorates revealed that eight categories exhibited these differences (Table 1). BIO abstracts contained proportionally fewer mentions of equity, case studies, international links, and workshops for conservation practitioners. SBE abstracts contained proportionally fewer mentions of dissemination to local community, data sharing, websites, and curriculum. In the following sections, we first share examples from the directorate where a particular BI was less prevalent, to spur ideas about how to address those particular BI in contexts in which they are less common. We then describe a few proposed BI that we found particularly innovative-the "novel or creative BI."

| Examples of equity concerns within BIO abstracts
Abstracts that met our criteria for the BI of equity described research results that seek to help address inequities. Abstracts in SBE more commonly addressed equity, but a few BIO abstracts did so as well. One example of an abstract from BIO focused on conservation of the plant genus Dombeya. This proposal incorporated a focus on equity through recognition of inequity as a threat to conservation. The authors stated, "… widespread poverty compels unsustainable land use making immediate conservation action imperative."

| Examples of international links or issues within BIO abstracts
Abstracts met our criteria for the BI of international links or issues if the research process will create international links or research will address international or non-U.S. issues. This was more common for SBE abstracts, but some BIO abstracts did address international issues. One BIO abstract, for example, described research at the Gump South Pacific Research Station in French Polynesia. In addition to the research taking place in an international setting, the researchers described how the study will offer innovative ways to communicate scientific findings to Polynesian communities, particularly K-12 schools. Many abstracts in this category described partnerships with non-U.S. researchers. One abstract, for instance, described research to investigate ant species diversity and relationships in Madagascar; researchers collaborated closely with Malagasy Co-PIs and students.

| Examples of minority inclusion within BIO abstracts
Abstracts met our criteria for the BI of "minority inclusion" if the research process will involve traditionally underserved populations. This was slightly more common for SBE abstracts, but BIO abstracts also noted many instances of minority inclusion. One BIO abstract focused on DNA sequencing, for example, noted that the project will create educational opportunities through partnerships with Historically Black Colleges. Another example was an abstract focused on invasive species in California grassland systems, which made specific mention of recruitment of and outreach to Latina women.

| Examples of dissemination to local community or 'the public' within SBE abstracts
Abstracts met our criteria for the BI of dissemination to local community or public if they broadly stated an intent to share results with local people or local community with scant specifics as to dissemination technique. This was more common for BIO abstracts, but SBE abstracts also broadly stated intentions to disseminate results. One example of an abstract from SBE that mentioned dissemination in broad terms focused on the analysis of archaeological plant remains from modern Turkey. The researchers stated that the results will be publically presented in Los Angeles, Philadelphia, and Turkey. Another abstract focused on an effort to analyze remains and depictions of animals excavated since 2003 at the Classic Period (ca. 200-750 CE) sites of El Perú-Waka and La Corona, Guatemala. The researchers described plans to disseminate the findings to the surrounding community via bilingual brochures.

| Examples of internet data sharing within SBE abstracts
Abstracts met our criteria for the BI of data sharing if they discussed plans to share data, source code, or software via internet. This was more common for BIO abstracts, but a few SBE abstracts described this type of BI. One SBE project worked with Tsimane, a native Amazonian population in Bolivia, and developed new methods to characterize parental knowledge of medical plants used to treat illness in children. The researchers intended to make their dataset publicly available and allow others to test different hypotheses. Another example was a project that involved observations of monkeys to understand primate-predator relationships. The researchers committed to adding their data to an existing public internet database.

| Examples of websites within SBE abstracts
Abstracts met our criteria for the BI of websites if they described an intent to make a website for sharing processed results and findings. This was more common for BIO abstracts, but a few SBE abstracts described plans to create websites. One SBE abstract focused on an effort to understand the value of conservation science through ethnographic and archival research. The project will share findings by creating an online platform for hosting research data and video production. Another SBE abstract described research to examine the possible negative ecosystem consequences of new food safety measures. The researchers will make their findings publically available on a blog.

| Examples of workshops for conservation people within BIO abstracts
Abstracts met our criteria for the BI of workshops for conservation people if the researchers described plans to host workshops for conservation practitioners, including government, non-governmental organizations, or both. This was more common for SBE abstracts, but BIO abstracts described this BI as well. One BIO abstract that described research on N deposition stated that it would organize a regional conference for managers, policy-makers, and researchers. Another project examined finches to investigate the broad question of how biodiversity develops. The researchers proposed sharing their findings in workshops with amateur birders, wildlife biologists, and conservationists.

| Examples of K-12 outreach within SBE abstracts
Abstracts met our criteria for K-12 outreach BI if they proposed to create or deliver lessons or curricula for K-12 students. BIO abstracts more often proposed K-12 outreach, but a few SBE abstracts did so as well. One SBE abstract described research on the relationship between Rocky Mountain Juniper growth and regional weather patterns. The researchers planned to incorporate their results into existing educational programs that serve K-12 students and teachers and conservation decision-makers. Another SBE project studied the diet and nutrition of four monkey species to map tree-provided nutrients in Uganda. The research team planned to lead hands-on lessons on wildlife and conservation for middle-school students in their U.S. hometown.

| Novel or creative BI
In addition to the associations described above, we noted, across all abstracts, practices that were particularly innovative and creative. We present those below, divided into BI related to the research process and BI related to dissemination of results. We hope that the examples below may inform future broader impacts work.

| Research process
A few abstracts emphasized the importance of community-based or engaged research-which, broadly, refers to many levels and ways of engaging nonacademics in the research process . We recognize that this approach is, in the broader research world, no longer truly novel (e.g., Fortmann (2008) and Stoecker (2012)). However, the approach is still relatively rare, especially in natural science research. This was very evident in our abstracts. We thus include it here, because our goal in pointing out novel impacts is to share ideas that are not yet widely adopted in conservation work. A few abstracts in our sample not only mentioned that they engaged in community-based research, but also discussed its importance. Some pointed out equity issues inherent in research that does not involve the community. A couple of abstracts mentioned that the goal of their work was to build consensus around conservation priorities, and the research process itself (participatory mapping processes) served as a two-way channel of information exchange with local communities. Another abstract noted the need and BI of changing who is perceived as "expert," which would, among other outcomes, increase the perceived legitimacy of under-represented groups to craft science and research that impacts their lives. A number of abstracts also incorporated citizen science, an evolving concept that most often refers to volunteer involvement in scientific data collection (Dickinson et al., 2012). As with community-based research more generally, citizen science is no longer novel (e.g., Hannibal, 2016). Although it is increasingly popular in the natural sciences, it is far from a majority practice. One abstract described a project working with high school teachers to collect and analyze data, then disseminate findings. Another abstract described a project on salmonids that used citizen science to increase access to local knowledge and observations. Multiple abstracts mentioned enhancing existing citizen science programs or creating new programs. As one example, one team partnered with the Lincoln Park Zoo (Chicago, IL) to work to increase participation in the zoo's pollination-related citizen science program (https://www.lpzoo.org/bumblebeeeffect-pollinator-power).

| Dissemination
Multiple abstracts described plans to connect their results with relevant policy. These abstracts described BI such as holding workshops targeting government agencies near study sites or creating more accurate, policy-relevant projections of future impacts. Creative examples of connections to decision-making-those we saw as particularly novel-went beyond sharing their results with decision-makers, to create tools or aids relevant to policy. In one case, for instance, a project on fish population biology planned to develop an open-source computational tool for fishery managers.
Abstracts also described using the power of the web as a creative approach to dissemination. For example, one abstract described a dedicated Youtube channel sharing charismatic footage from avian "nest cams." The channel features high-interest moments selected from thousands of hours of video footage, and predated the blossoming of live "nest cam" video footage on the internet (Figure 2). Another team proposed a professional, multi-faceted website offering informal education and formal long-distance learning opportunities (http://lemur.duke.edu/). A team working on webspinners (a type of insect that produces silk) proposed creating digital image galleries (though did not specify how the galleries would be publicized).
A few abstracts described dissemination through art. For example, one abstract proposed working with the community involved with the study to create a children's book that would convey the findings of the research (Figure 3). Another team proposed collaborating with IMAX filmmakers to inform a film about the study region and freshwater conservation. Another stated in general terms that "public displays and art projects" would help to share their findings with the public.
A few abstracts included plans to disseminate findings through experiential opportunities for community members. One project planned to offer free researcher-lead walks through the study site, exposing visitors to the content of the study and ecosystem science more generally. Another abstract described a two-week field class for students who are either excelling or students who are at risk (defined as those with low grades and/or low attendance), paired with opportunities for those students to participate in research.
Finally, some abstracts proposed creating programming or exhibits to be displayed at museums. For example, one team proposed to work with residents of the study region to create exhibits for local museums as a way to share their results with the local community. Another team, this one developing a research facility, proposed an "Educational Center" as part of the new laboratory facility.

| DISCUSSION
The U.S. National Science Foundation's Broader Impacts criterion is a commanding, contentious, and (for some) confusing modern manifestation of classic debates about the reasons for doing science. What is the point of science? Should scientists have to explain why their work is relevant? Because not everyone agrees on the answers to these questions, the BI criterion inspires praise and raises hackles. Since its inception, the requirement has been the subject of analysis and critique (Holbrook, 2012). Researchers who seek NSF funding have struggled to make sense of the criterion (Lok, 2010;NSF, 2018), particularly around the question of diversity (Intemann, 2009). Some scholars advocate for its elimination (Frodeman et al., 2013).
Yet the BI criterion, and the global conversation about research impact of which it is a part, seem here to stay. The NSF BI criterion will likely morph as understanding of coproduction (with its various meanings [Miller & Wyborn, 2018]), community-engaged research, and non-deficitmindset approaches to dissemination further infuse academic work. We hope that this study, which highlights the dissemination-focused, and often deficit-mindset-focused, nature of many BI in the conservation field can help inform this development.
As BI develop and evolve, creativity may play an important role. Our results show that the vast majority of NSF grant recipients propose a small range of BI in their abstracts, and that somewhat predictable differences exist in the types of BI that are more and less common for conservation-related work in the biological and social sciences (below, we reflect on these distinctions). We paid special attention to novel or creative methods, and, like Frodeman et al. (2013), were particularly interested in those most likely to reach audiences unlikely to be affected by actions that researchers engage in as part of their usual prescribed activity-notably, training young researchers and publishing in academic journals.
We found that mentions of BI are dominated by a few types of activity. The process-related impact of involving young and under-represented scientists is extremely common (as previously found [Nadkarni & Stasch, 2013]). Similarly, dissemination-related impacts mostly involve either sharing information "with the public" (described in that generic way) or targeted, short-term interactions with schoolchildren.
We did find, however, that a small subset of scientists gets creative about the BI of their work. These examples excite and inspire us. That said, almost all of the creativity we highlight was in the realm of dissemination-of sharing results. It infrequently addresses what may be seen as deeper issues about how researchers determine what to study and how they study it. These can be seen as future directions for BI, as discussed below.

| Differences by directorate
Just over a third of our BI categories had unequal representation between BIO and SBE conservation-related abstracts (Table 1). In our results section, we presented examples of BI from the directorate for which they were less common, with hope that these unique efforts may inspire innovative thinking. Here, we reflect on these results.
One particularly important difference was that BIO abstracts less commonly described BI that address issues of equity in ways other than involving under-represented people. This is not surprising; socially focused projects more directly relate to social issues such as equity. Yet attention to equity and justice is crucial for the future of conservation (Gould, Phukan, Mendoza, Ardoin, & Panikkar, 2018;Taylor, 2016). Since much conservation research focuses on biology, we hope the examples and ideas we provide in this article can increase attention to equity in all types of conservation research. SBE abstracts were also more likely to incorporate international links and work with underserved populations. These characteristics are more common for social science work, but they need not be. Biophysical science would benefit tremendously from more global engagement and deeper integration of diverse populations in research processes.
The BI more commonly found in BIO abstracts align with disciplinary conventions and practices, yet also offer interesting fodder or encouragement for social science work. BIO abstracts were more likely to propose dissemination in broad, unspecific terms; in other words, SBE abstracts were often more specific about their BI than BIO abstracts. The other findings of difference offer more rich insight. BIO abstracts were more likely to describe sharing data or finished results on the internet. Human subjects privacy concerns offer an important hurdle to free data sharing of social science work, but the hurdle is far from insurmountable. Social scientists have developed scores of techniques for deidentification of data. These methods could allow social science data and results to be more widely shared.
The final two differences by directorate suggest interesting variation in conceptions of outreach or dissemination between SBE and BIO. SBE abstracts were more likely to propose workshops for conservation practitioners, and BIO abstracts more likely to propose work with K-12 students. This may reflect conventions of various fields, or the idea that BIO results are not as directly relevant to decisionmakers, but may be more appropriate as "basic knowledge" shared with young learners. We suggest that research in both fields would do well to consider how to expand to techniques less common in their circles.

| Future directions in BIs
Future work in BI may take innumerable forms; here, we reflect on our results to suggest promising future directions. Our selection of "novel or creative BI" presents creative approaches to both the research process and dissemination, and inspired this section.
Though the process/dissemination distinction is helpful, one future direction may be to weaken that distinction-to better integrate the doing and the sharing of scholarship. Participatory and community-based research moves in this direction. Very few of the abstracts in our sample discussed participatory research, but some of these conveyed a more holistic perception of what a BI could be. These researchers did not reference the idea of a continuum of involvement of the public (e.g., Shirk et al., 2012, table 2), but the concept is highly relevant. Different projects and contexts encompass scores of details that determine appropriate levels of involvement; a future direction may be to become more intentional about how, when, and why to involve non-scientists in the research process.
In addition, the scant references to government officials or policy-makers in our results are somewhat surprising. If one of the goals of conservation science is to inform policy, we would expect more explicit engagement with government agencies, representatives, or other decision-makers.
These moves toward more engagement, in various forms, relate closely to an idea we find crucial to the discussion of BI: avoidance of a knowledge deficit model. A deficit model assumes "the problem" to be a deficit of knowledge (and sometimes scientific literacy) held by the public, and "the solution" as simply providing more, better information. Extensive scholarly work details the problems with this approach (Frodeman & Parker, 2009). Possibly drawing on this work, a recent NSF publication about BI describes that they should strive to communicate "…with the public rather than at the public…" (NSF, 2015, p. 4). Yet our data suggest that this is still rather rare.
Many of the abstracts in our sample implicitly exhibited a deficit mindset. They assumed that information simply existing or being published in a publically accessible forum was sufficient. The scholarly work cited above suggests that this is not where we want to be. The move away from a deficit model seems crucial to developing an active civic community around conservation issues, and BI, if conceived expansively, could be an important mechanism toward that shift.
Our focus on particularly novel impacts foreshadows a suggestion for future directions that intertwines with rejecting the deficit model: a stronger focus on creativity. Indeed, creativity is formally one of the five main factors used to judge proposals (specifically, the factor addresses whether proposals "suggest and explore creative, original, or potentially transformative concepts" [NSF, n.d.]). Our analysis suggests that this criterion may, in practice, be considered only for the intellectual merit of projects; our analysis clearly indicates that the vast majority of successful proposals are stunningly uncreative when it comes to the BI of their work.
To help inspire creativity, we have compiled examples and tools that might inspire outside-the-box research and dissemination techniques, as it is more feasible, in the space available here, to suggest widely applicable dissemination techniques than changes in research process. Figure 4 F I G U R E 4 Possible approaches to dissemination. Approaches that are well designed for two-way communication, and thus which may be more effective, are indicated by an asterisk presents a wide array of possible BI mechanisms, arranged to reflect two important considerations: the size of the group involved and the use of audio, visual, or written media. Figure 5 presents a series of considerations for dissemination-based BI, and conveys the idea that a kaleidoscope of possible BI is possible.

| Limitations
Here we note four primary limitations of our study: the constraints of our search criterion; a mismatch between the timing of the abstracts sampled and the revised BI criteria we used in our coding; the dichotomous nature of our between-directorate comparison; and omission from our analysis of the amorphous "benefits to society in the future" type of BI.
Our results are based on a one-word search of "conservation" in abstracts funded by NSF, followed by manual screening of abstracts related to ecosystem conservation. This selection process presents obvious complications. We surely missed some abstracts that are related to conservation but did not use that exact word. Though this is a problem, we do not expect that it is a debilitating one. Conservation could, in itself, be considered a BI, so it is likely that researchers would use the term to describe their work. There may, however, be a slight tendency for social science work, or more basic science that might have conservation implications, to have not used this term.
We also note a mismatch between the timing of the abstracts we analyzed and the release of the BI criteria that formed the backbone of our coding categories. The abstracts we analyze extend from the implementation of the BI F I G U R E 5 Guidance for researchers considering creative approaches to broader impacts. Reflection on the suite of questions may lead to a kaleidoscope of possible creative BI techniques criterion in 1997 to December 2012. In 2012-2013, NSF revised its merit review criteria (of which BI are part), and we created our a priori coding categories based on BI criteria that NSF published in 2015 (NSF, 2015). Thus the list of BI we used to code differed slightly from the list available to applicants when they prepared the abstracts we analyzed. Importantly, however, the 2012-2013 revisions changed conceptualizations or definitions of BI very little; scholars argue that the revisions did not substantially impact NSF's expectations of BI and selection of projects (Holbrook, 2012). Though emphasis and categorizations differ, the content is extremely similar. As a result, the codes we used would have changed little had we used NSF's pre-2012 list of BI. For this reason, the mismatch in timing should not affect our conclusions.
A shortcoming of our statistical analysis is that it does not address interdisciplinary work. It simplifies the complexity of much conservation research by labeling it with whatever directorate administered the grant. Our analysis creates what may be, in some cases, a false division into natural and social science. As interdisciplinary scholars, we are deeply aware of the need for integrative, interdisciplinary work in conservation.
A final shortcoming relates to a class of BI that involves future benefit to society, broadly speaking (Table 1, final point). We initially included this category in our coding, but about halfway through our coding process, we found that essentially every abstract claimed to lead to this type of BI in some way. Many claims were general (e.g., that the findings would benefit conservation), and some were more specific (e.g., increasing understanding about how climate change will impact alpine ecosystems). These claims were so diverse that focusing an analysis specifically on them would constitute a study in its own right; a thoughtful analysis of them could provide a fascinating addition to the scholarly discussion on BI.

| CONCLUSION
NSF is not prescriptive about BIs; indeed, recent discussions on BI indicate that the concept is constantly evolving (NSF, 2018). Yet our analysis suggests that BI work still falls within a fairly narrow range of options. In other words, we as researchers are neither taking advantage of the freedom we are being given, nor are we fulfilling the goal of innovation in all realms of our work. The relative scarcity of outside-the-box approaches to BI is understandable given the mounting pressures on academics (Berg & Seeber, 2016) and the fact that learning how to creatively achieve BI is not part of most scholarly training. That said, we advocate that researchers allow their brains to roam wild and free as they consider the most equitable, effective, innovative, engaging, and interesting ways to conduct and share their research. We feel that expanding how we achieve BI may be one way for academics to reclaim bits of the joy that accompanies new understanding and discovery (Berg & Seeber, 2016), and, crucially, to share that joy with the wider society. We hope that our analysis and sharing of ideas may help us all to think more broadly about BIs.