Quantifying habitat losses and gains made by U.S. Species Conservation Banks to improve compensation policies and avoid perverse outcomes

Compensation policies seek to counterbalance biodiversity losses caused by development; however, their effectiveness is rarely tested. We examined U.S. Species Conservation Banks (SCBs) in California, a compensation program initiated 30 years ago. We quantified the effect of 59 SCBs (15,350 ha) on habitat extent using statistical matching methods. SCBs averted a small, yet significant, amount of habitat loss (62 ha) between 2001 and 2011. However, unexpectedly, SCBs also averted significant habitat gains (1,424 ha). It is not possible to determine if losses averted by SCBs equaled losses caused by development for which credits were sold (because records of the latter do not exist), but estimated averted gains were 35 times greater than averted losses. To improve practice, SCBs must be designed to achieve outcomes that are additional and avoid crowding out other programs incentivizing statewide conservation goals.


INTRODUCTION
Compensation policies aim to counterbalance residual biodiversity losses caused by development, following efforts to first avoid and minimize biodiversity losses. An estimated 115 countries now have compensation policies (IUCN, 2017) and other individual compensatory programs and projects exist (Madsen, Carroll, & Moore Brands, 2010). Unsurprisingly, this has also corresponded with a huge research effort to understand the effectiveness of compensation. Studies have scrutinized differences in policy goals  and project implementation (Sonter et al., 2018), predicted future effects of compensation on biodiversity (Gordon et al., 2011;Sonter, Barrett, & Soares-Filho, 2014), and This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. demonstrated outcomes of individual projects (Lindenmayer et al., 2017). However, with the exception of U.S. wetland compensation activities (Matthews & Endress, 2008), ex-post assessments of policy-wide outcomes are rare, and so policy effectiveness remains unclear and debate continues over whether compensation has positive or negative outcomes.
Quasi-experimental approaches, such as statistical matching, provide an opportunity to fill this gap (Morgan & Winship, 2015). Although matching methods have not yet been applied in a compensation context, they have been used to evaluate effects of protected areas on environmental outcomes (Andam, Ferraro, Pfaff, Sanchez-Azofeifa, & Robalino, 2008;Pfaff et al., 2007). The approach compares treatment sites (e.g., areas protected for compensation) with matched controls (e.g., unprotected areas, with similar biophysical and socioeconomic characteristics). Matched controls represent a counterfactual scenario against which differences in an outcome variable (e.g., biodiversity loss) can be determined.
Estimating the effect of compensation is particularly challenging because counterfactual scenarios are complicated (Peterson, Maron, Moillanen, Bekessy, & Gordon, 2018). A treatment site might differ from a matched control site in terms of biodiversity losses-but if losses at the treatment site trigger compensatory requirements themselves, they should be excluded from counterfactual scenarios ; failing to do so would overestimate the benefits of compensation (Maron, Bull, Evans, & Gordon, 2015). Further, compensation often occurs in a context of other environmental interventions, and might even reduce or crowd out their benefits (Gordon, Bull, Wilcox, & Maron, 2015). A carefully calibrated statistical matching approach accounts for this complexity to estimate the true effect of compensatory policies.
We use statistical matching to examine the effect of the U.S. Species Conservation Banking program on the extent of natural vegetation. Species Conservation Banks (SCBs) seek to counterbalance negative impacts of development (typically mining, infrastructure, and urban expansion) on threatened and endangered species and habitats (US FWS, 2003). Conservation bankers, using easements or other mechanisms, establish SCBs. Management plans are developed to benefit target species (Fox & Nino-Murcia, 2005) and sometimes to improve ecosystem services and achieve no net loss of biodiversity (Gamarra & Toombs, 2017). Bank credits are generated through habitat protection or, in some cases, restoration. Methods used to determine credits generated by SCBs differ among SCBs; many use habitat extent alone, some consider the conservation value of the site, none assess the additionality of gains made by compensation (Gamarra & Toombs, 2017). Credits are then sold to developers who are required to compensate for their impacts on listed species, within the same "service area" (i.e., a nearby area with similar physical and ecological attributes).
A comprehensive database of SCBs exists (RIBITS, 2018) and several studies have reviewed their implementation (DOI, 2016;Fox & Nino-Murcia, 2005;Gamarra & Toombs, 2017). However, none has examined the effect of SCBs on habitat extent, with the purpose of evaluating policy effectiveness and consequences for statewide conservation goals. Here, we examine SCBs in California, home to 90% of SCBs, to ask three questions: (1) Which species and habitats do SCBs conserve? (2) What explains the distribution of SCBs across California? and (3) To what extent do SCBs affect habitat losses and gains? Although we were unable to determine whether SCBs counterbalance biodiversity losses due to development (due to data limitations), results highlight potential perverse outcomes, and reveal challenges in comprehensive assessments of policy effectiveness, particularly when considered in the context of other, potentially competing, environmental policies.

Data summary
We linked spatial data (point locations) on California's 98 SCBs (RIBITS, 2018) to information on project size, approval date, targeted species and habitats, and management goals (Gamarra & Toombs, 2017). We approximated SCBs' spatial footprints, using their area (available for 72 SCBs) and the map buffer tool in ArcGIS (Figure 1). We obtained modeled distributions of species and habitats targeted by SCBs when available (Comer et al., 2003;USGS, 2011) and overlaid them with our map of SCBs to determine spatial coincidence. We also obtained and overlaid data on land cover in 2001 and land cover change between 2001 and 2011 (Homer et al., 2015). Land cover maps were reclassified as natural (forest, shrubland, and herbaceous vegetation), agricultural (grazing and cultivated areas), and developed (artificial land cover) land. Land cover changes were reclassified as gains in natural land cover, losses in natural cover to agriculture, and losses in natural cover to developed land. We performed analyses at 1 ha resolution.

Quantifying effects of SCBs on biodiversity
We used statistical matching to quantify the effect of SCBs established between 2001 and 2011 on biodiversity. The outcome variable used to evaluate conservation should relate to the policy objective, which here was to compensate for development impacts on threatened species and habitats (US FWS, 2003). It was not possible to measure this outcome directly, because statewide data were unavailable for most species. Instead, we examined the effect of SCBs on natural land cover, as a proxy for habitat extent. Our analysis examined habitat extent alone, rather than also changes in habitat quality. Within SCBs, we assumed natural cover indicated species' habitat, given that (when data were available) most banks coincided with modeled distributions of target species and that most (79%) SCBs calculate credits using habitat area alone (Gamarra & Toombs, 2017). Outside SCBs, we assumed natural cover with similar landscape characteristics would provide similar habitat and, when these areas were near SCBs, also contained appropriate habitat for targeted species. To make these assumptions, we chose matched controls proximate to SCBs and used covariates that represented species distributions (see sections below).  Table S2). (c) Treatments, matched controls (see Methods) and modelled distribution of Swainson's hawk (USGS, 2011) We examined the effect of SCBs on habitat losses and gains separately. We expected SCBs to negatively affect habitat loss (indicating SCBs avert losses) and positively affect habitat gains (indicating SCBs undertake restoration). Losses were indicated by transitions from natural cover to agricultural or developed land; gains were indicated by transitions from agricultural or developed to natural, which could represent natural recovery or active restoration.
Treatments were any observation (1 ha grid cells) within SCBs approved prior to 2011 (59 SCBs). To quantity effects on habitat losses, we examined treatment observations classified as natural cover in 2001 (10,475 observations). We also separately examined treatment observations located within SCBs approved prior to 2001 (24 SCBs) to strictly align with our analysis period (2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011); and observations more than 100 m from SCB edge boundaries (3,408 observations) to exclude observations misclassified as SCBs when delimiting spatial footprints. To quantify effects of SCBs on habitat gains, we examined treatments classified as developed or agricultural land in 2001 (192 observations). For each set of treat-ments, we randomly selected 1,000 observations except when examining habitat gains, where we used all observations. We used 11 covariates hypothesized to explain SCB allocation across California and drivers of habitat losses and gains. These included seven datasets from USGS (2011): aspect, human disturbance (dummy variable for high/medium vs. low/no human alteration), elevation, ecotone type, forest edge width, canopy cover, and slope. These covariates were used to parameterize threatened species distribution models (US FWS, 2003) and thus improve chances that treatments and matched controls were of similar habitat type. They also explained the distribution of U.S. conservation easements (Baldwin & Leonard 2015). We also included four indicators for demand for ecosystem services (carbon storage, sediment export from agriculture, recreational bird watching, and crop pollination; US EPA, 2016) because some SCBs aimed to conserve them (Gamarra & Toombs, 2017).
Potential control observations were located outside protected areas (IUCN GAP status 1 or 2; GreenInfo Network, 2017). Control observations for examining effects on habitat loss were natural cover in 2001, whereas controls for examining habitat gains were developed or agricultural. We randomly sampled 100,000 observations for each analysis in R version 3.4 with the package "Matching" (Ho, Imai, King, & Stuart, 2011;Sekhon, 2011). We calculated covariate means for treatment and control observations prematching and used the Nearest Neighbor method to select matched controls that minimized differences. We matched each treatment observation to a unique control observation, but also assessed three alternative matching methods: (1) controls matched to multiple treatments; (2) five controls matched to each treatment; and (3) using calipers to constrain matches within 0.25 standard deviations of propensity scores (Rosenbaum & Rubin, 1985;Stuart, 2010).
Matching removed all significant differences in covariate values between treatments and controls ( Table 1). All covariates had lower absolute mean differences postmatching, except for agricultural sediment export, which slightly increased (-0.07 to -0.14 tons; Table 1). Alternative matching methods did not improve match statistics (Supporting Information Table S3); results were not sensitive to hidden bias (Supporting Information Table S1). To determine effects of SCBs, we compared outcome variables between treatments and matched controls, using Chi square tests. For habitat losses, we analyzed two levels of loss: all losses and unregulated losses. Unregulated losses were transitions from natural cover to agricultural land, and thus did not completely remove all potential habitat (i.e., some species can utilize agricultural landscapes).

Which species and habitats do SCBs conserve?
California's 59 SCBs approved prior to 2011 differed in size (2-2,450 ha; mean: 240 ha; total: 16,350 ha) and land cover (natural: 12,490 ha; agricultural: 2,000 ha; developed: 1,010 ha; Figure 2). SCBs lost 37 ha, but gained 17 ha, of natural cover between 2001 and 2011. All SCBs listed at least one species or habitat target (Supporting Information Table S2). Sixty-six SCBs (95%) listed species targets. Species distribution models were available for seven (all vertebrates) of the 44 listed species, which were targeted by 29 SCBs (42%). All but one of these SCBs coincided with their species' modeled distribution. The SCBs that did not coincide targeted the bluntnosed leopard lizard (Gambelia sila), which has a restricted range. Additionally, 46 SCBs (67%) listed habitat targets (34 habitat types in total). Most targeted vernal pools (18 SCBs) or wetlands (15 SCBs); 11 SCBs (24%) with listed habitat targets did not coincide with known distributions of these habitats.

What explains the distribution of SCBs across California?
SCBs were nonrandomly distributed across the landscape, according to variables in Table 1, all of which were controlled for by matching (see next paragraph). SCBs were established in areas of high human disturbance, low elevation, near narrow forest edges, low canopy cover, and on flat slopes. SCBs were also located in areas that stored less carbon, retained less sediment, contained less pollinator habitat, and had higher demand for bird watching. Differences held for SCBs established pre-2001 ( 2 = 3,530; P < 0.001; Supporting Information Table S3A) and when SCB edges were excluded ( 2 = 2,948; P < 0.001; Supporting Information Table S3A).

To what extent do SCBs affect habitat losses and gains?
SCBs had a significant effect on habitat loss (Table 2; P < 0.05). The rate of habitat loss within SCBs established pre-2011 was significantly less than the rate observed in matched controls. In total, SCBs averted 6.3% of their natural land cover from being lost between 2001 and 2011. However, excluding regulated losses (i.e., to developed land) from calculations reduced their apparent effects. In this case, SCBs averted only 0.5% of their natural land cover (62.5 ha; 21-228 ha) from being lost between 2001 and 2011 ( Figure 3). SCBs established pre-2001 averted 0.1% more unregulated loss than all SCBs established pre-2011 (Figure 2; Supporting Information Table S3). SCBs also had a significant effect on habitat gains; the rate of habitat gain was significantly less than that observed in matched controls ( Table 2); 45% of developed or agricultural land within SCBs would have otherwise transitioned to natural cover, amounting to 1,424 ha (743-2,075 ha) between 2001 and 2011 ( Figure 3). Results were similar for alternative matching methods (Supporting Information Table S3).

DISCUSSION
The 59 SCBs established pre-2011 in California conserved 15,350 ha of land and a range of species and habitats. We T A B L E 1 Covariate balance, comparing treatments to potential (U), and matched (M) controls. Lower mean differences postmatching indicate good performance for each covariate. Table also shows mean (eQQ Mean) and max (eQQ Max) difference from empirical quantile-quantile plots. Treatments were SCBs established pre-2011 (59 SCB); matching method was nearest Neighbor, 1-to-1, without caliper. Supporting Information  found SCBs averted some habitat loss, but also apparently negatively affected rates of habitat gain. We discuss consequences for achieving compensation objectives, identify factors limiting comprehensive assessments, and recommend ways to strengthen policy outcomes. SCBs coincide with modeled distributions of target species (Figure 1c; Supporting Information Table S2); however, spatial coincidence does not demonstrate positive effects on biodiversity. Such assessments require data on biodiversity trajectories-that is, species losses and gains across California-which were unsurprisingly unavailable. Instead, we assessed effects on habitat extent, indicated by natural land cover, assuming habitat within SCBs was appropriate for targeted species and that treatments and matched controls contained similar habitat. These assumptions were reasonable given that our covariates were used to model species distributions (USGS, 2011) and that most SCBs calculate credits from habitat area alone (Gamarra & Toombs, 2017). If these assumptions were untrue, our results may be biased, particularly if small habitat gains yielded large benefits for species. However, even in these cases, examining effects on natural cover still illustrates consequences of SCBs for other statewide conservation priorities, such as carbon storage (Anderson, Field, & Mach, 2017).
Statistical matching controlled for the nonrandom allocation of SCBs across California. Failing to account for this bias would have suggested SCBs cause (rather than avert) losses, because SCBs were established in areas of development pressure (Table 1) with high background rates of loss. This finding is similar to other privately protected areas (Baldwin and Leonard 2015), but differs from publicly funded protected areas, which often have low agricultural or development value (Andam et al., 2008).
Most SCBs conserved existing habitat (Figure 2), averting habitat losses equivalent to 0.4% of natural land cover within SCBs and less than 0.01% of regulated losses across T A B L E 2 Impact of Species Conservation Banks' (SCBs) on habitat losses and gains (Diff in mean), reported as the proportion of natural cover that would have otherwise been lost or gained, respectively. For natural cover losses, we show subset results to all losses and unregulated loss Natural cover losses Natural cover gains We expected SCBs to elevate rates of habitat gains because some had restoration goals (Figure 2), but we found the opposite, suggesting SCBs also avert 35 times more habitat gains than losses (Table 2; Figure 3). Several factors may explain this counterintuitive finding. First, within SCBs, preserving agricultural land may benefit some target species (e.g., Swainson's Hawk, Giant Garter Snake; Supporting Information Table S2), or be less expensive than returning it to a natural state (Merenlender, Huntsinger, Guthey, & Fairfax, 2004;Rissman et al., 2013). Second, outside SCBs, habitat gains may be in response to other habitat restoration programs (NRCS, 2018), and because SCBs must create additional gains, they would not qualify for them. Whether SCBs truly avert the gains that these programs would have made depends on whether programs generate a capped amount of restoration (e.g., a fixed budget) or not (e.g., tax incentives). If the former dominates, our results may reflect a redistribution of restoration among properties. If the latter, averted gains may be quite real, revealing a perverse outcome. Further research should examine factors explaining averted gains and how they relate to species targeted by SCBs, and whether results hold when using information on species gains achieved by restoration.
Regulators have two options to mitigate impacts of development on biodiversity: deny development that cannot remove impacts through avoidance and minimization measures or improve compensation so that SCBs counterbalance impacts without causing additional losses. To improve practice, SCBs must have an explicit goal against which to assess effectiveness. Goals currently differ among SCBs and are sometimes absent entirely (Gamarra & Toombs, 2017). A consistent metric is required to calculate additional credits generated by SCBs, rather than using habitat area alone. Generating gains through habitat creation rather than preservation (such as the UK's proposed Net Gain policy; DEFRA, 2018) would help; however, these policies must avoid displacing other restoration programs that incentivize restoration. Finally, detailed management and monitoring plans, with feedback to a central authority, are required so that management can respond to project failure.
At the program scale, compensation can be evaluated using statistical matching methods, so long as sufficient information exists. In California, data gaps limited our conclusions. To overcome these limitations, we recommend regulators: (1) Map compensation sites. We mapped SCBs using point locations and the map buffer tool in ArcGIS, and removed some uncertainties by excluding edges; however, an accurate map of SCBs would eliminate uncertainty entirely.
(2) Record biodiversity transactions. Data are available on SCBs, but information on regulated losses is missing. (3) Monitor threatened species inside and outside SCBs, to obtain baseline trajectories. Gaps in data and policy goals limit policy evaluations and prevent adaptive management. Compensation programs must be designed with clear, testable objectives to permit monitoring, evaluation, and adaptive management.

ACKNOWLEDGMENTS
Discussions with Carla Archibald, Becky Chaplin-Kramer, Megan Evans, Maria Jose Carreras Gamarra, Alex Pfaff, Taylor Ricketts, and Ted Toombs greatly improved our manuscript. All data are publicly available, as referenced throughout.