A pragmatic approach for comparing species distribution models to increasing confidence in managing piping plover habitat

Conservation management often requires decision‐making without perfect knowledge of the at‐risk species or ecosystem. Species distribution models (SDMs) are useful but largely under‐utilized due to model uncertainty. We used an ensemble modeling approach of two independently derived SDMs to explicitly address common modeling impediments and directly inform conservation decision‐making for piping plovers in a heavily populated mid‐Atlantic (USA) coastal zone. We summarized previously published Bayesian network and maximum entropy models to highlight similarities and differences in structure, and we compared the relative importance of predictors used. Despite differences in analytical approach, relative importance of factors driving nest‐site selection was consistent. Models demonstrated considerable agreement when comparing a binary (suitable/unsuitable) measure of suitability. Instances of model consensus (i.e., overlapping areas of predicted piping plover nesting habitat between models) provide a stronger “signal” in model results, reducing uncertainty related to biases or errors associated with either model. We tested model accuracy using a common dataset of plover nests initiated within the focal areas between 2013 and 2015, and we examined congruency in model outputs. Nearly, 90% of all nests occurred in areas predicted suitable by at least one model, and at least 33% of the total nests were predicted in areas suitable by both. Because models predominantly agreed on what drives piping plover nest‐site selection, areas predicted suitable by a single model should not be discounted. This case study demonstrates how models can effectively inform conservation planning by explicitly identifying the management objective, presenting robust evidence to allow managers to evaluate outcomes of alternative management decisions, and clearly communicating results that address real‐world conservation problems. Our results can greatly increase the piping plover management community's ability to prioritize candidate sites for future protection, manage existing nesting habitat appropriately, and make a compelling case for conservation actions against competing land use objectives.

greatly increase the piping plover management community's ability to prioritize candidate sites for future protection, manage existing nesting habitat appropriately, and make a compelling case for conservation actions against competing land use objectives.

K E Y W O R D S
conservation planning, environmental suitability, research-implementation gap, model uncertainty, piping plover, science communication, species distribution modeling, habitat management 1 | INTRODUCTION Addressing conservation problems with precise scientific evidence is an ideal shared by both managers and the research community. However, conservation management often requires decision-making without perfect knowledge of the at-risk species or ecosystem and with high uncertainty in the predicted outcomes of specific interventions (Addison et al., 2013). Such decisions can have major ramifications, as complex trade-offs exist among conservation goals and other economic, social, and political objectives (McShane et al., 2011). Probabilistic models can provide critical decision support because they are simplifications of complex systems aimed at recognizing patterns between the dependent variable (i.e., species' occurrence or vital rate) and a range of physical or environmental conditions (Schmolke, Thorbek, DeAngelis, & Grimm, 2010). Species distribution models (SDMs) are particularly relevant to conservation management, with many potential applications in decisionmaking processes (Moilanen, Wilson, & Possingham, 2009;Possingham, Ball, & Andelman, 2000). For instance, SDMs can offer important suitability predictions for sites within a species' range that have not been surveyed, determine which sites could become suitable given specific management interventions, or facilitate prioritization of sites for conservation protection (Guisan et al., 2013). Unfortunately, these models are seldom considered in management decisions (Guisan et al., 2013;Pullin, Knight, Stone, & Charman, 2004;Sutherland, Pullin, Dolman, & Knight, 2004). Rather, a common practice among managers is to base conservation decisions largely on personal experience, unstructured subjective judgment, or intuition (Knight et al., 2008;Pullin et al., 2004).
Previous studies have shown that under-utilization of models in conservation planning can be attributed both to modeling methodologies and outputs (Addison et al., 2013). For instance, decision-makers frequently cite that models do not accurately represent their understanding of the decision context, fail to incorporate all factors that influence conservation decisions, are too simple or too complicated to represent the system, or are inadequately parameterized due to data gaps (Fazey, Fischer, & Lindenmayer, 2005;Knight et al., 2008;Linklater, 2003;Young & Van Aarde, 2011). Similarly, decision-makers often discount model results either because they are not effectively communicated or because outputs are too uncertain for the problem at hand (Addison et al., 2013;Arlettaz et al., 2010;Guisan et al., 2013). Uncertainty in modeling results can particularly challenge meaningful translation of science to inform decision-making (Bainbridge, 2014). Uncertainty can arise from multiple sources, including inherent randomness and natural variation within an ecological system, measurement error and bias in sampling approaches, and subjectivity in interpretations of modeling results (Uusitalo, Lehikoinen, Helle, & Myrberg, 2015). Even in cases where managers opt to use predictive modeling to address specific conservation questions, model choice often brings unacknowledged biases and assumptions that can influence results (Burgman, Lindenmayer, & Elith, 2005).
Some studies have advocated for a more pragmatic modeling approach that reduces the researchimplementation gap (Guisan et al., 2013;Schmolke et al., 2010;Sofaer et al., 2019). An ideal modeling approach should address previous impediments to the use of models in decision-making by: (a) explicitly identifying the management problem and objectives; (b) defining and evaluating the consequences of alternative actions; (c) assessing model sensitivity, accuracy, and uncertainty; and (d) clearly communicating results in a manner that directly addresses the natural resource problem (Addison et al., 2013;Guisan et al., 2013;Sofaer et al., 2019). Here, we provide a case study that utilizes an ensemble modeling approach, incorporates these considerations, and can directly inform conservation decision-making in a heavily populated mid-Atlantic (USA) coastal zone.
Conservation planning along the mid-Atlantic coast is often particularly contentious; land use along existing coastlines is almost entirely designated for human development or recreation (Wescott, 2004). High human demand for housing and infrastructure, and for maximizing recreational opportunities, restricts opportunities for increasing habitat extent (McFadden, 2007;Tarlock, 2007). In addition, the ecological functions of sandy beaches are usually undervalued (Beatley, 2012;Jones, Schlacher, Schoeman, Weston, & Withycombe, 2017), leading to strong conservation conflicts between biodiversity conservation and human activities. Effective balancing of tradeoffs between conservation and socioeconomic considerations could likely be achieved through the adoption of an evidence-based framework for identifying high-quality areas for habitat protection and species management, while also reducing restrictions on lower quality areas for human use.
The focal species addressed here is the federally threatened U.S. Atlantic Coast piping plover (Charadrius melodus) that breeds along the mid-Atlantic coastline. Due to dense human population and development, piping plovers in this region are severely threatened by both direct and indirect anthropogenic impacts that remove or degrade habitat (Defeo et al., 2009;McFadden, 2007;Schlacher et al., 2016). Habitat availability can be increased through restoration activities or protections against human disturbance (Maslo, Handel, & Pover, 2011;Maslo, Leu, Pover, Weston, & Schlacher, 2018); yet, rarely do arguments for piping plover conservation overcome those pertaining to anthropogenically focused coastal land use planning (Jones et al., 2017;Mainka & McNeely, 2011). Therefore, one of the key problems in this system is the conflict between beach habitat conservation and human recreational use. Increasing confidence in identifying suitable piping plover habitat versus unsuitable beach areas, as well as generating quantifiable indices of its quality, could lead to significant conservation benefits for this species.
Here, we compare the outputs of two independently derived SDMs that predict piping plover nesting habitat suitability. Model approaches include maximum entropy  and a Bayesian network . Each modeling effort used independently derived underlying data, parameters, and suitability thresholds for defining habitat. To determine whether models captured important predictors of piping plover nesting habitat suitability, we summarize the two modeling approaches and highlight similarities and differences in model structure. We also compare the relative importance of predictor variables used in each modeling approach. To increase confidence in model-derived identification of suitable piping plover habitat, we test each model's accuracy by using a common dataset of piping plover nests initiated within two focal areas between 2013 and 2015, and we examine congruency in outputs from models separately by region. We then discuss the implications of our analysis to piping plover management, as well as the use of ensemble modeling to improve conservation management.

| Focal species
The piping plover is a migratory shorebird that nests along the western Atlantic coast, northern Great Plains, and Great Lakes of Canada and the United States (USA). The Atlantic coast population considered here breeds along~2,000 km of shoreline from northern Canada south to North Carolina, USA. Piping plovers are solitary nesters that typically select flat, open dry sand or pebble beaches with sparse vegetation (Cohen, Wunker, & Fraser, 2008;Maslo et al., 2011;Maslo, Leu, et al., 2016;Zeigler et al., 2017). Nesting sites are restricted to areas that are beyond the high-tide line but near low-energy foraging habitats (Loegering & Fraser, 1995;Maslo, Burger, & Handel, 2012). Breeding pairs lay up to four eggs in small depressions made in the sand, and precocial chicks hatch after 27-30 days of incubation (reviewed in U.S. Fish and Wildlife Service, 1996). Adults and chicks forage mostly along the ocean-or bay-side intertidal zone and ephemeral pools, where they consume various insects and marine arthropods (Elias, Fraser, & Buckley, 2000;Loegering & Fraser, 1995;Patterson, Fraser, & Roggenbuck, 1991). Chicks fledge~25 days after hatching, and by August individuals begin migrating back to wintering grounds in the Caribbean and the southern coasts of the Atlantic Ocean and Gulf of Mexico (reviewed in Gratto-Trevor et al., 2012;U.S. Fish and Wildlife Service, 1996). Piping plovers exhibit rapid population-level responses to habitat change (Cohen, Houghton, & Fraser, 2009) and are considered an umbrella species for multiple beach-dependent avifauna (Maslo, Leu, et al., 2016).

| Study areas
We compared predicted piping plover nesting habitat extent across two study regions: (a) Rockaway Peninsula and Fire Island off the coast of Long Island, New York; and (b) Pullen Island and Long Beach Island, New Jersey, USA ( Figure 1). The study areas were chosen based on geographic overlap between the two original models (Maslo, Leu, et al., 2016;. The 18-km Rockaway Peninsula study region spans from the Breezy Point Unit of the Gateway National Recreation Area and Rockaway Point to the Far Rockaways ( Figure 1). The majority of the Rockaway Peninsula is densely developed, with housing and commercial developments and infrastructure spanning all but the westernmost point, which is managed by the U.S. National Park Service. Fire Island extends from Democrat Point (at the Fire Island Inlet) in Robert Moses State Park past the Moriches Inlet to Smith Point. This island is micro-tidal and wave-dominated (Hayes, 1979). The approximately 50-km study area contains a mix of anthropogenically modified coastline with communities, housing developments, roads, and infrastructure interspersed with minimally developed federal, state, and county lands. Together, these sites comprise the Long Island study region referenced in this paper.
Long Beach Island (34 km) is located off the New Jersey coast and is heavily developed with residential and commercial structures and associated infrastructure. The southern tip of the island, however, is undeveloped and federally protected, and it has experienced very little direct anthropogenic shoreline modification (Rice, 2015). This area, along with Pullen Island (6 km), is part of the Edwin B. Forsythe National Wildlife Refuge. The southernmost protected portion of Long Beach Island is characterized by fringe marshes on remnant storm-surge platforms while the oceanside contains dune systems dissected by overwash-characteristics indicative of the strong influence waves have on shaping this island (Oertel & Kraft, 1994). Pullen Island is completely undeveloped and tide-dominated with extensive marsh development and a narrow sandy shoreline (Oertel & F I G U R E 1 Geographic extent of the two focal areas across New Jersey and New York, USA across which two species distribution models predicted suitable U.S. Atlantic Coast piping plover (Charadrius melodus) nesting habitat Kraft, 1994). Together, Long Beach and Pullen islands comprise the New Jersey study region referenced in this paper. Maslo, Leu, et al. (2016) predicted the distribution of piping plovers across New Jersey using a presence-only, spatially explicit maximum entropy modeling approach in the program Maxent (version 3.3.3 k; Phillips, Dudík, & Schapire, 2017). Maxent is a widely employed species distribution modeling platform that uses a deterministic, maximum-likelihood framework to analyze presenceonly species occurrence data along with spatial data representing relevant environmental conditions. Spatial areas are split into equal "cells," with the total area (or resolution) of each cell defined by the user. Maxent generates a relative probability of occurrence, or suitability score (from 0-1), for each cell. Scores are displayed visually as a map of predicted occurrence, reflecting the species' relative probability of occurrence by color.

| Maximum entropy model
The Maxent model considered here is described in more detail, including variable definitions and training data, elsewhere (Maslo, Leu, et al., 2016). In brief, the model was trained on 606 piping plover nests initiated in New Jersey during the years 2007-2011 as well as 10,000 "pseudoabsence" points sampled from spatial datasets of the environmental predictors. These background points were sampled from within the study area boundaries, not including water. The predictive accuracy of the Maxent model was tested on an independent dataset of nest occurrences. Nest occurrence data were obtained from the New Jersey Endangered and Nongame Species Program.  tested eight environmental predictor variables in the models (Table 1), representing (a) elevation, (b) slope, (c) land cover, distance to both the (d) high tide line and to (e) non-ocean foraging habitat, (f) total area of sandy beach within 100 m (to account for adequate breeding territory size), (g) total area of marsh within 100 m (to approximate for beach width), and (h) human disturbance level. Human disturbance was accounted for by classifying a given area as either protected (i.e., with some level of conservation action, such as beach closures or vehicle restrictions) or unprotected. For New Jersey, these landcover characteristics were determined through the geospatial analysis of LiDAR elevation data, aerial imagery, land use data, shoreline data observed in 2012 (see Maslo, Leu, et al., 2016 for data sources and methodologies) as part of a separate study of habitat change due to Hurricane Sandy (Maslo et al., 2019). Therefore, habitat suitability maps for New Jersey represent conditions present in 2012. We extended this analysis to the New York sites as part of the present study. Model input data for the New York region was determined following Maslo, Leu, et al. (2016) through analysis of digital elevation data (United States Geological Survey, 2015) and aerial imagery (United States Department of Agriculture, 2015) representing the 2015 environmental conditions. Therefore, habitat maps for New York sites represent conditions present in 2015.
Additionally, relative probability of nest occurrence was mapped under two scenarios of human disturbance for New Jersey and under one scenario for New York. In each case, we modified the human disturbance predictor variable in Maxent to account for the amount of land within the study area protected for conservation purposes. Under Scenario 1 (hereafter, Maxent-existing habitat), only areas under conservation protection experienced little human disturbance. The configuration of protected sites mirrored the existing conservation regime within the study area, as regulated by the United States Fish and Wildlife Service and the New Jersey Division of Fish and Wildlife. In Scenario 2 (hereafter, Maxent-potential habitat), we hypothetically designated all of coastal New Jersey as under conservation protection with minimal human disturbance for beach-nesting birds. Previous comparisons between these scenarios (Maslo et al., 2018) show that human disturbance significantly reduces the quality of otherwise geomorphically suitable nesting habitat, rendering it unsuitable for breeding. Because adopted practices for managing human disturbance vary significantly between New York and New Jersey, we quantified the relative probability of nest occurrence on Fire Island and the Rockaway Peninsula assuming both New York study areas were entirely under conservation protection. Thus, we considered only the Maxent-potential habitat scenarios for these sites.
Maxent utilizes the presence-only data and maps of the predictor variables to determine the conditional density of the predictor variables at piping plover nest locations as well as the marginal density of predictor variables across the entire study area. The trained Maxent model returned a relative probability of piping plover nest occurrence (or habitat suitability score) for each 10-m cell across each study area, based on the landcover characteristics in that cell. For all sites, we interpreted the relative probability of nest occurrence as the probability that a given raster cell supported suitable piping plover nesting habitat given the underlying landscape conditions and relative to surrounding areas. We assume, therefore, that a cell was considered to be "suitable" nesting habitat if its relative nesting occurrence probability was greater than or equal to the calculated 10-percentile training presence threshold (i.e., the minimum suitability score above which 90% of the training occurrence data fell, or 0.474 for piping plovers) based on the Maxent model results for New Jersey (Maslo, Leu, et al., 2016). The same threshold was used for both New Jersey and New York.

| Bayesian network model
The second model evaluated in this study was constructed as a Bayesian network (hereafter, BN) with the program Netica (version 512). Methodology is described in more detail in Zeigler et al. (2017.  connected (a) four nodes representing discretized continuous variables (distance to ocean, beach width, elevation, and distance to foraging); (b) four nodes representing categorical landcover characteristics (geomorphic setting, substrate type, vegetation type, vegetation density); and (c) an output node for the probability that the specific combination of landscape characteristics represented by the input nodes would be associated with landcover suitable for nesting. Variable definitions are given in Table 1 and explained in more detail in .
The BN's prior probability distributions were established in Netica through the "expectation-maximization" algorithm using presence/absence occurrence data. These data were collected through the smartphone application "iPlover"  and were supplemented with remotely sensed information. A simple habitat assessment was conducted at the coordinates of 287 nest and 269 random points observed during the breeding seasons (March-July) of 2014 and 2015 at each of the study areas considered here (Fire  (2017), and Sturdivant et al. (2016). Within this dataset, it was assumed that landcover characteristics associated with nest points delineated habitat, while characteristics associated with random points represented unsuitable landcover, or non-habitat. To derive prior probability distributions for the remaining continuous variables in the BN, we supplemented the dataset with additional landscape data using remotely sensed LiDAR and aerial photographs captured during the appropriate study periods . The trained BN was then used to map habitat. For each study area and year,  used orthoimagery and LiDAR to create eight geographic information system (GIS) layers in ArcGIS (version 10.4) that represented each input node in the BN. The eight layers were combined to form a single GIS layer, where every 5 × 5 m landscape cell had an attribute for each of the input nodes. The BN was then used to calculate the probability that a given cell contained piping plover nesting habitat, given that cell's underlying landscape characteristics. Because orthoimagery and LiDAR were used to map habitat probabilities, mapped habitat availability reflects the conditions at the time remote data were captured. BN model results described in the present study reflect 2014 conditions for the Rockaway peninsula, a mixture of 2014 and 2015 conditions for Fire Island, and fall 2012 conditions for the New Jersey site . Thus, the BN and Maxent models characterize the habitat conditions of the study area over the same time period.
In post-processing of BN results, we defined thresholds for which probabilities indicated the presence of habitat based on the Intergovernmental Panel on Climate Change's (IPCC's) likelihood scale (Intergovernmental Panel on Climate Change [IPCC] 2014). Under this scale, a landscape cell was considered "very likely habitat" if it had a probability >0.90 of being habitat, "likely habitat" with a probability 0.66-0.90, "uncertain" with a probability 0.33-0.66, and "unlikely habitat" with a probability <0.33. Any landscape cell with a probability >0.66 was considered habitat, while cells with a probability <0.33 were considered non-habitat.

| Model comparison
We used an ensemble modeling approach, meaning that we used more than one model to determine patterns and reduce uncertainty in understanding areas of high piping plover habitat suitability. "Consensus predictions" from an ensemble of models can illuminate the signal of interest and reduce noise from errors, biases and uncertainties associated with any one model considered (Araújo, Thuiller, & Pearson, 2006). We compared several aspects of each model, including structure, suitability threshold criteria, predictive accuracy, variable sensitivity, and predictions.
We first summarized the two modeling approaches, including data inputs, predictor variables, and suitability criteria to highlight similarities and differences in model structure (Table 1). We classified individual model inputs, procedures, and outputs according to the rubric established by Sofaer et al. (2019) in Table S1. We also summarized the relative importance of predictor variables used in each modeling approach based on sensitivity analyses. Maxent ranks predictor variables in order of permutation importance, or the normalized drop in area under the curve (AUC) score when training the model on only a single predictor variable. For the BN, we explored the influence of individual nodes on habitat predictions and the causal relationships of importance using the mutual information (entropy reduction) values in Netica. Entropy reduction measures the magnitude with which a finding at one node is expected to alter the beliefs at another node (in this case, habitat designation).
Although both models were independently tested for accuracy elsewhere (Maslo, Leu, et al., 2016;, we also tested model accuracy using a common testing dataset. For this exercise, we overlaid piping plover nest locations recorded by state and federal wildlife agencies within our focal areas for the years 2013-2015 onto the habitat suitability maps. We extracted the value from the underlying habitat maps for each piping plover nest point such that each nest was associated with a predicted suitability score from both the BN and Maxent models. We counted the number of nests occurring each year in areas predicted to be suitable by a single model, both models, or neither model. Model predictions were binary and simplified to suitable habitat or unsuitable landcover based on the model-specific thresholds for suitability (Table 1).
It is important to note that the BN was trained using some of the same nests present in the common testing dataset in New York and New Jersey during the breeding seasons of 2014 and 2015. The Maxent model was trained on presence data from 2007-2011, which did not overlap with data from the common testing dataset. However, the BN's training dataset also included absence data not included in the common testing dataset as well as presence-absence data from other locations. Furthermore, the probability distributions in the BN were derived from a learning algorithm embedded in Netica. Therefore, we believe there to be sufficient differences between the testing dataset and the training dataset/algorithm used to develop the BN, allowing a fair comparison between BN model predictions and actual nesting patterns. It is possible that overlap in testing and training datasets may make the BN inherently more accurate in predicting the common testing dataset compared to Maxent; however, there is no evidence, to our knowledge, that year has an effect on piping plover nesting patterns in previous studies that would result in Maxent being handicapped in its predictions of nesting patterns in 2013-2015 compared to the BN.
Finally, we examined congruency in outputs from the BN and Maxent models separately by region. For the New Jersey region, we assessed model congruence among both the Maxent-existing and Maxent-potential habitat models and the BN. For the New York region, we compared the Maxent-potential and BN models. In order to directly compare BN and Maxent results, we clipped Maxent suitability maps to the same extent covered by the BN maps, which included all land to the interior of the 2012 shoreline for Pullen and Long Beach Islands, New Jersey and all land to the interior of the 2014 shoreline for the Rockaway Peninsula and Fire Island, New York . Additionally, the resolution of Maxent suitability maps was resampled from 10 to 5-m to match the resolution of BN results.
To quantify similarity of raw suitability scores generated by each model, we first randomly selected 10,000 grid cells separately from each coverage area and recorded the raw suitability scores for those cells based on each of the habitat maps produced by the Maxentpotential, Maxent-existing (New Jersey only), and BN. We plotted the suitability scores for each pair of models on an X:Y scatterplot such that perfect model agreement would result in all records sitting on the diagonal 1:1 line. We then conducted a paired samples t-test of raw suitability scores for each model pair to assess statistical similarity in predicted suitability. Next, we converted each model output raster into binary suitability maps depicting suitable vs. unsuitable habitat across the entire focal area. For each model output, all cells with suitability scores above the respective suitability threshold (Maxent threshold = 0.474; BN threshold = 0.66) were considered suitable habitat, and cells with scores falling below the respective threshold were considered unsuitable landcover. Across the full coverage area, we calculated the percentage of total cells with common classifications between each model pair and performed a McNemar's test for paired samples to examine statistical similarity in classifications. We also visually examined model predictions by overlaying habitat suitability maps and identifying areas of agreement across the focal area. All geospatial analyses were performed using ArcMap (version 10.2.2), and all statistical analyses were performed in R (R Core Team, 2013).

| Model comparison and testing
The independently derived Maxent and BN models differed markedly in their analytical approach and data inputs (Table 1). Both models employed similar habitat predictors of piping plover use, particularly elevation, distance to high-energy (ocean) shorelines, distance to low-energy shorelines, landcover, and beach width. However, there were subtle differences in how these predictors were defined in each model. Furthermore, the BN explicitly considered geomorphic setting, substrate type, and vegetation characteristics as drivers of habitat suitability, while the Maxent model approximated for these attributes within a single land use parameter. The Maxent model also explicitly considered human disturbance as well as surrounding marsh area and slope, for which there were no equivalents in the BN (Table 1).
We also observed similarities in how predictor variables were ranked in independent sensitivity analyses ( Table 2). Tests of the Maxent model for habitat use in New Jersey (Maslo, Leu, et al., 2016) showed that land use and distance to the high tide line were the most important predictors, followed by conservation status (protected vs. unprotected), and beach area within a 100-m radius. The remaining variables had substantially lower permutation importance scores (Table 2). Sensitivity tests of the BN also showed that variables describing land cover (i.e., substrate type, geomorphic setting, and vegetation density) were the strongest drivers of nest-site selection. There were marked differences between models in how informative the remaining variables were. Maxent ranked distance to the high tide line as the second-most informative variable while this variable was of lowest importance in the BN. Conservation status had no equivalent in the BN but was the third-ranked variable in the Maxent model. Finally, the beach area within 100 m was ranked fourth in the Maxent model, but the BNequivalent beach width was ranked sixth (Table 2).
Other differences in the structures of the two models included training datasets. The Maxent models were trained on presence-only occurrence data consisting of T A B L E 2 Predictor variables for two species distribution models generated for piping plovers (Charadrius melodus) along the For the Bayesian network, entropy reduction is a measure of the magnitude with which a finding at one node (i.e., at the landscape predictor nodes) is expected to alter the beliefs at another node (i.e., the habitat designation node). 606 piping plover nest locations observed in New Jersey over a 5-year period (2007)(2008)(2009)(2010)(2011) and 10,000 pseudoabsence points extracted from spatial data reflecting 2012 (NJ) landscape conditions. Tested with an independent dataset from 2012, the Maxent model returned an areaunder-the-curve (AUC) of 0.965 (Maslo, Leu, et al., 2016). In contrast, the BN was trained on both nest presence (287 nests) and randomly selected absence (269) points from select barrier islands in New Jersey, New York and Virginia. The BN had an error rate of 23% based on 10-fold cross validation as well as an AUC value of 0.90 based on a receiver operating curve analysis . Across the New Jersey study region, the Maxent-potential habitat model predicted 287.6 ha of suitable piping plover nesting habitat (Table 3), which was measurably higher than the Maxent-existing habitat (160.4 ha) and BN models (106.5 ha). When we tested both models as part of this study using a common testing dataset containing documented piping plover nests within the focal regions for 2013 through 2015, we found that 50 nests (34%) occurred in locations predicted suitable by both models (Table 4). Seventy nests (48%) occurred in locations predicted suitable by the Maxent-potential model alone; 11 nests (8%) occurred in habitat predicted suitable by the BN model alone (Table 4). Across the New York study region, 79 of the 153 nests initiated (52%) occurred in habitat predicted suitable by both models (Table 4; e.g., Figures 4 and 5). Twenty-two percent and 18% of nests occurred in habitat predicted suitable by only the BN or Maxent model, respectively. As standalone models, the Maxent-potential model had a substantially higher predictive accuracy (82%) than the BN model (42%) in New Jersey, but not in New York, where the BN had a slightly higher predictive accuracy (73%) than the Maxent-potential model (70% ;  Table 4). In general, 90% of all nests in New Jersey were located in areas predicted to be suitable by at least one model; nearly 34% of the total nests were found in areas predicted to be suitable by both models (Table 4). In New York, 92% of documented nests occurred in areas predicted to be suitable by at least one model, and 52% of nests were located in areas predicted to be suitable by both models (Table 4).
For both study regions, comparison of 10,000 randomly selected grid cells revealed high dissimilarity in raw suitability scores between modeling approaches (Table 5; Figure 2). However, they demonstrated considerable agreement when compared after being converted to a binary measure of suitability. Across the New Jersey study region, all three models agreed on whether or not a given cell within the randomly selected subset was suitable ≥90% of the time (gray and orange colors combined in Figure 3). Overlap in predicted habitat occurred across 160.4 and 71.6 ha when comparing the Maxent-potential habitat model to the Maxent-existing habitat and BN models, respectively ( Figure 3; Table 5). Where models disagreed, the Maxent-potential habitat model predicted a significantly higher amount than either the Maxentexisting habitat (McNemar's χ 2 = 524, df = 1, p < .001) or BN models (McNemar's χ 2 = 475.5, df = 1, p < .001), and the Maxent-existing habitat model also predicted a significantly larger amount of habitat than the BN model (McNemar's χ 2 = 36.3, df = 1, p < .001). Overlap in predicted habitat suitability for these models occurred across 52.5 ha (Figure 3; Table 5).
Across the New York regions, model agreement on whether a given cell was suitable occurred in 72% of the paired subset of grid cells (gray and orange colors combined in Figures 4 and 5) with no significant differences T A B L E 3 Habitat availability a for piping plovers according two independent species distribution models on Fire Island and the Habitat availability was based on landscape conditions present in remote imagery and other geospatial data for each study area, and remote imagery was selected based according to availability at the time each modeling study was conducted. On the Rockaway Peninsula, the BN approach used geospatial data in the amount of predicted suitable habitat (McNemar's χ 2 = 0.56, df = 1, p = .45) than the BN. The area predicted to be suitable by both models was 196.3 ha in the New York study region (e.g., Figures 4 and 5; Table 5).

| DISCUSSION
Our case study of Atlantic Coast piping plover nesting habitat suitability demonstrates that ensemble modeling approaches can inform conservation by strengthening the credibility of independent models and increasing confidence in the identification of high-quality habitat. Models are by nature simplifications of a complex reality, thus, carrying with them some level of uncertainty, which can deter their adoption in on-the-ground conservation planning (Addison et al., 2013). Recognizing that welldeveloped models do accurately represent ecological conditions, but that no single model is "perfect" (Qiao, Soberón, & Peterson, 2015), ensemble approaches offer a strategy for increasing effectiveness of conservation decisions by reducing uncertainty. The models evaluated employed similar environmental inputs and training data sets but markedly different statistical approaches and thresholds for what constituted suitable habitat (Tables 1 and 2). The difference in statistical approaches, in part, led to high dissimilarity between models in raw suitability scores in both study regions (Figure 2c). This result was likely due to the fact that model outputs (suitability scores) have different interpretations among the two statistical approaches. In Maxent models, suitability score refers to the relative probability that a nest will occur in a given cell (e.g., a score of 0.75 means that there is a 75% higher chance that a nest will occur at that location compared to a location with a score of 0). In contrast, the Bayesian suitability scores represent the level of certainty the model applies to its prediction of suitable habitat (e.g., a score of 0.75 means that there is 75% confidence that the habitat is suitable for nesting in that location). Therefore, lack of agreement on raw suitability score across SDMs does not signify incongruence, only inherent differences in the methodological approach.
Indeed, the models tested demonstrated considerable agreement when comparing a binary measure of suitability after predicted habitat was delineated from unsuitable landcover based on model-specific thresholds of suitability (i.e., BN > 0.66,Maxent >0.474). The two approaches were in 90% agreement across cells in New Jersey ( Figure 3) and 72% in New York (Figures 4 and 5). Models generally predicted habitat in areas characterized by wide sandy beaches or overwash fans, with sparse vegetation and access to low-energy shorelines (Maslo, Leu, et al., 2016;. The similarity in model predictions when viewed as a binary value and not a raw probability value underscores the importance of the selection and application of suitability threshold values in species distribution and other habitat models (Clemens, Weston, Haslem, Silcocks, & Ferris, 2010). When viewed as a binary measure of suitability, we observed core areas of habitat predicted by both models (e.g., Figure 3-5; and 196.3 ha of habitat in the New Jersey and New York study areas, respectively. These areas of overlap, where both models predicted high suitability for piping plover nesting, potentially reduces uncertainty in model predictions. Experiential evidence from managers along with these combined model results could thereby increase a manager's confidence in seeking conservation protection of those locations while providing empirical support for such actions. Of significance, both models exhibited high similarity in identifying drivers of nest-site selection. The predictor variables used, although represented in different ways, were both effective at capturing the environmental conditions important in determining nesting habitat suitability, as established by previously calculated error rates for both models (Maslo, Leu, et al., 2016;. Given that predictive variables considered in both models were based on expert opinion and a review of published literature (e.g., Burger, 1987;Cohen et al., 2008;Flemming, Chiasson, & Austin-Smith, 1992;Maslo et al., 2011;Zeigler et al., 2017), this result may seem intuitive. Yet, this consistency in variable importance increases the applicability of model-derived outputs to areas outside its geographic extent, potentially further reducing the research-implementation gap.
Differences in model predictions, in addition to their similarities, also present important information for conservation. When we examined how often actual piping plover nests overlapped with predicted habitat, we found that nests occurred in locations predicted to be habitat by both models in only 52 and 34% of cases in New York and New Jersey, respectively (Table 3). Additionally, F I G U R E 3 Comparisons of habitat availability for nesting piping plovers as predicted by three models in the Edwin B. Forsythe National Wildlife Refuge on Long Beach and Pullen islands, New Jersey, based on 2012 conditions. Models included (i) a Bayesian network (BN), (ii) a maximum entropy model in which only the current network of protected areas have minimal human disturbance (Maxentexisting), and (iii) a maximum entropy model in which all habitat is hypothetically protected from human disturbance (Maxent-potential). Thresholds at which landcover was considered habitat were 0.66 for the BN and 0.474 for the Maxent model neither model alone captured all habitat used by piping plovers, and whether more nests were found in BNpredicted habitat or in Maxent-predicted habitat varied by study area (Table 3). Some of the differences in whether one or both models captured true nest occurrences likely results from the use of different training data sets. Sandy beaches are highly dynamic landscapes that naturally erode and accrete over short timeframes in response to weather and tides (Davis & FitzGerald, 2009). Thus, even slight inconsistencies in timescales may produce marked differences in model predictions at a local scale. However, over 90% of the nests were found in areas predicted to be habitat by at least one of the models, and together both the Maxent and BN models encompassed the majority of habitat used during the study period in New York and New Jersey.

| Management implications for piping plovers
Particularly in the New York-New Jersey region, where this research was focused, piping plover population growth is inhibited by a lack of suitable habitat resulting from dense coastal development, significant human disturbance, pressure from non-native and human-adapted predators, and the consequences of habitat-insensitive beach stabilization practices (e.g., Greene, 2002;Maslo et al., 2018;Maslo & Lockwood, 2009;Robinson et al., 2019). Typically, habitat protections are not instituted in new areas unless a federally or state-listed species is observed there. However, areas without protections in place are too highly disturbed in most cases to be considered attractive to piping plovers (Maslo et al., 2018), creating a conservation catch-22. Therefore, proactively adding new habitat zones has tremendous potential for increasing population viability. For piping plovers, the addition of protected sites can release competition for breeding territories within designated natural areas and provide dispersal areas for juvenile birds (Cairns, 1982;Cohen et al., 2009). It can also provide in-season refugia when primary nesting areas are inundated with storm tides, and ultimately reduce the dependency on any single site to achieve annual reproductive output goals. Model-predicted habitat delineations can greatly facilitate a wildlife manager's ability to identify areas with a high probability of nest occurrence and to manage those areas to promote reproductive success. Areas predicted suitable by multiple models may serve as core areas (and thus should be highly prioritized), while adjoining sections identified by a single model or not identified by either may provide room for compromise F I G U R E 4 Comparisons of habitat availability for nesting piping plovers as predicted by two models on (a) the Rockaway Peninsula, New York, including (b) the Breezy Point Unit of the Gateway National Recreation Area. Models included (i) a Bayesian network (BN) that was based on 2014 landscape conditions and (ii) a maximum entropy model, where all of the study area is assumed to be protected (Maxentpotential), that was based on 2015 landscape conditions. Thresholds at which landcover was considered habitat were 0.66 for the BN and 0.474 for the Maxent model during negotiations with stakeholders representing recreational interests.
Modeling approaches can inform site management activities within protected sites as well. Despite the known dynamics of beaches, land-use planning continues to promote the stabilization of coastal landscapes (Tarlock, 2007). These practices speed the normal rate of succession, resulting in larger, more densely vegetated dunes that eliminate nesting habitat and encourage occupancy by mammalian nest predators Nordstrom, 2005). Along highly developed shorelines, natural physical processes (changes in sediment supply, vegetation successional rates, etc.) in conservation areas are affected by stabilization and dredging practices nearby, causing suitable nesting areas to degrade. In reporting suitability scores, models provide insight into when management action may be required. Habitat enhancement, such as removing vegetation, lowering dune elevations, or adding shell could restore the quality of protected sites (Maslo et al., 2011); models can quantify how these changes affect habitat quality, which can be incorporated into an adaptive management framework (Wilhere, 2002). Furthermore, model outputs can be used by federal and state regulatory agencies when assessing impacts of stabilization, dredging, and other activities on habitat quality as part of the regulatory consultation and permit issuance process.
It is important to articulate clearly that we do not advocate the elimination of expert biological opinion in conservation planning. Indeed, models cannot capture all subtleties of a conservation conflict. Political and social norms, human use patterns, or the beach culture mean that not every site will be an equally strong candidate for conservation protection. Although the habitat suitability models provide the best means of evaluating suitability on a sliding scale, some qualitative assessments factoring in these concepts may be required. Upon completion of a model-based identification and ranking of sites, wildlife F I G U R E 5 Comparisons of habitat availability for nesting piping plovers as predicted by two models on Fire Island, New York. Models included (i) a Bayesian network (BN) that was based on 2014 elevation and 2015 landcover conditions and (ii) a maximum entropy model, where all of the study area is assumed to be protected (Maxent-potential), that was based on 2015 landscape conditions. Thresholds at which landcover was considered habitat were 0.66 for the BN and 0.474 for the Maxent model agencies should apply expert opinion to determine the feasibility and potential for conservation success of each identified location.

| CONCLUSIONS
We quantify nesting habitat extent of the Atlantic Coast piping plover in two regions of New Jersey and New York using an ensemble modeling approach to compare two independently derived SDMs. Areas of model consensus (i.e., overlap in predicted piping plover nesting habitat) provide a stronger "signal" in model results, reducing uncertainty related to biases or errors associated with either model. Because models predominantly agreed on what drives piping plover nest-site selection, areas predicted suitable by a single model should not be discounted. This case study demonstrates how models can effectively inform conservation planning by: (a) explicitly identifying management objectives; (b) using an ensemble modeling approach to allow managers to evaluate the outcomes of alternative management decisions; and (c) providing robust evidence that addresses the conservation problem. Ultimately, managers will determine how results from ensemble models are used in conservation planning, but direct comparisons provide an evidence-based framework upon which to support management decisions.