A modeler's manifesto: Synthesizing modeling best practices with social science frameworks to support critical approaches to data science

In the face of the "crisis of reproducibility" and the rise of "big data" with its associated issues, modeling needs to be practiced more critically and less automatically. Many modelers are discussing better modeling practices, but to address questions about the transparency, equity, and relevance of modeling, we also need the theoretical grounding of social science and the tools of critical theory. I have therefore synthesized recent work by modelers on better practices for modeling with social science literature (especially feminist science and technology studies) to offer a "modeler’s manifesto": a set of applied practices and framings for critical modeling approaches. Broadly, these practices involve 1) giving greater context to scientific modeling through extended methods sections, appendices, and companion articles, clarifying quantitative and qualitative reasoning and process; 2) greater collaboration in scientific modeling via triangulation with different data sources, gaining feedback from interdisciplinary teams, and viewing uncertainty as openness and invitation for dialogue; and 3) directly engaging with justice and ethics by watching for and mitigating unequal power dynamics in projects, facing the impacts and implications of the work throughout the process rather than only afterwards, and seeking opportunities to collaborate directly with people impacted by the modeling

California Wildfire Resilience Core Metrics Rating Process and Results

The California Wildfire & Forest Resilience Task Force (Task Force) has developed regionally-adapted resources to lessen wildfire risk to communities and enhance broader statewide ecosystem resilience. This includes a large set of metrics intended to support a wide range of organizations in prioritizing, planning and/or implementing management actions. The Science Advisory Panel to the Task Force (SAP) was asked to provide expert advice to inform the selection of a subset of “core” metrics for reporting outcomes and progress towards resilience goals. We used survey tools to collect and synthesize the scientific expertise of the SAP as well as other experts regarding existing “Regional Resource Kit” (RRK) metrics as well as potential suggested metrics. This report summarizes the survey process and results. We used two rounds of surveys to collect expert opinion on metrics. Round 1 asked respondents to 1) identify criteria for core metrics, 2) identify metrics from the Regional Resource Kits (RRKs) that were believed to be inadequate based on those criteria, and 3) recommend additional metrics not included in the original set. Round 2 asked respondents to rate the resulting set of 115 metrics. The Round 1 results indicated that selecting a useful set of core metrics depended on their intended application (e.g., planning vs.reporting), the intended audience (e.g., policy makers, scientists, and/or the public), and resilience outcomes (e.g., immediate wildfire risk reduction versus long-term ecological health). In Round 2 we asked respondents to rate each metric on how well it measured each of three broad, overlapping resilience goals identified by the Task Force: 1) reducing wildfire risk, 2) improving ecological integrity, and/or 3) supporting social and/or cultural wellbeing. The Task Force specified the purpose for the metrics: reporting progress to policymakers and the public. Therefore, respondents also evaluated three attributes for each metric: 1) how realistic it was to remeasure (i.e., feasibility), 2) how easy it was to explain to wide audiences (i.e., understandability), and 3) how well it represented the process of interest (i.e., sensitivity). In addition, respondents identified relevant region(s) of California (Sierra Nevada, Southern California, Central Coast, and Northern California) for each metric. For the 115 metrics collectively considered in Round 2 (81 from the RRKs and 34 novel metrics), 66 received an average rating of greater than 4 out of 5 on one or more of the three resilience goals (reduce fire risk, improve ecological resilience, support social/cultural wellbeing). Of these, 13 metrics were rated above 4 out of 5 for two of the three goals, and only “probability of high-severity fire” was rated that highly for all three. Metrics on topics relating to vegetation structure and composition as well as fire behavior and history were most abundant in the RRKs and in our list of highly rated metrics. Topics relating to air quality, water supply, economics, community readiness, environmental justice, and community wellbeing were less abundant in the RRKs and not as highly rated in our surveys; these topic areas could benefit from further expert feedback and development. Top-rated metrics already present in the RRKs included: probability of high severity fire, damage potential in the Wildland-Urban Interface (WUI), standing dead and ladder fuels, vegetative stress during extreme drought, tree mortality, and shrub resilience. Metrics that are not yet in the RRKs but do exist elsewhere include Cal EnviroScreen scores and areas of low potential shrub regeneration. Highly rated novel metrics suggested by the survey respondents include (among others): health outcomes related to air quality/smoke and insurance availability/price. Some considerations for proceeding with selection of core metrics arose through this process. First, metrics selected (and targeted desirable ranges for these metrics) might be ecosystem specific. Second, the tradeoff between the logistical aspects of the metrics (feasibility of remeasurement, understandability) with their scientific accuracy and value needs to be evaluated with the target audience in mind. Third, many metrics can address resilience across multiple topic areas and therefore framing this overlap carefully is important when determining how to track progress towards resilience goals

Using MPA Watch Data to Analyze Human Activities Along the California Coast

Information about human activities along the coast can help us understand human impacts on natural resources and the benefits people derive from marine protected areas (MPAs). In this project we examined human activities along the California coast from 2012 to 2020 using data from the MPA Watch community science network, gathered by more than 1,900 volunteer participants, and a handful of program staff. MPA Watch is gathering useful data at a statewide scale, and has successfully grown its network of volunteer monitoring programs over the last decade to include 12 local programs, 104 monitoring sites, and hundreds of volunteer surveyors each year. Among the observations recorded by MPA Watch surveyors, non-consumptive recreational activities vastly outnumber consumptive activities like fishing, both inside and outside of MPAs. This highlights the value of MPA Watch data for understanding human coastal use, underscores the importance of recreational activities in California’s coastal economy, and reinforces the need to monitor and understand non-consumptive uses in and around MPAs. Our analysis confirms that MPA Watch data can detect broad, statistically robust patterns in human activities along the coast, including among recreational activities that relate to Goal 3 of the Marine Life Protection Act, but have not been the focus of other socioeconomic monitoring projects. MPA Watch data can also be used to detect statistically significant differences between activities inside and outside of MPAs: we used occupancy modeling to investigate the likelihood of occurrence of seven different categories of human activities. From 2012 to 2020 at the statewide level, onshore fishing was less likely inside of MPAs, and tidepooling and recreational boating were more likely inside of MPAs (we found no difference in probability of occurrence for offshore fishing, domestic animals, or onshore or offshore recreation). We conclude with some recommendations for improving MPA Watch data collection protocols, for expanding the use of the data by managers and law enforcement, and for future in-depth analyses that incorporate more of the richness of the datase

Community and Citizen Science on the Elwha River: Past, Present, and Future

This report reflects on the past, present, and potential future of community and citizen science (CCS) in the Elwha River watershed, with particular focus on the years before and after a major restoration event: the removal of two dams that had impacted the river system for a century. We ask: how does CCS feature in the Elwha story and how could it feature? We use the term CCS to reference the broad range of ways in which members of the public might participate in authentic science and monitoring processes, including students and both paid and unpaid interns: participants are individuals contributing to scientific projects without prior formal training in the topic. We present insights about Elwha CCS from an academic literature review and discussions with scientists and many others that have been working in the Elwha. We found that the history of CCS on the Elwha is important but understated, with few scientific papers acknowledging support by volunteers of various kinds. Recent and ongoing CCS projects on the Elwha tend to be focused on biological phenomena, and most are associated with educational opportunities (across many types of institutions) and paid internships. We also noted that most Elwha CCS projects required volunteers with particular pre-existing skill sets (e.g., botanical knowledge) or time to impart specialized training (e.g. boat use), leading many projects towards engagement with a smaller number of volunteers. Many new projects would broaden public involvement in terms of the opportunities available, and increase the variety of focal topics for research and monitoring. This increased breadth is promising: there are indications that the local community’s interests also range widely, from fish recovery after dam removal to dam removal impacts on humans. Elwha CCS projects have encountered some challenges and barriers, including the administrative burden of coordinating volunteers and managing liability concerns. But Elwha ScienceScape scientists are committed to the value that CCS brings both to the research itself as well as to those who participate in these projects. CCS can be a way to increase equity in science and engage people who would not otherwise participate in research, and in many cases the research simply wouldn’t be possible without their help. Support with project administration, volunteer management, and data management could help in expanding CCS efforts and broadening their inclusivity. More systematic tracking of CCS projects to assess how they contribute to research and to community and participant benefit could be helpful in establishing and maintaining a long-term CCS strategy in the Elwha