index.qmd

# eLetter {.unnumbered} 

Avoided deforestation conservation projects necessitate comparisons between observations in protected forests and what-would-have-happened if the project did not exist (often referred to as the baseline, or counterfactual scenario) @balmford_realizing_2023. Nevertheless, it is imperative that forest carbon projects deliver the carbon emission reductions they report, and there are various methods that can be used to estimate these counterfactual scenarios. West et al. @west_action_2023 used the synthetic control method to choose pools of controls with similar covariate structure to *avoided unplanned deforestation conservation project* areas; they do not consider other methodologies such as *planned avoided deforestation*, or *improved forest management*. Synthetic controls were calculated from the mean deforestation rate of pools, weighted by covariate similarity with the project. We agree that the synthetic control method could potentially improve the evaluation of forest carbon projects. However, synthetic controls are also vulnerable to “gaming” or bias through inappropriate donor pool design/sampling and such econometric statistical methods require further context-specific refinement @balmford_realizing_2023 before they can improve upon existing approaches. 

West et al. @west_action_2023 claim that most forest carbon projects overestimate avoided deforestation or do not significantly reduce deforestation. These conclusions are poorly supported by their analysis because:

- The implemented synthetic control approach can suffer from bias @abadie_using_2021 due to substantial and incompatible differences between deforestation drivers, forest type and (bio)geography that are not adequately captured by the structure of covariates that are used to form the synthetic control. Important covariates are ignored, such as distance to roads and rivers [@laurance_roads_2017; @barber_roads_2014], fire risk @amador-jimenez_unintended_2020, or indicators of forest structure @duncanson_aboveground_2022 that predict biomass, timber value and logging effort. Using the authors’ code, we extracted the locations of controls used to create the synthetic control for project-856 (@fig-map), located in Colombia’s Pacific-montane-forest. No weighted donors were in the same political jurisdiction or ecoregion as the project and the highest weighted donor (0.46) was in the Amazon, 900 km away. We have also attempted to reproduce the synthetic controls for all considered project areas; interactive maps of these locations are presented at https://permian-global-research.github.io/science-letter-west-et-al/. 

- Calculations derived from Global Forest Change (GFC) @hansen_quantification_2010 inherit known sensitivity and accuracy issues [@kinnebrew_biases_2022; @galiatsatos_assessment_2020]. It is inappropriate to compare GFC-based calculations with those reported by projects, which are better calibrated for local contexts, because meaningful comparisons require deforestation rates to be derived using identical methods. Further, it is accepted that GFC is inappropriate for site-level deforestation assessment, in isolation [@galiatsatos_assessment_2020; @bos_global_2019]. 

- West et al. @west_action_2023 do not demonstrate that synthetic control counterfactuals are more accurate than project methods, undermining their ability to interpret differences between the synthetic control and project-reported deforestation rates. 

![Map of the donor pool and their weighted contribution to the synthetic control for Project-856. The project is located to the north of Colombia's Choco Department (political jurisdiction), in the Darien Mountain range and comprises a transition from lowland to montane rainforest. Also presented are ecoregions @dinerstein_ecoregion-based_2017 that intersect weighted donors. Two donors form more than 84% of the synthetic control, and no weighted donors were in the same ecoregion as the project with the highest weighted donor being 900 km away, in the Amazon. Project-856 was selected because it was the only one where fully reproducible code was made available. To see interactive maps of this and all other projects please see [https://permian-global-research.github.io/science-letter-west-et-al/](https://permian-global-research.github.io/science-letter-west-et-al/). Elevation data from https://registry.opendata.aws/terrain-tiles.](figures/colombia_856_donors.png){#fig-map}

The adoption of econometric methods to evaluate the efficacy of voluntary carbon projects is an important development in the field @balmford_realizing_2023, yet these methods need improvement. Critically, the drivers of deforestation are complex and highly variable across space and time. We believe that there is a need to more carefully consider the connections between spatial and econometric statistical techniques, in the context of remote sensing, to maximise the accuracy and utility of these methods @garcia_conservation_2022.  

No method of reconstructing a counterfactual can claim to represent absolute truth. In order to improve the evaluation of avoided deforestation conservation projects, one potential way forward could be to adopt a simulation framework to benchmark the efficacy of synthetic controls alongside alternative methods. Simulations to reconstruct counterfactuals on non-project areas will provide quantitative metrics for counterfactual reconstruction accuracy. We hope this letter will stimulate further discussion on this topic and encourage the development and adoption of improved inference methods in voluntary carbon projects.



## References {.unnumbered}

::: {#refs}
:::

<br/><br/>

*The pdf version of this eLetter is available here: [<i class="bi bi-file-earmark-pdf"></i>](letter/response-to-west.pdf).*