Extraction of mitigation-related text from Endangered Species Act documents using machine learning: a case study

Various industrial and development projects have the potential to adversely affect threatened and endangered species and their habitats. The federal Endangered Species Act (ESA) requires preparation of a biological assessment or habitat conservation plan before federal agencies can authorize, through decision documents and permits, unintentional and otherwise prohibited “take” (i.e., harm) of listed species. These documents describe the potential effects of proposed projects on listed species and include measures to mitigate those effects. Collectively, these assessments, plans, decision documents, and permits—termed ESA documents in our study—are valuable for identifying approved mitigation options that could apply to future projects. However, owing to the volume, length, and complexity of these documents, manual review would be time- and labor-intensive. In this study, we apply three supervised machine learning algorithms, including two based on state-of-the-art transfer learning, to develop and evaluate predictive models capable of extracting mitigation-related text from ESA documents. The machine learning models were developed based on a training dataset that was created as part of this study. The best performing model showed an estimated ROC-AUC score of 0.98 and a precision recall AUC score of 0.86 during cross-validation, indicating great potential for effectively extracting mitigation-related content from existing documents. To illustrate the utility of this technology, we present a simulated case study application in which the use of pretrained machine learning models capable of recognizing mitigation measures, coupled with a large historical corpus of ESA documents and keyword filters, provided a means to rapidly assess the commonly used mitigation measures for a given species. While this technology did not eliminate the requirement for biological expertise, it did allow for rapid scoping assessments and could serve as a supporting resource even for experienced biologists.