IDEAS home Printed from https://ideas.repec.org/a/eee/ecomod/v359y2017icp80-91.html
   My bibliography  Save this article

Sensitivity analysis and Bayesian calibration for testing robustness of the BASGRA model in different environments

Author

Listed:
  • Hjelkrem, Anne-Grete Roer
  • Höglind, Mats
  • van Oijen, Marcel
  • Schellberg, Jürgen
  • Gaiser, Thomas
  • Ewert, Frank

Abstract

Proper parameterisation and quantification of model uncertainty are two essential tasks in improvement and assessment of model performance. Bayesian calibration is a method that combines both tasks by quantifying probability distributions for model parameters and outputs. However, the method is rarely applied to complex models because of its high computational demand when used with high-dimensional parameter spaces. We therefore combined Bayesian calibration with sensitivity analysis, using the screening method by Morris (1991), in order to reduce model complexity by fixing parameters to which model output was only weakly sensitive to a nominal value. Further, the robustness of the model with respect to reduction in the number of free parameters were examined according to model discrepancy and output uncertainty. The process-based grassland model BASGRA was examined in the present study on two sites in Norway and in Germany, for two grass species (Phleum pratense and Arrhenatherum elatius). According to this study, a reduction of free model parameters from 66 to 45 was possible. The sensitivity analysis showed that the parameters to be fixed were consistent across sites (which differed in climate and soil conditions), while model calibration had to be performed separately for each combination of site and species. The output uncertainty decreased slightly, but still covered the field observations of aboveground biomass. Considering the training data, the mean square error for both the 66 and the 45 parameter model was dominated by errors in timing (phase shift), whereas no general pattern was found in errors when using the validation data. Stronger model reduction should be avoided, as the error term increased and output uncertainty was underestimated.

Suggested Citation

  • Hjelkrem, Anne-Grete Roer & Höglind, Mats & van Oijen, Marcel & Schellberg, Jürgen & Gaiser, Thomas & Ewert, Frank, 2017. "Sensitivity analysis and Bayesian calibration for testing robustness of the BASGRA model in different environments," Ecological Modelling, Elsevier, vol. 359(C), pages 80-91.
  • Handle: RePEc:eee:ecomod:v:359:y:2017:i:c:p:80-91
    DOI: 10.1016/j.ecolmodel.2017.05.015
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304380016307281
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ecolmodel.2017.05.015?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Confalonieri, R. & Bellocchi, G. & Bregaglio, S. & Donatelli, M. & Acutis, M., 2010. "Comparison of sensitivity analysis techniques: A case study with the rice model WARM," Ecological Modelling, Elsevier, vol. 221(16), pages 1897-1906.
    2. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    3. Oomen, Roelof J. & Ewert, Frank & Snyman, Hennie A., 2016. "Modelling rangeland productivity in response to degradation in a semi-arid climate," Ecological Modelling, Elsevier, vol. 322(C), pages 54-70.
    4. Campbell, Katherine, 2006. "Statistical calibration of computer simulations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1358-1363.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hjelkrem, Anne-Grete Roer & Eikemo, Håvard & Le, Vinh Hong & Hermansen, Arne & Nærstad, Ragnhild, 2021. "A process-based model to forecast risk of potato late blight in Norway (The Nærstad model): model development, sensitivity analysis and Bayesian calibration," Ecological Modelling, Elsevier, vol. 450(C).
    2. Höglind, Mats & Cameron, David & Persson, Tomas & Huang, Xiao & van Oijen, Marcel, 2020. "BASGRA_N: A model for grassland productivity, quality and greenhouse gas balance," Ecological Modelling, Elsevier, vol. 417(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    2. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.
    3. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    4. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    5. Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    6. Wilkinson Richard David, 2013. "Approximate Bayesian computation (ABC) gives exact results under the assumption of model error," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 12(2), pages 129-141, May.
    7. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    8. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    9. Francisco A. Buendia-Hernandez & Maria J. Ortiz Bevia & Francisco J. Alvarez-Garcia & Antonio Ruizde Elvira, 2022. "Sensitivity of a Dynamic Model of Air Traffic Emissions to Technological and Environmental Factors," IJERPH, MDPI, vol. 19(22), pages 1-17, November.
    10. Matthias Katzfuss & Joseph Guinness & Wenlong Gong & Daniel Zilber, 2020. "Vecchia Approximations of Gaussian-Process Predictions," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(3), pages 383-414, September.
    11. Jakub Bijak & Jason D. Hilton & Eric Silverman & Viet Dung Cao, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.
    12. Hao Wu & Michael Browne, 2015. "Random Model Discrepancy: Interpretations and Technicalities (A Rejoinder)," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 619-624, September.
    13. Kanapaux, William & Kiker, Gregory A., 2013. "Development and testing of an object-oriented model for adaptively managing human disturbance of least tern (Sternula antillarum) nesting habitat," Ecological Modelling, Elsevier, vol. 268(C), pages 64-77.
    14. Villez, Kris & Del Giudice, Dario & Neumann, Marc B. & Rieckermann, Jörg, 2020. "Accounting for erroneous model structures in biokinetic process models," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    15. Xiaoyu Xiong & Benjamin D. Youngman & Theodoros Economou, 2021. "Data fusion with Gaussian processes for estimation of environmental hazard events," Environmetrics, John Wiley & Sons, Ltd., vol. 32(3), May.
    16. Petropoulos, G. & Wooster, M.J. & Carlson, T.N. & Kennedy, M.C. & Scholze, M., 2009. "A global Bayesian sensitivity analysis of the 1d SimSphere soil–vegetation–atmospheric transfer (SVAT) model using Gaussian model emulation," Ecological Modelling, Elsevier, vol. 220(19), pages 2427-2440.
    17. Merlin Keller & Guillaume Damblin & Alberto Pasanisi & Mathieu Schumann & Pierre Barbillon & Fabrizio Ruggeri, 2022. "Validation of a Computer Code for the Energy Consumption of a Building, with Application to Optimal Electric Bill Pricing," Post-Print hal-04071903, HAL.
    18. David Breitenmoser & Francesco Cerutti & Gernot Butterweck & Malgorzata Magdalena Kasprzak & Sabine Mayer, 2023. "Emulator-based Bayesian inference on non-proportional scintillation models by compton-edge probing," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    19. Yuan, Jun & Nian, Victor & Su, Bin & Meng, Qun, 2017. "A simultaneous calibration and parameter ranking method for building energy models," Applied Energy, Elsevier, vol. 206(C), pages 657-666.
    20. Barde, Sylvain, 2024. "Bayesian estimation of large-scale simulation models with Gaussian process regression surrogates," Computational Statistics & Data Analysis, Elsevier, vol. 196(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecomod:v:359:y:2017:i:c:p:80-91. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/ecological-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.