IDEAS home Printed from https://ideas.repec.org/p/aiz/louvad/2022018.html
   My bibliography  Save this paper

A Quadrature Rule combining Control Variates and Adaptive Importance Sampling

Author

Listed:
  • Leluc, Rémi
  • Portier, François
  • Segers, Johan

    (Université catholique de Louvain, LIDAM/ISBA, Belgium)

  • Zhuman, Aigerim

    (Université catholique de Louvain, LIDAM/ISBA, Belgium)

Abstract

Driven by several successful applications such as in stochastic gradient descent or in Bayesian computation, control variates have become a major tool for Monte Carlo integration. However, standard methods do not allow the distribution of the particles to evolve during the algorithm, as is the case in sequential simulation methods. Within the standard adaptive importance sampling framework, a simple weighted least squares approach is proposed to improve the procedure with control variates. The procedure takes the form of a quadrature rule with adapted quadrature weights to reflect the information brought in by the control variates. The quadrature points and weights do not depend on the integrand, a computational advantage in case of multiple integrands. Moreover, the target density needs to be known only up to a multiplicative constant. Our main result is a non-asymptotic bound on the prob- abilistic error of the procedure. The bound proves that for improving the estimate’s accuracy, the benefits from adaptive importance sampling and control variates can be combined. The good behavior of the method is illustrated empirically on synthetic examples and real-world data for Bayesian linear regression.

Suggested Citation

  • Leluc, Rémi & Portier, François & Segers, Johan & Zhuman, Aigerim, 2022. "A Quadrature Rule combining Control Variates and Adaptive Importance Sampling," LIDAM Discussion Papers ISBA 2022018, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
  • Handle: RePEc:aiz:louvad:2022018
    as

    Download full text from publisher

    File URL: https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A261036/datastream/PDF_01/view
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Pierre Del Moral & Arnaud Doucet & Ajay Jasra, 2006. "Sequential Monte Carlo samplers," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 411-436, June.
    2. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-1339, November.
    3. Chris J. Oates & Mark Girolami & Nicolas Chopin, 2017. "Control functionals for Monte Carlo integration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(3), pages 695-718, June.
    4. Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
    5. Portier, Francois & Segers, Johan, 2019. "Monte Carlo integration with a growing number of control variates," LIDAM Reprints ISBA 2019035, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    6. Plassier, Vincent & Portier, François & Segers, Johan, 2020. "Risk bounds when learning infinitely many response functions by ordinary linear regression," LIDAM Discussion Papers ISBA 2020019, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    7. Leluc, Rémi & Portier, François & Segers, Johan, 2021. "Control variate selection for Monte Carlo integration," LIDAM Reprints ISBA 2021024, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    8. repec:dau:papers:123456789/6072 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markku Lanne & Jani Luoto, 2015. "Estimation of DSGE Models under Diffuse Priors and Data-Driven Identification Constraints," CREATES Research Papers 2015-37, Department of Economics and Business Economics, Aarhus University.
    2. Leluc, Rémi & Portier, François & Zhuman, Aigerim & Segers, Johan, 2023. "Speeding up Monte Carlo Integration: Control Neighbors for Optimal Convergence," LIDAM Discussion Papers ISBA 2023019, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    3. Hajargasht, Gholamreza & Rao, D.S. Prasada, 2019. "Multilateral index number systems for international price comparisons: Properties, existence and uniqueness," Journal of Mathematical Economics, Elsevier, vol. 83(C), pages 36-47.
    4. Cranfield, John A.L. & Preckel, Paul V. & Liu, Songquan, 1997. "Approximating Bayesian Posteriors using Multivariate Gaussian Quadrature," 1997 Annual Meeting, July 13-16, 1997, Reno\ Sparks, Nevada 35791, Western Agricultural Economics Association.
    5. Ardia, David & Hoogerheide, Lennart F., 2010. "Efficient Bayesian estimation and combination of GARCH-type models," MPRA Paper 22919, University Library of Munich, Germany.
    6. Holloway, Garth J. & Hertel, Thomas W., 1991. "Comparing Hypotheses About Competition," Working Papers 225867, University of California, Davis, Department of Agricultural and Resource Economics.
    7. Koopman, Siem Jan & Lucas, André, 2008. "A Non-Gaussian Panel Time Series Model for Estimating and Decomposing Default Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 510-525.
    8. Fuentes-Albero, Cristina & Melosi, Leonardo, 2013. "Methods for computing marginal data densities from the Gibbs output," Journal of Econometrics, Elsevier, vol. 175(2), pages 132-141.
    9. Arnaud Dufays, 2016. "Evolutionary Sequential Monte Carlo Samplers for Change-Point Models," Econometrics, MDPI, vol. 4(1), pages 1-33, March.
    10. Ardia, David & Hoogerheide, Lennart F. & van Dijk, Herman K., 2009. "Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 29(i03).
    11. Nalan Basturk & Cem Cakmakli & S. Pinar Ceyhan & Herman K. van Dijk, 2014. "On the Rise of Bayesian Econometrics after Cowles Foundation Monographs 10, 14," Tinbergen Institute Discussion Papers 14-085/III, Tinbergen Institute, revised 04 Sep 2014.
    12. Christophe Planas & Alessandro Rossi & Gabriele Fiorentini, 2008. "The marginal likelihood of Structural Time Series Models, with application to the euro area and US NAIRU," Working Paper series 21_08, Rimini Centre for Economic Analysis.
    13. Hajivassiliou, Vassilis A. & Ruud, Paul A., 1986. "Classical estimation methods for LDV models using simulation," Handbook of Econometrics, in: R. F. Engle & D. McFadden (ed.), Handbook of Econometrics, edition 1, volume 4, chapter 40, pages 2383-2441, Elsevier.
    14. Baştürk, Nalan & Grassi, Stefano & Hoogerheide, Lennart & Opschoor, Anne & van Dijk, Herman K., 2017. "The R Package MitISEM: Efficient and Robust Simulation Procedures for Bayesian Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 79(i01).
    15. Schwaab, Bernd & Koopman, Siem Jan & Lucas, André, 2011. "Systemic risk diagnostics: coincident indicators and early warning signals," Working Paper Series 1327, European Central Bank.
    16. Geweke, J. & Joel Horowitz & Pesaran, M.H., 2006. "Econometrics: A Bird’s Eye View," Cambridge Working Papers in Economics 0655, Faculty of Economics, University of Cambridge.
    17. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    18. Vijverberg, Wim P. M., 1997. "Monte Carlo evaluation of multivariate normal probabilities," Journal of Econometrics, Elsevier, vol. 76(1-2), pages 281-307.
    19. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aiz:louvad:2022018. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Nadja Peiffer (email available below). General contact details of provider: https://edirc.repec.org/data/isuclbe.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.