IDEAS home Printed from https://ideas.repec.org/p/aiz/louvad/2022018.html
   My bibliography  Save this paper

A Quadrature Rule combining Control Variates and Adaptive Importance Sampling

Author

Listed:
  • Leluc, Rémi
  • Portier, François
  • Segers, Johan

    (Université catholique de Louvain, LIDAM/ISBA, Belgium)

  • Zhuman, Aigerim

    (Université catholique de Louvain, LIDAM/ISBA, Belgium)

Abstract

Driven by several successful applications such as in stochastic gradient descent or in Bayesian computation, control variates have become a major tool for Monte Carlo integration. However, standard methods do not allow the distribution of the particles to evolve during the algorithm, as is the case in sequential simulation methods. Within the standard adaptive importance sampling framework, a simple weighted least squares approach is proposed to improve the procedure with control variates. The procedure takes the form of a quadrature rule with adapted quadrature weights to reflect the information brought in by the control variates. The quadrature points and weights do not depend on the integrand, a computational advantage in case of multiple integrands. Moreover, the target density needs to be known only up to a multiplicative constant. Our main result is a non-asymptotic bound on the prob- abilistic error of the procedure. The bound proves that for improving the estimate’s accuracy, the benefits from adaptive importance sampling and control variates can be combined. The good behavior of the method is illustrated empirically on synthetic examples and real-world data for Bayesian linear regression.

Suggested Citation

  • Leluc, Rémi & Portier, François & Segers, Johan & Zhuman, Aigerim, 2022. "A Quadrature Rule combining Control Variates and Adaptive Importance Sampling," LIDAM Discussion Papers ISBA 2022018, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    • Handle: RePEc:aiz:louvad:2022018
    as

    Download full text from publisher

    File URL: https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A261036/datastream/PDF_01/view
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Pierre Del Moral & Arnaud Doucet & Ajay Jasra, 2006. "Sequential Monte Carlo samplers," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 411-436, June.
    2. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-1339, November.
    3. Chris J. Oates & Mark Girolami & Nicolas Chopin, 2017. "Control functionals for Monte Carlo integration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(3), pages 695-718, June.
    4. Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
    5. Portier, Francois & Segers, Johan, 2019. "Monte Carlo integration with a growing number of control variates," LIDAM Reprints ISBA 2019035, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    6. Plassier, Vincent & Portier, François & Segers, Johan, 2020. "Risk bounds when learning infinitely many response functions by ordinary linear regression," LIDAM Discussion Papers ISBA 2020019, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    7. repec:dau:papers:123456789/6072 is not listed on IDEAS
    8. Leluc, Rémi & Portier, François & Segers, Johan, 2021. "Control variate selection for Monte Carlo integration," LIDAM Reprints ISBA 2021024, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markku Lanne & Jani Luoto, 2015. "Estimation of DSGE Models under Diffuse Priors and Data-Driven Identification Constraints," CREATES Research Papers 2015-37, Department of Economics and Business Economics, Aarhus University.
    2. Hajargasht, Gholamreza & Rao, D.S. Prasada, 2019. "Multilateral index number systems for international price comparisons: Properties, existence and uniqueness," Journal of Mathematical Economics, Elsevier, vol. 83(C), pages 36-47.
    3. Cranfield, John A.L. & Preckel, Paul V. & Liu, Songquan, 1997. "Approximating Bayesian Posteriors using Multivariate Gaussian Quadrature," 1997 Annual Meeting, July 13-16, 1997, Reno\ Sparks, Nevada 35791, Western Agricultural Economics Association.
    4. Ardia, David & Hoogerheide, Lennart F., 2010. "Efficient Bayesian estimation and combination of GARCH-type models," MPRA Paper 22919, University Library of Munich, Germany.
    5. Koopman, Siem Jan & Lucas, André, 2008. "A Non-Gaussian Panel Time Series Model for Estimating and Decomposing Default Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 510-525.
    6. Arnaud Dufays, 2016. "Evolutionary Sequential Monte Carlo Samplers for Change-Point Models," Econometrics, MDPI, vol. 4(1), pages 1-33, March.
    7. John Geweke & Joel Horowitz & M. Hashem Pesaran, 2006. "Econometrics: A Bird’s Eye View," CESifo Working Paper Series 1870, CESifo.
    8. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    9. Hajivassiliou, Vassilis A. & Ruud, Paul A., 1986. "Classical estimation methods for LDV models using simulation," Handbook of Econometrics, in: R. F. Engle & D. McFadden (ed.), Handbook of Econometrics, edition 1, volume 4, chapter 40, pages 2383-2441, Elsevier.
    10. Baştürk, Nalan & Grassi, Stefano & Hoogerheide, Lennart & Opschoor, Anne & van Dijk, Herman K., 2017. "The R Package MitISEM: Efficient and Robust Simulation Procedures for Bayesian Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 79(i01).
    11. Lennart F. Hoogerheide & Johan F. Kaashoek, 2004. "Functional Approximations to Likelihoods/Posterior Densities: A Neural Network Approach to Efficient Sampling," Computing in Economics and Finance 2004 74, Society for Computational Economics.
    12. Fok, Dennis & Franses, Philip Hans, 2007. "Modeling the diffusion of scientific publications," Journal of Econometrics, Elsevier, vol. 139(2), pages 376-390, August.
    13. Gael M. Martin & David T. Frazier & Ruben Loaiza-Maya & Florian Huber & Gary Koop & John Maheu & Didier Nibbering & Anastasios Panagiotelis, 2023. "Bayesian Forecasting in the 21st Century: A Modern Review," Monash Econometrics and Business Statistics Working Papers 1/23, Monash University, Department of Econometrics and Business Statistics.
    14. Liesenfeld, Roman & Richard, Jean-François, 2008. "Improving MCMC, using efficient importance sampling," Computational Statistics & Data Analysis, Elsevier, vol. 53(2), pages 272-288, December.
    15. Kleibergen, Frank & Paap, Richard, 2002. "Priors, posteriors and bayes factors for a Bayesian analysis of cointegration," Journal of Econometrics, Elsevier, vol. 111(2), pages 223-249, December.
    16. Edward Herbst & Frank Schorfheide, 2014. "Sequential Monte Carlo Sampling For Dsge Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(7), pages 1073-1098, November.
    17. Villani, Mattias, 2006. "Bayesian point estimation of the cointegration space," Journal of Econometrics, Elsevier, vol. 134(2), pages 645-664, October.
    18. Heckelei, Thomas & Mittelhammer, Ronald C., 1996. "Bayesian Bootstrap Analysis of Systems of Equations," Discussion Papers 18786, University of Bonn, Institute for Food and Resource Economics.
    19. Garland Durham, 2004. "Likelihood-based estimation and specification analysis of one- and two-factor SV models with leverage effects," Econometric Society 2004 North American Summer Meetings 294, Econometric Society.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aiz:louvad:2022018. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Nadja Peiffer (email available below). General contact details of provider: https://edirc.repec.org/data/isuclbe.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.