IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v111y2016i513p377-393.html
   My bibliography  Save this article

An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants

Author

Listed:
  • Faming Liang
  • Ick Hoon Jin
  • Qifan Song
  • Jun S. Liu

Abstract

Sampling from the posterior distribution for a model whose normalizing constant is intractable is a long-standing problem in statistical research. We propose a new algorithm, adaptive auxiliary variable exchange algorithm, or, in short, adaptive exchange (AEX) algorithm, to tackle this problem. The new algorithm can be viewed as a MCMC extension of the exchange algorithm, which generates auxiliary variables via an importance sampling procedure from a Markov chain running in parallel. The convergence of the algorithm is established under mild conditions. Compared to the exchange algorithm, the new algorithm removes the requirement that the auxiliary variables must be drawn using a perfect sampler, and thus can be applied to many models for which the perfect sampler is not available or very expensive. Compared to the approximate exchange algorithms, such as the double Metropolis-Hastings sampler, the new algorithm overcomes their theoretical difficulty in convergence. The new algorithm is tested on the spatial autologistic and autonormal models. The numerical results indicate that the new algorithm is particularly useful for the problems for which the underlying system is strongly dependent. Supplementary materials for this article are available online.

Suggested Citation

  • Faming Liang & Ick Hoon Jin & Qifan Song & Jun S. Liu, 2016. "An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 377-393, March.
    • Handle: RePEc:taf:jnlasa:v:111:y:2016:i:513:p:377-393
    DOI: 10.1080/01621459.2015.1009072
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2015.1009072
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/01621459.2015.1009072?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Liang, Faming & Liu, Chuanhai & Carroll, Raymond J., 2007. "Stochastic Approximation in Monte Carlo Computation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 305-320, March.
    2. Ming Gao Gu & Hong‐Tu Zhu, 2001. "Maximum likelihood estimation for spatial models by Markov chain Monte Carlo stochastic approximation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(2), pages 339-355.
    3. Christophe Andrieu & Arnaud Doucet & Roman Holenstein, 2010. "Particle Markov chain Monte Carlo methods," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(3), pages 269-342, June.
    4. J. Møller & A. N. Pettitt & R. Reeves & K. K. Berthelsen, 2006. "An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants," Biometrika, Biometrika Trust, vol. 93(2), pages 451-458, June.
    5. Jin, Ick Hoon & Liang, Faming, 2014. "Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 402-416.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Park, Jaewoo & Jin, Ick Hoon & Schweinberger, Michael, 2022. "Bayesian model selection for high-dimensional Ising models, with applications to educational data," Computational Statistics & Data Analysis, Elsevier, vol. 165(C).
    2. Tetyana Kosyakova & Thomas Otter & Sanjog Misra & Christian Neuerburg, 2020. "Exact MCMC for Choices from Menus—Measuring Substitution and Complementarity Among Menu Items," Marketing Science, INFORMS, vol. 39(2), pages 427-447, March.
    3. Bochao Jia & Suwa Xu & Guanghua Xiao & Vishal Lamba & Faming Liang, 2017. "Learning gene regulatory networks from next generation sequencing data," Biometrics, The International Biometric Society, vol. 73(4), pages 1221-1230, December.
    4. Wanchuang Zhu & Yanan Fan, 2023. "A synthetic likelihood approach for intractable markov random fields," Computational Statistics, Springer, vol. 38(2), pages 749-777, June.
    5. Chih‐Sheng Hsieh & Lung‐Fei Lee & Vincent Boucher, 2020. "Specification and estimation of network formation and network interaction models with the exponential probability distribution," Quantitative Economics, Econometric Society, vol. 11(4), pages 1349-1390, November.
    6. Babkin, Sergii & Stewart, Jonathan R. & Long, Xiaochen & Schweinberger, Michael, 2020. "Large-scale estimation of random graph models with local dependence," Computational Statistics & Data Analysis, Elsevier, vol. 152(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jin, Ick Hoon & Liang, Faming, 2014. "Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 402-416.
    2. Jonathan U Harrison & Ruth E Baker, 2020. "An automatic adaptive method to combine summary statistics in approximate Bayesian computation," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-21, August.
    3. Wanchuang Zhu & Yanan Fan, 2023. "A synthetic likelihood approach for intractable markov random fields," Computational Statistics, Springer, vol. 38(2), pages 749-777, June.
    4. Chih-Sheng Hsieh & Michael D. König & Xiaodong Liu, 2012. "Network formation with local complements and global substitutes: the case of R&D networks," ECON - Working Papers 217, Department of Economics - University of Zurich, revised Feb 2017.
    5. Rajala, T. & Penttinen, A., 2014. "Bayesian analysis of a Gibbs hard-core point pattern model with varying repulsion range," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 530-541.
    6. Ioannis Bournakis & Mike Tsionas, 2024. "A Non‐parametric Estimation of Productivity with Idiosyncratic and Aggregate Shocks: The Role of Research and Development (R&D) and Corporate Tax," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 86(3), pages 641-671, June.
    7. Laleh Tafakori & Armin Pourkhanali & Riccardo Rastelli, 2022. "Measuring systemic risk and contagion in the European financial network," Empirical Economics, Springer, vol. 63(1), pages 345-389, July.
    8. S. Bogan Aruoba & Pablo Cuba-Borda & Kenji Higa-Flores & Frank Schorfheide & Sergio Villalvazo, 2021. "Piecewise-Linear Approximations and Filtering for DSGE Models with Occasionally Binding Constraints," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 41, pages 96-120, July.
    9. Del Negro, Marco & Schorfheide, Frank, 2008. "Forming priors for DSGE models (and how it affects the assessment of nominal rigidities)," Journal of Monetary Economics, Elsevier, vol. 55(7), pages 1191-1208, October.
    10. Arellano, Manuel & Blundell, Richard & Bonhomme, Stéphane & Light, Jack, 2024. "Heterogeneity of consumption responses to income shocks in the presence of nonlinear persistence," Journal of Econometrics, Elsevier, vol. 240(2).
    11. Jaehee Kim & Sooyoung Cheon, 2010. "A Bayesian regime‐switching time‐series model," Journal of Time Series Analysis, Wiley Blackwell, vol. 31(5), pages 365-378, September.
    12. Diana Giurghita & Dirk Husmeier, 2018. "Statistical modelling of cell movement," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 265-280, August.
    13. Joshua Chan & Arnaud Doucet & Roberto León-González & Rodney W. Strachan, 2018. "Multivariate stochastic volatility with co-heteroscedasticity," CAMA Working Papers 2018-52, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    14. McKinley, Trevelyan J. & Ross, Joshua V. & Deardon, Rob & Cook, Alex R., 2014. "Simulation-based Bayesian inference for epidemic models," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 434-447.
    15. Arnaud Dufays, 2016. "Evolutionary Sequential Monte Carlo Samplers for Change-Point Models," Econometrics, MDPI, vol. 4(1), pages 1-33, March.
    16. Giesecke, K. & Schwenkler, G., 2019. "Simulated likelihood estimators for discretely observed jump–diffusions," Journal of Econometrics, Elsevier, vol. 213(2), pages 297-320.
    17. Andrew Hoegh & Frank T. Manen & Mark Haroldson, 2021. "Agent-Based Models for Collective Animal Movement: Proximity-Induced State Switching," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 26(4), pages 560-579, December.
    18. Kouritzin, Michael A., 2017. "Residual and stratified branching particle filters," Computational Statistics & Data Analysis, Elsevier, vol. 111(C), pages 145-165.
    19. Lux, Thomas, 2020. "Bayesian estimation of agent-based models via adaptive particle Markov chain Monte Carlo," Economics Working Papers 2020-01, Christian-Albrechts-University of Kiel, Department of Economics.
    20. Zhang, Yixiao & Yu, Cindy L. & Li, Haitao, 2022. "Nowcasting GDP Using Dynamic Factor Model with Unknown Number of Factors and Stochastic Volatility: A Bayesian Approach," Econometrics and Statistics, Elsevier, vol. 24(C), pages 75-93.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:111:y:2016:i:513:p:377-393. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.