IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v138y2019icp13-26.html
   My bibliography  Save this article

A flexible sequential Monte Carlo algorithm for parametric constrained regression

Author

Listed:
  • Ng, Kenyon
  • Turlach, Berwin A.
  • Murray, Kevin

Abstract

An algorithm is proposed that enables the imposition of shape constraints on regression curves, without requiring the constraints to be written as closed-form expressions, nor assuming the functional form of the loss function. This algorithm is based on Sequential Monte Carlo–SimulatedAnnealing and only relies on an indicator function that assesses whether or not the constraints are fulfilled, thus allowing the enforcement of various complex constraints by specifying an appropriate indicator function without altering other parts of the algorithm. The algorithm is illustrated by fitting rational function and B-spline regression models subject to a monotonicity constraint. An implementation of the algorithm using R is freely available on GitHub.

Suggested Citation

  • Ng, Kenyon & Turlach, Berwin A. & Murray, Kevin, 2019. "A flexible sequential Monte Carlo algorithm for parametric constrained regression," Computational Statistics & Data Analysis, Elsevier, vol. 138(C), pages 13-26.
    • Handle: RePEc:eee:csdana:v:138:y:2019:i:c:p:13-26
    DOI: 10.1016/j.csda.2019.03.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947319300787
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2019.03.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pierre Del Moral & Arnaud Doucet & Ajay Jasra, 2006. "Sequential Monte Carlo samplers," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 411-436, June.
    2. J. O. Ramsay, 1998. "Estimating smooth monotone functions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 365-375.
    3. Matzkin, Rosa L, 1991. "Semiparametric Estimation of Monotone and Concave Utility Functions for Polychotomous Choice Models," Econometrica, Econometric Society, vol. 59(5), pages 1315-1327, September.
    4. Faming Liang & Yichen Cheng & Guang Lin, 2014. "Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 847-863, June.
    5. Wolters, Mark A., 2012. "A particle swarm algorithm with broad applicability in shape-constrained estimation," Computational Statistics & Data Analysis, Elsevier, vol. 56(10), pages 2965-2975.
    6. Enlu Zhou & Xi Chen, 2013. "Sequential Monte Carlo simulated annealing," Journal of Global Optimization, Springer, vol. 55(1), pages 101-124, January.
    7. Roberto Andreani & Ellen H. Fukuda & Paulo J. S. Silva, 2013. "A Gauss–Newton Approach for Solving Constrained Optimization Problems Using Differentiable Exact Penalties," Journal of Optimization Theory and Applications, Springer, vol. 156(2), pages 417-449, February.
    8. Dimitris Fouskakis & David Draper, 2002. "Stochastic Optimization: a Review," International Statistical Review, International Statistical Institute, vol. 70(3), pages 315-349, December.
    9. Hazelton, Martin L. & Turlach, Berwin A., 2011. "Semiparametric regression with shape-constrained penalized splines," Computational Statistics & Data Analysis, Elsevier, vol. 55(10), pages 2871-2879, October.
    10. Kevin Murray & Samuel Müller & Berwin Turlach, 2013. "Revisiting fitting monotone polynomials to data," Computational Statistics, Springer, vol. 28(5), pages 1989-2005, October.
    11. Mary Meyer & Amber Hackstadt & Jennifer Hoeting, 2011. "Bayesian estimation and inference for generalised partial linear models using shape-restricted splines," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 23(4), pages 867-884.
    12. M. Locatelli, 2000. "Simulated Annealing Algorithms for Continuous Global Optimization: Convergence Conditions," Journal of Optimization Theory and Applications, Springer, vol. 104(1), pages 121-133, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abdelaati Daouia & Hohsuk Noh & Byeong U. Park, 2016. "Data envelope fitting with constrained polynomial splines," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 3-30, January.
    2. Geweke, John & Durham, Garland, 2019. "Sequentially adaptive Bayesian learning algorithms for inference and optimization," Journal of Econometrics, Elsevier, vol. 210(1), pages 4-25.
    3. Wu, Ximing & Sickles, Robin, 2018. "Semiparametric estimation under shape constraints," Econometrics and Statistics, Elsevier, vol. 6(C), pages 74-89.
    4. Enlu Zhou & Xi Chen, 2013. "Sequential Monte Carlo simulated annealing," Journal of Global Optimization, Springer, vol. 55(1), pages 101-124, January.
    5. Taeryon Choi & Hea-Jung Kim & Seongil Jo, 2016. "Bayesian variable selection approach to a Bernstein polynomial regression model with stochastic constraints," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(15), pages 2751-2771, November.
    6. Christophe Abraham & Khader Khadraoui, 2015. "Bayesian regression with B-splines under combinations of shape constraints and smoothness properties," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 69(2), pages 150-170, May.
    7. Hazelton, Martin L. & Turlach, Berwin A., 2011. "Semiparametric regression with shape-constrained penalized splines," Computational Statistics & Data Analysis, Elsevier, vol. 55(10), pages 2871-2879, October.
    8. Minggen Lu, 2015. "Spline estimation of generalised monotonic regression," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 27(1), pages 19-39, March.
    9. Wang, Fan & Zhang, Shengfan & Henderson, Louise M., 2018. "Adaptive decision-making of breast cancer mammography screening: A heuristic-based regression model," Omega, Elsevier, vol. 76(C), pages 70-84.
    10. Bin Liu, 2017. "Posterior exploration based sequential Monte Carlo for global optimization," Journal of Global Optimization, Springer, vol. 69(4), pages 847-868, December.
    11. Kevin Murray & Samuel Müller & Berwin Turlach, 2013. "Revisiting fitting monotone polynomials to data," Computational Statistics, Springer, vol. 28(5), pages 1989-2005, October.
    12. Golchi, Shirin & Campbell, David A., 2016. "Sequentially Constrained Monte Carlo," Computational Statistics & Data Analysis, Elsevier, vol. 97(C), pages 98-113.
    13. Yining Chen & Richard J. Samworth, 2016. "Generalized additive and index models with shape constraints," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(4), pages 729-754, September.
    14. James Martin & Ajay Jasra & Emma McCoy, 2013. "Inference for a class of partially observed point process models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 65(3), pages 413-437, June.
    15. M. Bierlaire & M. Thémans & N. Zufferey, 2010. "A Heuristic for Nonlinear Global Optimization," INFORMS Journal on Computing, INFORMS, vol. 22(1), pages 59-70, February.
    16. Weitao Sun & Yuan Dong, 2011. "Study of multiscale global optimization based on parameter space partition," Journal of Global Optimization, Springer, vol. 49(1), pages 149-172, January.
    17. John Geweke, 2016. "Sequentially Adaptive Bayesian Learning for a Nonlinear Model of the Secular and Cyclical Behavior of US Real GDP," Econometrics, MDPI, vol. 4(1), pages 1-23, March.
    18. Gerber, Mathieu & Bornn, Luke, 2018. "Convergence results for a class of time-varying simulated annealing algorithms," Stochastic Processes and their Applications, Elsevier, vol. 128(4), pages 1073-1094.
    19. Asghari, M. & Afshari, H. & Jaber, M.Y. & Searcy, C., 2023. "Credibility-based cascading approach to achieve net-zero emissions in energy symbiosis networks using an Organic Rankine Cycle," Applied Energy, Elsevier, vol. 340(C).
    20. Lee Anthony & Caron Francois & Doucet Arnaud & Holmes Chris, 2012. "Bayesian Sparsity-Path-Analysis of Genetic Association Signal using Generalized t Priors," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(2), pages 1-31, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:138:y:2019:i:c:p:13-26. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.