IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i20p3190-d1496926.html
   My bibliography  Save this article

Smooth Sigmoid Surrogate (SSS): An Alternative to Greedy Search in Decision Trees

Author

Listed:
  • Xiaogang Su

    (Department of Mathematical Science, University of Texas at El Paso, El Paso, TX 79968, USA)

  • George Ekow Quaye

    (Division of Health Services and Outcomes Research, Children’s Mercy Kansas City, Kansas City, MO 64108, USA)

  • Yishu Wei

    (Reddit Inc., San Francisco, CA 94102, USA)

  • Joseph Kang

    (US Census Bureau, Washington, DC 20233, USA)

  • Lei Liu

    (Division of Biostatistics, Washington University in St. Louis, St. Louis, MO 63110, USA)

  • Qiong Yang

    (Department of Biostatistics, School of Public Health, Boston University, Boston, MA 02118, USA)

  • Juanjuan Fan

    (Department of Mathematics and Statistics, San Diego State University, San Diego, CA 92182, USA)

  • Richard A. Levine

    (Department of Mathematics and Statistics, San Diego State University, San Diego, CA 92182, USA)

Abstract

Greedy search (GS) or exhaustive search plays a crucial role in decision trees and their various extensions. We introduce an alternative splitting method called smooth sigmoid surrogate (SSS) in which the indicator threshold function used in GS is approximated by a smooth sigmoid function. This approach allows for parametric smoothing or regularization of the erratic and discrete GS process, making it more effective in identifying the true cutoff point, particularly in the presence of weak signals, as well as less prone to the inherent end-cut preference problem. Additionally, SSS provides a convenient means of evaluating the best split by referencing a parametric nonlinear model. Moreover, in many variants of recursive partitioning, SSS can be reformulated as a one-dimensional smooth optimization problem, rendering it computationally more efficient than GS. Extensive simulation studies and real data examples are provided to evaluate and demonstrate its effectiveness.

Suggested Citation

  • Xiaogang Su & George Ekow Quaye & Yishu Wei & Joseph Kang & Lei Liu & Qiong Yang & Juanjuan Fan & Richard A. Levine, 2024. "Smooth Sigmoid Surrogate (SSS): An Alternative to Greedy Search in Decision Trees," Mathematics, MDPI, vol. 12(20), pages 1-28, October.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:20:p:3190-:d:1496926
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/20/3190/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/20/3190/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Torsten Hothorn & Achim Zeileis, 2008. "Generalized Maximally Selected Statistics," Biometrics, The International Biometric Society, vol. 64(4), pages 1263-1269, December.
    2. Jushan Bai, 1997. "Estimation Of A Change Point In Multiple Regression Models," The Review of Economics and Statistics, MIT Press, vol. 79(4), pages 551-563, November.
    3. Hothorn, Torsten & Lausen, Berthold, 2003. "On the exact distribution of maximally selected rank statistics," Computational Statistics & Data Analysis, Elsevier, vol. 43(2), pages 121-137, June.
    4. Shih, Yu-Shan & Tsai, Hsin-Wen, 2004. "Variable selection bias in regression trees with constant fits," Computational Statistics & Data Analysis, Elsevier, vol. 45(3), pages 595-607, April.
    5. Xiaogang Su & Karen Meneses & Patrick McNees & Wesley O. Johnson, 2011. "Interaction trees: exploring the differential effects of an intervention programme for breast cancer survivors," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 60(3), pages 457-474, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gerhard Tutz & Moritz Berger, 2016. "Item-focussed Trees for the Identification of Items in Differential Item Functioning," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 727-750, September.
    2. Achim Zeileis & Torsten Hothorn, 2013. "A toolbox of permutation tests for structural change," Statistical Papers, Springer, vol. 54(4), pages 931-954, November.
    3. Carolin Strobl & Julia Kopf & Achim Zeileis, 2015. "Rasch Trees: A New Method for Detecting Differential Item Functioning in the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 289-316, June.
    4. Strobl, Carolin & Boulesteix, Anne-Laure & Augustin, Thomas, 2007. "Unbiased split selection for classification trees based on the Gini Index," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 483-501, September.
    5. Umar, Muhammad & Su, Chi-Wei & Rizvi, Syed Kumail Abbas & Lobonţ, Oana-Ramona, 2021. "Driven by fundamentals or exploded by emotions: Detecting bubbles in oil prices," Energy, Elsevier, vol. 231(C).
    6. Felix Pretis & Michael Mann & Robert Kaufmann, 2015. "Testing competing models of the temperature hiatus: assessing the effects of conditioning variables and temporal uncertainties through sample-wide break detection," Climatic Change, Springer, vol. 131(4), pages 705-718, August.
    7. Koo, Bonsoo & Seo, Myung Hwan, 2015. "Structural-break models under mis-specification: Implications for forecasting," Journal of Econometrics, Elsevier, vol. 188(1), pages 166-181.
    8. Martin T. Bohl & Alexander Pütz & Pierre L. Siklos & Christoph Sulewski, 2018. "Information Transmission under Increasing Political Tension – Evidence for the Berlin Produce Exchange 1887-1896," CQE Working Papers 7618, Center for Quantitative Economics (CQE), University of Muenster.
    9. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    10. Ronald Bewley & Minxian Yang, 2006. "A hybrid forecasting approach for piece-wise stationary time series," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(7), pages 513-527.
    11. Du, Xiaodong & Yu, Cindy L. & Hayes, Dermot J., 2011. "Speculation and volatility spillover in the crude oil and agricultural commodity markets: A Bayesian analysis," Energy Economics, Elsevier, vol. 33(3), pages 497-503, May.
    12. Marotta, Giuseppe, 2009. "Structural breaks in the lending interest rate pass-through and the euro," Economic Modelling, Elsevier, vol. 26(1), pages 191-205, January.
    13. Luca Benati, 2003. "Evolving Post-World War II U.K. Economic Performance," Computing in Economics and Finance 2003 171, Society for Computational Economics.
    14. Boldea, Otilia & Hall, Alastair R., 2013. "Estimation and inference in unstable nonlinear least squares models," Journal of Econometrics, Elsevier, vol. 172(1), pages 158-167.
    15. Pastore, Chiara & Jones, Andrew M., 2023. "Human capital consequences of missing out on a grammar school education," Economic Modelling, Elsevier, vol. 126(C).
    16. Alastair R. Hall & Denise R. Osborn & Nikolaos Sakkas, 2017. "The asymptotic behaviour of the residual sum of squares in models with multiple break points," Econometric Reviews, Taylor & Francis Journals, vol. 36(6-9), pages 667-698, October.
    17. Timothy Besley & Thiemo Fetzer & Hannes Mueller, 2015. "The Welfare Cost Of Lawlessness: Evidence From Somali Piracy," Journal of the European Economic Association, European Economic Association, vol. 13(2), pages 203-239, April.
    18. Xu Cheng & Zhipeng Liao & Frank Schorfheide, 2016. "Shrinkage Estimation of High-Dimensional Factor Models with Structural Instabilities," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 83(4), pages 1511-1543.
    19. Magazzino, Cosimo & Drago, Carlo & Schneider, Nicolas, 2023. "Evidence of supply security and sustainability challenges in Nigeria’s power sector," Utilities Policy, Elsevier, vol. 82(C).
    20. D R Osborn & M Sensier, 2004. "Modelling UK Inflation: Persistence, Seasonality and Monetary Policy," Centre for Growth and Business Cycle Research Discussion Paper Series 46, Economics, The University of Manchester.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:20:p:3190-:d:1496926. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.