IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v35y2023i6p1491-1510.html
   My bibliography  Save this article

ℓ 0 Trend Filtering

Author

Listed:
  • Canhong Wen

    (International Institute of Finance, School of Management, University of Science and Technology of China, Hefei 230000, China)

  • Xueqin Wang

    (International Institute of Finance, School of Management, University of Science and Technology of China, Hefei 230000, China)

  • Aijun Zhang

    (Department of Statistics and Actuarial Science, The University of Hong Kong, 999077 Hong Kong)

Abstract

The ℓ 0 trend filtering ( ℓ 0 -TF) is a new effective tool for nonparametric regression with the power of automatic knot detection in function values or derivatives. It overcomes the drawback of ℓ 1 -TF that is known to have bias issues. To solve the ℓ 0 -TF problem, we propose an alternating minimization induced active set (AMIAS) search method based on the necessary optimality conditions derived from an augmented Lagrangian framework. The proposed method takes full advantage of the primal and dual variables with complementary supports, and decouples the high-dimensional problem into two subsystems on the active and inactive sets, respectively. A sequential AMIAS algorithm with warm start initialization is developed for efficient determination of the cardinality parameter, along with the output of solution paths. Theoretically, the oracle estimator of ℓ 0 -TF is justified to behave like regression splines under the continuous time setting with mild conditions. Our numerical experiments include simulation studies for comparing ℓ 0 -TF to ℓ 1 -TF and free-knot splines on several synthetic examples, and a real data application of time series segmentation on Hong Kong PM2.5 indexes.

Suggested Citation

  • Canhong Wen & Xueqin Wang & Aijun Zhang, 2023. "ℓ 0 Trend Filtering," INFORMS Journal on Computing, INFORMS, vol. 35(6), pages 1491-1510, November.
    • Handle: RePEc:inm:orijoc:v:35:y:2023:i:6:p:1491-1510
    DOI: 10.1287/ijoc.2021.0313
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2021.0313
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2021.0313?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    2. Peter C. B. Phillips & Zhentao Shi, 2021. "Boosting: Why You Can Use The Hp Filter," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 62(2), pages 521-570, May.
    3. Hodrick, Robert J & Prescott, Edward C, 1997. "Postwar U.S. Business Cycles: An Empirical Investigation," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 29(1), pages 1-16, February.
    4. Yao, Yi-Ching, 1988. "Estimating the number of change-points via Schwarz' criterion," Statistics & Probability Letters, Elsevier, vol. 6(3), pages 181-189, February.
    5. Rafal Baranowski & Yining Chen & Piotr Fryzlewicz, 2019. "Narrowest‐over‐threshold detection of multiple change points and change‐point‐like features," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 81(3), pages 649-672, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cho, Haeran & Kirch, Claudia, 2024. "Data segmentation algorithms: Univariate mean change and beyond," Econometrics and Statistics, Elsevier, vol. 30(C), pages 76-95.
    2. Hall, Viv B & Thomson, Peter, 2022. "A boosted HP filter for business cycle analysis: evidence from New Zealand’s small open economy," Working Paper Series 9473, Victoria University of Wellington, School of Economics and Finance.
    3. Morana, Claudio, 2024. "A new macro-financial condition index for the euro area," Econometrics and Statistics, Elsevier, vol. 29(C), pages 64-87.
    4. Viv B. Hall & Peter Thomson, 2021. "Does Hamilton’s OLS Regression Provide a “better alternative” to the Hodrick-Prescott Filter? A New Zealand Business Cycle Perspective," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 17(2), pages 151-183, November.
    5. Melina Dritsaki & Chaido Dritsaki, 2022. "Comparison of HP Filter and the Hamilton’s Regression," Mathematics, MDPI, vol. 10(8), pages 1-18, April.
    6. Baffes, John & Kabundi, Alain, 2023. "Commodity price shocks: Order within chaos?," Resources Policy, Elsevier, vol. 83(C).
    7. Fryzlewicz, Piotr, 2020. "Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection," LSE Research Online Documents on Economics 103430, London School of Economics and Political Science, LSE Library.
    8. Venkata Jandhyala & Stergios Fotopoulos & Ian MacNeill & Pengyu Liu, 2013. "Inference for single and multiple change-points in time series," Journal of Time Series Analysis, Wiley Blackwell, vol. 34(4), pages 423-446, July.
    9. Lee, Sokbae & Liao, Yuan & Seo, Myung Hwan & Shin, Youngki, 2021. "Sparse HP filter: Finding kinks in the COVID-19 contact rate," Journal of Econometrics, Elsevier, vol. 220(1), pages 158-180.
    10. Haitham A. Al-Zoubi, 2024. "An affine model for short rates when monetary policy is path dependent," Review of Derivatives Research, Springer, vol. 27(2), pages 151-201, July.
    11. Todd E. Clark & Michael W. McCracken, 2006. "Forecasting of small macroeconomic VARs in the presence of instabilities," Research Working Paper RWP 06-09, Federal Reserve Bank of Kansas City.
    12. Baffes, John & Kabundi, Alain, 2024. "Do supercycles dominate commodity price movements?," Economics Letters, Elsevier, vol. 237(C).
    13. Giuliano Queiroz Ferreira & Leonardo Bornacki Mattos, 2022. "Regime-dependent price puzzle in the Brazilian economy: evidence from VAR and FAVAR approaches," SN Business & Economics, Springer, vol. 2(9), pages 1-28, September.
    14. Chao Du & Chu-Lan Michael Kao & S. C. Kou, 2016. "Stepwise Signal Extraction via Marginal Likelihood," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 314-330, March.
    15. Haeran Cho & Claudia Kirch, 2022. "Two-stage data segmentation permitting multiscale change points, heavy tails and dependence," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(4), pages 653-684, August.
    16. McGonigle, Euan T. & Cho, Haeran, 2023. "Robust multiscale estimation of time-average variance for time series segmentation," Computational Statistics & Data Analysis, Elsevier, vol. 179(C).
    17. Viv B Hall & Peter Thomson, 2020. "Does Hamilton’s OLS regression provide a “better alternative†to the Hodrick-Prescott filter? A New Zealand business cycle perspective," CAMA Working Papers 2020-71, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    18. Hiroshi Yamada & Ruixue Du, 2018. "Some Results on ℓ 1 Polynomial Trend Filtering," Econometrics, MDPI, vol. 6(3), pages 1-10, July.
    19. Holger Dette & Theresa Eckle & Mathias Vetter, 2020. "Multiscale change point detection for dependent data," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1243-1274, December.
    20. Won Son & Johan Lim & Donghyeon Yu, 2023. "Path algorithms for fused lasso signal approximator with application to COVID‐19 spread in Korea," International Statistical Review, International Statistical Institute, vol. 91(2), pages 218-242, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:35:y:2023:i:6:p:1491-1510. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.