IDEAS home Printed from https://ideas.repec.org/a/wly/apsmbi/v36y2020i5p783-794.html
   My bibliography  Save this article

Covariate‐dependent control limits for the detection of abnormal price changes in scanner data

Author

Listed:
  • Youngrae Kim
  • Sangkyun Kim
  • Johan Lim
  • Sungim Lee
  • Won Son
  • Heejin Hwang

Abstract

Currently, large‐scale data for consumer goods, called scanner data, are obtained by scanning the bar codes of individual products at the points of sale of retail outlets. Many national statistical offices use scanner data to build consumer price statistics. In this process, as in other statistical procedures, the detection of abnormal transactions in sales prices is an important step in the analysis. Popular methods for conducting such outlier detection are the quartile method, the Hidiroglou‐Berthelot method, the resistant fences method, and the Tukey algorithm. These methods are based solely on information about price changes and not on any of the other covariates (eg, sales volume or types of retail shops) that are also available from scanner data. In this article, we propose a new method to detect abnormal price changes that take into account an additional covariate, namely, sales volume. We assume that the variance of the log of the price change is a smooth function of the sales volume and estimate the function from previously observed data. We numerically show the advantages of the new method over existing methods. We also apply the methods to real scanner data collected at weekly intervals by the Korean Chamber of Commerce and Industry between 2013 and 2014 and compare their performance.

Suggested Citation

  • Youngrae Kim & Sangkyun Kim & Johan Lim & Sungim Lee & Won Son & Heejin Hwang, 2020. "Covariate‐dependent control limits for the detection of abnormal price changes in scanner data," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 36(5), pages 783-794, September.
    • Handle: RePEc:wly:apsmbi:v:36:y:2020:i:5:p:783-794
    DOI: 10.1002/asmb.2529
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asmb.2529
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asmb.2529?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Fan, Jianqing & Yao, Qiwei, 1998. "Efficient estimation of conditional variance functions in stochastic regression," LSE Research Online Documents on Economics 6635, London School of Economics and Political Science, LSE Library.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chernozhukov, Victor & Fernández-Val, Iván & Hoderlein, Stefan & Holzmann, Hajo & Newey, Whitney, 2015. "Nonparametric identification in panels using quantiles," Journal of Econometrics, Elsevier, vol. 188(2), pages 378-392.
    2. Hoderlein, Stefan & White, Halbert, 2012. "Nonparametric identification in nonseparable panel data models with generalized fixed effects," Journal of Econometrics, Elsevier, vol. 168(2), pages 300-314.
    3. Franke, Jurgen & Neumann, Michael H. & Stockis, Jean-Pierre, 2004. "Bootstrapping nonparametric estimators of the volatility function," Journal of Econometrics, Elsevier, vol. 118(1-2), pages 189-218.
    4. Jing Lv & Chaohui Guo, 2017. "Efficient parameter estimation via modified Cholesky decomposition for quantile regression with longitudinal data," Computational Statistics, Springer, vol. 32(3), pages 947-975, September.
    5. Martins-Filho, Carlos & Yao, Feng & Torero, Maximo, 2018. "Nonparametric Estimation Of Conditional Value-At-Risk And Expected Shortfall Based On Extreme Value Theory," Econometric Theory, Cambridge University Press, vol. 34(1), pages 23-67, February.
    6. Yanchun Jin, 2016. "Nonparametric tests for the effect of treatment on conditional variance," KIER Working Papers 948, Kyoto University, Institute of Economic Research.
    7. Holger Dette & Kay Pilz, 2009. "On the estimation of a monotone conditional variance in nonparametric regression," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 61(1), pages 111-141, March.
    8. Dingshi Tian & Zongwu Cai & Ying Fang, 2018. "Econometric Modeling of Risk Measures: A Selective Review of the Recent Literature," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 201807, University of Kansas, Department of Economics, revised Oct 2018.
    9. Otsu, Taisuke & Xu, Ke-Li & Matsushita, Yukitoshi, 2015. "Empirical likelihood for regression discontinuity design," Journal of Econometrics, Elsevier, vol. 186(1), pages 94-112.
    10. Sami MESTIRI, 2022. "Modeling the volatility of Bitcoin returns using Nonparametric GARCH models," Journal of Academic Finance, RED research unit, university of Gabes, Tunisia, vol. 13(1), pages 2-16, June.
    11. Čížek, Pavel & Koo, Chao Hui, 2021. "Jump-preserving varying-coefficient models for nonlinear time series," Econometrics and Statistics, Elsevier, vol. 19(C), pages 58-96.
    12. Nicoleta Serban, 2008. "Estimating and clustering curves in the presence of heteroscedastic errors," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 20(7), pages 553-571.
    13. Dette, Holger & Pilz, Kay F., 2004. "On the estimation of a monotone conditional variance in nonparametric regression," Technical Reports 2004,42, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    14. Heather L. R. Tierney, 2019. "Forecasting with the Nonparametric Exclusion-from-Core Inflation Persistence Model Using Real-Time Data," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 25(1), pages 39-63, February.
    15. Likai Chen & Ekaterina Smetanina & Wei Biao Wu, 2022. "Estimation of nonstationary nonparametric regression model with multiplicative structure [Income and wealth distribution in macroeconomics: A continuous-time approach]," The Econometrics Journal, Royal Economic Society, vol. 25(1), pages 176-214.
    16. Holger Dette & Mareen Marchlewski & Jens Wagener, 2012. "Testing for a constant coefficient of variation in nonparametric regression by empirical processes," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(5), pages 1045-1070, October.
    17. Tran, Kien C. & Tsionas, Efthymios G., 2009. "Estimation of nonparametric inefficiency effects stochastic frontier models with an application to British manufacturing," Economic Modelling, Elsevier, vol. 26(5), pages 904-909, September.
    18. Christina Korting & Carl Lieberman & Jordan Matsudaira & Zhuan Pei & Yi Shen, 2023. "Visual Inference and Graphical Representation in Regression Discontinuity Designs," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(3), pages 1977-2019.
    19. León Beleña & Ernesto Curbelo & Luca Martino & Valero Laparra, 2024. "Second-Moment/Order Approximations by Kernel Smoothers with Application to Volatility Estimation," Mathematics, MDPI, vol. 12(9), pages 1-15, May.
    20. Yuan, Ming & Wahba, Grace, 2004. "Doubly penalized likelihood estimator in heteroscedastic regression," Statistics & Probability Letters, Elsevier, vol. 69(1), pages 11-20, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:apsmbi:v:36:y:2020:i:5:p:783-794. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1526-4025 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.