IDEAS home Printed from https://ideas.repec.org/p/boj/bojwps/wp18e13.html
   My bibliography  Save this paper

Compilation of Experimental Price Indices Using Big Data and Machine Learning:A Comparative Analysis and Validity Verification of Quality Adjustments

Author

Listed:
  • Nobuhiro Abe

    (Bank of Japan)

  • Kimiaki Shinozaki

    (Bank of Japan)

Abstract

This paper compiles experimental price indices for 20 home electrical appliances and digital consumer electronic products using big data obtained from Kakaku.com, the largest price comparison website in Japan, and a machine-learning algorithm which pairs legacy and successor products with high precision. In so doing, authors examine the validity of quality adjustment methods by performing comparative analyses on the difference these methods have on price indices. Findings from the analyses are as follows: Indices applied with the Webscraped Prices Comparison Method--the quality adjustment method newly developed and introduced by the Bank of Japan--are more cost-effective than those applied with the Hedonic Regression Method which is known to possess high accuracy in index creation. Indices applied with the Matched-Model Method, which is frequently applied to price indices using big data is unable to precisely reflect price increases intended to ensure the profitability often seen in home electronics at time of product turnover. This indicates the significant downward bias in price indices. These findings once again highlight the importance of selecting the appropriate quality adjustment method when compiling price indices.

Suggested Citation

  • Nobuhiro Abe & Kimiaki Shinozaki, 2018. "Compilation of Experimental Price Indices Using Big Data and Machine Learning:A Comparative Analysis and Validity Verification of Quality Adjustments," Bank of Japan Working Paper Series 18-E-13, Bank of Japan.
    • Handle: RePEc:boj:bojwps:wp18e13
    as

    Download full text from publisher

    File URL: http://www.boj.or.jp/en/research/wps_rev/wps_2018/data/wp18e13.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kozo Ueda & Kota Watanabe & Tsutomu Watanabe, 2016. "Product Turnover and Deflation: Evidence from Japan," CARF F-Series CARF-F-400, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    2. Gautam Gowrisankaran & Marc Rysman, 2012. "Dynamics of Consumer Demand for New Durable Goods," Journal of Political Economy, University of Chicago Press, vol. 120(6), pages 1173-1219.
    3. Alberto Cavallo & Roberto Rigobon, 2016. "The Billion Prices Project: Using Online Prices for Measurement and Research," Journal of Economic Perspectives, American Economic Association, vol. 30(2), pages 151-178, Spring.
    4. Ana M. Aizcorbe & Carol Corrado & Mark Doms, 2003. "When do matched-model and hedonic techniques yield similar measures?," Working Paper Series 2003-14, Federal Reserve Bank of San Francisco.
    5. Daniel Melser & Iqbal A. Syed, 2016. "Life Cycle Price Trends and Product Replacement: Implications for the Measurement of Inflation," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 62(3), pages 509-533, September.
    6. Nobuhiro Abe & Yojiro Ito & Ko Munakata & Shinsuke Ohyama & Kimiaki Shinozaki, 2016. "Pricing Patterns over Product Life-Cycle and Quality Growth at Product Turnover: Empirical Evidence from Japan," Bank of Japan Working Paper Series 16-E-5, Bank of Japan.
    7. Abe, Naohito & Enda, Toshiki & Inakura, Noriko & Tonogi, Akiyuki, 2015. "Effects of New Goods and Product Turnover on Price Indexes," RCESR Discussion Paper Series DP15-2, Research Center for Economic and Social Risks, Institute of Economic Research, Hitotsubashi University.
    8. Research and Statistics Department, 2017. "Rebasing the Corporate Goods Price Index to the Base Year 2015 -- Main features of the rebasing and price developments in the 2015 base index --," Bank of Japan Research Papers 17-02-03, Bank of Japan.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dmytro Krukovets, 2020. "Data Science Opportunities at Central Banks: Overview," Visnyk of the National Bank of Ukraine, National Bank of Ukraine, issue 249, pages 13-24.
    2. Bogdan Oancea, 2023. "Automatic Product Classification Using Supervised Machine Learning Algorithms in Price Statistics," Mathematics, MDPI, vol. 11(7), pages 1-32, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuki Teranishi, 2017. "Product Cycles and Prices:Search Foundation," UTokyo Price Project Working Paper Series 079, University of Tokyo, Graduate School of Economics.
    2. Kozo Ueda & Kota Watanabe & Tsutomu Watanabe, 2016. "Product Turnover and Deflation: Evidence from Japan," UTokyo Price Project Working Paper Series 073, University of Tokyo, Graduate School of Economics.
    3. Nobuhiro Abe & Yojiro Ito & Ko Munakata & Shinsuke Ohyama & Kimiaki Shinozaki, 2016. "Pricing Patterns over Product Life-Cycle and Quality Growth at Product Turnover: Empirical Evidence from Japan," Bank of Japan Working Paper Series 16-E-5, Bank of Japan.
    4. Kozo Ueda & Kota Watanabe & Tsutomu Watanabe, 2018. "Product Turnover and the Cost of Living Index: Quality vs. Fashion Effects," Globalization Institute Working Papers 337, Federal Reserve Bank of Dallas.
    5. David Argente & Munseob Lee & Sara Moreira, 2018. "How do Firms Grow? The Life Cycle of Products Matters," 2018 Meeting Papers 1174, Society for Economic Dynamics.
    6. repec:grz:wpaper:2016-13 is not listed on IDEAS
    7. Crystal G. Konny & Brendan K. Williams & David M. Friedman, 2019. "Big Data in the US Consumer Price Index: Experiences and Plans," NBER Chapters, in: Big Data for Twenty-First-Century Economic Statistics, pages 69-98, National Bureau of Economic Research, Inc.
    8. Jan de Haan & Rens Hendriks & Michael Scholz, 2021. "Price Measurement Using Scanner Data: Time‐Product Dummy Versus Time Dummy Hedonic Indexes," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 67(2), pages 394-417, June.
    9. Fernando Alvarez & Francesco Lippi & Juan Passadore, 2017. "Are State- and Time-Dependent Models Really Different?," NBER Macroeconomics Annual, University of Chicago Press, vol. 31(1), pages 379-457.
    10. Klaus Adam & Henning Weber, 2023. "Estimating the Optimal Inflation Target from Trends in Relative Prices," American Economic Journal: Macroeconomics, American Economic Association, vol. 15(3), pages 1-42, July.
    11. Fernando Alvarez & Francesco Lippi, 2020. "Temporary Price Changes, Inflation Regimes, and the Propagation of Monetary Shocks," American Economic Journal: Macroeconomics, American Economic Association, vol. 12(1), pages 104-152, January.
    12. John M. Abowd & Ian M. Schmutte & William Sexton & Lars Vilhuber, 2019. "Suboptimal Provision of Privacy and Statistical Accuracy When They are Public Goods," Papers 1906.09353, arXiv.org.
    13. Mira Frick & Ryota Iijima & Tomasz Strzalecki, 2019. "Dynamic Random Utility," Econometrica, Econometric Society, vol. 87(6), pages 1941-2002, November.
    14. Tsutomu Watanabe & Tomoyoshi Yabu, 2023. "The demand for money at the zero interest rate bound," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(6), pages 968-976, September.
    15. Néstor Duch-Brown & Lukasz Grzybowski & André Romahn & Frank Verboven, 2023. "Evaluating the Impact of Online Market Integration—Evidence from the EU Portable PC Market," American Economic Journal: Microeconomics, American Economic Association, vol. 15(4), pages 268-305, November.
    16. Erica L. Groshen & Brian C. Moyer & Ana M. Aizcorbe & Ralph Bradley & David M. Friedman, 2017. "How Government Statistics Adjust for Potential Biases from Quality Change and New Goods in an Age of Digital Technologies: A View from the Trenches," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 187-210, Spring.
    17. Patrick Bajari & Zhihao Cen & Victor Chernozhukov & Manoj Manukonda & Jin Wang & Ramon Huerta & Junbo Li & Ling Leng & George Monokroussos & Suhas Vijaykunar & Shan Wan, 2023. "Hedonic prices and quality adjusted price indices powered by AI," CeMMAP working papers 08/23, Institute for Fiscal Studies.
    18. Hui Li & Yijin Kim & Kannan Srinivasan, 2022. "Market Shifts in the Sharing Economy: The Impact of Airbnb on Housing Rentals," Management Science, INFORMS, vol. 68(11), pages 8015-8044, November.
    19. Steven T. Berry & Philip A. Haile, 2021. "Foundations of Demand Estimation," Cowles Foundation Discussion Papers 2301, Cowles Foundation for Research in Economics, Yale University.
    20. Kozo Ueda & Kota Watanabe & Tsutomu Watanabe, 2021. "Household Inventory, Temporary Sales, and Price Indices," CARF F-Series CARF-F-520, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.

    More about this item

    Keywords

    price index; quality adjustment method; hedonic approach; support vector machine;
    All these keywords.

    JEL classification:

    • C43 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Index Numbers and Aggregation
    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • E31 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Price Level; Inflation; Deflation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boj:bojwps:wp18e13. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bank of Japan (email available below). General contact details of provider: https://edirc.repec.org/data/bojgvjp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.