IDEAS home Printed from https://ideas.repec.org/a/spr/astaws/v18y2024i2d10.1007_s11943-024-00340-6.html
   My bibliography  Save this article

Flat rent price prediction in Berlin with web scraping

Author

Listed:
  • Camilo Meyberg

    (Freie Universität Berlin)

  • Ulrich Rendtel

    (Freie Universität Berlin)

  • Holger Leerhoff

    (Amt für Statistik Berlin-Brandenburg)

Abstract

Internet data pose a challenge to the traditional system of official statistics, which relies on more conventional sources such as surveys and registers, not readily adaptable to rapid changes. Expanding this system to include internet data is currently at an experimental stage, exploring these sources’ potentials and benefits. This paper describes a project conducted within the ESSnet Trusted Smart Statistics – Web Intelligence Network framework. It investigates the use of online apartment listings to analyze the rental market. We used web scraping to extract information from two online real estate portals for flats in the city of Berlin. Using this data, we developed a model to predict rental prices per square meter based on the accommodation’s features and location within the city. We detected offers which appear in both portals by means of statistical matching and removed duplicate offers. Missing values were treated by multiple imputation. The prediction model is a semi-parametric approach where the postal districts are used to describe the location effect. Comparisons with microcensus results and the local rent index reveal significant differences between the market of online flat offers and the stock of existing flat contracts. Interested readers will find the commented programming code in the internet supplement.

Suggested Citation

  • Camilo Meyberg & Ulrich Rendtel & Holger Leerhoff, 2024. "Flat rent price prediction in Berlin with web scraping," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 18(2), pages 245-278, June.
    • Handle: RePEc:spr:astaws:v:18:y:2024:i:2:d:10.1007_s11943-024-00340-6
    DOI: 10.1007/s11943-024-00340-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11943-024-00340-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11943-024-00340-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rebecca C. Steorts & Timo Schmid & Nikos Tzavidis, 2020. "Smoothing and Benchmarking for Small Area Estimation," International Statistical Review, International Statistical Institute, vol. 88(3), pages 580-598, December.
    2. Edward E. Leamer, 2007. "Housing is the business cycle," Proceedings - Economic Policy Symposium - Jackson Hole, Federal Reserve Bank of Kansas City, pages 149-233.
    3. Adam Czerniak & Michał Rubaszek, 2018. "The Size of the Rental Market and Housing Market Fluctuations," Open Economies Review, Springer, vol. 29(2), pages 261-281, April.
    4. Michal Rubaszek & Margarita Rubio, 2020. "Does the rental housing market stabilize the economy? A micro and macro perspective," Empirical Economics, Springer, vol. 59(1), pages 233-257, July.
    5. Heij, Christiaan & de Boer, Paul & Franses, Philip Hans & Kloek, Teun & van Dijk, Herman K., 2004. "Econometric Methods with Applications in Business and Economics," OUP Catalogue, Oxford University Press, number 9780199268016.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michal Rubaszek & Margarita Rubio, 2020. "Does the rental housing market stabilize the economy? A micro and macro perspective," Empirical Economics, Springer, vol. 59(1), pages 233-257, July.
    2. Adam Czerniak & Michał Rubaszek, 2018. "The Size of the Rental Market and Housing Market Fluctuations," Open Economies Review, Springer, vol. 29(2), pages 261-281, April.
    3. Mateusz Tomal, 2022. "Testing for overall and cluster convergence of housing rents using robust methodology: evidence from Polish provincial capitals," Empirical Economics, Springer, vol. 62(4), pages 2023-2055, April.
    4. Michał Hebdzyński, 2024. "Price-setting factors or revealed preferences? How to understand the results of hedonic models and hedonic indices of the housing rental market that base on listings data?," Bank i Kredyt, Narodowy Bank Polski, vol. 55(4), pages 519-550.
    5. Robert J. Shiller, 2007. "Understanding recent trends in house prices and homeownership," Proceedings - Economic Policy Symposium - Jackson Hole, Federal Reserve Bank of Kansas City, pages 89-123.
    6. Navarro-García, Juan Carlos & Madrid-Guijarro, Antonia, 2016. "Real economic activity and accounting information in Spanish construction and real estate firms," Revista de Contabilidad - Spanish Accounting Review, Elsevier, vol. 19(1), pages 21-30.
    7. Michael T. Kiley & Jean-Philippe Laforte & Rochelle M. Edge, 2008. "The Sources of Fluctuations in Residential Investment: A View from a Policy-Oriented DSGE Model of the U.S. Economic," 2008 Meeting Papers 990, Society for Economic Dynamics.
    8. Mariarosaria Comunale, 2017. "Synchronicity of real and financial cycles and structural characteristics in EU countries," CEIS Research Paper 414, Tor Vergata University, CEIS, revised 25 Sep 2017.
    9. Franses, Ph.H.B.F. & van Oest, R.D., 2006. "Testing changes in consumer confidence indicators," Econometric Institute Research Papers EI 2006-18, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    10. Hertrich Markus, 2019. "A Novel Housing Price Misalignment Indicator for Germany," German Economic Review, De Gruyter, vol. 20(4), pages 759-794, December.
    11. Blum, Bianca, 2018. "Ausgestaltung einer Steuerpolitik zur Förderung von LED-Beleuchtung," The Constitutional Economics Network Working Papers 01-2018, University of Freiburg, Department of Economic Policy and Constitutional Economic Theory.
    12. Nyakabawo, Wendy & Miller, Stephen M. & Balcilar, Mehmet & Das, Sonali & Gupta, Rangan, 2015. "Temporal causality between house prices and output in the US: A bootstrap rolling-window approach," The North American Journal of Economics and Finance, Elsevier, vol. 33(C), pages 55-73.
    13. Yépez, Carlos A., 2018. "Financial intermediation and real estate prices impact on business cycles: A Bayesian analysis," The North American Journal of Economics and Finance, Elsevier, vol. 45(C), pages 138-160.
    14. Seitz, Franz & Baumann, Ursel & Albuquerque, Bruno, 2015. "The information content of money and credit for US activity," Working Paper Series 1803, European Central Bank.
    15. Matthew Rognlie & Andrei Shleifer & Alp Simsek, 2018. "Investment Hangover and the Great Recession," American Economic Journal: Macroeconomics, American Economic Association, vol. 10(2), pages 113-153, April.
    16. Goodness C. Aye & Stephen M. Miller & Rangan Gupta & Mehmet Balcilar, 2016. "Forecasting US real private residential fixed investment using a large number of predictors," Empirical Economics, Springer, vol. 51(4), pages 1557-1580, December.
    17. Crowe, Christopher & Dell’Ariccia, Giovanni & Igan, Deniz & Rabanal, Pau, 2013. "How to deal with real estate booms: Lessons from country experiences," Journal of Financial Stability, Elsevier, vol. 9(3), pages 300-319.
    18. Hassan Belkacem Ghassan & Abdelkrim Ahmed Guendouz, 2019. "Panel modeling of z-score: evidence from Islamic and conventional Saudi banks," International Journal of Islamic and Middle Eastern Finance and Management, Emerald Group Publishing Limited, vol. 12(3), pages 448-468, July.
    19. Greg Howard & Carl Liebersohn, 2019. "What Explains U.S. House Prices? Regional Income Divergence," 2019 Meeting Papers 1054, Society for Economic Dynamics.
    20. Mehmet Balcilar & Rangan Gupta & Stephen M. Miller, 2015. "The out-of-sample forecasting performance of nonlinear models of regional housing prices in the US," Applied Economics, Taylor & Francis Journals, vol. 47(22), pages 2259-2277, May.

    More about this item

    Keywords

    Web scraping; Flat offers; Official statistics; Rent indices; Statistical matching; Multiple imputation; Semi-parametric regression;
    All these keywords.

    JEL classification:

    • R21 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - Household Analysis - - - Housing Demand
    • R31 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - Real Estate Markets, Spatial Production Analysis, and Firm Location - - - Housing Supply and Markets
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:astaws:v:18:y:2024:i:2:d:10.1007_s11943-024-00340-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.