IDEAS home Printed from https://ideas.repec.org/a/bla/reesec/v51y2023i3p754-778.html
   My bibliography  Save this article

Testing machine learning systems in real estate

Author

Listed:
  • Wayne Xinwei Wan
  • Thies Lindenthal

Abstract

Uncertainty about the inner workings of machine learning (ML) models holds back the application of ML‐enabled systems in real estate markets. How do ML models arrive at their estimates? Given the lack of model transparency, how can practitioners guarantee that ML systems do not run afoul of the law? This article first advocates a dedicated software testing framework for applied ML systems, as commonly found in computer science. Second, it demonstrates how system testing can verify that applied ML models indeed perform as intended. Two system‐testing procedures developed for ML image classifiers used in automated valuation models (AVMs) illustrate the approach.

Suggested Citation

  • Wayne Xinwei Wan & Thies Lindenthal, 2023. "Testing machine learning systems in real estate," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 51(3), pages 754-778, May.
    • Handle: RePEc:bla:reesec:v:51:y:2023:i:3:p:754-778
    DOI: 10.1111/1540-6229.12416
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/1540-6229.12416
    Download Restriction: no
    File URL: https://libkey.io/10.1111/1540-6229.12416?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gabriel M. Ahlfeldt & Kristoffer Moeller & Sevrin Waights & Nicolai Wendland, 2017. "Game of Zones: The Political Economy of Conservation Areas," Economic Journal, Royal Economic Society, vol. 127(605), pages 421-445, October.
    2. Gabriel Ahlfeldt & Alexandra Mastro, 2012. "Valuing Iconic Design: Frank Lloyd Wright Architecture in Oak Park, Illinois," Housing Studies, Taylor & Francis Journals, vol. 27(8), pages 1079-1099, November.
    3. Marc K. Francke & Alex M. Minne, 2017. "Land, Structure and Depreciation," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 45(2), pages 415-451, April.
    4. Daniel W. Apley & Jingyu Zhu, 2020. "Visualizing the effects of predictor variables in black box supervised learning models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(4), pages 1059-1086, September.
    5. Jing Lei & Max G’Sell & Alessandro Rinaldo & Ryan J. Tibshirani & Larry Wasserman, 2018. "Distribution-Free Predictive Inference for Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1094-1111, July.
    6. Sendhil Mullainathan & Ziad Obermeyer, 2017. "Does Machine Learning Automate Moral Hazard and Error?," American Economic Review, American Economic Association, vol. 107(5), pages 476-480, May.
    7. Nikhil Naik & Ramesh Raskar & César A. Hidalgo, 2016. "Cities Are Physical Too: Using Computer Vision to Measure the Quality and Impact of Urban Appearance," American Economic Review, American Economic Association, vol. 106(5), pages 128-132, May.
    8. Duncan Simester & Artem Timoshenko & Spyros I. Zoumpoulis, 2020. "Targeting Prospective Customers: Robustness of Machine-Learning Methods to Typical Data Challenges," Management Science, INFORMS, vol. 66(6), pages 2495-2522, June.
    9. Shen, Lily & Ross, Stephen, 2021. "Information value of property description: A Machine learning approach," Journal of Urban Economics, Elsevier, vol. 121(C).
    10. Coulson, N. Edward & McMillen, Daniel P., 2008. "Estimating time, age and vintage effects in housing prices," Journal of Housing Economics, Elsevier, vol. 17(2), pages 138-151, June.
    11. Edwin Buitelaar & Frans Schilder, 2017. "The Economics of Style: Measuring the Price Effect of Neo†Traditional Architecture in Housing," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 45(1), pages 7-27, February.
    12. Aubry, Mathieu & Kräussl, Roman & Manso, Gustavo & Spaenjers, Christophe, 2019. "Machine learning, human experts, and the valuation of real assets," CFS Working Paper Series 635, Center for Financial Studies (CFS).
    13. Gabriel M. Ahlfeldt & Kristoffer Moeller & Sevrin Waights & Nicolai Wendland, 2017. "Game of Zones: The Political Economy of Conservation Areas," Economic Journal, Royal Economic Society, vol. 127(605), pages 421-445, October.
    14. Edward L. Glaeser & Michael Scott Kincaid & Nikhil Naik, 2018. "Computer Vision and Real Estate: Do Looks Matter and Do Incentives Determine Looks," NBER Working Papers 25174, National Bureau of Economic Research, Inc.
    15. Yi Fan & Ho Pin Teo & Wayne X. Wan, 2021. "Public transport, noise complaints, and housing: Evidence from sentiment analysis in Singapore," Journal of Regional Science, Wiley Blackwell, vol. 61(3), pages 570-596, June.
    16. Julia Freybote & Lauren Simon & Lauren Beitelspacher, 2016. "Understanding the contribution of curb appeal to retail real estate values," Journal of Property Research, Taylor & Francis Journals, vol. 33(2), pages 147-161, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Timothy L. Hamilton & Erik B. Johnson, 2023. "The amenity value of natural views," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 51(5), pages 1079-1107, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wan, Wayne Xinwei & Lindenthal, Thies, 2022. "Towards accountability in machine learning applications: A system-testing approach," ZEW Discussion Papers 22-001, ZEW - Leibniz Centre for European Economic Research.
    2. Ka Shing Cheung & Chung Yim Yiu, 2022. "The economics of architectural aesthetics: Identifying price effect of urban ambiences by different house cohorts," Environment and Planning B, , vol. 49(6), pages 1741-1756, July.
    3. Koster, Hans R.A. & van Ommeren, Jos & Volkhausen, Nicolas, 2021. "Short-term rentals and the housing market: Quasi-experimental evidence from Airbnb in Los Angeles," Journal of Urban Economics, Elsevier, vol. 124(C).
    4. Sevrin Waights, 2019. "The preservation of historic districts—is it worth it?," Journal of Economic Geography, Oxford University Press, vol. 19(2), pages 433-464.
    5. Liu, Ruiming & Yan, Haosheng & Zhang, Zebang, 2024. "Does historic preservation affect firms' output? Evidence from the awarding of the Historic City title in China," China Economic Review, Elsevier, vol. 85(C).
    6. van Vuuren, Aico, 2022. "Is There a Diminishing Value of Urban Amenities as a Result of the COVID-19 Pandemic?," IZA Discussion Papers 15025, Institute of Labor Economics (IZA).
    7. Alan Sage & Mike Langen & Alex van de Minne, 2023. "Where is the opportunity in opportunity zones?," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 51(2), pages 338-371, March.
    8. van Vuuren, Aico & Kjellander, Josef & Nilsson, Viktor, 2019. "Refugees and apartment prices: A case study to investigate the attitudes of home buyers," Regional Science and Urban Economics, Elsevier, vol. 77(C), pages 20-37.
    9. Zhou, Yang, 2021. "The political economy of historic districts: The private, the public, and the collective," Regional Science and Urban Economics, Elsevier, vol. 86(C).
    10. Hilber, Christian A.L. & Palmer, Charles & Pinchbeck, Edward W., 2019. "The energy costs of historic preservation," Journal of Urban Economics, Elsevier, vol. 114(C).
    11. Yoshida, Jiro, 2020. "The economic depreciation of real estate: Cross-sectional variations and their return implications," Pacific-Basin Finance Journal, Elsevier, vol. 61(C).
    12. Rolheiser, Lyndsey & van Dijk, Dorinth & van de Minne, Alex, 2020. "Housing vintage and price dynamics," Regional Science and Urban Economics, Elsevier, vol. 84(C).
    13. Temel, Anna & Thiel, Patrick & Vance, Colin, 2024. "Open space amenities: Is closer better?," Ruhr Economic Papers 1109, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    14. Green, Colin P. & Heywood, John S. & Navarro Paniagua, Maria, 2020. "Did the London congestion charge reduce pollution?," Regional Science and Urban Economics, Elsevier, vol. 84(C).
    15. Waights, Sevrin, 2018. "Does gentrification displace poor households? An ‘identification-via-interaction’ approach," LSE Research Online Documents on Economics 88691, London School of Economics and Political Science, LSE Library.
    16. Fugang Gao & Huub Ploegmakers & Erwin van der Krabben & Xiaoping Shi, 2022. "Impacts of the political incentive for environmental protection on industrial land supply: Evidence from the cadre evaluation system reform in China," Papers in Regional Science, Wiley Blackwell, vol. 101(4), pages 1001-1025, August.
    17. Erik B Johnson & Alan Tidwell & Sriram V Villupuram, 2020. "Valuing Curb Appeal," The Journal of Real Estate Finance and Economics, Springer, vol. 60(1), pages 111-133, February.
    18. Portillo, Javier E. & Wagner, Gary A., 2021. "Do cultural districts spur urban revitalization: Evidence from Louisiana," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 651-673.
    19. Galster, George C. & Osland, Liv, 2024. "Educational and gender heterogeneity of the rural-urban earnings premium: New evidence from Norway," Regional Science and Urban Economics, Elsevier, vol. 105(C).
    20. Lu, Jiaxuan, 2023. "The economics of China’s between-city height competition: A regression discontinuity approach," Regional Science and Urban Economics, Elsevier, vol. 100(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:reesec:v:51:y:2023:i:3:p:754-778. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/areueea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.