Enhancing the Validation of Automated Valuation Models (AVMs) through Wisdom of the Crowd

Automated Valuation Models (AVMs) play a pivotal role in appraisal of real estate as collateral for mortgage loans by banks. However, the verification of these models poses a significant challenge. When relying solely on a single appraiser to determine the AVM's validity, concerns may arise regarding a potential subjectivity bias, stemming from the individual preferences of the appraiser. Conversely, if verification is based on transaction prices, critics may argue that it relies on outdated prices due to the time lag between contract conclusion and final sale which can invoke a so-called temporal lag or smoothing bias. Therefore, we address the question of which validation mechanism is most suitable for verifying AVM accuracy. Over a period of three years, approximately 250 properties were evaluated each year. Each property was assessed by two students from the real estate management field. All students had already received training in property valuation, but they had diverse backgrounds, such as being undergraduate or graduate students, and either working full-time or part-time. About half of the total 30 students were already employed in the real estate industry. The over 750 evaluated properties represent transactions collected within the scope of loan approvals and were assessed using an AVM model. Additionally, the actual transaction price is available. This means that for each property, there is the transaction price, the AVM value, and two values from different students.In our analysis we consider three methods: single appraiser verification, transaction price verification, and an verification by two less-experienced students. The aim is to demonstrate a comparison of the valuations. We examine potential benefits and limitations and look into the effect on potential biases which can be improved by the students-based approach and influence the overall reliability of AVMs. Initial results already indicate that the average from both students' assessments significantly represents an improvement.