CoreLogic’s Chief Analytics Architect, Scott Matthews, explores whether man and machine can work together in valuation
Chess, played since the 7th century, is a mastermind game steeped in tradition. Arguably, it represents the ultimate battle of strategy and wits, played at the highest level between grand masters who are touted as the epitome of human intelligence at its best. Over the years, the chess game strategy has also been the focus of many attempts to build a machine that can outplay the game’s best.
Since the rise of the digital computer in the 1950s, chess has been the centre of increasingly more complex computational implementations to play the game, adhering to the rules but also being able understand the opponents strategy and outplay them. In 1997, IBM’s Deep Blue supercomputer famously defeated chess Grand Master Kasparov, more recently Google’s AlphaZero which, equipped with no more human input apart from the fundamental rules of chess, was able to self-learn and defeat the current incumbent in under four hours.
This prompts us to pose the question, “Are the machines now smart enough to take over the game?” Skynet, as far as we know, is not yet self-aware, but we are seeing many implementations of AI becoming incredibly adept at performing certain tasks, such as chess. In the case of chess, whilst Alpha Zero’s efforts are undoubtedly impressive, chess is a discrete task. The rules are clear; the basic moves of the game are clearly defined.
Clearly in the case of AlphaZero, AI has advanced to the point where the machines are supremely adept at the problem of winning a game of chess. Real world problems, however, are far more complex as the rules are not always clear, or for that matter, well understood. Take the task we are faced with at CoreLogic each day, estimating market value of residential properties with increasing levels of accuracy and speed. The task we have is to use the price signal information we receive, coupled with property attribute, spatial and market information to estimate what an arms-length transaction for a property in today’s market might fetch.
Automated Valuation Models (AVMS) have been used for some time as a means to value collateral held against a residential home loan and advanced to the point where, for a large portion of properties, a very precise estimate of market value can be generated. Increasingly, AVMs are incorporating machine learning and AI approaches to help improve these predictions.
Our newest AVM IntelliVal does exactly this and is showing great promise as the next generation of AVM.
AVMS and AI
Moving down the AI / Machine learning path has allowed us to understand many things about these approaches – their strengths and importantly their weaknesses.
For AI to work well it has to be well informed as to the problem it is solving – what are the rules of the game? What matters when it comes to valuing property? In other words – ‘how do people think about property and estimating its value?’
When it comes to accuracy, AVMs are pretty good, at around 80 per cent, as detailed in the chart below. Of course, these approaches also need well-curated and cleansed data to be effective. The questions, though, are these: ‘how do we best capture the rules of the game and how do we best understand how humans think about property and interact in the market?
AVMs are designed to work effectively across all property types and regions. However, due to inhomeneous nature of property markets, property design and/or data availability, and differences at a regional level can, and do, exist. As the table above shows States perform relatively consistently over 80% within +/- 15% with the exception of WA and TAS, both of which performs at 78% within +/- 15%.
The Human Approach
The answers lie in those humans who are expert at valuing property, valuers. Valuers have spent years understanding how the property market works and what to consider when valuing a property in a given location and where, in many instances, a valuer is able to do what an AVM cannot – that is to observe and interpret that which is not observed in the data we capture. Quality is an esoteric element that is difficult to quantify and capture, but something that a valuer can intuitively estimate.
Models, on the other hand, are able to assimilate a larger volume of price signal information in understanding likely market price based on previous sales, and potential market risks. The question is ‘How can models/AI and humans work together to better estimate market value?’ Good AI, done well, provides an intuitive experience for the human with the outcome improving over time (think of the Google experience!). With valuing property, it is possible for valuers to inform the models and, importantly, for the model to inform valuers. Both working together are likely to give the best outcomes – better accuracy, faster turnaround times, and greater efficiency.
As is the case with chess, prior to the self-learning AlphaZero’s of today, advances were made with ‘Centaurs’, that is part machine, part man, improving on the solely machine based approaches. The humans were able to help inform the machine of things it could not observe directly itself, and the machine was able to see moves that the human player had not considered.
Of course, now the machines are able to understand themselves the dynamics of the game. With property, the ‘game’ is far more complex, and the rules less defined. The machines need human input to help better understand how the game is played, and importantly, the machines can help humans better understand market dynamics that may not be obvious. We have an active research and development program that is yielding interesting results in improving AVM accuracies by utilising property imagery and descriptions to improve the estimation of quality.
We are also looking into approaches whereby we can assist valuers in better understanding the dynamics of an area and considering their feedback to better understand the local market. This is not to replace the work of valuers with a machine, but to enhance the role of both in performing the task in a more efficient and effective way.
The path we are treading is about enlightening both the man and machine as to what is driving the market so we can better estimate property value and ultimately provide a superior customer experience and management of collateral risks for the financial institutions lending against this property.
There are interesting times ahead, and for now, the machines are not poised to completely take over this space, but they are sufficiently advanced to interact with valuers to deliver a better result for everyone.