How property data products win and why many real estate portals are falling out of step

What thirty-five years of building property models taught me about repeatability, infrastructure, and why artefact-based data no longer scales

By Malcolm Treanor, Co-founder & COO, WINR

Early in my career, I was responsible for sourcing real property data to support quantitative models estimating homeowner equity by comparing purchase price with current market value. Purchase prices had to be sourced from individual state land titles offices, each with its own schema, requiring us to create internal persistent property identifiers to group transactions over time, while market value inputs were only available at coarse postcode or census collection area level and delivered as one-off extracts because producing anything repeatable was considered too labour-intensive.

For property portals, duplicate listings are often the first visible symptom of deeper data identity problems. Much of our efforts were spent wrangling data delivered via email, CD, and, in one case, a box of floppy disks, a reminder that nothing about the pipeline was designed for reuse.

It worked, but only because a disciplined data products team absorbed the complexity that the data itself never should have imposed. Nothing about the data was designed to behave predictably under reuse, and nothing we built could reliably compound.

Twenty-five years later, I expected that to have changed.

Recently, I worked on a project that required sourcing granular property intelligence from several major real estate portals in Europe. The mechanics were familiar in an uncomfortable way. Access was still framed around subscriptions to Excel files. Outputs were largely aggregated. Household-level property data was not available. Geographic units were often too coarse, inconsistent, or unstable to support serious modelling requirements.

Once again, the issue was not coverage or insight. It was usability.

What has changed, however, is the buyer. A new generation of product managers is far more attuned to how customers actually consume data. They are designing property data products to be integrated into customer systems, not delivered as artefacts to be handled manually. These teams think in terms of schemas, identifiers, versioning, and repeatability, and they are increasingly winning pitches against slower incumbents whose offerings still assume analyst-led consumption.

This is the quiet failure mode of property data products. They look valuable, but they are not designed as reusable data products. They are delivered as artefacts rather than infrastructure, with each customer expected to do the hard work of making the data operational inside their own systems.

For most property portals, this problem surfaces first as duplicate listings. The same home appears multiple times, often with slightly different addresses, different listing IDs, or coming from different feeds. Analysts recognise it immediately, but the underlying cause is structural. Listings are transient artefacts. They come and go, they are relisted, reformatted, and rekeyed. Without a persistent way to recognise the same underlying property or household, duplication is inevitable. Teams can suppress it in the UI, but it quietly breaks downstream analytics, price indices, and any attempt at longitudinal modelling.

As consumption shifts from people to systems, tolerance for inconsistency collapses. Quantitative models, automated pipelines, and production workflows require stable inputs. Data that is easier to integrate is increasingly chosen over data that is richer but harder to use.

This is why property data monetisation so often stalls. Not because demand is weak, but because the data does not behave predictably when reused.

The root cause is a flawed mental model. Property data continues to rely on address text and aggregation as if they were stable references. Address text is a description, not an identifier. It changes with formatting, enrichment, and re-entry. When the description changes, systems assume the underlying property or household has changed as well.

That assumption is fatal for modelling, automation, and repeatable use.

The Real Estate Property Intelligence Playbook was written to address this problem directly. It is not a sales document. It is a practical operating guide for product leaders and Chief Data Officers who want to build property data products that customers can integrate once and rely on thereafter.

The playbook argues for treating property and households as first-class reference entities. It sets out how to design property data infrastructure with stable identifiers, deterministic joins, versioned datasets, and predictable behaviour under reuse. It also explains why household-level property data is the missing foundation for many use cases, from quantitative modelling and forecasting through to event and change signals.

If you lead data or product at a real estate portal and recognise this pattern, the playbook will feel uncomfortably familiar. It describes problems most teams live with but rarely name, and it lays out a practical execution path to fix them.

Frequently Asked Questions

What type of property models are used to estimate household equity?

The models referenced are established quantitative approaches used in housing economics to estimate household equity by comparing a known purchase price with an estimated current market value over time. They rely on transaction history and market movement rather than listings or opinion-based valuations, and have been used by economists, banks, and regulators for decades.

Are these property models new or AI-driven?

No. These models pre-date modern machine learning by many years and are mathematically orthodox. A widely cited example is the Case-Shiller Home Price Index, which applies repeat-sales methodology to measure how property values change using actual transaction data rather than advertised prices.

How do repeat-sales models work in practice?

Repeat-sales models estimate value change by tracking the same property across multiple transactions and adjusting for broader market movement. This approach avoids distortions caused by changes in property mix or listing behaviour and provides a more stable basis for time-series analysis than simple geographic averages.

Where does the repeat-sales methodology come from?

Modern repeat-sales indices build on academic work such as the Bailey–Muth–Nourse (BMN) framework. This methodology underpins several official house price indices, including those published by national housing and financial regulators. The Federal Housing Finance Agency (FHFA) documents its repeat-sales approach as part of its House Price Index methodology.

Why weren’t these models the hardest part?

The modelling itself was rarely the constraint. The harder problem was always the data. Without stable property or household references, consistent schemas, and repeatable delivery, even well-established models become fragile, labour-intensive, and difficult to reuse at scale.

How does this relate to modern property data products and household identifiers?

For teams arriving here looking for how this actually gets operationalised, the key shift is separating entity recognition from analysis. Instead of repeatedly inferring households from address text, modern property data architectures introduce a stable household reference that allows the same physical household to be recognised consistently across time, datasets, and refresh cycles. When that reference exists, repeat-sales and equity models stop being bespoke exercises and become reusable components. The mathematics stays the same. What changes is that the data can finally support deterministic joins, automation, and scale without constant manual correction.

Why do real estate portals struggle with duplicate listings?

Duplicate listings are a structural consequence of how property data is produced and exchanged. Listings are created for marketing purposes, not as long-lived reference records. When properties are relisted, syndicated across multiple feeds, or slightly reformatted, new listing identifiers are often issued. Without a persistent way to recognise the same underlying property or household across time and sources, duplication is unavoidable.

This challenge is widely acknowledged by large-scale property aggregators. A well-known internal presentation from Nestoria described deduplication as one of the hardest problems in property aggregation, driven by inconsistent formats, changing identifiers, and the absence of stable reference entities.

Isn’t duplicate listing detection solved by standards like RESO?

Standards such as RESO significantly improve interoperability by standardising field names, enumerations, and data transport between MLSs and downstream systems. However, they do not solve duplication on their own. RESO standardises listings, not identity. Listing identifiers remain scoped to individual feeds and listing lifecycles, and they do not persist across relisting, aggregation, or time.

As a result, even fully RESO-compliant feeds can still contain duplicate representations of the same property. This is why portals continue to invest heavily in internal deduplication logic and identity layers above their ingestion pipelines.