When you type in a home’s street address to obtain an online valuation from the two biggest players in the field — Zillow’s Zestimate or Redfin’s Estimate tools — how good is what you get?
Both are used by millions of home shoppers, owners, realty agents, anyone curious about what a house in their neighborhood might be worth. Both also have been criticized for estimates that are off the mark; some homeowners have actually sued Zillow over their Zestimates, though unsuccessfully. Zillow’s own CEO, Spencer Rascoff, famously sold his Seattle home for 40 percent below its Zestimate.
Accuracy matters a lot in this arena because many buyers and sellers use the online estimates to price their homes or make purchase offers, literally handing sellers or buyers the estimates as part of their bargaining strategy. This is despite both companies’ warnings that these are not appraisals, only algorithm-based computer estimates. They are starting points, not holy writ.
So which company’s estimate is the more accurate? For two years, Redfin has claimed that it produces estimates that are superior, based on the results of an independent study. When it values homes that are on the market, Redfin says its median national error rate is just 1.77 percent. That is, the selling price, compared with the estimate, is within that margin of error half the time. On houses that are not for sale, Redfin’s median error rate is 6.66 percent. Redfin has a total of 74.4 million properties in its valuation database — 1.3 million on the market and listed for sale, 73.1 million off the market.
But now it looks like bragging rights for accuracy could be shifting to Zillow. Following an international contest involving teams of data scientists, Zillow announced that its median error rate on valuations of the 110 million U.S. homes in its database will soon drop to 4.0 percent or even below, from the current 4.5 percent. Zillow does not provide a breakout that distinguishes between its error rates for homes already listed on the market and off-market homes, so there is no direct comparison to Redfin’s claimed 1.77 percent figure for listed houses. But the overwhelming majority of homes in Zillow’s 110 million-property database are off-market, which are more challenging to value because there’s usually less detailed information available on them. Note the difference in Redfin’s 1.77 percent error rate for listed homes versus its 6.66 percent rate for off-market homes. Given this, Zillow’s claim that it will have a 4 percent composite error rate on 110 million homes — the vast majority of them off-market — looks better.
An error rate of 4 percent or less would put Zillow close to a standard that many appraisers consider passable for their own work. Ryan Lundquist, an appraiser in the Sacramento, California, area, told me that for many colleagues, a 4 percent median error rate “would be a fairly acceptable range.”
Pat Turner, an appraiser in the Richmond, Virginia, market and a longtime skeptic about automated valuations, says the only way Zillow could ever get to a median error rate of 4 percent would be in “cookie cutter” subdivisions, where houses are similar and comparable properties are plentiful. In neighborhoods with greater diversity of home types, ages, interior improvements and land sizes — or in non-urban areas where comparable homes and data are hard to find — he seriously doubts the claim.
Does it really matter what these companies say about improvements in their error rates? Absolutely — if you make use of Zillow Zestimates or Redfin Estimates. If they don’t produce value estimates you can rely on within their published error rates, why would you waste your time looking at them?
But remember: “median national error rate” can be a tricky concept. “National” does not mean your local market. Your neighborhood may have a much better — or far worse — error rate than the national medians. Before using either tool, it’s a good idea to go their web pages and check how far off their estimates tend to be where you live. You can find them at www.zillow.com/#acc and www.redfin.com/redfin-estimate.
And focus on the key term “median.” In Chicago, the median Zestimate error rate is an impressive-looking 3.8 percent; but 41.4 percent of Zestimates are not within 5 percent of the actual sale price. That’s sobering. In Washington, D.C., the median error rate is 3.1 percent. But fully a third of Zestimates aren’t within 5 percent of being accurate.