The Redfin Advantage: Bigger, Broader, Higher Statistical Confidence

April 24, 2008 addition: Redfin is correcting how it calculates the weighted average in the Redfin Advantage report.
Greg Wharton, the general counsel at my last job, liked to answer questions enigmatically. When I asked him if a software pawnshop in Florida was trying to buy us, he would smile and say, “answering that question now would obligate me to answer that question in the future.”

That can be a heavy burden to bear, especially when the question you’ve answered is a contentious one. Last spring, Redfin analyzed the MLS data available to all brokers to show that our home-buyers negotiated a better deal on a home — saving an extra $4,000 off the list price — over and above our commission refund. We called the total savings, of $14,080, the Redfin Advantage. We promised to make calculating it a regular thing, which meant that saying we saved customers money in one year would obligate us to admit it was a fluke if we didn’t repeat our performance in the next.

So when Redfin set out to evaluate our performance over the last 12 months, in a downturn that has brokers of all stripes competing ferociously at the negotiating table, I was sick to my stomach. As transaction volume grows, we should regress towards the mean. But the numbers came in for both Seattle and the Bay Area, and they only show the Redfin Advantage got bigger, with a smaller range of error (thank goodness!):

• Redfin home-buyers over the past twelve months paid on average 1.015% below homes’ asking price, while customers of other brokerages paid .087% below asking price.
• This difference in negotiating results saved Redfin customers nearly 1% of the home’s final price, for an average savings of $5,048.
• In addition, Redfin refunded each of these customers an average of $10,520 in commissions.
• The data is statistically significant, with p-values for each of the three counties we evaluated between .00004 and .02. To a statistician, this means there is less than a 2% chance that our results could be the result of chance. The p-value for last year was closer to .03 or 3%.
• It would be hard for us to exaggerate or fabricate this data, since other brokers can and will challenge the result. There was a huge brouhaha last year. One intrepid broker found an error on one transaction, and we immediately issued a .01% correction.
• Redfin’s customer satisfaction rate was again 95%, for home-buyers whose offers succeeded or failed. Our demographic broadened, with the number of high-technology customers dropping from 48% to 33% of our total, and the number of first-time home-buyers increasing to 45%. Unlike the negotiating advantage, the satisfaction and demographic results come from our own surveys, which are less reliable.

The report analyzes data from February 6, 2007 – February 5, 2008, based on the anniversary of our launch of our home-buying service, Redfin Direct for Buyers. We didn’t analyze LA, San Diego, Boston or Washington, D.C. because we hadn’t served those markets for the full year, and we didn’t have a statistically significant numbers of sales there either.

Because the negotiating advantage is consistent across different Redfin agents, different customers, different counties, different years, in markets that were healthy and slumping alike, it seems fair to conclude that the advantage stems from the Redfin business model itself. What does that mean?

Customers on the prowl for a deal are a big reason we negotiate effectively. The partnership we try to set up with customers gives them a more active role in negotiations and, because they have the most skin in the game, they come to the negotiating table armed to the teeth with data from our site. Last year we tried to give all the credit to our agents, but now, we think Greg Swann was right: a lot of credit goes to our customers too.

We still think the results validate the skill of our real estate agents too, whom we pay customer satisfaction bonuses rather than commissions to avoid creating any pressure for customers to close on a bad deal. Alone among any national brokerages, we require our agents to have experience with at least 20 transactions before representing a client. Hats off to all our Redfin agents, and thanks for your hard work, now being recognized on TechCrunch, in our favorite real estate blogs and the local papers.

Many thanks to Redfin star Chris Glew for preparing the report, his first big business project. An anthropology M.A., Chris previously studied ancient Mexican turds and fabulous jungle-buried relics. In his interview for the job, he explained that the diameter of an empire’s tortilla-making griddles increased with its ability to enslave people in the fanatical construction of monuments (which in turn prompted us to expand Redfin’s Costco order.)