Blog/ 2026-10-07

How to validate a new AVM feature without appraiser labels

A two-story brick house with a neatly trimmed lawn and landscaped entrance.

Appraiser labels are the gold standard for checking whether a new feature earns its place in an AVM, and they're also the slowest, most expensive way to find out. If you're testing a curb-appeal signal, a condition proxy, or anything derived from imagery, you usually need an answer before you can justify commissioning a labeling panel. The good news is you don't need appraiser sign-off to get a defensible first read.

Start with the labels you already have

Closed sale prices are ground truth too, just noisier and less granular than an appraiser's line-item condition grade. Most shops already have what they need:

  • Arm's-length sale price, which captures the net effect of condition, location, and curb appeal even when it doesn't decompose them.
  • Repeat sales on the same parcel, which let you isolate the change in a feature's value over time while holding the structure constant.
  • Assessor condition codes and permit history, which correlate with physical condition even though they're updated on a different cadence than a sale.
  • Time-on-market and price-cut frequency from MLS feeds, which track buyer perception of a property in ways tabular fields rarely do.

None of these is a perfect appraiser label. Together they let you build a residual-based validation framework that doesn't wait on a panel.

A residual-based validation workflow

Hold out a sample of recent closed sales the model hasn't seen, score each one with and without the candidate feature, and compare absolute percentage error between the two runs. If the feature is doing real work, the error reduction should concentrate in the segments where it theoretically matters: older housing stock with wide condition variance, rural parcels where comps are thin, properties with recent exterior renovation. A feature that improves overall MAPE but shows no segment-level pattern is probably just absorbing noise or leaking location information you already have elsewhere.

From there, run the checks a reviewer will ask for:

  1. Monotonicity and sanity direction. A curb-appeal or parcel-condition score should move the value estimate the way a human would expect. Plot partial dependence and look for a score that trends overgrown lot, deferred exterior maintenance, or vacant-looking structure toward lower value, and well-kept landscaping or recent re-roofing toward higher value. If the direction flips or wobbles, the feature isn't ready regardless of what it does to MAPE.
  2. Feature attribution shift. Compare SHAP values before and after adding the feature. You want to see it pull weight away from proxies that were standing in for it, like zip-code fixed effects or a stale "effective age" field, not just stack on top of them.
  3. Stability over time. Run a population stability index on the feature's distribution month over month. A feature sourced from annual imagery refreshes should be stable between refresh cycles and show a clean, explainable step when a new capture lands, not drift that looks like a data pipeline problem.
  4. Time-window backtest. Train through the end of last year, score sales from this year. This catches features that look strong in-sample because they're correlated with something that happened to be true for that cohort, not because they actually generalize.

This is also the stage to look at where the model disagrees with price the most. If large residuals cluster on properties where you had no curb-appeal or parcel-context signal at all, that's your evidence the gap is real, and it's a reasonable basis to scope a small appraiser-reviewed sample later, once the feature has already earned its place on sale-price evidence. For a parcel and neighborhood embedding sourced from very-high-resolution imagery and built specifically to slot into this kind of validation, an embedding delivered as a per-property vector keyed to your own record IDs is worth a look if curb-appeal and context are the gap you're trying to close.

None of this replaces an appraiser's eye forever. It gets you a defensible, sale-price-backed answer on whether a feature belongs in the model before you spend the budget to find out the expensive way.

If parcel-level imagery signal is the piece missing from your validation set, AVM Imagery is built to be that input.

Start a pilot

← Back to the blog