Measured
13 cars against the carrier’s own estimates
An outside shop ran the estimator over two evenings and the carrier’s own estimates of record for the same vehicles followed each time, from the adjuster who wrote them. 13 pair on VIN. Priced at the $56 an hour Progressive printed on those documents, this is how the estimator did.
Median absolute error
25.3%
Written under
10 of 13
Within 25%
6 of 13
What this is measured against, and what it is not
Carrier estimates of record. Not settlements, and not completed repair orders. That distinction is the first thing worth knowing and it is not a technicality. The 2024 Porsche published elsewhere on this site was written by its carrier at $429.04 and settled at $6,003.64 seventeen days later after teardown. A figure measured against first writes is a real figure about first writes, and nothing more. Accuracy against completed repair orders has still not been measured, and this page does not claim it.
Every car, including the worst
Ordered by what the carrier allowed. Hours and parts come from the stored estimate; only the labor rate is changed, to the one Progressive printed.
| Vehicle | Progressive allowed | The estimator | Difference |
|---|---|---|---|
| 2017 Mercedes-Benz GLS The outlier, and the reason the median is the honest figure. This is a rear bumper repair of 13 lines: the carrier repaired the cover at 1.0 hour and the estimator replaced it. One repair-or-replace call decides the whole percentage when the total is this small. | $862.80 | $2,310.62 | +167.8% |
| 2007 Land Rover LR3 | $1,222.65 | $951.68 | -22.2% |
| 2023 Nissan Versa | $1,529.30 | $1,511.66 | -1.2% |
| 2025 Volkswagen Taos | $3,203.02 | $2,644.40 | -17.4% |
| 2018 Chevrolet Suburban | $3,540.65 | $3,176.39 | -10.3% |
| 2016 Chevrolet Suburban | $3,940.78 | $5,006.67 | +27.0% |
| 2024 Ford Transit | $3,967.08 | $3,193.08 | -19.5% |
| 2026 Ford Mustang Mach-E The largest miss after the Mercedes, and the only battery-electric car in the set. The comparison corpus behind the estimator is almost entirely combustion collision work. | $4,546.66 | $2,488.80 | -45.3% |
| 2012 Ford F-150 | $4,685.51 | $3,501.93 | -25.3% |
| 2019 Nissan Rogue Sport | $5,873.50 | $5,676.45 | -3.4% |
| 2021 Lexus GX | $1,120.86 | $1,454.15 | +29.7% |
| 2025 Kia Soul | $3,166.27 | $1,944.08 | -38.6% |
| 2023 Ford F-150 Two things are wrong with this row and both are stated rather than fixed. The carrier figure is a FOURTH supplement, written after the truck was apart, while the estimator wrote from four photographs before any teardown, so part of the gap is that rather than error. This VIN was also estimated twice, 10 minutes apart, and the two runs are far apart: $2,410.63 from 4 photographs and $5,227.55 from 8. That is not the estimator disagreeing with itself, it is the estimator pricing the damage it can see, and it is the clearest thing on this page about how much photo coverage matters. The 4-PHOTOGRAPH run is the one shown, against a carrier figure written after teardown, so this row is the least flattering comparison available on this car. It is kept, and flagged, because dropping the rows that flatter least is the whole failure this page exists to avoid. | $4,253.33 | $1,468.74 | -65.5% |
Why the median and not the mean
The mean absolute error across these 13 is 36.4% and the median is 25.3%. The whole of that gap is the Mercedes. It is an $862.80 rear bumper repair, and on a total that small one repair-or-replace decision moves the percentage by more than a hundred points. The signed bias moves the same way for the same reason: the mean reads -1.8% and the median -17.4%. With the flagged cars removed, the median absolute error of the remaining 10 is 20.8%. All three numbers are printed because choosing one and omitting the others is how an honest table becomes a dishonest headline.
The estimator writes under, not over
It came in below the carrier on 10 of 13 cars, median -17.4%. That is worth saying because it used to be the opposite. An earlier version of the estimator ran far over on refinish hours, and the fix for it was shipped on 1 September 2026 after being measured. This is the first time the direction has been checked on another shop’s cars rather than on our own, and it is the direction the fix predicted. A tool that writes under is a tool that costs a shop money on a supplement rather than one that embarrasses it in front of an adjuster, and neither is good enough.
What makes this set unusually clean
The estimates were written on 2026-10-01 and the carrier’s documents reached us on 2026-10-02. The carrier wrote most of them earlier than that, in late September, so the documents existed before the estimates did. That is not what makes the comparison clean.
What makes it clean is that the estimator reads four reference indexes, and all four were built on 2026-09-11 and were not rebuilt until after every estimate in this table had been written. Not one of these documents was in any of them at the time. The estimator could not have been influenced by a document that was not in anything it read. A date comparison between two things that both move proves nothing; the build date on the artefact the model actually consults does.
Those indexes were rebuilt on 2026-10-02 and these documents are in them now, which is the entire reason for collecting them. It also fixes this table: no car written after that date can be added to it, because its estimate would have been informed by the documents it is being scored against. The next measurement needs the next set of documents.
All 13 were written by one build of the estimator, gemini-3.5-flash+pe4b4404be300. Every earlier comparison on this site was written before the estimator recorded which version produced a given estimate, which means none of them can be attributed to any particular one. This is the first that can.
What 13 cars does not support
It does not support a per-carrier figure, a per-make figure, or a claim about any vehicle not in the table. One carrier is represented, and two pairs are the same model in different years. There is one battery-electric car and it is among the largest misses, which is suggestive and is not evidence.
These are smaller claims than average. The median total across these 13 is $3,540.65, against $7,374.16 across the other 78 other Progressive estimates in the corpus. That matters for reading the number rather than for trusting it: a small claim is exactly where a single repair-or-replace call swings the percentage, which is what the Mercedes at the top of the table shows. The labor rates are the same — every one of them was written at $56 an hour for body and refinish, which is the median across all 93 of them.
This figure will move as the set grows, including upward, and it will be republished when it does rather than frozen at whatever reads best.
The harder test is the one you run yourself. Photograph a job you have already closed and compare it to the estimate you were paid on.