Read the calibration results
How to tell whether the calibrated model is accurate enough, using its goodness of fit.
Where: Home › Building setup › Stats and the top of the Benchmarking page
Goodness of fit
If a model is calibrated within the threshold limits, it is considered sufficiently accurate in simulating the physical reality it is intended to represent. These thresholds are a preliminary guide for the building calibration and should not be treated as absolute values. The statistical indices here relate only to the predicted building energy consumption.
- CV(RMSE) - How much the model’s hourly or monthly values scatter around the measured values.
- NMBE - Whether the model uses more or less energy than measured, on average.
Hourly calibration
Threshold limits of statistical criteria for calibration:
| Statistical Indices | ASHRAE Guideline 14 | IPMVP | FEMP |
|---|---|---|---|
| NMBE [%] | ±10 | ±5 | ±10 |
| CV(RMSE) [%] | 30 | 20 | 30 |
| CV(RMSE) | Less than 20% | Between 20% and 30% | Greater than 30% |
|---|---|---|---|
| Calibration | Good-fitting | Acceptable | Not acceptable |
| Required procedure | Not required | Not required | Double-check meter data and user inputs and then rerun a calibration |
Monthly calibration
Threshold limits of statistical criteria for calibration:
| Statistical Indices | ASHRAE Guideline 14 | IPMVP | FEMP |
|---|---|---|---|
| NMBE [%] | ±5 | ±20 | ±5 |
| CV(RMSE) [%] | 15 | - | 15 |
| CV(RMSE) | Less than 8% | Between 8% and 15% | Greater than 15% |
|---|---|---|---|
| Calibration | Good-fitting | Acceptable | Not acceptable |
| Required procedure | Not required | Not required | Double-check meter data and user inputs and then rerun a calibration |
The Stats tab
Open a calibrated building on Home and choose Stats in the Building setup panel.
Fit to the meter data says whether the calibrated model meets ASHRAE Guideline 14, then shows CV(RMSE) and NMBE as gauges: the dark mark is the limit for the building’s meter data, and a bar past it turns orange. A positive NMBE means the model uses less than measured; the caption says which way it goes.
Electricity (kWh):
- Measured and Calibrated model, year weather - The meter year’s electricity and the model’s over the same hours (interval data) or billing periods (bills). Their difference is the NMBE.
- Calibrated model, typical weather - The calibrated model run on a typical year. This is the baseline the energy audit uses, so it differs from the meter year by the weather too.
Weather: the climate zone; Calibration weather, the actual weather of the meter year, measured hourly at the building’s location, which the calibration used; and Typical weather, the station of the typical year and its data set (for example TMY3), which the audit’s results are based on.
Three charts open from the buttons at the top:
- Heatmap - Each day’s hours as colors, the measured beside the model with interval data. It shows working hours, weekends and gaps at a glance.
- Time series - The hourly measured and model lines, to zoom into a week.
- Monthly (Monthly comparison) - The meter data and the model month by month: electricity, peak demand and, with gas bills, natural gas. With interval data each calendar month is compared over the hours that have a reading; with bills, each bill against the model over the same billing period, the months the CV(RMSE) and NMBE are calculated from. Each month’s difference is shown above its bars, in red at 15% or more, so a season the model misses stands out even when the year’s statistics look fine.
The Progress tab
The Progress tab, next to Stats, shows the calibration’s search: each simulation’s fit score, and the best one’s CV(RMSE) and NMBE. It keeps the last calibration’s search until the next one. See What you’ll see next.
On the Benchmark page
The Benchmarking page shows the same two values as bars. The dark mark is the ASHRAE Guideline 14 limit, and the bar turns orange when the value goes past it. The label next to the bars shows whether the calibrated model meets the guideline.