Simulation and evaluation

Seven reference days and a cold one, hardware simulators that all know how to say no, a seeded weather the planner never sees, and multi-day sweeps that can falsify a day.

Nothing in hems talks to household hardware yet, so hems-sim stands in for every device. That makes the simulator the thing every figure on this site rests on — which is why the design question it answers is not “does it run?” but can it fail?

The reference household

Every reference day is the same house unless the day says otherwise: a common German single-family home in 2026.

That last row is the ordinary German household of 2026 and the two halves are deliberately different answers. The meter has been in for years, so § 51 EEG has been taking the negative quarter hours since the start of 2025 — and the § 9 Abs. 2 cap is still on, because that one lifts only after the network operator’s first successful Ansteuerbarkeit test, which is a different event on a different clock. --imsys is that test happening.

Seven days

$ just demo-all

Only the winter day’s full transcript is held to the binary by a test; the savings in this table are not, and a figure nothing compares is wrong within a fortnight. just day-report prints the one that is checked.

autumn is also the only one of the seven where the seam between the arbiter and the wiring shows: a switching wallbox spends the afternoon being asked for power that falls between what three conductors and one can hold, and hems_device::realisable answers each of those with zero — correctly, and silently. The day reports 19 ticks and 0,18 kWh of it against nought on the other six, and its own test pins that, which is what keeps the number from being structurally zero.

Two of those are chosen rather than obvious. capped is in May, not June, because the cap is a fraction of direct-current power and how close a roof gets to it is decided by cell temperature. shared removes the store rather than adding a device, because a household with no battery is the one where a § 14a ceiling actually binds — and that is where the shadow price of relief stops being zero.

And the comparisons

Six of these run as part of just demo-all; the last two are their own recipes, because each costs minutes rather than seconds.

Every simulator knows how to say no

A simulator that agrees with its controller flatters it and hides exactly the bugs worth finding. So each one has at least one refusal:

Every one of those lags is a time, not a number of calls, and that is not a detail: a dynamic expressed per tick is a dynamic whose speed belongs to whoever is calling it. These days tick once a minute and a box ticks once a second, so a “two-second” inverter counted in steps would take four minutes to obey a feed-in limit. Every simulator here takes the step duration, and a day run at any cadence describes the same hardware.

Two of those found real defects. Without the heat pump’s own thermostat, a manager that stopped planning cooked the reference house to 64 °C and reported it as a saving — energy nobody used is energy nobody bought. And the compressor is the clearest case in the workspace: a minimum runtime can be implemented, documented, cited and unit-tested and still be enforced on no day, because its rows need the slot before them and a receding horizon executes only the slot that has none. A constraint no simulator can be watched breaking is one no simulator can be watched obeying.

The day that happens is not the day that was forecast

This is the part that decides whether any saving figure means anything.

flowchart LR
  SEED["seed + instant"] --> R["Realisation
4 octaves of value noise:
~4 h, 1 h, 15 min, 4 min"]
  R --> SIM["the day the house lives
hems-sim"]
  HIST["six weeks of the box's
own metering"] --> FC["what the planner is told
hems-forecast"]
  FC --> PLAN["plan"]
  PLAN --> SIM
  SIM --> SCORE["CRPS, coverage,
and the money"]
  FC --> SCORE

weather::Realisation is a seeded process: four octaves of correlated noise on the cloud cover at about four hours, one hour, a quarter hour and four minutes — a front, a haze, a cumulus field, one cloud — plus a diurnal temperature shape with a slow error, a load multiplier and a hot-water draw that varies.

The noise is correlated rather than white, deliberately. White noise averages out inside a quarter hour, so a planner working in quarter hours never sees it and the arbiter has nothing to catch up on. The fastest octave is the one that earns its keep, because it moves below the planner’s own grain.

And it is a pure function of (seed, instant) — no generator state, no iteration order to depend on — so the day still replays to the last euro cent on any machine under any number of threads. Determinism was never the thing that had to go; being told the answer was.

The simulated roof delivers 92 % of what its geometry says: soiling, a little shading, mismatch, an ordinary German roof after three years. Nothing tells the model, and the residual corrector has to find it — which it does, reporting 90 %. A figure sitting at exactly 100 % would mean the corrector is not being fed.

A test whose job is to fail if the simulator gets too good

Three of the workspace’s guards are worth naming because of what they guard against:

• one asserts the reference day’s forecasts were wrong — a day the planner cannot be surprised by measures a planner that was shown the answer, and a test that only checks “the day saves money” passes either way;
• one runs the day’s own quarter-hour registers through the § 42c allocation;
• one checks that every asset the arbiter commands can be described in S2.

A rule module can be implemented, cited, tested and reached by nothing at all, and no property test catches that: a property is a statement about code that runs.

One day cannot answer some questions

Two of them, and both have a command.

Is the forecast band the width it claims to be?

Forecast error is correlated across a day, so ninety-six quarter hours of one Tuesday are close to one draw: a day’s coverage figure is a coin toss reported to three significant figures.

$ just backtest summer 20

Twenty seeded weathers, each one episode, their scores merged. It is the only thing in the workspace that can say whether the band the planner hedges against is the width it claims — 80 % coverage on the January day, 75 % on the June one, against a nominal 80 %. Calibration carries an episode count beside its sample count and is_well_calibrated asks for twenty days, so a single day that happened to land inside its band cannot report itself calibrated.

What is a hedge worth?

A single realisation pays a hedge’s premium every time and makes its claim never, so measured once, insurance is always a pure loss.

$ just risk deadline 20

  policy                mean     worst      best    unserved     solve
  one median           1.09€    -0.58€     1.75€       0.10€       81s
  three futures        1.22€     0.54€     1.60€       0.00€      651s
  …and the tail        1.20€     0.62€     1.69€       0.00€      567s
  only when at risk     1.18€     0.61€     1.69€       0.00€      480s

Scenarios pay where a service is at risk, cost about ninety cents a day where nothing is, and — since the baseline was given the battery it owns — the hedge does buy a better bad day: on this evening it takes the worst weather from −€0,58 to €0,62, where the median plan costs the household money. That last finding read the opposite way until the counterfactual was corrected, which is the sharpest argument here for owning the sweep rather than footnoting it: a handicapped baseline did not merely flatter a figure, it reversed a conclusion.

What the reference days are not

Two limits, stated here rather than discovered by somebody else.

The box’s history is generated by the same process the day is. Six weeks of metering, produced by the same simulator, means the forecasts are scored against a world whose statistics they were fitted to. That is the friendliest possible test — and the weather-foresight premium it leaves on the table is cents rather than the 54 % a broken comparison once reported. A real box faces a distribution that shifts: a season, a new tenant, a roof that gets cleaned. The field number is worse than this one and never better, which is the safe direction for a claim.

And for six weeks there is no history at all. Every day above starts warm; a household’s box does not. An eighth day runs the same January cold — no profile, no correction its roof has earned, no session history — and asserts what a box owes on its first evening rather than a saving: that it plans at all, that it respects the § 14a reduction and the § 9 EEG ceiling, that the car is charged and nobody is cold, and that it is honest about knowing nothing, with a roof correction of exactly 1,0 and a forecast score strictly worse than the warm box’s.

The saving is the finding rather than the assertion, and it is negative: a box on its first evening is €1,64 worse than no box. A household with no profile falls back to persistence from one meter reading, and that reading is usually the night trough, so the plan is made for a house that uses a third of what it will. The published answer is BDEW’s H0 standard load profile hung on the measured level, and writing a plausible curve instead would be worse than the defect — this simulator’s own household is two-peaked, so a hand-fitted prior would score well here and say nothing about a real one.

A reference day is calibrated. Each one is tuned to isolate one mechanism, so adding a device to a tuned scenario makes it measure two things at once and attribute the sum to one. That is why offline and autumn leave the dishwasher unloaded: they are controlled experiments on the surplus, and an appliance that eats a kilowatt-hour of exactly that surplus would make both days unreadable.

Reproducibility has a price, and it is named

The solver’s time budget is measured against the wall clock, so the answer depends on how busy the machine was. Two runs of the same day on the same inputs can differ — which is exactly what “replay the day and compare” needs not to happen. That is how it was found: the determinism test passed on its own and failed under a parallel test run.

So a box in the field keeps the budget, and anything that has to be reproducible sets it to zero and waits. The relative gap is unaffected: it is a property of the search, not of the clock.