MeterStore

Metering time series at settlement scale,
in an open format.

PostgreSQL holds the recent interval window at low latency. Apache Iceberg holds the history at analytical scale. One explicit timestamp separates them, and one SQL statement spans both.

Get started View on GitHub

  MeterInterval.from ─────────────────────────────────────▶

  │◀──── Iceberg (cold, settled) ────▶│
                                      │◀── Postgres (hot) ──▶│
  epoch                       tiering_watermark            now

The wall this exists for

An intelligent measuring system produces one value per measuring point, per OBIS code, per interval, and fifteen minutes is the German settlement grain:

ScaleRows / dayRows / year
10 k measuring points~1 M~350 M
100 k (mid-size utility)~9.6 M~3.5 B
1 M (metering operator)~96 M~35 B

Retention runs to years or decades for the settlement record. PostgreSQL handles the first row of that table comfortably, the second with care, and the third not without becoming a full-time job. Yet recent data is written continuously, corrected, and read transactionally — a PostgreSQL workload — while settlement, forecasting and grid analysis scan years across hundreds of thousands of meters, an object-storage-and-columnar problem.

The data has a natural split most systems refuse to exploit: recent intervals are hot and still being corrected; historical intervals are cold and settled. The boundary between them is a timestamp.

Four decisions that carry the weight

The watermark lives inside the Iceberg snapshot. Archival writes the tier boundary into the snapshot summary, in the same commit as the data it describes. Iceberg commits are a compare-and-swap, so the rows and the boundary become durable together or not at all, with no external checkpoint store to fall out of sync.

Purge is DROP TABLE, never DELETE. The hot table is time-partitioned, so an archived window is exactly one partition: detached for archival, dropped a cycle later once no query planned against the old boundary can still need it. Deleting a day for 100 k meters row by row would leave ~9.6 M dead tuples for autovacuum.

Corrections are versions, not overwrites. In MSCONS a correction versions the value [MSCONS AHB 3.2 Kap. 5.1]. Nothing is updated in place, so the store needs only Iceberg’s append, and a past settlement stays reproducible because the value it used is still there.

Nothing on the archival path holds a window. The detached partition is paged by keyset into the Parquet writer, so peak memory is the chunk size, not the ~9.6 M rows of a day at 100 k measuring points.

One statement, both tiers

let result = store.query(r#"
    SELECT meter_local_day("from") AS day, SUM(value) AS kwh
    FROM readings
    WHERE malo_id = '41373559241'
      AND "from" >= '2025-01-01' AND "from" < '2026-01-01'
    GROUP BY 1 ORDER BY 1
"#).await?;

// Every result carries the boundary it was computed against.
result.watermark();         // where cold ended and hot began
result.tiers_scanned();     // [Cold, Hot]
result.touched_hot_tier();  // whether the answer is only valid for now

The range is cut at the watermark, each half is read from the tier that holds it, and the halves are concatenated — not merged, not deduplicated, because they are disjoint by construction.

meter_local_day matters: the UTC day boundary sits at 01:00 or 02:00 in Berlin, so grouping on UTC days is wrong every day, not only at the DST transitions. The calendar arithmetic is metering’s.

For gas it is the wrong function: a Gastag runs 06:00 to 06:00 local, so meter_balancing_day("from", sparte) picks the day each row’s commodity is settled on. An external engine has neither function, so every row also stores its balancing_day, and date_trunc('month', balancing_day) gives the Bilanzierungsmonat too.

What you get that a general lakehouse does not

An open format for regulated data. Standard Iceberg v2 on object storage, readable by Spark, Trino, DuckDB, Snowflake and PyIceberg with MeterStore nowhere in the data path — and read back by DuckDB and PyIceberg in the test suite.

Time travel as compliance. MaBiS settlement must be reproducible. Pin an Iceberg snapshot plus a version ceiling to reconstruct what was known at a point in time, or pin the row-level transaction-time axis and reproduce across both tiers.

A correct domain model, stored correctly. Readings are metering’s types: DST-correct calendars, exact decimals, quality flags, all four Sparten, stored at 35-billion-row scale without losing a decimal, a quality flag or an interval boundary.

No server extension, no server configuration. A library, not a PostgreSQL extension. No shared_preload_libraries, no restart and no superuser: the one extension it creates, btree_gist, ships in contrib and is trusted, so it deploys on RDS, Cloud SQL and Azure Postgres unchanged. Getting started lists the privileges.

Start here

Getting started How tiering works API reference