The implementation

Edge infrastructure integrated with TSDB and OLAP systems; over 50,000 connected clients and 120M messages per day.

A daily total hides the hard moments

Our edge and data work includes infrastructure integrated with time-series databases and OLAP systems, supporting over 50,000 connected clients and 120 million messages every day. Here, clients means connected participants in the data system, not business customers.

Dividing 120 million by the seconds in a day gives roughly 1,389 messages per second. That is an average, not a capacity target. It says nothing about payload size, simultaneous reconnections, a burst from one site, or the traffic created when a disconnected gateway catches up.

The interesting architecture question is what happens during those uneven moments. A system must protect the usefulness of current data while making progress on the backlog.

Give every event enough identity

A sensor value needs context: which device, which measurement, which unit, and which point in time? The time an event happened and the time it arrived can differ substantially at the edge. Treating them as interchangeable can distort the operational picture.

Duplicates are another normal design concern when clients retry. A practical event contract should support a deliberate deduplication strategy, make schema changes visible, and preserve a way to trace questionable readings back to their origin.

These are architecture considerations, not claims that every message has the same delivery guarantee. The contract should make that guarantee explicit. Otherwise, each downstream consumer ends up inventing its own interpretation.

Let time-series and analytical systems do different jobs

A time-series database is well suited to questions framed around measurements and time: what changed in this zone, during this window, for this device? Time-based partitioning is one established approach; the Timescale hypertable documentation explains how it divides data into time ranges.

OLAP serves a different pattern of work: comparing sites, analysing cohorts, exploring longer histories, and aggregating across operating dimensions. Integrating the two is about query responsibilities and data contracts. It does not require pretending that a single storage layout is ideal for every question.

Raw events, operational views, and analytical summaries should have clear purposes. Retention and aggregation decisions are product decisions too: discard too much and an investigation loses its evidence; keep everything indiscriminately and cost becomes its own operational problem.

Design for reconnection

The edge is where connectivity becomes a variable rather than an assumption. Local buffering needs a storage budget, a retention policy, and a defined response when the budget is exhausted. Reconnection needs pacing so that old data does not overwhelm current work.

A useful review asks about the ugly cases: clocks that drift, devices that repeat a reading, a schema that changes mid-rollout, and a gateway that returns with a large backlog. An ingestion benchmark alone cannot answer those questions.

The system also needs to reveal its own condition. Freshness, backlog age, rejection rates, and missing clients can be more informative than a single green health indicator.

Make the operating picture trustworthy

The value of a large telemetry platform is the confidence with which a team can use its output. Can an operator distinguish a healthy reading from a stale one? Can an engineer follow an anomaly through the pipeline? Can an analyst explain why a summary differs from the raw history?

Those questions connect infrastructure to the business using it. At 120 million messages a day, the job is to make sure the next decision is informed by the right data, with its history and limitations intact.

Throughput gets the data in. Context makes it worth keeping.

Working through a similar problem?

Bring us the operating context, the constraints, and the part that needs to work better.

Talk to Wireone Labs ↗