Guide · Resilience & Architecture

High Availability and Redundancy for Rail Monitoring Systems

A monitoring overlay only earns its place while it is watching. The moment a server, a network switch, a power supply, or a communications link fails, the control room can lose sight of dozens of trackside sites at once — often exactly when a developing fault means visibility matters most. High availability is the discipline of removing those single points of failure so the system keeps reporting through a component failure. This guide covers where availability is lost, how redundancy is built at the central platform, across the network, and at the wayside, and how to prove the failover works before a real fault relies on it.

What high availability means for a monitoring overlay

Availability is a measurable property, not a marketing word. It is the proportion of time a system is able to do its job, and it is usually written as A = MTBF / (MTBF + MTTR) — mean time between failures over the sum of that and mean time to repair. The same relationship sits at the heart of the railway RAMS framework in EN 50126, where reliability, availability and maintainability are specified and demonstrated together rather than left to chance.

There are only two ways to move that number. You can make components fail less often, or you can make repair — from the system's point of view — effectively instant by having a standby take over. Beyond a certain point, buying more reliable single components gets expensive fast, so high availability is overwhelmingly a story about redundancy: arranging that no one failure takes the whole system down.

It is worth being clear about scope. This guide is about the availability of the monitoring platform itself — the overlay that collects, stores, and presents wayside data. That overlay is non-vital and advisory. The vital signalling has its own fail-safe design and its own assurance, and the availability target for the monitoring layer is an operational decision driven by what a blind spot costs the operator, not by the signalling safety case. The rest of this guide assumes that target has been set deliberately and asks how to design to it.

Where availability is lost: the single points of failure

A monitoring system is a chain that runs from a sensor in a location case, through a site data logger or RTU, over a communications link, into a central server and its database, and onto an operator's screen. Every link in that chain is a candidate single point of failure — a component whose failure alone removes visibility. Designing for high availability starts with naming them honestly.

Layer	Single point of failure	What is lost when it fails
Central platform	Single application or database server	All sites go dark at once; live view and history unavailable
Central network	One core switch or firewall	Server is up but unreachable — the same blind spot
Power	Single supply or UPS to the server room or a location case	Everything downstream of it stops
Communications	One link or bearer to a site or region	A site, or a whole area, drops off the map
Time reference	Single GNSS clock source	Event ordering and correlation degrade across the estate
Wayside	Single RTU or data logger at a site	That site is unmonitored until someone attends

Not every one of these justifies a redundant pair. The point of listing them is to make the choice explicit: for each, decide whether the consequence of its failure is acceptable, and duplicate the ones that are not. A useful discipline is to walk the chain and ask, at each link, "if this dies right now, what does the control room stop seeing, and for how long?"

Redundancy at the central platform

The central server and its database are the most consequential single point of failure, because their loss darkens the whole estate at once. The standard answer is a redundant pair — an active node carrying the load and a standby ready to take over — but "ready" comes in degrees, and the degree you choose is a direct trade-off between cost and how much you can lose in a switchover.

Standby type	Readiness	Typical takeover	Suited to
Hot standby	Runs continuously, kept synchronised with live data, alarms and history	Seconds or less, no loss of state	A control room that must never lose visibility
Warm standby	Powered and configured, not fully synchronised	A short start-up or catch-up	Where a brief interruption is tolerable
Cold standby	Spare equipment, powered up and configured on demand	Minutes to hours	Low-consequence or cost-sensitive sites

A hot standby is the norm for a wide-area monitoring platform. The active and standby servers exchange a heartbeat every few seconds; the standby continuously receives the same live values, alarms and historical writes so that its state mirrors the active node. When the heartbeat stops and the failure is confirmed, the standby promotes itself to active and operators keep working, ideally without noticing. The database deserves the same treatment — replicated synchronously to a second instance so that no acknowledged write is lost when the primary fails.

Network redundancy: dual paths and zero-recovery protocols

Redundant servers achieve nothing if a single switch failure isolates them. Network redundancy is what keeps the paths between components alive, and it comes in two broad grades.

The general-purpose approach is dual, physically separate paths managed by a protocol that reconverges after a break. Rapid Spanning Tree (RSTP) is the common example: it blocks the redundant path in normal operation and opens it when the primary fails. That works, but reconvergence takes a short but real interval during which frames are lost — acceptable for routine telemetry, less so for time-critical data.

Where a switchover gap is unacceptable, the relevant standard is IEC 62439-3, which defines two zero-recovery protocols already widely used in electrical substation and railway networks:

• PRP (Parallel Redundancy Protocol) — each frame is sent simultaneously over two independent LANs. The receiver keeps whichever copy arrives first and discards the duplicate. Because both paths are always live, a failure of one causes zero lost frames, not a reconnection delay.
• HSR (High-availability Seamless Redundancy) — the same duplicate-and-discard principle applied to a ring topology, so devices need only two ports and the ring itself provides the second path.

There is a failure mode that catches out otherwise careful designs, and it is worth stating plainly: two supposedly independent paths that both run through the same switch, the same cable duct, or the same power supply are not independent. A single failure takes both down and the redundancy is bypassed. Independence has to be physical — separate hardware, separate routes, separate power — or it is not redundancy at all.

Redundancy and graceful degradation at the wayside

Trackside sites cannot all be built as fully redundant pairs; there are hundreds of them and the economics will not allow it. The wayside strategy is therefore a mix of selective redundancy at the sites that warrant it and graceful degradation everywhere — designing so that a failure narrows what the system can do rather than stopping it dead.

Three techniques do most of the work:

• Redundant RTUs sharing the site I/O — at a high-consequence location such as a major junction, a second data logger monitors the same inputs and stands ready to take over, so a single unit failure does not blind the site.
• Redundant power — dual supplies and an N+1 arrangement of UPS units, so that losing one supply or battery string does not stop the location case. Battery health is itself worth monitoring, as covered in the wayside battery guide.
• Store-and-forward buffering — the single most valuable resilience feature at the wayside. Each logger holds its own timestamped buffer, so when the link to the centre or the central platform itself is unavailable, data is retained locally and back-filled in order once connectivity returns.

Store-and-forward is what turns a central or network outage from a permanent hole in the record into a delay. It works hand in hand with source time-stamping: because each record is tagged with an accurate time at the point of capture — the principle behind sequence-of-events recording and the reason a historian buffers at the edge — the back-filled data slots into the timeline correctly rather than arriving as an undated lump. On links that are intermittent by nature, this is essential rather than optional, as the remote diagnostics guide discusses.

Automatic failover without split-brain

Redundancy is only useful if the switch to the standby is automatic and safe. Two problems have to be solved together: detecting the failure quickly, and making sure the pair never both act as active at once.

Detection is done with a heartbeat — a regular signal between the active and standby nodes, typically every one to five seconds, backed up by health checks on the things that actually matter (is the database reachable, is the application responding, is the link up). A missed heartbeat starts a short confirmation window so that a momentary blip does not trigger an unnecessary switchover.

The harder problem is split-brain: if the two nodes lose contact with each other but both keep running, each may conclude the other has failed and both promote to active. The result is two systems issuing conflicting commands and writing duplicate or contradictory data — usually worse than a clean outage. The defence is an independent tie-breaker:

• A quorum or witness — a third, independent vote that only one node can hold, so a node that cannot secure it knows it must not act as active.
• A separate heartbeat path — routed differently from the main data path, so the two are unlikely to fail at the same instant and a data-path break alone does not look like a node failure.
• Fencing — a firm rule that a node which cannot confirm it holds the quorum steps down and stays down until the situation is resolved.

Data integrity across a failover

A switchover should be invisible in the data as well as on the screen. Two failure patterns matter: a gap, where records are lost during the transition, and a duplicate, where the same event is written twice by both nodes. Synchronous replication of acknowledged writes prevents the gap; idempotent handling — writing each record against a stable key so a repeat is recognised and ignored — prevents the duplicate. Combined with the wayside buffer described above, the goal is a record that a post-incident investigation can trust: continuous, correctly ordered, and free of the artefacts a failover would otherwise introduce.

Design to a target, then prove it

High availability is a specification, not an aspiration. The design should start from a stated availability target for the monitoring service, chosen by consequence — a busy metro control room and a lightly used branch line do not need the same number — and every redundancy decision should trace back to it. Over-engineering a low-consequence site wastes money that would be better spent hardening a critical one.

The single most common way high availability fails in the field is that the redundancy was never tested. A standby that has sat idle for a year, a replication link that silently stopped weeks ago, or the two "independent" paths quietly sharing one switch — none of these show up until the day the primary fails and the backup is asked to work for the first time. Redundancy you do not test is a hope, not a design. Scheduled failover drills, continuous monitoring of standby health and replication lag, and an event on every switchover are what turn a redundant architecture on paper into one you can rely on.

What to monitor on the monitoring system itself

A high-availability platform should watch its own health as diligently as it watches the wayside. Instrument at least the following:

Signal	Purpose
Heartbeat between active and standby	Detect a failed node and trigger failover
Failover / switchover events	Record every promotion so silent loss of a backup is visible
Standby synchronisation lag	Confirm the standby is current and can take over without loss
Replication status (database)	Catch a stalled replica before it is needed
Network path / link state (each path)	Prove both redundant paths are genuinely up, not one masking the other
Power source in use and UPS state	Show when a site or server room is running on its backup supply
Store-and-forward buffer depth	Expose a link outage and confirm back-fill on recovery
Time reference / clock source health	Guard event ordering across the estate

Frequently asked questions