Our Live Liquidation Feed streams every futures liquidation across 7 exchanges to your browser the moment it happens. Behind what looks like a simple scrolling table is a pipeline that starts with persistent WebSocket connections to exchange APIs written in Rust, batches events through an async channel, pushes them into a Cloudflare Durable Object acting as a pub/sub hub, and fans out over WebSockets to every connected browser — all with sub-second latency.
This post walks through the full architecture: how the data is collected, how it flows through the system, and the design decisions we made along the way.
The problem
Crypto futures exchanges publish liquidation events — forced closures of leveraged positions — in real time over their WebSocket APIs. Each exchange has its own data format, its own quirks, and its own connection lifecycle. We wanted to:
- Aggregate liquidations from 7 exchanges (Binance, Bybit, OKX, Bitget, BitMEX, Kraken, Aster) into a single unified stream
- Deliver them to any number of browser clients with sub-second latency
- Maintain rolling statistics (1h, 4h, 12h, 24h windows) without a traditional database
- Keep the infrastructure cheap to run even with idle connections
The result is a three-layer pipeline: Collectors → Relay → Clients.
Layer 1: Rust collectors
The data collection layer runs as a long-lived Rust process that maintains persistent WebSocket connections to every exchange. Each exchange module implements a common trait and emits a CommonLiquidation struct:
pub struct CommonLiquidation {
pub symbol: String,
pub side: String, // "SELL" = long liquidated, "BUY" = short liquidated
pub price: String,
pub average_price: String,
pub quantity: String,
pub filled_quantity: String,
pub event_time: i64,
pub trade_time: i64,
}
When a liquidation is parsed from any exchange, a single function call sends it to the relay:
// Standard exchanges (linear contracts)
relay_liquidation(&common_liq, "binance_futures");
// Exchanges where USD value needs special handling
relay_liquidation_with_usd_value(&common_liq, "bitmex", usd_value);
The relay_liquidation call is non-blocking. Under the hood it writes to a bounded async channel (capacity: 10,000 items). If the channel is full — which would mean the relay is severely backed up — the event is dropped and a warning is logged. In practice this never happens, but the design ensures the collector loop is never stalled waiting on the relay.
Why a channel, not direct HTTP?
Each exchange fires liquidations at unpredictable rates. During a cascade event, Binance alone can emit hundreds of liquidations per second. Making an HTTP POST for each one would be wasteful and create head-of-line blocking. Instead, a background task drains the channel and batches events before sending.
Layer 2: the batch sender
A dedicated Tokio task runs in the background, pulling liquidations from the channel and batching them:
async fn batch_sender_task(
mut receiver: mpsc::Receiver<RelayLiquidation>,
client: Client,
config: LiquidationRelayConfig,
) {
let mut batch: Vec<RelayLiquidation> = Vec::with_capacity(config.batch_size);
let mut flush_interval = interval(config.batch_timeout);
loop {
tokio::select! {
liq = receiver.recv() => {
match liq {
Some(l) => {
batch.push(l);
if batch.len() >= batch_size {
send_batch(&client, &api_url, &api_key, &mut batch).await;
}
}
None => {
if !batch.is_empty() {
send_batch(&client, &api_url, &api_key, &mut batch).await;
}
break;
}
}
}
_ = flush_interval.tick() => {
if !batch.is_empty() {
send_batch(&client, &api_url, &api_key, &mut batch).await;
}
}
}
}
}
Two flush triggers:
- Size-based: when the batch reaches 50 items, send immediately. During high-volume periods this keeps latency low.
- Time-based: every 1 second, flush whatever has accumulated. During quiet periods this ensures events don't sit in the buffer.
The tokio::select! macro lets both conditions race. Whichever fires first wins. This gives us the best of both worlds: low latency under load, bounded delay during calm markets.
The batch is POSTed as JSON to the relay API endpoint with an internal auth header. If the POST fails, the events are lost — we chose simplicity over guaranteed delivery here because liquidation data is ephemeral and the dashboard is for real-time observation, not historical analysis.
Layer 3: Cloudflare Durable Object
This is where it gets interesting. The relay layer is a single Cloudflare Durable Object — a stateful singleton that lives on Cloudflare's edge network. It serves two roles: it receives liquidation batches over HTTP from the collectors, and it maintains WebSocket connections to browser clients.
Why Durable Objects?
We needed something that could:
- Hold WebSocket connections open for extended periods
- Maintain in-memory state (recent liquidations, aggregates)
- Persist state across restarts
- Scale to many concurrent clients without running a server
Durable Objects are a natural fit. A single instance (keyed by idFromName('global')) handles all connections. Cloudflare's runtime gives us built-in SQLite-backed storage, WebSocket management, and automatic lifecycle handling.
The Hibernation API
A key detail: we use Cloudflare's WebSocket Hibernation API. Standard Durable Objects charge for wall-clock time the object is alive. With hibernation, the object can go to sleep when there are no incoming requests, and wake up when a WebSocket message or HTTP request arrives — without dropping WebSocket connections.
This means we can have hundreds of idle browser connections without paying for compute time. The WebSocket connections survive hibernation because Cloudflare manages them at the platform level.
// Accept WebSocket with hibernation support
this.state.acceptWebSocket(server);
The handlers webSocketMessage, webSocketClose, and webSocketError are called by the runtime when activity occurs — the object doesn't need to poll.
Processing a liquidation
When a batch arrives, each liquidation is processed sequentially:
async processLiquidation(liq) {
// 1. Calculate USD value (with exchange-specific logic)
const usdValue = this.calculateUsdValue(liq);
// 2. Normalize into a standard format
const normalizedLiq = {
id: `${liq.exchange}-${liq.symbol}-${liq.event_time}-${randomId}`,
exchange: liq.exchange,
symbol: liq.symbol,
side: isLong ? 'LONG' : 'SHORT',
price, quantity, usdValue,
timestamp: liq.event_time,
receivedAt: Date.now(),
};
// 3. Prepend to recent list (capped at 500)
this.recentLiquidations.unshift(normalizedLiq);
// 4. Update rolling aggregates
this.updateAggregates(normalizedLiq);
// 5. Broadcast to all connected WebSocket clients
this.broadcastLiquidation(normalizedLiq);
// 6. Persist to storage
await this.state.storage.put({
recentLiquidations: this.recentLiquidations,
aggregates: this.aggregates,
});
}
The USD value calculation has exchange-specific logic. BitMEX inverse contracts (XBTUSD, ETHUSD) express quantity in USD already, while linear contracts need price * quantity. Some exchanges like OKX and Bitget provide an explicit USD value from the collector side.
Rolling window aggregates
Rather than storing every individual event forever, we maintain minute-level buckets for the last 24 hours (up to 1,440 buckets). Each bucket stores total volume, count, and long/short breakdown:
const minuteKey = Math.floor(liq.timestamp / 60000) * 60000;
if (!this.aggregates.minuteBuckets[minuteKey]) {
this.aggregates.minuteBuckets[minuteKey] = {
timestamp: minuteKey,
volumeUsd: 0, count: 0,
longVolumeUsd: 0, shortVolumeUsd: 0,
};
}
bucket.volumeUsd += liq.usdValue;
bucket.count += 1;
When a client requests stats for the 1h, 4h, 12h, or 24h window, we dynamically sum the relevant buckets:
calculateWindowStats() {
const now = Date.now();
const windows = {};
for (const [windowKey, windowMs] of Object.entries(this.AGGREGATE_BUCKETS)) {
const cutoff = now - windowMs;
let volumeUsd = 0, count = 0, longVolumeUsd = 0, shortVolumeUsd = 0;
for (const bucket of Object.values(this.aggregates.minuteBuckets)) {
if (bucket.timestamp >= cutoff) {
volumeUsd += bucket.volumeUsd;
count += bucket.count;
longVolumeUsd += bucket.longVolumeUsd;
shortVolumeUsd += bucket.shortVolumeUsd;
}
}
windows[windowKey] = { volumeUsd, count, longVolumeUsd, shortVolumeUsd };
}
return windows;
}
This is a simple but effective approach. Minute granularity is more than enough for dashboard display, and summing 1,440 objects is near-instant. Old buckets are pruned on every write to keep memory bounded.
Broadcasting with per-client filtering
Clients can subscribe with filters (exchanges, symbols, minimum USD threshold). These filters are stored in the WebSocket attachment — a Durable Object feature that associates arbitrary data with a connection:
ws.serializeAttachment({ filters: data });
On broadcast, we check each client's filters before sending:
broadcastLiquidation(liq) {
const message = JSON.stringify({ type: 'liquidation', data: liq });
for (const ws of this.state.getWebSockets()) {
try {
const attachment = ws.deserializeAttachment();
if (attachment?.filters) {
if (filters.exchanges && !filters.exchanges.includes(liq.exchange)) continue;
if (filters.minUsd && liq.usdValue < filters.minUsd) continue;
}
ws.send(message);
} catch (e) {
// Client disconnected — cleaned up automatically
}
}
}
The filtering happens server-side in the Durable Object, so clients with a $1M+ minimum threshold don't receive noise from small liquidations. Failed sends are silently caught — disconnected clients are automatically cleaned up by the runtime.
Layer 4: the browser
The frontend is a React component that connects via WebSocket and manages state locally.
Connection lifecycle
const ws = createLiquidationsWebSocket({
onSnapshot: (data) => {
// Receive initial state: stats + last 100 liquidations
setStats(data.stats);
setLiquidations(data.recent);
},
onLiquidation: (liq) => {
// Prepend new liquidation, cap at 500
setLiquidations(prev => [liq, ...prev].slice(0, 500));
},
onClose: () => {
// Reconnect after 3 seconds
setTimeout(reconnect, 3000);
},
});
On connect, the Durable Object sends a snapshot message containing the current stats and 100 most recent liquidations. This means a new client immediately sees the current state without waiting for the next event. After the snapshot, individual liquidation messages stream in.
Heartbeats
A ping/pong mechanism runs every 30 seconds to keep the connection alive through proxies and load balancers:
// Client sends
ws.send(JSON.stringify({ type: 'ping' }));
// Durable Object responds
ws.send(JSON.stringify({ type: 'pong', timestamp: Date.now() }));
Stats polling as backup
As a safety net, the frontend also polls the REST /liquidations/stats endpoint every 10 seconds. If a WebSocket message is missed or the connection hiccups, the stats still converge to the correct values. The WebSocket provides low-latency updates; the polling provides consistency.
Architecture diagram
┌─────────────────────────────────────────────────────┐
│ Exchange WebSocket APIs │
│ Binance · Bybit · OKX · Bitget · BitMEX · │
│ Kraken · Aster │
└──────────────────────┬──────────────────────────────┘
│ persistent WS connections
▼
┌─────────────────────────────────────────────────────┐
│ Rust Data Collector │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Binance │ │ Bybit │ │ OKX │ ... │
│ │ module │ │ module │ │ module │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ └──────┬───────┴──────┬───────┘ │
│ ▼ ▼ │
│ CommonLiquidation struct │
│ │ │
│ ▼ │
│ mpsc channel (cap: 10,000) │
│ │ │
│ ▼ │
│ Batch sender task │
│ (flush: 50 items or 1 sec) │
└──────────────────────┬──────────────────────────────┘
│ HTTP POST (batched JSON)
▼
┌─────────────────────────────────────────────────────┐
│ Cloudflare Worker │
│ (API router + internal auth) │
└──────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────┐
│ Durable Object: LiquidationRelay │
│ (single global instance) │
│ │
│ ┌─────────────────┐ ┌──────────────────────┐ │
│ │ In-memory state │ │ SQLite storage │ │
│ │ • recent (500) │ │ (persisted on write) │ │
│ │ • minute buckets │ │ │ │
│ │ • exchange aggs │ └──────────────────────┘ │
│ │ • symbol aggs │ │
│ │ • largest 24h │ │
│ └────────┬────────┘ │
│ │ broadcast │
│ ▼ │
│ ┌──────────────────────────────────┐ │
│ │ WebSocket connections │ │
│ │ (Hibernation API) │ │
│ │ • per-client filters │ │
│ │ • snapshot on connect │ │
│ │ • ping/pong heartbeat │ │
│ └──────────────────────────────────┘ │
└──────────────────────┬──────────────────────────────┘
│ WebSocket frames
▼
┌─────────────────────────────────────────────────────┐
│ Browser (React) │
│ │
│ • Animated liquidation table │
│ • Rolling stat cards (1h/4h/12h/24h) │
│ • Exchange & symbol breakdowns │
│ • Client-side filtering + pause │
│ • Automatic reconnection │
│ • Stats polling fallback (10s) │
└─────────────────────────────────────────────────────┘
Design tradeoffs
A few deliberate choices worth calling out:
Fire-and-forget delivery. If a batch POST from the collector to the Durable Object fails, those liquidations are lost. We don't queue or retry. For a real-time dashboard, stale data is worse than missing data. Our historical data collectors are much less forgiving — there we strictly care about capturing every single event — but for a live dashboard, freshness matters more than completeness.
Single Durable Object instance. All traffic goes through one global instance. Durable Objects can handle thousands of concurrent WebSocket connections and significant HTTP throughput. If we ever hit the ceiling, we could shard by exchange or by region — but a single instance keeps the code dramatically simpler and avoids coordination overhead.
Minute-level aggregation. We chose minute buckets over per-event storage for aggregates. This caps memory at ~1,440 buckets regardless of throughput. The tradeoff is that window boundaries are approximate to the minute, which is fine for a dashboard.
Server-side filtering. Filters are applied in the Durable Object, not the client. This reduces bandwidth for clients with restrictive filters but adds CPU to the broadcast loop. With the current scale (hundreds, not millions, of concurrent clients) this is the right call.
No message queue. We intentionally skipped Kafka, Redis Pub/Sub, or any external message broker. The Rust async channel + Durable Object combination provides the fan-out we need without operational overhead. One less thing to monitor.
What we'd do differently at 100x scale
If the system needed to handle 100x more liquidation events or 100x more clients:
- Shard the Durable Object by exchange or geographic region, with a fan-out layer that routes clients to the right shard
- Add a message queue (e.g., Cloudflare Queues) between the collector and the Durable Object for retry and backpressure
- Move aggregation to a separate worker so the broadcast path stays fast
- Use Cloudflare Pub/Sub (when available for WebSocket fan-out) to move the broadcast loop out of the Durable Object
For now, the current architecture handles our traffic with headroom to spare, and the simplicity makes it easy to debug and extend.
Try it
The live feed is running at cryptohftdata.com/liquidations. Open it during a volatile market and watch the cascades unfold in real time.
If you want to work with the data programmatically, our REST API and Python SDK give you access to historical liquidation data at tick level across all supported exchanges.