Skip to main content

D111 — sFOX real-time ingestion: REST polling + reconciliation first; WebSocket listener deferred

Architecture decision record. Status, thematic clusters, and how to record a new ADR: the decision log index. The mechanics live in packages/stables/sfox-sdk/ (message layer, transport-ready), app-modules/rails/ (sync adapter, when a consumer exists), and docs/tracking/ (operational posture, polling cadence, reconciliation heuristics).

Context​

The sFOX SDK (packages/stables/sfox-sdk, PR #114) landed as a complete transport layer (Saloon connector + DTOs + typed WebSocket message builders/parsers) for both the sFOX Connect API and the main sFOX API. sFOX has no webhooks anywhere — the platform's only push channel is a WebSocket feed (Connect four feeds: Onboarding, Balances, OpenOrders, Trades; main sFOX four feeds: OpenOrders, Trades, Balances, PostTradeSettlement, plus market data).

The immediate integration task is real-time sFOX data visibility into Stables' mirrors (balances, orders, positions, etc. — the REST read surfaces for tenant-facing APIs and compliance/ops views). This ADR records the decision on WHEN and HOW to activate the WebSocket listener, and why the SDK's message layer ships now despite the listener being deferred.

Decision​

Real-time sFOX data ingestion starts with REST polling + mirror reconciliation only. The WebSocket listener (a dedicated single-task ECS daemon consuming the four Connect partner feeds via the typed message layer) is explicitly deferred until a concrete product need demands sub-minute event latency (e.g. trading-UX balance freshness or transaction settlement streaming to clients).

  • Initial posture (now). Domain code reads sFOX state via REST poll-on-demand or a scheduled reconciliation job that reconciles the canonical sFOX state against the tenant mirror tables at a fixed cadence (e.g. hourly for balances, reconcile-on-settlement for orders). REST is already the correctness backbone for all provider mirrors in this app (see AbstractMirrorReconciler in app-modules/provider-mirror/ and the mirrors:reconcile-stale command); sFOX mirrors ride that same reconciliation guardrail.

  • WebSocket message layer future-proofing (now). The SDK ships the full message layer — ConnectMessageBuilder for outbound command serialization, ConnectFeedMessageFactory for inbound frame parsing + typed payload DTOs. The bridge between the layer and a live socket is the only gap: a thin async-client wrapper (Amphp or Reactphp) that pipes raw frame bytes → json_decode → ConnectFeedMessageFactory::make() → domain command dispatch. The SDK itself needs zero changes when the listener is built; the message layer is complete and production-ready today.

  • Listener shape (when needed). A single desired_count=1 ECS daemon task on one of the four Connect partner feeds (or fan-out to all four on separate task definitions if that becomes necessary). The listener runs a thin event loop: socket connects → authenticate with the partner token → subscribe → for each inbound frame: parse via factory → dispatch a queued mirror-upsert job (via the task-runner). The job executes in a Horizon queue worker through the same idempotent mirror-upsert path (out-of-order guards) used by REST reconciliation — the daemon never writes directly. On reconnect, re-sync the mirror from REST (a GET-all-state read) to heal the gap during downtime. The daemon never owns the write — it queues jobs that Horizon workers execute, preserving the single-write-path invariant.

Rationale: operational burden vs. latency gain. A WebSocket consumer is a new long-running ECS service + a new async-client dependency + reconnection/gap-healing operational toil (monitoring uptime, heartbeat loss, reauth races, backpressure queuing). REST reconciliation is already the repo's correctness backbone; the listener would only buy latency, never truth. Until a feature explicitly needs sub-minute event visibility (not just correctness), the cost of the daemon + the operational surface area is not justified by the latency delta vs. a 60-second REST poll + reconcile.

Consequences​

  • Latency bound. Mirror state lags sFOX state by the poll/reconcile cadence (initially 60s for balances, longer for orders). Responses to client balance reads come from the mirror, not sFOX real-time. This is acceptable for retail product features; trading UX or margin-call velocity would justify revisiting.
  • No new dependency now. The app stays on its current async-client/stream ecosystem; a WS listener adds one (Amphp or Reactphp). This is deferred pending product need.
  • No new service now. ECS task definition + monitoring + on-call runbooks for a daemon are non-trivial. Reconciliation via the existing task-runner is a lower-surface-area mechanism.
  • Message layer is complete. When the listener is built, it will consume the existing typed frame factories + payload DTOs with zero SDK changes — the bridge is the only new code.
  • Mirror write path stays single. Whether the write arrives via REST poll, scheduled reconciliation, or (future) WebSocket queue, the domain module takes exactly one path (the rails module's idempotent mirror-upsert commands via the CQRS bus, e.g. the ManagesWallets or OnboardsCustomers capability interfaces), preserving the CQRS bus + command-idempotency invariant.

Revisit trigger​

Add the WebSocket listener to the roadmap when:

  1. A product feature needs sFOX event latency < poll cadence (e.g. "show real-time balance updates in the app"; "settle trades on the platform immediately upon sFOX notification"), or
  2. Compliance/ops workflows need sub-minute event streaming (e.g. regulatory reporting that consumes live feed data, not snapshot reconciliation).

Short of either, the REST + reconciliation posture is production-ready and operationally simpler.

← Decision log index