LB Cluster Solution Documentation

Prerequisites

FIXEdge servers are often used in business critical systems for interchange of financial information between clients and providers. Usually such systems require high-availability because downtime leads to significant money losses. Besides being high-load systems they often require good scalability that is a problem for non-cluster systems. Having several standalone FIXedge servers may be a reasonable approach in some cases but it has some significant drawbacks - client sessions must be distributed manually, it may be hard to manage configuration of several FIXEdge nodes, system will not be failure tolerant. To meet these needs Load-Balancing Cluster Solution is provided to:

  • Enable customers to distribute the workload over several FIXEdge nodes transparently to their clients.
  • Enable customers to provide service that is tolerant to FIXEdge node failures.
  • Enable customers to use shared configuration to avoid possibility of inconsistent configuration of FIXEdge nodes.

Solution overview

Load-Balancing cluster is a solution based on FIXEdge product. Several FIXEdge nodes (minimum two) are logically combined and work as one for clients. Client sessions contact Load-Balancer which decides what FIXEdge node will be serving each incoming session. The distribution of incoming sessions depends on load-balancing algorithm. Once FIX session is established it is handled by the same node until logout or TCP connection termination. Failover procedure is automatic - once the node serving the session fails session connection is rejected and on client reconnect it will be forwarded to one of the remaining active nodes. 

FIXEdge nodes can be dynamically added and removed to/from a cluster without a need to restart cluster as a whole. 

Cluster health is monitored by checking health of services/components. Healthchecks are used for maintaining work of cluster services, besides it can be used for integration with external monitoring systems in the future.

Cluster is able to run several instances of some critical services such as Configuration Service, Scheduler Service, Cluster Coordination Service to increase availability. A brief description of cluster components and logical diagram are available here.

Deployment of cluster is described here.

Configuration tool (command-line tool) is used to change cluster configuration. User manual is available here. Other properties are configured in local property files for FIXEdge nodes (see).

LB Cluster Components

Component diagram illustrates high-level structure of LB-Cluster Solution and interconnection with other systems. Two rectangles indicate that there may be several instances of a service.


Component typeDescriptionImplementation notes
FIXEdge nodeThe FIXEdge nodes process the messages passing from and to the clientsFunctionally it is FIXEdge server/FIXEdge Java server
Load BalancerLoad balancer redirects incoming connections to the one of the FIXEdge nodes. It is a single entry point for clients initiator sessions.3-rd party product is used - HA-proxy
Configuration Service Configuration Service is the centralized source of the cluster configuration. It manages sessions and schedules configurations.
Cluster Coordination ServiceCluster Coordination Service is used to coordinate FIXEdge nodes with each other. It is user to register other services/components, manage healthchecks.3-rd party product is used - Consul
Scheduler ServiceScheduler Service allows to manage sessions states according to the specified events.
Shared FIX StorageShared FIX Storage is used to store FIX messages and sessions' last incoming and outgoing sequence numbers.

Oracle DB is used. Other DB systems may be used - MSSQL, PostgreSQL, etc.

FO is used for FIXEdge Java solution.

Configuration StorageConfiguration Storage is used to store cluster configurationOracle DB  is used. Other DB systems may be used - MSSQL, PostgreSQL, etc.
Configuration Tool (CLI)Configuration Tool is used to configure the clusterCommand Line Tool
Log Collector (+Log Storage)FIXEdge application logs are sent to Log Collector which provide single point to search and analyze the logs. 3-rd party product is used - Splunk