FIX Antenna C++ 2.19.0 Results
Overview
FIX Antenna C++ performance was measured using the sample from the FA package (samples\Benchmark\Latency2). Configuration properties can also be found in the package.
You can reproduce the measurement on your hardware by compiling and executing the sample described above using the following instruction:
How to measure FA performance using the FA package sample
The following instruction provides the steps to measure the FIX Antenna C++ performance using the Benchmark samples on the FA package.
Two Linux machines should be used for the test, one as the Sender Host (server), the other as the Receiver Host (client).
Download the FIX Antenna C++ package, unzip it and go to the \Samples\Benchmark\Latency2 folder.
- Run the makefile.ind to compile all samples.
Go to the \Samples\Benchmark\Latency2\bin folder.
Open the settings.sh script and change the IP address from the localhost to the IP of the Server Host in the ServerAdress parameter.
On the Server side run the run.Server.<configuration_name>.sh script and on the Client side - the run.Client.<configuration_name>.sh script. As soon as both of them are executed, the RTT is automatically measured. The latency.csv file with results will appear in the same folder.In the optimized.hard configuration the number of cores used by the FIX sessions is directly written in the configuration. Please change it according to your machines. It should be changed in the settings.sh file both in the TasksetServerCores and TasksetClientCores (please see the picture above) parameters.
In the optimized.balanced.onload configuration the Solarflare OpenOnload profile is specified. Please specify your existing configuration or set up the new one in the OpenOnload. It should be changed in the settings.sh file both in the OnloadServerProfile and OnloadClientProfile (please see the picture above) parameters.
Environment
Sender Host:
- Intel(R) Xeon(R) CPU E5-2643 v3 @ 3.40GHz (2 CPU Hyper-Trading Enabled, 24 Cores)
- RAM 128 GB, 2133 MHz
- NIC Solarflare Communications SFC9120 (Firmware-version: 4.2.2.1003 rx1 tx1)
- Linux (CentOS 7.0.1406 kernel 3.10.0-123.el7.x86_64)
- SolarFlare driver version: 4.1.0.6734a
Receiver Host:
- Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz (2 CPU Hyper-Trading Enabled, 20 Cores)
- RAM 128 GB, 2133 MHz
- NIC Solarflare Communications SFC9120 (Firmware-version: 4.2.2.1003 rx1 tx1)
- Linux (CentOS 7.0.1406 kernel 3.10.0-123.el7.x86_64)
- SolarFlare driver version: 4.1.0.6734a
Test scenario
The test scenario is the following:
- Two test servers are connected via 10GB link.
- An Initiator FIX session is established on the Sender Host, an Acceptor FIX session is established on the Receiver Host.
- The Initiator sends the New Order Single (35=D) message to the Acceptor; the Acceptor receives, validates and parses the message and sends the Execution report (35=8) message back to the Initiator.
During the test Round-trip time (RTT) is measured. The first measurement, t1, is made before the message is sent by the Initiator, the second, t2, is made after the received message is parsed by the Initiator.
RTT=t2-t1.
Test configurations
Properties | Default | Aggressive | Balanced | Optimized.hard.spin | Balanced.onload | Optimized.hard | |
---|---|---|---|---|---|---|---|
Prepared messages1 | No | No | No | Yes | No | Yes | |
Affinity | No | No | Yes | No | Yes | Yes | |
Thread Pool or Aggressive Mode | Thread Pool (10 Threads) | Aggressive Mode2 | Aggressive Mode | Aggressive Mode | Aggressive Mode | Aggressive Mode | |
Message validation parameters | IgnoreUnknownFields | ||||||
ProhibitUnknownTags | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
CheckDuplicateTags | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
MessageMustBeValidated | ✔ | ✔ | |||||
VerifyRepeatingGroupBounds | ✔ | ✔ | |||||
Validation.CheckRequiredGroupFields | ✔ | ✔ | |||||
Validation.VerifyTagsValues | ✔ | ✔ | |||||
Validation.ProhibitTagsWithoutValue | ✔ | ✔ | |||||
Storage type | PersistentMM3 | PersistentMM | PersistentMM | PersistentMM | PersistentMM | PersistentMM | |
Solarflare OpenOnload4 | No | No | No | No | Yes | Yes |
1 Prepared messages - a mechanism that preallocates storage for messages (buffers) and then reuses it.
2 Aggressive Mode - 2 dedicated threads are allocated for the FIX session: one for sending messages, another - for receiving them.
3 PersistentMM storage type = a file on the disc.
4 Solarflare OpenOnload - the kernel bypass technique by Solarflare that is activated within the test.
The following description will help to choose the most relevant configuration of FIX Antenna C++:
- Default. It is the good quick start variant with balanced performance and security. The configuration may be effectively applied both to the Sender Host and the Receiver Host.
- Aggressive. It is the optimized version of the default configuration. The aggressive mode is used, so that there are allocated two dedicated threads for each FIX session. The configuration is advised to be used for limited amount of sessions (say, tens or hundreds sessions) and may be effectively applied both to the Sender Host and the Receiver Host.
- Balanced. Most validation parameters are switched off to improve performance. The configuration is advised to be used in the controlled environment and is preferable for the Sender Host.
- Optimized.hard.spin. The spinning mode is activated in the configuration by the following setting: Session.Default.AggressiveReceiveDelay = 0, it forces threads to spin. It is not recommended to use the configuration if the number of threads is significant as it is highly consumable for the CPU. It is advised to be used for the Sender Host.
- Balanced.onload. The Solarflare OpenOnload technique activated within the configuration optimizes the TCP stack. However, the configuration may be applied only if there is a Solarflare Network interface controller installed on the machine. It is advised to be used for the Sender Host.
- Optimized.hard. For the RTT reduction, the affinity masks are used in the configuration. Also few validation parameters, which are not necessary in the fully controlled environment, were switched off. It is advised to be used for the Sender Host.
Results
FIX Antenna C++ configurations comparison
Configuration | Default | Aggressive | Balanced | Optimized.hard.spin | Balanced.onload | Optimized.hard |
---|---|---|---|---|---|---|
RTT (microseconds) | ||||||
Min | 47,1 | 46,1 | 27,3 | 19,0 | 11,3 | 8,3 |
Max | 190,7 | 218,0 | 217,5 | 223,7 | 199,5 | 13,2 |
Average | 51,6 | 51,1 | 31,5 | 21,6 | 13,2 | 8,7 |
RTT distribution (percentiles) | ||||||
50% | 49,8 | 48,8 | 29,9 | 21,0 | 11,8 | 8,5 |
95% | 59,9 | 59,0 | 38,9 | 24,0 | 18,6 | 9,3 |
99% | 67,1 | 90,4 | 45,8 | 25,3 | 24,6 | 10,4 |
FIX Antenna C++ 2.19.0 vs QuickFIX C++ 1.14.3
The benchmark code was ported to QuickFIX:
Results can be compared in the following table:
Configuration | FIXAntenna default | FIXAntenna balanced | QuickFIX no validation | QuickFIX validation |
---|---|---|---|---|
RTT (microseconds) | ||||
Min | 47,1 | 27,3 | 155,0 | 157 |
Max | 190,1 | 217,5 | 371,5 | 377,8 |
Average | 51,6 | 31,5 | 158,9 | 161,8 |
RTT distribution (Percentiles) | ||||
50% | 49,8 | 29,9 | 157,6 | 159,5 |
95% | 59,9 | 38,9 | 165,0 | 167,1 |
99% | 67,1 | 45,9 | 177,4 | 183,5 |