FIX Antenna C++ 2.19.0 Results
Overview
FIX Antenna C++ performance was measured using the sample from the FA package (samples\Benchmark\Latency2). Configuration properties can also be found in the package.
You can reproduce the measurement on your hardwareĀ by compiling andĀ executingĀ the sample described above using the following instruction:
How to measure FA performance using the FA package sample
The following instruction provides the steps to measure the FIX Antenna C++ performance using the Benchmark samples on the FA package.
Two LinuxĀ machines should be used for the test, one as the Sender Host (server), the other as the Receiver Host (client).
Download the FIX Antenna C++ package, unzip it and go to the \Samples\Benchmark\Latency2 folder.
- Run the makefile.ind to compile all samples.
Go to the \Samples\Benchmark\Latency2\bin folder.
Open the settings.sh script and change the IP address from the localhost to the IP of the Server Host in the ServerAdress parameter.
On the Server side run the run.Server.<configuration_name>.sh script and on the Client side - the run.Client.<configuration_name>.sh script. As soon as both of them are executed, the RTT is automatically measured. The latency.csv file with results will appear in the same folder.In the optimized.hard configuration the number of cores used by the FIX sessions is directly written in the configuration. Please change it according to your machines. It should be changed in the settings.sh file both in the TasksetServerCores and TasksetClientCores (please see the picture above) parameters.Ā
In the optimized.balanced.onload configurationĀ the Solarflare OpenOnload profile is specified. Please specify your existing configuration orĀ set upĀ the new one in the OpenOnload. It should be changed in the settings.sh file both in theĀ OnloadServerProfile andĀ OnloadClientProfile (please see the picture above) parameters.Ā
Environment
Sender Host:
- Intel(R) Xeon(R) CPU E5-2643 v3 @ 3.40GHz (2 CPU Hyper-Trading Enabled, 24 Cores)
- RAM 128 GB, 2133 MHz
- NIC Solarflare Communications SFC9120 (Firmware-version: 4.2.2.1003 rx1 tx1)
- Linux (CentOS 7.0.1406 kernelĀ 3.10.0-123.el7.x86_64)
- SolarFlare driver version: 4.1.0.6734a
Receiver Host:
- Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHzĀ (2 CPUĀ Hyper-Trading Enabled, 20 Cores)
- RAM 128 GB, 2133 MHz
- NIC Solarflare Communications SFC9120 (Firmware-version: 4.2.2.1003 rx1 tx1)
- Linux (CentOS 7.0.1406 kernelĀ 3.10.0-123.el7.x86_64)
- SolarFlare driver version: 4.1.0.6734a
Test scenario
The test scenario is the following:
- Two test servers are connected via 10GB link.
- AnĀ Initiator FIX session is established on theĀ Sender Host,Ā an Acceptor FIX session is established on the Receiver Host.
- The InitiatorĀ sends the New Order Single (35=D)Ā message to the Acceptor; the Acceptor receives, validates and parses the message and sendsĀ the Execution reportĀ (35=8) message backĀ to the Initiator.
During the test Round-trip time (RTT) is measured. The first measurement, t1,Ā is made before the messageĀ isĀ sentĀ by the Initiator, the second, t2, is madeĀ after the received message is parsed by theĀ Initiator.
RTT=t2-t1.
Test configurations
Properties | Default | Aggressive | Balanced | Optimized.hard.spin | Balanced.onload | Optimized.hard | |
---|---|---|---|---|---|---|---|
Prepared messages1 | No | No | No | Yes | No | Yes | |
Affinity | No | No | Yes | No | Yes | Yes | |
Thread Pool or Aggressive Mode | Thread Pool (10 Threads) | Aggressive Mode2 | Aggressive Mode | Aggressive Mode | Aggressive Mode | Aggressive Mode | |
Message validation parameters | IgnoreUnknownFields | ||||||
ProhibitUnknownTags | ā | ā | ā | ā | ā | ā | |
CheckDuplicateTags | ā | ā | ā | ā | ā | ā | |
MessageMustBeValidated | ā | ā | |||||
VerifyRepeatingGroupBounds | ā | ā | |||||
Validation.CheckRequiredGroupFields | ā | ā | |||||
Validation.VerifyTagsValues | ā | ā | |||||
Validation.ProhibitTagsWithoutValue | ā | ā | |||||
Storage type | PersistentMM3 | PersistentMM | PersistentMM | PersistentMM | PersistentMM | PersistentMM | |
Solarflare OpenOnload4 | No | No | No | No | Yes | Yes |
1 Prepared messages - a mechanism that preallocates storage for messages (buffers)Ā and then reuses it.
2 Aggressive Mode - 2 dedicated threads are allocated for the FIX session: one for sending messages, another - for receiving them.
3 PersistentMM storage type =Ā a file on the disc.
4Ā Ā Solarflare OpenOnload - the kernel bypassĀ technique by Solarflare thatĀ is activated within the test.
The following description will help to choose the most relevant configuration of FIX Antenna C++:
- Default. It is the good quick start variant with balancedĀ performance and security. The configuration may be effectively applied both to the Sender Host and the Receiver Host.
- Aggressive. It is the optimized version of the default configuration. The aggressive mode is used, so that there are allocated two dedicated threads for each FIX session. The configuration is advised to be used for limited amount of sessionsĀ (say, tens orĀ hundreds sessions)Ā and may be effectively applied both to the Sender Host and the Receiver Host.
- Balanced. Most validation parameters are switched off to improve performance. The configuration is advised to be used in the controlled environment and is preferableĀ for the Sender Host.
- Optimized.hard.spin. The spinning mode is activated in the configuration by the following setting:Ā Session.Default.AggressiveReceiveDelay = 0, it forces threads to spin. It is not recommended to use the configuration if the number of threads is significant as itĀ is highly consumable for the CPU.Ā It is advised to be used for the Sender Host.
- Balanced.onload. The Solarflare OpenOnload technique activated within the configuration optimizes the TCP stack. However, the configuration may beĀ applied only if there is a Solarflare Network interface controller installed on the machine. It is advised to be used for the Sender Host.
- Optimized.hard. For theĀ RTT reduction, the affinity masksĀ are used in the configuration. Also few validation parameters, which are not necessaryĀ in the fully controlledĀ environment,Ā were switched off. It is advised to be used for the Sender Host.
Results
FIX Antenna C++ configurations comparison
Configuration | Default | Aggressive | Balanced | Optimized.hard.spin | Balanced.onload | Optimized.hard |
---|---|---|---|---|---|---|
RTTĀ (microseconds) | ||||||
Min | 47,1 | 46,1 | 27,3 | 19,0 | 11,3 | 8,3 |
Max | 190,7 | 218,0 | 217,5 | 223,7 | 199,5 | 13,2 |
Average | 51,6 | 51,1 | 31,5 | 21,6 | 13,2 | 8,7 |
RTTĀ distribution (percentiles) | ||||||
50% | 49,8 | 48,8 | 29,9 | 21,0 | 11,8 | 8,5 |
95% | 59,9 | 59,0 | 38,9 | 24,0 | 18,6 | 9,3 |
99% | 67,1 | 90,4 | 45,8 | 25,3 | 24,6 | 10,4 |
FIX Antenna C++ 2.19.0Ā vs QuickFIX C++ 1.14.3
TheĀ benchmarkĀ code was ported to QuickFIX:
Results can be compared in the following table:
Configuration | FIXAntenna default | FIXAntenna balanced | QuickFIX no validation | QuickFIX validation |
---|---|---|---|---|
RTT (microseconds) | ||||
Min | 47,1 | 27,3 | 155,0 | 157 |
Max | 190,1 | 217,5 | 371,5 | 377,8 |
Average | 51,6 | 31,5 | 158,9 | 161,8 |
RTTĀ distribution (Percentiles) | ||||
50% | 49,8 | 29,9 | 157,6 | 159,5 |
95% | 59,9 | 38,9 | 165,0 | 167,1 |
99% | 67,1 | 45,9 | 177,4 | 183,5 |