Benchmarking of FIX Antenna .NET Core 0.9.2 Linux
Environment
Server host (epam1)
- Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz (2 CPU, 20 Cores)
- RAM 128 GB, 2133 MHz
- NIC Solarflare Communications SFC9120 (Firmware-version: 4.2.2.1003 rx1 tx1)
- Linux (CentOS 7.0.1406 kernel 3.10.0-957.21.3.el7.x86_64)
- SolarFlare driver version: 4.1.0.6734a
Client host (epam2)
- Intel(R) Xeon(R) CPU E5-2643 v3 @ 3.40GHz (2 CPU, 12 Cores)
- RAM 128 GB, 2133 MHz
- NIC Solarflare Communications SFC9120 (Firmware-version: 4.2.2.1003 rx1 tx1)
- Linux (CentOS 7.0.1406 kernel 3.10.0-957.21.3.el7.x86_64)
- SolarFlare driver version: 4.1.0.6734a
Test scenario
Single Session Echo Scenario
- FA .NET Core has one acceptor session configured on server host.
- Client application has one initiator session configured on client host.
The overall process is following:
- Client application connects to the FA .NET Core instance and sends 200000 FIX 4.2 messages with a rate of 5000 messages per second.
- FA .NET Core receives the messages and matches it to the same session using business layer logic.
- FA .NET Core responds to the client application with the same message in the same session.
The response time measured by client application is the difference between timestamps:
- t1 - timestamp taken right before sending message to client session.
- t2 - timestamp taken right after received the same message in client back from FA .NET Core.
The round-trip time formula is: RTT=t2-t1 and measured in microseconds.
The scenario is the same for FIX Antenna C++. Please see details here
The Sender part latency will be the same in both cases (as for testing FIX Antenna C++ and FIX Antenna .Net Core) and roughly it can be considered as half of numbers from here for Optimized.hard configuration.
Results
Test configurations
- Nagle's algorithm -is disabled
- Affinity - sessions threads are pinned to the specific CPU core.
- Message validation- is disabled
- Storage type and Queue type - In memory
- Solarflare OpenOnload - the kernel bypass technique by Solarflare is used.
Echo benchmark for transient session
FA .NET Core 0.9.2, µsec | |||||
---|---|---|---|---|---|
#1 | #2 | #3 | #4 | #5 | |
Min | 13.6 | 13.6 | 13.8 | 13.8 | 13.8 |
Max | 334.8 | 228.0 | 212.9 | 800.8 | 350.7 |
Median | 14.3 | 14.2 | 14.4 | 14.4 | 14.3 |
Average | 14.8 | 14.7 | 14.8 | 14.9 | 14.8 |
50% | 14.3 | 14.2 | 14.4 | 14.4 | 14.3 |
95% | 15.7 | 15.6 | 15.7 | 15.8 | 15.7 |
99% | 18.5 | 18.3 | 18.4 | 18.8 | 18.4 |
The median for 0.9.2 is within interval 14.32±0.05 (coefficient of variation (CV) is 0.3%).
Iteration | 0.9.2 median, µsec |
---|---|
#1 | 14.3 |
#2 | 14.2 |
#3 | 14.4 |
#4 | 14.4 |
#5 | 14.3 |
avg | 14.3 |
std.dev. | 0.048 |
CV | 0.003 |
Echo benchmark for sessions with file storage
Test configurations
- Nagle's algorithm -is disabled
- Affinity - sessions threads are pinned to the specific CPU core.
- Message validation- is disabled
- Storage type and Queue type - In file
- Solarflare OpenOnload - the kernel bypass technique by Solarflare is used.
FA .NET 0.9.2, µsec | |||||
---|---|---|---|---|---|
#1 | #2 | #3 | #4 | #5 | |
Min | 25.9 | 26.0 | 25.9 | 25.8 | 25.8 |
Max | 412.2 | 422.9 | 377.6 | 402.9 | 399.1 |
Median | 27.1 | 27.2 | 27.1 | 27.1 | 27.0 |
Average | 27.8 | 27.9 | 27.8 | 27.8 | 27.7 |
50% | 27.1 | 27.2 | 27.1 | 27.1 | 27.0 |
95% | 30.0 | 30.2 | 30.1 | 30.1 | 30.0 |
99% | 33.8 | 33.8 | 33.8 | 33.9 | 33.8 |
The median for 0.9.2 is within interval 27.11±0.9 (coefficient of variation (CV) is 0.3%).
Iteration | 0.9.2 median, µsec |
---|---|
#1 | 27.1 |
#2 | 27.2 |
#3 | 27.1 |
#4 | 27.1 |
#5 | 27.0 |
avg | 27.1 |
std.dev. | 0.086 |
CV | 0.003 |
Echo benchmark compared to QuickFIX/n
This picture shows overall performance comparison for single session echo scenario.
Validation benchmark
Benchmark creates two validation engines (for FIX4.2 and for FIX4.4) to validate message.
Results show amount of time used to validate one message by validation engine initialized for FIX4.2 or FIX4.4.
Message:
public const string StandardMessage =
"8=FIX.4.2\u00019=153\u000135=D\u000149=BLP\u000156=SCHB\u000134=01\u000150=30737\u000197=Y\u000152=20000809-20:20:50
\u000111=90001008\u00011=10030003\u000121=2\u000155=TESTA\u000154=1\u000138=4000\u000140=2\u000159=0\u000144=30
\u000147=I\u000160=20000809-18:20:32\u000110=061\u0001";
Summary:
BenchmarkDotNet=v0.12.1, OS=Windows 8.1 (6.3.9600.0)
Intel Core i7-6700 CPU 3.40GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3328122 Hz, Resolution=300.4698 ns, Timer=TSC
.NET Core SDK=3.1.302
[Host] : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
Core3.1 : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
Net48 : .NET Framework 4.8 (4.8.4180.0), X64 RyuJIT
LaunchCount=5
Results:
Method | Runtime | Mean | Error | StdDev | Median |
---|---|---|---|---|---|
'Validate FIX4.2' | .NET Core 3.1 | 44.69 μs | 0.163 μs | 0.388 μs | 44.66 μs |
'Validate FIX4.4' | .NET Core 3.1 | 76.72 μs | 0.614 μs | 1.659 μs | 76.19 μs |
'Validate FIX4.2' | .NET 4.8 | 55.95 μs | 0.114 μs | 0.273 μs | 55.88 μs |
'Validate FIX4.4' | .NET 4.8 | 92.52 μs | 0.272 μs | 0.674 μs | 92.49 μs |