Configuring FIX Sessions
Overview
The settings of FIX sessions in the FIXEdge are defined in two configuration files: 'FIXEdge.properties' and 'engine.properties'. There are common settings for all FIX sessions and individual settings for each registered FIX session. The common settings are located in both configuration files and described in the part 'Configuring FIX Sessions#Common Settings of Sessions'. The individual settings of registered FIX session are located in the file 'FIXEdge.properties' and described in the part 'Settings of registered FIX Sessions'.
Basic concepts related to the FIX session and its configuring are described in part 'Configuring FIX Sessions#Introduction and Terminology'
Introduction and Terminology
A FIX session is a bi-directional stream of ordered FIX messages between two counterparties (FIX applications). The FIX message transmission is performed within the FIX connection. Each FIX connection comprises of three parts: logon, message exchange and logout. Parts logon and logout are used for initiation and termination of the FIX session, correspondingly.
Initiator and Acceptor
In the FIX session, one counterparty plays the role of an Initiator and other - plays the role of an Acceptor. The Initiator is responsible for establishing the connection (TCP/IP) and sends the first Logon message to the Acceptor. The Acceptor has a responsibility to perform authentication and replies the confirming Logon message to the Initiator as a declaration that FIX session has been established. In case of a connection failure, the Acceptor waits when the Initiator will initiate reconnection.
Figure 1. FIX connection
When the counterparty decides to terminate the FIX session, it sends the Logout message to other party. The confirming Logout message should be generated in response.
Session Identification
Both the Initiator and the Acceptor have own logical textual identifiers that must be bilaterally agreed between counterparties. These identifiers are included in each FIX message: SenderCompID (49) and TargetCompID (56) tags.
All FIX messages must have the same version of the FIX protocol (BeginString (8) tag) in the one FIX session. Therefore, message's tags (SenderCompID, TargetCompID and BeginString) identify a FIX session.
Registered and Unregistered FIXEdge Sessions
FIXEdge can work with registered FIX sessions as well as with unregistered FIX sessions.
Working with registered sessions is a standard mode. This mode assumes that before using, each FIX session must be registered in the configuration file ‘FIXEdge.properties’, where the session has own session name and a set of properties that describes specifics of the session behavior. Refer to part ‘Settings of Registered FIX Sessions’ for more information.
Working with unregistered sessions is an auxiliary mode for FIXEdge. The configuration property Configuring FIX Sessions#UnregisteredAcceptor.CreateSession can be set to ‘true’ or ‘false’ to turn this mode on or off, correspondingly.
All registered FIX sessions are created in the FIXEdge memory at the time of FIXEdge starting.
The session - initiator sends the Logon messages to establish the connection and waits for a confirming Logon message.
The session-acceptor waits for incoming logon right after creation. When FIXEdge receives incoming Logon message it tries to find the corresponding session-acceptor in the session list, i.e. session-acceptor with SenderCompID and TargetCompID equal to the TargetCompID and SenderCompID extracted from the received Logon message accordingly. If such a session exists FIXEdge checks credentials and then links incoming message flow to the found session-acceptor. Otherwise, FIXEdge treats this situation as an "unknown connection".
If UnregisteredAcceptor.CreateSession property is set to ‘false’, then incoming Logon messages from unknown connections are ignored.
If UnregisteredAcceptor.CreateSession property is set to ‘true’ then each incoming Logon message from unknown connection initiates creation of the session-acceptor.
Such FIX sessions that are created without preliminary settings are called ‘unregistered acceptor.’ All unregistered sessions have similar behavior because is described with the same set of properties (refer to the part ‘Configuring FIX Sessions#Settings of Unregistered Acceptors’ for more information).
Note: All incoming application messages having been received before session initiation (before Logon message) are rejected
Session Name
Each FIX session registered in FIXEdge has its own name. The session name must be unique among names of FIX Sessions and Transport Adaptor Sessions because the session name can be also used for session definition as a Source or a Destination in the business layer rules.
It must contain characters with codes from ASCII [30] to ASCII [128] and must begin with a letter, digit or underlining symbol ‘_’.
Full list of Session names are registered in property Configuring FIX Sessions#FixLayer.FixEngine.Sessions.
Session Time Schedule
It is recommended that a new FIX session is established once within each 24 hour period. The counterparties must bilaterally agree as to when FIX sessions are to be started/stopped based upon individual system and time zone requirements. There are DefaultStartTime and DefaultTerminateTime properties to specify the default session working time.
Multiple logins
The several FIX connections could be sequentially established (not concurrent) in one FIX session. In other terms, the counterparties can connect and disconnect multiple times while maintaining a single FIX session, meaning that the FIX session spans multiple logins.
For registered sessions, the RecreateOnLogout property defines the possibility of multiple logins. If the property is set to ‘false’, the session is removed from the list of sessions after successful disconnection (by Logout message). If set to 'true', the session will be recreated after disconnection.
Note: Disconnection without the exchange of Logout messages is interpreted as an abnormal condition (failure connection). In case when the connection was broken without a Logout message, the session is considered as deliberately disconnected "non-gracefully".
The Initiator is responsible for restoring the connection once it is broken. When a connection error is detected, the Initiator starts a reconnection process: restoring the telecommunication link and exchanging of Logon messages. The number of reconnection attempts and delay can be set in the Reconnect.MaxTries and Configuring FIX Sessions#Reconnect.Interval properties.
Username and Password
Username (553) and Password (554) can be sent in the Logon message for security purposes for both Active and Backup connections. It is also possible to send a Username and Password in different tags other than standard ones (for instance to increase security). It can be done by setting the UsernameTag and PasswordTag parameters (available for both Active and Backup connections).
Example: FixLayer.FixEngine.Session.Session_Name.UsernameTag=10553
FixLayer.FixEngine.Session.Session_Name.PasswordTag=10554
FixLayer.FixEngine.Session.Session_Name.Username=user
FixLayer.FixEngine.Session.Session_Name.Password=111
In this case, the Logon message would contain 10553 and 10554 custom tags instead of standard Username (553) and Password (554) tags: 10553=user and 10554=111.
UsernameTag and PasswordTag properties can be defined in a custom Logon message for active connections. In case they are not present in a custom Logon message but exist in the FixEdge.properties file, the actual Logon message would contain those two predefined tags and corresponding values.
If any of these parameters are not set, the default value will be used. The default value for the UsernameTag property is 553 and the default value for the PasswordTag property is 554.
The ability to export the Username and Password property values from the vault (i.e. Azure Key Vault) is available since the FIXEdge C++ 6.15.4 release.
FIXEdge C++ provides the ability to export the Username and Password property values from the vault using scripts. This can be achieved by defining these properties in the FIXEdge.properties file in the following format:
- FixLayer.FixEngine.Session.<Session_Name>.Username = script://@[timeout=<value>, trim]<path_to_script>
- FixLayer.FixEngine.Session.<Session_Name>.Password = script://@[timeout=<value>, trim]<path_to_script>
Where
- timeout - is an optional attribute that defines the time interval to wait for successful script execution. If this attribute is not defined, then 5 sec will be used by default.
- trim - is an optional attribute that defines whether values returned by the script should be trimmed. If this attribute is present, then the line break symbol will be deleted from the Username and Password values received from the vault.
- path to script - path to the script that will be used to get credentials from the vault. If the path is relative, it must be relative to the current working directory.
This feature additionally allows the execution of procedural scripts, if necessary.
Sequence Number Handling
The FIX protocol assumes complete ordered delivery of messages between counterparties. It means there is no acknowledgment of individual message delivery. Instead of this, the following control mechanism is used:
- Each FIX message has a unique sequence number (MsgSecNum (34) tag).
- Counterparties must control the sequence numbers of outgoing messages and incoming messages, which are continuous and sequential.
For this, the counterparty establishes an independent incoming and outgoing sequence numbers for each FIX session. Sequence numbers are initialized at the start of the FIX session starting at 1 (one) and increment throughout the session.
Incoming sequence number is a counter for incoming messages.
Outgoing sequence number is a counter for outgoing messages.
Note: It is possible to change starting number of the sequence counters, refer to InSeqNum and OutSeqNum for details.
Monitoring of incoming sequence number allows counterparties to identify and react onto disturbing the synchronization. There are two types of sequence numbers desynchronization:
- Sequence number is too high, i.e. a gap in sequence numbers is detected. It indicates that some messages were missed. The counterparty initiates standard message recovery process.
- Sequence number is too low – it indicates some serious problem and leads to immediate session termination
In order to maintain the integrity of connection Heartbeat and Test Request messages are used.
In accordance with FIX protocol standard, after the correct FIX session termination (by the Logout message) sequence numbers are reset to 1. If the connection is terminated 'non-gracefully' sequence numbers will continue when the session is restored.
In fact, a lot of service providers never reset sequence numbers during the day. There are also some, who reset sequence numbers once per week. There are several cases to handle such deviation from standard in FIXEdge:
Intraday Logout Tolerance mode makes that FIX session sequence numbers always will continue on the next connection.
Configuring FIX Sessions#Force Sequence Number Reset mode makes the session reset sequence number every time on logon and forces the counterparty to do the same.
Intraday Logout Tolerance Mode
In the Intraday Logout Tolerance mode, the FIXEdge will continue sequence numbers even in case the FIX session is re-established after correct termination (by the Logout message). The configuration property IntradayLogoutTolerance can be set to ‘true’ or ‘false’ to turn this mode on or off, correspondingly.
If set to ‘false’ then the session does not exist after correct termination and hence a newly created session with the same SenderCompID and TargetCompID starts with 1 as it is specified by the FIX protocol.
In the Intraday Logout Tolerance mode, the sequence number is never reset during the day It means that the Initiator should initiate session recovery by sending Logon message with MsgSeqNum = <last outgoing sequence number> + 1; and will expect confirming Logon message with MsgSeqNum = <last incoming sequence number> + 1. If a gap is detected, the standard message recovery or gap filling process takes place.
Note: It is crucial that both sides – the initiator and the acceptor – work in the same mode. Otherwise, the ‘sequence number is too low’ fatal error appears when the session is reestablished.
During the end-of-day procedure, session logs are archived. So, because the FIXEdge stores session state in logs files, absence of such file will be treated as the session being created from scratch and hence sequence numbers start from 1.
Force Sequence Number Reset Mode
In the Force Sequence Number Reset mode, the FIXEdge will reset outgoing and incoming sequence number on every time on logon and force the counterparty to do the same via sending the tag ResetSeqNumFlag (141) = Y in Logon message.
The configuration property ForceSeqNumReset is used to set the mode. The following values can be applied for the property:
Value ‘0’ or ‘false’ – mode is off, do not use tag ResetSeqNumFlag (141)
Value ‘1’ or ‘true’ – Enable SeqNumreset at the first time of session initiation
Value ‘2’ – reset sequence number at every logon
This mode is used to resolve ‘sequence number is too low’ problem. It usually occurs as a result of applying the Configuring FIX Sessions#Intraday Logout Tolerance mode only by one of a counterparty or after ‘non-graceful' session termination log-files are cleared on the one side and kept on the other side.
Note: This is not a recommended way since messages sent during inactivity time will be lost. Also, it is recommended to make sure that other counterparty supports this functionality before relying on it.
The matrix of different combinations
ForceSeqNumReset | 0 | 1 | 2 |
---|---|---|---|
IntradayLogoutTolerance | |||
true | FIXEdge will always use the same set of logs and will never reset sequence numbers. This is the most safe combination, but you have to make sure that your counterparties don’t reset sequence numbers on their side. Otherwise, they will send you confirming Logon with SeqNum=1 and FIXEdge will terminate the session. | FIXEdge will send only first Logon with ResetSeqNum flag (141=Y), after that it will always use the same set of logs and will reset sequence numbers only after logs cleanup. | FIXEdge will always send Logon with ResetSeqNum flag (141=Y). |
false | FIXEdge will create new set of logs after Logout. | FIXEdge will create new set of logs after Logout. | Each Logon will be sent with SeqNum=1 and with ResetSeqNum flag (141=Y). |
Other Cases of Sequence Number Handling
The property ResetSeqNumAfter24hourscan be used for sequence number resetting after each 24 hour period via sending the tag ResetSeqNumFlag (141) = Y in Logon message.Heartbeat and Test Request Messages
During periods of message inactivity, counterparties have to generate the Heartbeat messages at a regular pre-defined time interval (Heartbeat Interval). The recipient also has to generate the Heartbeat message in response.
The heartbeat interval is declared by the Initiator using the HeartBtInt (108) tag in the Logon message. If HeartBtInt tag is set to zero then no regular Heartbeat messages will be generated.
When either counterparty has not received any messages for some period of inactivity (heartbeat interval + delta (reasonable transmission time)), it has to generate a Test Request message. The other counterparty has to send Heartbeat message in response to the Test Request message.
In this manner, counterparties regularly are checking sequence numbers or communication line status:
- When a gap in sequence numbers is detected, the counterparty initiates the standard message recovery process.
- It is considered that FIX Connection is lost (state "Telecommunication link failed") when response Heartbeat message is not received after the heartbeat interval plus delta.
Standard Message Recovery Process
When the gap in sequence numbers is detected each counterparty can initiate a request for message retransmission - Resend Request message. The counterparty can request a single message or range of missed messages: BeginSeqNo (7) and EndSeqNo (16) tags are used.
In response to the Resend Request message the other counterparty retransmits the application messages or sends the Sequence Reset messages. The Sequence Reset message (in ‘Gap Fill’ mode) is used when sender wants to skip some messages for the following reasons: when missed messages were application messages and became obsolete or when missed messages were just session messages.
Figure ... Message recovery
The properties OutgoingMessagesStorageSize and ResendMessagesBlockSize are used for limiting the number of requested and sent messages.
Logging Storage Type
According to the FIXprotocol session is responsible for outgoing message flow. This means that any time counterparty may request retransmission of the previously sent messages. Therefore, the session must store all sent messages to some storage.
Moreover, to successful restore the session after failure the following logs of data have to be stored:
- Session parameters (SenderCompID, TargetCompID, FIX version, etc.)
- Session incoming and outgoing sequence numbers.
- Incomming messages. Incoming message is stored to the log after the business layer processes it (optional).
- Outgoing messages. Outgoing message is stored to the log before sending.
Incoming messages are stored in files with the ‘.in’ extension. Outgoing messages are stored in files with the ‘.out’ extension.
There are the following modes of storage that can be defined for the FIX session:
Type | Description |
---|---|
Persistent | In case of the persistent session, the logs are stored to the files. This guarantees that session can recover its state (sent messages and latest sequence numbers) after connection failure but it decreases the performance. |
PersistentMM | In this case, all related information is stored to the persistent memory mapped files. |
SplitPersistent | Like persistent session, in this case the information is stored to the log-files but there is a log-file size limitation. When log-file size exceeds the limit then new log-file will be created. At the same time, the previous log-files become available to change or delete because the FIXEdge does not lock them. So, it is possible to recover the session state and to re-transmit of the previously sent messages from old log-files if these files have not been changed. |
Transient | Transient session stores all related information in memory. This means that this type of session has greater throughput and lower latency. However, such session cannot be restored after failure, and hence message loss is possible. This session type is recommended when recovery after failure is not required, i.e. when every time session is connected from scratch. This applicable for market data sessions, when client has to subscribe and request snapshot on every logon and hence recovery is not required. Another variant is when persistent already exists in application so having yet another storage of recovery information is redundant. In the latter case the application is responsible to restore session state after creation and before connect. |
Null | No storage is used. This means that incoming and outgoing messages are not stored. This session type is recommended when messages resend, recovery after failure and logging are not required. |
The mode of storage is defined by the different properties FixLayer.FixEngine.Session.Session_Name.StorageType and Configuring FIX Sessions#UnregisteredAcceptor.SessionStorageType, for registered and unregistered sessions correspondingly. Refer also to part ‘Configuring FIX Sessions#Logging and Backup Settings’ for more details.
Encryption
The exchange of sensitive data across public carrier networks may make it advisable to employ data encryption techniques to mask the application messages. FIXEdge can perform message encryption in the session by means of several cryptographic technologies: PGP, PEM, PKCS, DES or DES-MD5.
The choice of encryption method is determined by mutual agreement of the two counterparties of FIX session.
The required encryption method is defined in the session parameter EncryptMethod. FIXEdge requires GNU Privacy Guard command-line utility to use encryption. Others settings of encryption are set in a special configuration file.
Note: Problems in use of encryption may occur in case of separated configuration of sessions that will be connected subsequently (i.e. when the related sessions have different encryption methods). In this case a session will be started, but errors will be logged during the action of message sending. The server does not decrypt the encrypted messages on the business layer, so only unencrypted tags (such as TargetCompID, SenderCompID, etc.) could be used for routing purposes.
Usage the SSL is an alternative to encryption of FIX messages. In case of SSL, the protection will be spread to the whole connection, and encryption of messages is not required.
Common Settings of Sessions
This part contains a list of common properties and their detailed description related to registered and unregistered sessions. These properties are stored in both configuration files 'FIXEdge.properties' and 'engine.properties'.
Business Rules parameters
BusinessLayer.RoutingRules
Mandatory.PathtoBusiness Layer routing rules configuration file in XML format.
Default value: BusinessLayer.RoutingRules = FIXEdge1/conf/BL_Config.xml
BusinessLayer.JS.ExecuteInParallel
Optional.Upperlimit of JS scripts that can be executed in parallel. Each parallel executed script requires separate JS environment. Set a positive number or "auto".
Defaultvalueisauto.
Configuration File FIXEdge.properties
The ‘FIXEdge.properties’ file contains the following common properties that describe registered sessions specified below:
FixLayer.FixEngine.Sessions
Mandatory. Contains a comma-separated list of names of FIX Sessions registered in the FIXEdge
Example: FixLayer.FixEngine.Sessions = TestFIXAcceptor, TestFIXInitiator
Default value: FixLayer.FixEngine.Sessions = TestFIXAcceptor - this session is defined in installation procedure
FixLayer.FixEngine.Sessions.ArchivePath
Optional. Defines the path to move log files after the session is closed.
Note: directory must exist otherwise the backup procedure will fail.
Attention: it shouldn't point to the path from BackupDirectory (defined in engine.properties), otherwise an error message will occur on session start, and session won't start
Default value: FixLayer.FixEngine.Sessions.ArchivePath = FixEdge1/log/archive
FixLayer.FixEngine.Sessions.DefaultIgnoreDuplicateResendRequests
Optional. This property allows to ignore resend request duplicates and to respond to multiple similar resend request messages only once.
The recommended value is 'true'.
Example: FixLayer.FixEngine.Sessions.DefaultIgnoreDuplicateResendRequests = false
Default value: FixLayer.FixEngine.Sessions.DefaultIgnoreDuplicateResendRequests = true
FIXEdge.RootDir
The path is used as a root directory for relative paths in the other parameters. The path from a directory where FIXEdge binary was run (usually it's a FIXEdge/bin).