How to configure logging and logs rotation in FIXEdge
Overview
FIXEdge provides the possibility to set up logging and logs rotation by configuring the engine.properties and the FIXEdge.properties files.
Configuring engine.properties file
The following parameters related to logging can be set up in the engine.properties file:
Parameter | Description | Example |
---|---|---|
Engine parameters | ||
LogFileName | Engine's log file name. If this parameter is commented or empty, the engine.log" will be used. | LogFileName = FIXAntenna_C++.log |
LogDirectory | This property is the path of the directory in which the logs for all incoming (if LogIncomingMessages is set to 'true') and outgoing FIX messages are stored. It is possible to specify a path related to the EngineRoot directory. For example if LogDirectory is set to '\logs\' then the real path is $(EngineRoot)/logs. The specified directory must exist. | LogDirectory = logs |
Session parameters | ||
Debug.LogSessionExtraParameters | If this property value is true, general and extra session parameters of primary and backup connections will be printed to the log file. If this property value is false, only general session parameters of primary and backup connections will be printed to the log file. All FIXEdge-specific session parameters will be printed to the log regardless of Debug.LogSessionExtraParameters property value. | Debug.LogSessionExtraParameters = true |
Session parameters: Storages | ||
LogIncomingMessages | This property provides an option to log incoming FIX messages (those received) from a counterparty FIX Engine. They will be stored in the directory specified by the LogDirectory parameter in a file with the extension 'in'. Changing this value will impact the performance of FIX Engine. In the development should be true. In production should be false. | LogIncomingMessages = true |
TimestampsInLogs | An option to write timestamps in the log files. This parameter is optional, the default value is true. Changing this value will impact the performance of FIX Engine. In the development should be true. In production should be false. | TimestampsInLogs = true |
EnableIncrementalLogFileCreation | Setting property "true" will reserve 10Mb of disc space for logging. If the log file reaches 10Mbs, another 10Mbs will be reserved, and so on. When the property is set to "true" FixAntenna performance is greatly increased. | EnableIncrementalLogFileCreation = true |
Persistents.LogIncrementSize | This property specifies the amount of memory reserved for memory-mapped file message storage for out/in file. Changing this value will impact upon the performance of FIX Engine. A large value is better. | Persistents.LogIncrementSize = 1000000 |
Persistents.IdxIncrementSize | This property specifies the amount of memory reserved for memory-mapped file message storage for idx file. Changing this value will impact upon the performance of FIX Engine. A large value is better. | Persistents.IdxIncrementSize = 10000 |
SplitPersistentMsgStorage.MaxSliceSize | This property specifies the maximum size of one slice for sliced message storage. Note: Storage slices are created with step, specified with Persistents.LogIncrementSize parameter value. If the MaxSliceSize value is no multiple of Persistents.LogIncrementSize, the result file size will be less, than specified. If value is less than LogIncrementSize, LogIncrementSize value will be used. Recommended value: any multiple of Persistents.LogIncrementSize Default value is 500000000 - 500Mb. | SplitPersistentMsgStorage.MaxSliceSize = 500000000 |
SplitPersistentMsgStorage.BackupDir | This property is the path of the directory, where sliced message storage will search for log files in case they are not found in the primary location. It is the user's responsibility to move files from the primary location to the backup. The default value is empty - backup path disabled. | SplitPersistentMsgStorage.BackupDir = logs/BackupDir |
MessageStorage.IndexRebuildEnabled | Optional. If this parameter is set to 'true', FIX Engine will try to restore the index file from the log file in case the index file is missed or corrupted. The default value is false. Please be careful when you enable this property: restoring process uses some heuristic algorithms, and some data (not messages) could be missed. | MessageStorage.IndexRebuildEnabled = false |
NewFeatures.EnableOptimizedPersistentMMIndex | Optional. This property enables the method of maintaining the start-up of FIX sessions with log files in the PersistentMM storage marked with the 'correctly stored' flag. If this property is set to 'true' FIXEdge will skip the check of the 'stored correctly' log files in order to improve performance. Formats of PersistentMM log files with and without enabled NewFeatures.EnableOptimizedPersistentMMIndex property are incompatible. The default value is 'true'. The NewFeatures.EnableOptimizedPersistentMMIndex property is available since FIXEdge 6.14.0 release. | NewFeatures.EnableOptimizedPersistentMMIndex = true |
Session parameters: Unregistered Acceptors parameters | ||
UnregisteredAcceptor.SessionStorageType | Optional. Commented. Defines the storage type for unregistered sessions. Valid values: "persistent" | "persistentMM" | "splitPersistent" | "transient" | "null" | UnregisteredAcceptor.SessionStorageType = persistent |
Backup parameters | ||
BackupDirectory | Relative path to the backup folder. This folder will be used for message storage files of the backup connections. | BackupDirectory = logs/backup |
HiddenLogonCredentials | Optional. Provides an option to mask password in in/out log files. Valid values: "true" | "false". Default value: HiddenLogonCredentials = false | HiddenLogonCredentials = true |
Logging parameters: refer to Logging parameters page for details. |
Logging parameters of the backup sessions
If the backup parameters of the session are the same as the primary parameters of the session then Host, Port, HeartbeatInterval, and INFO message "The backup session has the same parameters as the primary session" will be printed in the log.
If at least one backup parameter of the session differs from the primary one, then Host, Port, HeartbeatInterval, and all the backup parameters of the session will be printed in the log.
Configuring FIXEdge.properties file
The following parameters related to logging can be set up in FIXEdge.properties file:
Parameter | Description | Example |
---|---|---|
FixLayer.FixEngine.Sessions.ArchivePath | Defines the path to move log files after the session is closed. Note: directory must exist otherwise the backup procedure will fail. It shouldn't point to the path from BackupDirectory (defined in engine.properties), otherwise error message will occur on session start, and the session won't start. | FixLayer.FixEngine.Sessions.ArchivePath = FixEdge1/log/archive |
FixLayer.FixEngine.Session.Session_Name.StorageType | Mandatory. Defines the session storage type. Valid values: "persistent" | "persistentMM" | "splitPersistent" | "transient" | "null" | FixLayer.FixEngine.Session.TestFIXAcceptor.StorageType = persistentMM |
FixLayer.FixEngine.Session.Session_Name.LogDirectory | Optional. Defines the path to the directory in which logs of the primary connection of the session are stored. If this property was not specified with value then the value of the LogDirectory will be used. Since FIXEdge C++ 6.14.3 release, If this property was specified with the non-existent path then FIXEdge C++ will try to create a new directory in the specified path. This parameter is available since FIXEdge C++ 6.14.0 release. | FixLayer.FixEngine.Session.TestFIXAcceptor.LogDirectory = FixEdge1/logs |
FixLayer.FixEngine.Session.Session_Name.Backup.LogDirectory | Optional. Defines the path to the directory in which logs of the backup connection of the session are stored. If this property was not specified with value then the value of the BackupDirectory will be used. This parameter is available since FIXEdge C++ 6.14.0 release. | FixLayer.FixEngine.Session.TestFIXAcceptor.Backup.LogDirectory = FixEdge1/logs/backup |
Logging parameters: refer to Logging parameters page for details. |
If you want FIXEdge to store logs in system logs or print it to the console you should change Log.Device property:
Accepted values:
- File - messages will be written to the file.
- Console - messages will be written to the console.
- EventLog - messages will be written to the Windows Event Log (Windows-specific).
- WinDebug - messages will be written to the OutputDebugString (Windows-specific)
- Syslog - messages will be written to the system log (Linux/Unix specific).
It also could be configured per category.
Configuring logs rotation
In order to set up logs rotation use the following parameters in the FIXEdge.properties file:
Log.File.Rotate
Enables/disables log file rotation.
Valid values:
- true - to enable rotation;
- false - to disable rotation.
Log.File.Rotate.SizeLimit
Size of file in bytes. When log file reaches this size it is renamed (follow standard rotation approach) and the new file is created.
Example: Log.File.Rotate.SizeLimit = 104857600
Log.File.Rotate.FileLimit
A number of files to be involved in the rotation. When the number is exceeded the older file is removed.
Example: Log.File.Rotate.FileLimit = 10
Log.File.Rotate.Dir
Rotation directory. Directory to store rotated files.
Example: Log.File.Rotate.Dir = <%SERVICE_NAME%>/log
Division by categories and severities
Please see How to divide different categories and severities of log files into different files in the Logging section for details.
Best practices and commonly used cases
Below are examples of client's logging configuration which are set up for production.
engine.properties logging parameters
# The FIX Engine configuration file. # # $Revision: FA 2.8 $ # For UNIX platforms, the filename-separator character is "/". # For Win32 platforms, the filename-separator character is "\\" or "/". ####################################################################### # Engine settings ####################################################################### # The top of the directory tree under which the engine's configuration, # and log files are kept. ... # Engine's log file name. # If this parameter is commented or empty, the 'engine.log' will be used. # LogFileName = FIXAntenna_C++.log ... # Default storage type of the created unregistered sessions. By default persistent storage # type used. Use "transient" value to use transient storage for the sessions. # UnregisteredAcceptor.SessionStorageType = ... # This property is the path of the directory in which the logs for all incoming # (if LogIncomingMessages is set to "true") and outgoing FIX messages are stored. # It is possible to specify a path related to the EngineRoot directory. # For example if LogDirectory is set to \"logs\" then the real path is $(EngineRoot)/logs. # The specified directory must exist. LogDirectory = FIXOEGW1/log # Relative path to the backup folder. This folder will be used for message storage files # of the backup connections. BackupDirectory = FIXOEGW1/log/backup ####################################################################### # Session settings ####################################################################### # This property provides an option to log incoming FIX messages (those received) from # a counterparty FIX Engine. They will be stored in the directory specified by # the LogDirectory parameter in a file with extension "in". LogIncomingMessages = true ... ####################################################################### # Validation ####################################################################### ... ####################################################################### ... # An option to write timestamps in the log files. # This parameter is optional, the default value is true. TimestampsInLogs = true ... # Setting property "true" will reserve 10Mb of disc space for logging. # If log reaches 10Mbs, another 10Mbs will be reserved and so on. # When property is "true" FixAntenna performance is greately increased. EnableIncrementalLogFileCreation = false # Setting property "true" will enable logging session parameters on creation. Debug.LogSessionExtraParameters = true ####################################################################### # Engine monitoring ####################################################################### ... ####################################################################### # Engine measuring ####################################################################### ... ####################################################################### # Logger settings ####################################################################### # Default target devices # # Valid values: # * File - messages will be written to the file. # * Console - messages will be written to the console. # * EventLog (Windows specific) - messages will be written to the Windows Event Log. # * WinDebug (Windows specific) - messages will be written to the OutputDebugString. # * Syslog (Linux/Unix specific) - messages will be written to the system log. # # The properties of the device are in the "Log.<name>" section. For instance Log.File contains # all properties of the File device. Log.Device = File # Enable Debug messages. # Valid values: # * true - to enable Debug messages # * false - to disable Debug messages Log.DebugIsOn = true Log.TraceIsOn = true # Enable Note messages. # Valid values: # * true - to enable Note messages # * false - to disable Note messages Log.NoteIsOn = true # Enable Warning messages. # Valid values: # * true - to enable Warning messages # * false - to disable Warning messages Log.WarnIsOn = true # Enable Error messages. # Valid values: # * true - to enable Error messages # * false - to disable Error messages Log.ErrorIsOn = true # Enable Fatal Error messages. # Valid values: # * true - to enable Fatal Error messages # * false - to disable Fatal Error messages Log.FatalIsOn = true # Enables/disables log file cycling # Valid values: # * true - to enable cycling # * false - to disable cycling Log.Cycling = false ####################################################################### # Logger settings: File device ####################################################################### # Timezone of the log message timestamp # Valid values: # * Local - display time as local # * UTC - display time as UTC Log.File.TimeZone = UTC # Root directory for log files. It will be concatenated with # each file name specified by 'Log.File.Name' Log.File.RootDir = ./logs # File name. If more than one category uses files with the same name # the same file will be used simultaniously. Log.File.Name = _.log # Separated log for 'Version' category Log.Version.File.Name = version.log # If set to true then lock file <Log.File.RootDir/Log.File.Name>.lock will be created. This # will prevent other instance of the application use the same file. # # Valid values: # * true - lock log file # * false - do not lock log file Log.File.Locked = false # Enables/disables log file rotation # Valid values: # * true - to enable rotation # * false - to disable rotation Log.File.Rotate = false # Local time when logs should ba backed up. Shoudl be in format 'hh:mm' # or 'hh:mm:ss'. # Optional. Log.File.Backup.Time = # If true then file will be recreated on each start. # If false then new records will be appended to the existent file. Log.File.Recreate = true # If set to true then buffer will be flushed after each logging call. # If set to false then flush is not called. # # Setting to true decreases program performance; setting to false increases # a risk of records loss in case of program failure. Log.File.AutoFlush = true # String that specifies the name of the source. The source name must be a # subkey of a logfile entry under the EventLog key in the registry. For # example, "WinApp" is a valid source name if the registry has the following # key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Event Log\Application\WinApp Log.EventLog.EventSource = TestService # Specifies amount of memory reserved for memory mapped file message storage for out/in file # WARNING: Changing this value will impact upon the performance of FIX Engine. # Large value is better. Persistents.LogIncrementSize=1000000 # Specifies amount of memory reserved for memory mapped file message storage for idx file # WARNING: Changing this value will impact upon the performance of FIX Engine. # Large value is better. Persistents.IdxIncrementSize=10000
FIXEdge.properties logging parameters
# The FIX Server configuration file. # # $Revision: 1.5.2.3 $ # The top of the directory tree under which the FIXEdge's configuration, # and log files are kept. #------------------------------------------------------------ # Control Centre Section #------------------------------------------------------------ ... #------------------------------------------------------------ # Business Layer Section #------------------------------------------------------------ ... #------------------------------------------------------------ # FIX Layer Section #------------------------------------------------------------ # This parameter determines the names of FIX session-acceptors. For each # such a session it is necessary to define the parameters described below. # The format is 'session.X.ParameterName' where 'X' is the name of session. FixLayer.FixEngine.Sessions = FIXEDGE-CONTROL, TEST001, ... # Path to move log files after the appointed session was closed. # Warning: directory must exist otherwise the backup procedure will fail. # Note: Optional. FixLayer.FixEngine.Sessions.ArchivePath = /log/FIXEdge1/logs/archive ... #------------------------------------------------------------ # Test FIX 4.4 sessions #------------------------------------------------------------ FixLayer.FixEngine.Session.TEST001.Version = FIX44 FixLayer.FixEngine.Session.TEST001.StorageType = persistentMM ... #------------------------------------------------------------ # Transport Layer Section #------------------------------------------------------------ ... #------------------------------------------------------------ # The SMTP Adaptor configuration file. #------------------------------------------------------------ ... #------------------------------------------------------------ # Default log settings section #------------------------------------------------------------ # Target directory. # Please specify absolute directory path for NT Service application. Log.File.RootDir = /log/FIXEdge1/logs # Target devices Log.Device = File # Turns on/off logging on the debug level Log.DebugIsOn = false # Turns on/off logging on the trace level Log.TraceIsOn = false # Turns on/off logging on the notice level Log.NoteIsOn = true # Turns on/off logging on the warning level Log.WarnIsOn = true # Turns on/off logging on the error level Log.ErrorIsOn = true # Turns on/off logging on the fatal level Log.FatalIsOn = true # Turns cycling on/off. Log.Cycling = false # Number of repeating records to be placed to log before cycling is started. Log.Cycling.Ignore = 3 # Number of repeating records to be accumulated (hidden) before write the # "cycle record" to the log. Log.Cycling.BlockSize = 10 # Multiplier for the Block Size. If BlockSize number of messages is # accumulated and the same message still appears then next BlockSize # is calculated as the previous one multiplied by Multiplier. Log.Cycling.Multiplier = 10 # File name. If more than one category uses files with the same name # the same file will be used simultaneously. Log.File.Name = FixEdge.log # Separated log for 'Version' category. Not required Log.Version.File.Name = version.log # Separated log for level 'Trace' in 'CCAdminApplication' category. Not required. Log.CCAdminApplication.Trace.File.Name = FixEdge_CCAdminApplication_Trace.log Log.CCAdminApplication.TraceIsOn = false # Custom format of log file record. # Can be defined separately for each log file e.g. Log.Version.File.Format = ... # Format string can contain any symbols and keywords: # %% - put '%' symbol # %level - put severity level name: 'ERROR', 'WARN' etc. # %tablevel - put extra spaces for vertical alignment of records after %level # %date{FIX} - date in 'YYYYMMDD-HH:MM:SS.sss' format # %date{ISO8601} - date in 'YYYY-MM-DD HH:MM:SS,sss' format # %date - date in 'DD MMM YYYY HH:MM:SS,sss' format # %timezone - time zone abbreviation or name (setup TZ env variable to take abbreviation in windows) or 'UTC' # %logger - log category, component # %thread - thread id # %message - logged message Log.File.Format = %date{ISO8601} %timezone %level%tablevel [%logger] %thread %message #Log.File.Format = [%level] %tablevel%date{FIX} [%thread] [%logger] - %message # Timezone of the log message timestamp # Valid values: # * Local - display time as local # * UTC - display time as UTC Log.File.TimeZone = UTC # If true then file will be recreated on each start. # If false then new records will be appended to the existent file. Log.File.Recreate = false # If set to true then buffer will be flushed after each logging call. # If set to false then flush is not called. # Setting to true decreases program performance; setting to false increases # a risk of records loss in case of program failure. Log.File.AutoFlush = true # Enables rotation in the logs when size limit is reached Log.File.Rotate = true # Size of file in bytes. When log file reaches this size it is renamed # (follow standard rotation approach) and the new file is created Log.File.Rotate.SizeLimit = 104857600 # Number of files to be involved in rotation. When the number is exceeded # the older file is removed Log.File.Rotate.FileLimit = 0 # Rotation directory. Directory to store rotated files. Log.File.Rotate.Dir = /log/FIXEdge1/logs