How to set strategy for processing out-of-sequence messages
Since 2.23 version of FIX Antenna, a new OutOfSequenceMessagesStrategy parameter is supported for different types of FIX engine reaction on emerging gaps. This parameter can be configured in engine.properties file.
The following values of OutOfSequenceMessagesStrategy parameter define four different types of strategies:
- RequestAlways - send resend request on any out-of-sequence message.
- RequestOnce - send the only resend request on a gap. Duplicates of resend requests for the same interval are not sent.
- IgnoreGap - send resend request on gap and process out-of-sequence messages.
- Queue - use a temporary queue for storing out-of-sequence messages and process them after the gap is closed.
Strategies can be defined for all sessions and/or for each session with parameter OutOfSequenceMessagesStrategy. Session parameters have higher priority over engine parameters.
RequestAlways strategy
Send resend request with BeginSeqNo (7) = <out-of-sequence message number>, EndSeqNo (16) =0 on any out-of-sequence message. Strategy RequestAlways is default.
Settings :
The strategy can be specified for an engine:
Engine parameters
|
The strategy can be specified for a single session:
Sessions parameters
|
RequestOnce strategy
Send resend request on a gap. Duplicates of resend requests for the same interval are not sent.
Settings:
The strategy can be specified for an engine:
Engine parameters
|
The strategy can be specified for a single session:
Sessions parameters
|
IgnoreGap strategy
Send resend request on gap and processing out-of-sequence messages.
Settings:
The strategy can be specified for an engine:
Engine parameters
|
The strategy can be specified for a single session:
Sessions parameters
|
Queue strategy
Use temporary queue for storing out-of-sequence messages and processing them after the gap is closed.
This strategy is useful when a counterparty sends messages without sequence synchronization.
Test request, Resend request and Logon messages are processed immediately and added to the queue for logging.
Messages are written to sessions logs regardles of have they processed or waiting in the queue. After sequence gap is resolved the messages from the queue are processed and written to the session logs one more time with Q mark after the timestamp. For example:
20171219-12:45:30.818 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=3.... 20171219-12:45:40.820 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=4... 20171219-12:45:50.824 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=6... 20171219-12:46:00.811 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=7... 20171219-12:46:00.811 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=5... 20171219-12:47:50.824 :Q8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=6... 20171219-12:47:00.811 :Q8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=7...
A queue should be cleared after exceeding the limit out-of-sequence messages and a new resend request should be sent. Queue size is specified by parameter OutOfSequenceMessagesStrategy.QueueSize (engine or session parameter). Queue is cleaned after overflowing.
Sequence Reset (35=4) message clean the Queue.
Settings:
The strategy can be specified for an engine:
Engine parameters
|
The strategy can be specified for a single session:
Sessions parameters
|
Troubleshooting
- The Queue strategy is not supported for FAST sessions
- ResendRequestBlockSize parameter is not supported for RequestAlways strategy. If ResendRequestBlockSize parameter is defined then RequestOnce strategy will be used instead.
- Old parameters for sessions (DeliverAppMessagesOutOfOrder, SuppressDoubleResendRequest) must be removed for using OutOfSequenceMessagesStrategy. Old parameters have higher priority over the new ones.
Comparison between strategies and old parameters prior FIX Antenna 2.23 version
New parameters | Map in old parameters |
|---|---|
| OutOfSequenceMessagesStrategy = AlwaysRequest | DeliverAppMessagesOutOfOrder = false |
| OutOfSequenceMessagesStrategy = AlwaysRequest ResendRequestBlockSize = 10 | Strategy AlwaysRequest not support with parameter ResendRequestBlockSize. Will be used strategy RequestOnce |
| OutOfSequenceMessagesStrategy = IgnoreGap | DeliverAppMessagesOutOfOrder = true SuppressDoubleResendRequest = false Parameter SuppressDoubleResendRequest is ignored if DeliverAppMessages is True. |
| OutOfSequenceMessagesStrategy = RequestOnce | DeliverAppMessagesOutOfOrder = false SuppressDoubleResendRequest = true |
| OutOfSequenceMessagesStrategy = Queue | DeliverAppMessagesOutOfOrder = true SuppressDoubleResendRequest = false Parameter SuppressDoubleResendRequest is ignored if DeliverAppMessages is True. SizeOutOfSequenceMessagesQueue = <Default value> ( for session parameter will be use engine parameter OutOfSequenceMessagesStrategy.QueueSize |
| OutOfSequenceMessagesStrategy = Queue OutOfSequenceMessagesStrategy.QueueSize = 100 | DeliverAppMessagesOutOfOrder = false SuppressDoubleResendRequest = false |
Update logging
| Event | Strategy | Level | Log record |
|---|---|---|---|
Detect gap and send resend request | Queue | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "Queue". The following out of sequence messages will be queued. The current queue size is 1 ( max value : OutOfSequenceMessagesStrategy.QueueSize = 10 ). |
| RequestAlways | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "RequestAlways". | |
| RequestOnce | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "RequestOnce". The following out of sequence messages will be ignored. | |
| IgnoreGap | WARN | WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "IgnoreGap". The out of sequence messages will be processed. | |
Start processing from temporary queue | Queue | DEBUG | [DEBUG] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The message with SeqNum 7 has closed gap. Start processing messages from the out of sequence queue. The current queue size is 10. |
Finish processing from temporary queue | Queue | DEBUG | [DEBUG] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Finish processing messages from the out of sequence queue. The current queue size is 5. Next expected SeqNum is 7. |
Owerflow temporary queue | Queue | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-10 is sent to counterparty. OutOfSequenceMessagesStrategy is set to "Queue". The following out of sequence messages will be queued. The queue has been cleared after overflow. The current queue size is 1 ( max value : OutOfSequenceMessagesStrategy.QueueSize = 10 ). |
Clear temporary queue | Queue | DEBUG | [DEBUG] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The out of sequence queue has been cleared. |
Catch exception on processing message from queue (incoming seqNum is not changed) | Queue | ERROR | [ERROR] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Error on processing message from the out of sequence queue with SeqNum 10. Reason :<Reason> . The out of sequence queue will be cleared. |
Catch exception on processing message from queue (incoming seqNum is changed) | Queue | ERROR | [ERROR] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Error on processing message from the out of sequence queue with SeqNum 10. Reason :<Reason> . |
Unexpected behavior: push message in the queue on processing from queue | Queue | ERROR | [ERROR] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Unxpected behavior on processing message with SeqNum 10 from the out of sequence queue . The queue has been cleared. |
| Used depricated parameter DeliverAppMessagesOutOfOrder = true | IgnoreGap | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The parameter DeliverAppMessagesOutOfOrder is deprecated. Use the parameter OutOfSequenceMessagesStrategy. Will be using strategy "OutOfSequenceMessagesStrategy = IgnoreGap". |
Used depricated parameter DeliverAppMessagesOutOfOrder = true | |||
Used depricated parameter SuppressDoubleResendRequest = true | RequestOnce | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The parameter SuppressDoubleResendRequest is deprecated. Use the parameter OutOfSequenceMessagesStrategy. Will be using strategy "OutOfSequenceMessagesStrategy = RequestOnce". |
| Strategy is used for FAST session | Queue | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The strategy "Queue" is not supported for the FAST session. Will be using strategy "OutOfSequenceMessagesStrategy = RequestAlways". |
ResendRequestBlockSize is set together with conflicting strategies: OutOfSequenceMessagesStrategy = RequestAlways or | RequestAlways | WARN | [WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The strategy "RequestAlways" can't be used with the parameter "ResendRequestBlockSize." Will be using strategy "OutOfSequenceMessagesStrategy = RequestOnce". |