How to set strategy for processing out-of-sequence messages

Since 2.23 version of FIX Antenna, a new OutOfSequenceMessagesStrategy parameter is supported for different types of FIX engine reaction on emerging gaps. This parameter can be configured in engine.properties file.

The following values of OutOfSequenceMessagesStrategy parameter define four different types of strategies:

RequestAlways - send resend request on any out-of-sequence message.
RequestOnce - send the only resend request on a gap. Duplicates of resend requests for the same interval are not sent.
IgnoreGap - send resend request on gap and process out-of-sequence messages.
Queue - use a temporary queue for storing out-of-sequence messages and process them after the gap is closed.

Strategies can be defined for all sessions and/or for each session with parameter OutOfSequenceMessagesStrategy. Session parameters have higher priority over engine parameters.

RequestAlways strategy

Send resend request with BeginSeqNo (7) = <out-of-sequence message number>, EndSeqNo (16) =0 on any out-of-sequence message. Strategy RequestAlways is default.

Settings :

The strategy can be specified for an engine:

Engine parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is "RequestAlways"
OutOfSequenceMessagesStrategy = RequestAlways

The strategy can be specified for a single session:

Sessions parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is engine parameter OutOfSequenceMessagesStrategy
Session.Sender/Target.OutOfSequenceMessagesStrategy = RequestAlways

RequestOnce strategy

Send resend request on a gap. Duplicates of resend requests for the same interval are not sent.

Settings:

The strategy can be specified for an engine:

Engine parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is "RequestAlways"
OutOfSequenceMessagesStrategy = RequestOnce

The strategy can be specified for a single session:

Sessions parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is engine parameter OutOfSequenceMessagesStrategy
Session.Sender/Target.OutOfSequenceMessagesStrategy = RequestOnce

IgnoreGap strategy

Send resend request on gap and processing out-of-sequence messages.

Settings:

The strategy can be specified for an engine:

Engine parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is "RequestAlways"
OutOfSequenceMessagesStrategy = IgnoreGap

The strategy can be specified for a single session:

Sessions parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is engine parameter OutOfSequenceMessagesStrategy
Session.Sender/Target.OutOfSequenceMessagesStrategy = IgnoreGap

Queue strategy

Use temporary queue for storing out-of-sequence messages and processing them after the gap is closed.

This strategy is useful when a counterparty sends messages without sequence synchronization.

Test request, Resend request and Logon messages are processed immediately and added to the queue for logging.

Messages are written to sessions logs regardles of have they processed or waiting in the queue. After sequence gap is resolved the messages from the queue are processed and written to the session logs one more time with Q mark after the timestamp. For example:

20171219-12:45:30.818 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=3....
20171219-12:45:40.820 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=4...
20171219-12:45:50.824 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=6...
20171219-12:46:00.811 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=7...
20171219-12:46:00.811 : 8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=5...
20171219-12:47:50.824 :Q8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=6...
20171219-12:47:00.811 :Q8=FIX.4.49=5935=049=FIXCLIENT56=FIXEDGE 34=7...

A queue should be cleared after exceeding the limit out-of-sequence messages and a new resend request should be sent. Queue size is specified by parameter OutOfSequenceMessagesStrategy.QueueSize (engine or session parameter). Queue is cleaned after overflowing.

Sequence Reset (35=4) message clean the Queue.

Settings:

The strategy can be specified for an engine:

Engine parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is "RequestAlways"
OutOfSequenceMessagesStrategy = Queue

// This parameter sets size temporary queue for processing out-of-sequence messages
// default value is "2000"
OutOfSequenceMessagesStrategy.QueueSize = 10

The strategy can be specified for a single session:

Sessions parameters

// This parameter sets strategy for processing out-of-sequence messages
// default value is engine parameter OutOfSequenceMessagesStrategy
Session.Sender/Target.OutOfSequenceMessagesStrategy = Queue

// This parameter sets size temporary queue for processing out-of-sequence messages
// default value is engine parameter OutOfSequenceMessagesStrategy.QueueSize
Session.Sender/Target.OutOfSequenceMessagesStrategy.QueueSize = 10

Troubleshooting

The Queue strategy is not supported for FAST sessions
ResendRequestBlockSize parameter is not supported for RequestAlways strategy. If ResendRequestBlockSize parameter is defined then RequestOnce strategy will be used instead.
Old parameters for sessions (DeliverAppMessagesOutOfOrder, SuppressDoubleResendRequest) must be removed for using OutOfSequenceMessagesStrategy. Old parameters have higher priority over the new ones.

Comparison between strategies and old parameters prior FIX Antenna 2.23 version

New parameters	Map in old parameters
OutOfSequenceMessagesStrategy = AlwaysRequest	DeliverAppMessagesOutOfOrder = false SuppressDoubleResendRequest = false
OutOfSequenceMessagesStrategy = AlwaysRequest ResendRequestBlockSize = 10	Strategy AlwaysRequest not support with parameter ResendRequestBlockSize. Will be used strategy RequestOnce
OutOfSequenceMessagesStrategy = IgnoreGap	DeliverAppMessagesOutOfOrder = true SuppressDoubleResendRequest = false Parameter SuppressDoubleResendRequest is ignored if DeliverAppMessages is True.
OutOfSequenceMessagesStrategy = RequestOnce	DeliverAppMessagesOutOfOrder = false SuppressDoubleResendRequest = true
OutOfSequenceMessagesStrategy = Queue	DeliverAppMessagesOutOfOrder = true SuppressDoubleResendRequest = false Parameter SuppressDoubleResendRequest is ignored if DeliverAppMessages is True. SizeOutOfSequenceMessagesQueue = <Default value> ( for session parameter will be use engine parameter OutOfSequenceMessagesStrategy.QueueSize for engine parameter will be use 2000 )
OutOfSequenceMessagesStrategy = Queue OutOfSequenceMessagesStrategy.QueueSize = 100	DeliverAppMessagesOutOfOrder = false SuppressDoubleResendRequest = false

Update logging

Event	Strategy	Level	Log record
Detect gap and send resend request	Queue	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "Queue". The following out of sequence messages will be queued. The current queue size is 1 ( max value : OutOfSequenceMessagesStrategy.QueueSize = 10 ).
	RequestAlways	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "RequestAlways".
	RequestOnce	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "RequestOnce". The following out of sequence messages will be ignored.
	IgnoreGap	WARN	WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-4 is sent to the counterparty. OutOfSequenceMessagesStrategy is set to "IgnoreGap". The out of sequence messages will be processed.
Start processing from temporary queue	Queue	DEBUG	[DEBUG] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The message with SeqNum 7 has closed gap. Start processing messages from the out of sequence queue. The current queue size is 10.
Finish processing from temporary queue	Queue	DEBUG	[DEBUG] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Finish processing messages from the out of sequence queue. The current queue size is 5. Next expected SeqNum is 7.
Owerflow temporary queue	Queue	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Sequence number gap is detected on message with SeqNum 5. Resend request message for sequence numbers 1-10 is sent to counterparty. OutOfSequenceMessagesStrategy is set to "Queue". The following out of sequence messages will be queued. The queue has been cleared after overflow. The current queue size is 1 ( max value : OutOfSequenceMessagesStrategy.QueueSize = 10 ).
Clear temporary queue	Queue	DEBUG	[DEBUG] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The out of sequence queue has been cleared.
Catch exception on processing message from queue (incoming seqNum is not changed)	Queue	ERROR	[ERROR] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Error on processing message from the out of sequence queue with SeqNum 10. Reason :<Reason> . The out of sequence queue will be cleared.
Catch exception on processing message from queue (incoming seqNum is changed)	Queue	ERROR	[ERROR] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Error on processing message from the out of sequence queue with SeqNum 10. Reason :<Reason> .
Unexpected behavior: push message in the queue on processing from queue	Queue	ERROR	[ERROR] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : Unxpected behavior on processing message with SeqNum 10 from the out of sequence queue . The queue has been cleared.
Used depricated parameter DeliverAppMessagesOutOfOrder = true	IgnoreGap	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The parameter DeliverAppMessagesOutOfOrder is deprecated. Use the parameter OutOfSequenceMessagesStrategy. Will be using strategy "OutOfSequenceMessagesStrategy = IgnoreGap".
Used depricated parameter DeliverAppMessagesOutOfOrder = true SuppressDoubleResendRequest = true	IgnoreGap	WARN
Used depricated parameter SuppressDoubleResendRequest = true	RequestOnce	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The parameter SuppressDoubleResendRequest is deprecated. Use the parameter OutOfSequenceMessagesStrategy. Will be using strategy "OutOfSequenceMessagesStrategy = RequestOnce".
Strategy is used for FAST session	Queue	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The strategy "Queue" is not supported for the FAST session. Will be using strategy "OutOfSequenceMessagesStrategy = RequestAlways".
ResendRequestBlockSize is set together with conflicting strategies: OutOfSequenceMessagesStrategy = RequestAlways or SuppressDoubleResendRequest = false DeliverAppMessagesOutOfOrder = false and ResendRequestBlockSize used	RequestAlways	WARN	[WARN] 20180117-12:51:38.037 [242168] [Engine] - Session <Sender,Target> : The strategy "RequestAlways" can't be used with the parameter "ResendRequestBlockSize." Will be using strategy "OutOfSequenceMessagesStrategy = RequestOnce".