How to manage incoming FIX message flow with low latency TCP dispatcher

When FIX Antenna HFT is configured to use low latency TCP dispatcher via setting the UseTCPDispatcher property to 'true' the spinning reader threads deliver incoming FIX messages directly to the application. Since each spinning thread serves its own subset of sessions the application should return control from the callbacks as fast as possible to allow the thread to serve other sessions. The other reason to return control quickly is that the thread owns the session during callback, which prevents other threads from using this session (e.g. sending outgoing messages). So, the recommended way of processing incoming messages is to enqueue messages to process in the separate thread as shown in the 'Router' sample. 

To prevent the unlimited growth of the queue of incoming messages, the application can pause reading the session socket within the callback while writing (sending messages) is still kept active:

bool MyApp::process( Engine::FIXMessageProcessElem* work, const Engine::Session& sn )
{
    if( queueIsFull() )
    {
        work->pauseRead = 1;    // pause read
        return false;           // false to return back the message, otherwise true
    }
}

This will let TCP control the incoming flow and the remote side will be noticed that it should pause sending of new messages.

However, the session shouldn’t stay in this state longer than one heartbeat interval, otherwise, the engine will try to check if the session is lost and then disconnect it. To activate the socket reading the application can call session’s method resumeRead():

// somewere
if( !queueIsFull() )
{
    sn.resumeRead();
}

It is allowed to call this method multiple times without any negative effect.

The same workflow can also be applied to the SessionLevelReject message since it often has business meaning and should be processed the same way.