Inter-Process Communication (IPC) Handler

Inter-Process Communication (IPC) Handler has been available since the FIXEdge 6.18.0 release.

Overview

The FIXEdge IPC Handler is a component of the FIXEdge Business Layer designed to facilitate communication between the server and third-party applications using IPC mechanisms such as Unix domain sockets and Windows named pipes. This interaction involves the exchange of FIX messages and auxiliary data from the Server to the Client, with the possibility of modified FIX messages being returned to the Server, enhanced with additional auxiliary data.

Functionality

Message Relay: The IPC Handler acts as a relay for messages between FIXEdge and external clients.
Data Serialization and Processing: It serializes incoming source FIX messages into JSON format, converts them into IPC protocol entries for transmission, and routes them into client-specified systems.
Unix Socket / Named Pipe Communication: Leverages Unix sockets for Linux / Named Pipes for Windows to establish and maintain connections with external client systems.
Bidirectional Communication: Facilitates two-way communication, sending responses back to the business layer.
Protocol Agnostic Integration: Enables integration with any external system that adheres to the defined IPC protocol.
Customization: Allows users to configure the handler's behavior to suit specific needs and business processes.

Benefits

Enhanced System Integration: Enables communication and data exchange with various external client systems.
Improved Flexibility: Supports integration with future systems adopting the IPC protocol.
Increased Scalability: Enables concurrent execution of business tasks alongside external system interactions.
User Control: Provides customization options for users to tailor the handler's behavior to their specific requirements.

Terms and Abbreviations

Handler: An instance of the FIXEdge IPC handler.
Client: A third-party application connected to or attempting to connect to a Handler.
Protocol: The data communication and format specification used between the Handler and the Client.

Communication Protocol

The communication protocol comprises two main stages:

Handshake: Establishes the connection, verifies protocol compatibility, and ensures no other clients are connected. Upon successful handshake, the server provides the client with a unique clientTag for subsequent message identification.
Data Exchange: Facilitates the exchange of FIX messages and supplementary data between the server and client.

Detailed Protocol Specification:

For the official protocol specification, see the IPC/client/protocol.md file available in the FIXEdge SDK package.

Data Exchange

In the data exchange phase, the Handler sends JSON-serialized FIX messages to the Client together with supplementary data that includes the clientTag and any additional auxiliary data. The Client must respond with a result code (0 for success) and additional reply data within a specified timeout:

Success: The response can include an amended FIX message, a FIX parser template string, and auxiliary data.
Failure: An error message can be returned.

The auxiliary data from the client is stored in the session context key-value storage using a key derived from either AuxDataContextKeyName or a combination of ${Handler_Name}.${clientTag} (or simply ${Handler_Name} if clientTag is not specified).

Timeout Handling:

The client must send its response within a defined timeframe (specified in microseconds). Failure to respond within this timeout period will be treated as an error.

Business Layer Configuration Options

All configurations for the IPC Handler are defined within the BL_Config.xml file as shown below:

<FIXEdge>
  <DllHandlers>
    <Handler Name="IPC1"
      Description="IPC Handler"
      DllName="./libIPCHandler-MD-x64.so"
      ListenAddress="/tmp/unixsock"
      ClientTag="ClientTagValue"      
      TimeoutUsec="100000"
      ClientConnectionWaitTimeoutMsec="100"
      AuxData="StaticIpc1AuxData"
      AuxDataContextKeyName="ResultAuxDataKeyName"
    />
  </DllHandlers>
  <BusinessLayer>
    <Rule Description="Sending to IPC client">
      <Source Name="NORTH-FE"/>
      <Action>
        <HandlerAction Name="IPC1" AuxData="DynamicIpc1AuxData" TimeoutUsec="15000" AuxDataContextKeyName="AuxDataKvKey"/>
      </Action>
    </Rule>
  </BusinessLayer>
</FIXEdge>

This code demonstrates sending a FIX message to the "IPC1" handler. Clients can customize the timeout, include auxiliary data, and specify the key for storing client data in the session context.

FIXEdge.DllHandlers.Handler Element

Attribute	Meaning	Required	Default Value

Attribute	Meaning	Required	Default Value
Name	A unique handler name used in creating a rule.	Mandatory
Description	A mandatory description of the handler.	Mandatory
DllName	The name of the DLL/SO file containing the handler code.	Mandatory
ListenAddress	The Windows named pipe/Unix domain socket for client connections.	Mandatory
ClientTag	The client tag passed by the Server to the Client.	Optional	Empty string
TimeoutUsec	The maximum handling time (in microseconds) on the Client side for a message.	Optional	100,000
ClientConnectionWaitTimeoutMsec	The time (in milliseconds) to wait for a client to (re)connect if no client was connected when the message arrived	Optional	100
AuxData	The auxiliary data sent to the Client with each incoming message.	Optional	Empty string
AuxDataContextKeyName	The session context key name to save the auxiliary data received from the client.	Optional	%HandlerName% or %HandlerName%.%clientTag% if clientTag is not empty

FIXEdge.BusinessLayer.Rule.Action.HandlerAction Element

Attribute	Meaning	Required

Attribute	Meaning	Required
Name	Matches the name of the Handler from the FIXEdge.DllHandlers.Handler element.	Mandatory
AuxData	The auxiliary data passed to the Client for this specific HandlerAction.	Optional
Timeout	The maximum communication time (in microseconds) for this specific HandlerAction.	Optional
AuxDataContextKeyName	The session context key-value storage key name to save the auxiliary data received from the client.	Optional

Calling from JavaScript

To call a handler from JavaScript:

// Calling the IPC1 with the current FIX message, passing "HandlerAuxData" to the Client, binding the auxiliary data, sent by the client to the "IPC1_clientAuxData" key of the session KV storage.

handler("IPC1", [ 
	["TimeoutUsec", "150000"], 
	["AuxData", "HandlerAuxData"], 
	["AuxDataContextKeyName", "AuxDataKvKey"] 
]);

This example initiates a call to the IPC1 handler with the current FIX message, passing "HandlerAuxData" to the Client, and binds the auxiliary data sent by the Client to the IPC1_clientAuxData key in the session KV storage.

Scripting Abstractions for FIXEdge IPC Server

FIXEdge SDK provides IPC client samples for Linux and Windows implementing the current protocol version with Node.JS and Python. The samples consist of a simple FIXEdge server abstraction and some client code:

FIXEdge-SDK/ipc_client
FIXEdge-SDK/ipc_client/protocol.md                          # The actual protocol specification.
FIXEdge-SDK/ipc_client/Node.js                             # Node.js is similar for Windows and Linux.
FIXEdge-SDK/ipc_client/Node.js/fixedge_ipc_server.mjs      # Simple server abstraction.
FIXEdge-SDK/ipc_client/Node.js/ipc_client_example.mjs     # Simple client that uses the above abstraction.
FIXEdge-SDK/ipc_client/Python
FIXEdge-SDK/ipc_client/Python/Linux
FIXEdge-SDK/ipc_client/Python/Linux/fixedge_ipc_server.py  # Simple server abstraction.
FIXEdge-SDK/ipc_client/Python/Linux/ipc_client_example.py # Simple client that uses the above abstraction.
FIXEdge-SDK/ipc_client/Python/Windows/requirements.txt    # Dependencies to be installed.
FIXEdge-SDK/ipc_client/Python/Windows
FIXEdge-SDK/ipc_client/Python/Windows/fixedge_ipc_server.py  # Simple server abstraction.
FIXEdge-SDK/ipc_client/Python/Windows/ipc_client_example.py # Simple client that uses the above abstraction.

Node.js FIXEdge Server Abstraction

Node.js module fixedge_ipc_server.mjs provides a FIXEdge IPC server abstraction which is similar for both Windows and Linux. The module provides a FixEdgeIpcServerV1 class definition which implements the IPC Protocol V1 communication. The class provides the following methods:

FixEdgeIpcServerV1 Method	Result and Parameters	Description

FixEdgeIpcServerV1 Method	Result and Parameters	Description
connect(connPath)	connPath - FIXEdge IPC server listen address	A Unix socket path on Linux or a Windows named pipe name on Windows.
on(event, callback) once(event, callback)	event - one of 'message', 'clientTag', 'error', 'end'. 'message' callback will be called when a new message from the server arrives. The callback signature: onMessage(fixMessageRequest). The fixMessageRequest is described below. 'error' callback will be triggered when the server returns an error during the handshake phase. The callback signature: onError(message), where message is a string containing the error text. 'end' callback will be triggered when the server closes the connection. The callback signature: onError(). 'clientTag' callback will be triggered when the client receives a clientTag from the server. The callback signature: onClientTag(clientTag), where clientTag is a string containing the tag. callback - a callback function to be called when the specified event is received.	The on('message', ...) callback can be substituted with an onMessage(fixMessageRequest) method defined on the FixEdgeIpcServerV1 object. Please find the description of on('message', ...) requirements in the section dedicated to the onMessage() callback.
onMessage(fixMessageRequest)	fixMessageRequest is an object with the following structure: `{ 'message': 'a string containing the JSON FIX message', 'clientTag': 'a string containing the clientTag setup on server', 'auxData': 'a string auxiliary data passed with the message' }`	The onMessage call should always call replySuccess or replyError to wrap up the server communication. Otherwise, the FIXEdge IPC server will time out waiting for the reply and fail the message handling.
replySuccess(replyMessage, templateString, auxData)	replyMessage - a JSON string representing the modified FIX message (or empty if no modifications were made). templateString - a template string for FIXEdge to be able to parse the returned message. auxData - auxiliary data that is attached to the key specified by the AuxDataContextKeyName configuration parameter.	Upon receiving a success reply, the FIXEdge IPC server will try parsing the returned message (if any), and if successful, it will continue the business rule execution with the new message. If no messages are returned, it continues the business rule execution with the original message.
replyError(errorCode, errorMessage)	errorCode - a non-zero error code describing the error. errorMessage - a textual error description.	If the returned message is not parsed successfully, the rule is failed. The rule is failed, and the error code and message are logged and populated using the configured FIXEdge mechanisms.

Basic Usage Example

This simple example for Linux only prints the received message to the standard output.

import { FixEdgeIpcServerV1 } from './fixedge_ipc_server.mjs';
 
const fe = new FixEdgeIpcServerV1;
 
fe.on('error', (message)=>{
    console.log('FixEdge server communication error: ', message);
});
 
fe.on('end', ()=>{
    console.log('FixEdge server closed connection.');
});
 
fe.on('clientTag', (clientTag)=>{
    console.log('FixEdge server set up the client tag: ' + clientTag);
});
 
fe.onMessage = (fixMessageRequest)=>{
    console.log('FixEdge server message: ' + JSON.stringify(fixMessageRequest));
    fe.replySuccess(/*not returning any modified message*/ '', /*no template string*/ '',  /*no auxiliary data*/ '');
};
 
if(process.platform !== 'win32')
    fe.connect('/tmp/fixedge_unixdomainsocket_name');
else
    fe.connect('\\\\.\\pipe\\fixedge_namedpipe_name');

Python FIXEdge Server Abstraction

Python version 3 and later is necessary to run the examples.

Python FIXEdge server abstraction may require installing dependencies if specified in requirements.txt file. An example command that may be used is as follows:

python3 -m pip install -r requirements.txt

FixEdgeIpcServerV1 Method	Result and Parameters	Description

FixEdgeIpcServerV1 Method	Result and Parameters	Description
connect(connPath)	connPath - FIXEdge IPC server listen address. Result - the clientTag assigned by the server.	A Unix socket path on Linux or a Windows named pipe name on Windows. In case of an unsuccessful connection, the returned error message is thrown as an exception.
recv_fix_message_request()	Result - a dictionary with the following fields: `{ 'message': 'a string containing the JSON FIX message', 'clientTag': 'a string containing the clientTag setup on server', 'auxData': 'a string auxiliary data passed with the message' }`	The recv_fix_message_request blocks the script execution until a new message handling request is read from the Unix socket/named pipe. If a communication error occurs during the call, an exception with the error description is thrown within its body.
reply_success(replacement_fix_message, template_string, auxiliary_data)	replacement_fix_message - a JSON string representing the modified FIX message (or empty if no modifications were done). template_string - a template string for FIXEdge to be able to parse the returned message. auxiliary_data - auxiliary data that is attached to the key specified by the AuxDataContextKeyName configuration parameter.	Upon receiving a successful reply, the FIXEdge IPC server will try parsing the returned message (if any). If successful, it will continue the business rule execution with the new message. If no messages are returned, it continues the business rule execution with the original message.
reply_error(error_code, error_message)	error_code - a non-zero error code describing the error. error_message - a textual error description.	If the returned message is not parsed successfully, the rule is failed. The rule is failed, and the error code and message are logged and populated using the configured FIXEdge mechanisms.

Basic Usage Example

This simple example for Linux only prints the received message to the standard output.

import sys
import json
from fixedge_ipc_server import fixedge_ipc_server_v1
 
if sys.version_info[0] < 3:
    raise Exception("Python 3 is required for this code to run.")
 
server = fixedge_ipc_server_v1()
 
client_tag = ""
try:
    client_tag = server.connect('/tmp/fixedge_unixdomainsocket_name')
except Exception as error:
    print("Unable to connect due to:", error)
    exit(1)
 
print('FixEdge server set up the client tag: ', client_tag)
 
while True:
     fmr = {};
     try:
        fmr = server.recv_fix_message_request()
    except Exception as error:
        print("Unable to communicate due to:", error)
        exit(1)
 
    print(fmr)
    server.reply_success('', '', '')

Configuration and Scripting Examples

BL_Config for Windows IPC Handler Configuration

A minimal FIXEdge BL_Config entry for Windows could look like this:

<Handler
    Name="IPC1"
    Description="IPC Handler"
    DllName="IPCHandler-MD-x64.dll"
    ListenAddress="\\.\pipe\fixedge_namedpipe_name"
/>

Modifying the Incoming Message

import sys
import json
from fixedge_ipc_server import fixedge_ipc_server_v1

if sys.version_info[0] < 3:
    raise Exception("Python 3 is required for this code to run.")

server = fixedge_ipc_server_v1()

client_tag = ""
try:
	client_tag = server.connect('/tmp/fixedge_unixdomainsocket_name')
except Exception as error:
	print("Unable to connect due to:", error)
    exit(1)

print('FixEdge server set up the client tag: ', client_tag)

while True:
     fmr = {};
     try:
		fmr = server.recv_fix_message_request()
	except Exception as error:
        print("Unable to communicate due to:", error)
        exit(1)

    message = json.loads(fmr['message'])

	message['Body']['ListID'] = 'NewListID'

    server.reply_success(json.dumps(message), '', '')

import { FixEdgeIpcServerV1 } from './fixedge_ipc_server.mjs';

const fe = new FixEdgeIpcServerV1;

fe.on('error', (message) => {
    console.log('FixEdge server communication error: ', message); 
});

fe.on('end', () => {
    console.log('FixEdge server closed connection.'); 
});

fe.on('clientTag', (clientTag) => {
    console.log('FixEdge server set up the client tag: ' + clientTag); 
});

fe.onMessage = (fixMessageRequest) => {
    var msg = JSON.parse(fixMessageRequest.message);
    msg.Body.ListID = 'NewListID'; 
    fe.replySuccess(JSON.stringify(msg), '', '');
};

if (process.platform !== 'win32') 
    fe.connect('/tmp/fixedge_unixdomainsocket_name');
else
    fe.connect('\\\\.\\pipe\\fixedge_namedpipe_name');

Using Custom FIX Parsers

# Usual FE server abstraction initialization is skipped
 
while True:
     fmr = {};
     try:
        fmr = server.recv_fix_message_request()
    except Exception as error:
        print('Unable to communicate due to:', error)
        exit(1)
 
    message = json.loads(fmr['message'])
    message['Header']['BeginString'] = 'FIX.4.3'
    server.reply_success(json.dumps(message), 'FIX43', '')

// Usual FE server abstraction initialization is skipped
 
fe.onMessage = (fixMessageRequest)=>{
    var msg = JSON.parse(fixMessageRequest.message);
    msg.Header.BeginString = 'FIX.4.3';
    fe.replySuccess(JSON.stringify(msg), 'FIX43',  '');
};

Handling Auxiliary Data

Auxiliary data can be transferred to a client using a BL_Config rule configuration using Handler.AuxData (to specify per-Handler auxiliary data) or Rule.AuxData (to specify per-Rule auxiliary data) XML elements. Yet another option is to use a Script XML element of BL_Config.xml, as follows:

<Action> 
    <Script Language="JavaScript" FileName="aux-data.js"/> 
</Action>

The aux-data.js file could look like this:

// Calling the IPC1 with the current FIX message, passing "Hello, Client!" to 
// the Client, binding the auxiliary data sent by the client to the 
// "IPC1_clientAuxData" key of the session KV storage. 
var ctxKvKey = "IPC1_clientAuxData";

handler("IPC1", [
    ["AuxData", "Hello, Client!"], 
    ["AuxDataContextKeyName", ctxKvKey]
]);

print(getCtxKV(ctxKvKey));

This example assumes that auxiliary data in the string format "Hello, Client!" is passed to the client. The client also passes its auxiliary data to the Handler. The Handler attaches the auxiliary data to the session context key-value store under the key IPC1_clientAuxData, which is later read by the getCtxKv call, as demonstrated in the following code excerpts:

# Usual FE server abstraction initialization is skipped
while True:
    fmr = {}
    try:
        fmr = server.recv_fix_message_request()
    except Exception as error:
        print('Unable to communicate due to:', error)
        exit(1)

    print('Received from server:', fmr['auxData'])
    server.reply_success('', '', '', 'Hello, Server!')

// Usual FE server abstraction initialization is skipped
fe.onMessage = (fixMessageRequest) => {
    console.log(fixMessageRequest.auxData);
    fe.replySuccess(JSON.stringify(msg), '', 'Hello, Server!');
};

Both of these examples print "Hello, Client!" to the standard output, and a log entry similar to the following appears in the FIXEdge logs:

...
2024-10-18 14:28:09,759 UTC   DEBUG   [BL_Action]  138398241130048  Setting BL context key value store [IPC1._client_] = Hello, Server!
...
2024-10-18 14:28:09,760 UTC   INFO    [aux-data.js]  138398241130048  JavaScript 'aux-data.js' output: Hello, Server!
...

These messages may change when disabling log categories.

EPAM-B2BITS products for Capital Markets

Inter-Process Communication (IPC) Handler

Overview

Functionality

Benefits

Terms and Abbreviations

Communication Protocol

Data Exchange

Business Layer Configuration Options

FIXEdge.DllHandlers.Handler Element

FIXEdge.BusinessLayer.Rule.Action.HandlerAction Element

Calling from JavaScript

Scripting Abstractions for FIXEdge IPC Server

Node.js FIXEdge Server Abstraction

Basic Usage Example

Python FIXEdge Server Abstraction

Basic Usage Example

Configuration and Scripting Examples

BL_Config for Windows IPC Handler Configuration

Modifying the Incoming Message

Using Custom FIX Parsers

Handling Auxiliary Data

Related content