How to update entities stored in History with referenced sequence number

Overview

This document describes how to configure the following use case:

A client sends New Orders List (35=E) message to FIXEdge. FIXEdge stores the message content into DataBase and routes the messages to a counterparty.
Assumed that the following tags should be stored into DB:

  • ListID(66)
  • ClOrdId(11) from a repeating group
  • Symbol(55) from a repeating group
  • TransactTime(60) from a repeating group
  • Text (58) from a repeating group

If the counterparty rejects a message with business reject (35=j), the reject reason will be updated in Text entity.
If the counterparty rejects a message with session level reject (35=3), the record will be removed from the DB.

DB Schema

Each repeating group entry is represented as a row with ListID(66) value as a key field.

BL Configuration

Configure ODBC History

Used ODBC History for storing message into the DB table:

BL_Config.xml
		<!-- Orders Table: ("ListID"*; "ClOrdId"*; "Symbol"; "TransactTime"; "Text") -->
		<History Name="Orders" 
				StorageType="ODBC"
				TableName="orders" 
				ConnectionString="DSN=FIXEdgeDB;UID=user;PWD=password;>
			<KeyField ColumnName="ListID">66</KeyField>
			<KeyField ColumnName="ClOrdId">11</KeyField>
			<Field ColumnName="Symbol">55</Field>
			<Field ColumnName="TransactTime">60</Field>
			<Field ColumnName="Text">58</Field>
		</History>
Assumed that 66 and 11 tags define a DB record (or fix message entity) unambiguously

Configure messages routing

BL_Config.xml
		<Rule Description="Enrich, store, and route a New Order Single message">
			<Source Name="SENDER" />
			<Condition>
				<MatchField Value="E" Field="35"/>
			</Condition>
			<Action>
				<!-- The message can be modified/enriched before routing -->
				<Script Language="JavaScript" FileName ="FIXEdge1/conf/saveOrders"/> <!-- Save orders linked with ListID (66) tag -->
				<Send Name="RECIPIENT" />
			</Action>
		</Rule>
Message enrichments could be done there

Processing rejects

BL_Config.xml
		<OnBusinesslRejectEvent>
			<Source Name="RECIPIENT"/>
			<Condition>
				<MatchField Value="E" Field="372"/>
			</Condition>
			<Action>
				<Script Language="JavaScript" FileName ="FIXEdge1/conf/updateOrders"/>
			</Action>
		</OnBusinesslRejectEvent>
		
		<OnSessionLevelRejectEvent>
			<Source Name="RECIPIENT"/>
			<Condition>
				<MatchField Value="E" Field="372"/>
			</Condition>
			<Action>
				<Script Language="JavaScript" FileName ="FIXEdge1/conf/removeOrders"/>
			</Action>
		</OnSessionLevelRejectEvent>

It is user responsibility to insert and remove records to/from DB.

Scripts saveOrders and updateOrders should be in sync with DB structure.

Full BL_Config.xml example

BL_Config.xml
<FIXEdge>
	<BusinessLayer>

		<!-- Orders Table: ("ListID"*; "ClOrdId"*; "Symbol"; "TransactTime"; "Text") -->
		<History Name="Orders" 
				StorageType="File"
				WorkingDirectory="FIXEdge1/log/" 
				StorageFileName="Orders">
			<KeyField ColumnName="ListID" ColumnSize="128">66</KeyField>
			<KeyField ColumnName="ClOrdId" ColumnSize="256">11</KeyField>
			<Field ColumnName="Symbol" ColumnSize="256">55</Field>
			<Field ColumnName="TransactTime" ColumnSize="256">60</Field>
			<Field ColumnName="Text" ColumnSize="256">58</Field>
		</History>

		
		<Rule Description="Enrich, store, and route a New Order Single message">
			<Source Name="SENDER" />
			<Condition>
				<MatchField Value="E" Field="35"/>
			</Condition>
			<Action>
				<!-- The message can be modified/enriched before routing -->
				<Script Language="JavaScript" FileName ="FIXEdge1/conf/saveOrders"/> <!-- Save orders linked with ListID (66) tag -->
				<Send Name="RECIPIENT" />
			</Action>
		</Rule>
		
		
		<OnBusinesslRejectEvent>
			<Source Name="RECIPIENT"/>
			<Condition>
				<MatchField Value="E" Field="372"/>
			</Condition>
			<Action>
				<Script Language="JavaScript" FileName ="FIXEdge1/conf/updateOrders"/>
			</Action>
		</OnBusinesslRejectEvent>
		
		<OnSessionLevelRejectEvent>
			<Source Name="RECIPIENT"/>
			<Condition>
				<MatchField Value="E" Field="372"/>
			</Condition>
			<Action>
				<Script Language="JavaScript" FileName ="FIXEdge1/conf/removeOrders"/>
			</Action>
		</OnSessionLevelRejectEvent>
		
		
		<DefaultRule>
			<Action>
				<DoNothing/>
			</Action>
		</DefaultRule>

	</BusinessLayer>
</FIXEdge>


Scripts descriptions

Saving Orders to DB

The script does the following:

  1. Checks if message has required tags
  2. Searches for records in DB
  3. Prepares a new record 
  4. Inserts or Updates the record in DB
saveOrders.js
listID = getStringField(66);
noOrders = getNumField(73);
ordersGrp = getGroup(73)
if ( (listID != null) && (noOrders > 0) && isGroupValid(ordersGrp) )  // check if message is valid 
{
	for (i=0; i < noOrders; ++i) 
	{
		clOrdId = getStringField(ordersGrp, i, 11);	
		symbol = getStringField(ordersGrp, i, 55);
		transactTime = getStringField(ordersGrp, i, 60);
		text = getStringField(ordersGrp, i, 58);

		// ListID, ClOrdId are key fields and shouldn't be null
		if (clOrdId != null) 
		{
			orderKeys = new Array( noOrders, clOrdId ); 
			
			// convert missing values to empty string.
			if (symbol == null) {symbol="";}
			if (transactTime == null) {transactTime="";}
			if (text == null) {text="";}
			orderCommonValues = new Array(symbol, transactTime, text);
			
			//print("[DEBUG] getting data for listID=" + listID + " and clOrdId=" + clOrdId + " from 'Orders' table.");
			// get data from history
			records = getRecordFromHistory("Orders", orderKeys); // records array is {ListID*, ClOrdId*, Symbol, TransactTime, Text}
			if (records == null) // if there are no records - add new one
			{
				//print("[DEBUG] Saving order with ListID=" + listID + " and clOrdId=" + clOrdId + " to 'Orders' table.");
				saveToHistory("Orders", orderKeys, orderCommonValues, ""); 
			}
			else //or update stored record
			{
				//print("[DEBUG] record was found. Updating order with ListID=" + listID + " and clOrdId=" + clOrdId + " to 'Orders' table.");
				updateHistory("Orders", orderKeys, orderCommonValues, ""); 					
			}			
		}
	}
}

Updating Orders in DB

The script does the following:

  1. Prepares a record for update
  2. Uses RefSeqNum (45) tag value for getting data from the rejected message
  3. Checks if the rejected message has required tags
  4. Gets the records of  the rejected message from DB 
  5. Updates the record with prepared values
  6. Updates the records in DB.
updateOrders.js
rejectedText = "Rejected: " + getNumField(58); // Reject reason will be updated for rejected orders.

//getting rejected message from the storage.
rejectedSeqNum = getNumField(45);
sender = getStringField(56); // Get SenderCompId as TragetCompId in Reject message
target = getStringField(49); // Get TragetCompId as SenderCompId in Reject message
//print("[DEBUG] getting message with " + rejectedSeqNum + " sequence number from storage for session " + sender + "-" + target + ".");
getMsgBySeqNum(sender, target, Number(rejectedSeqNum));

listID = getStringField(66);
noOrders = getNumField(73);
ordersGrp = getGroup(73)
if ( (listID != null) && (noOrders > 0) && isGroupValid(ordersGrp) )  // check if message is valid 
{
	for (i=0; i < noOrders; ++i) 
	{
		clOrdId = getStringField(ordersGrp, i, 11);	
		// ListID, ClOrdId are key fields. clOrdId should be not null.
		if (clOrdId != null) 
		{
			orderKeys = new Array( noOrders, clOrdId ); 
			//print("[DEBUG] getting data for listID=" + listID + " and clOrdId=" + clOrdId + " from 'Orders' table.");
			// get data from history
			records = getRecordFromHistory("Orders", orderKeys); // records array is {ListID*, ClOrdId*, Symbol, TransactTime, Text}
			// records length should be 5
			if ( (records != null) && (records.length == 5) ) {
				// updating Text field with reject field.
				records[4] = rejectedText;
				// remove first 2 elemets as  key fields from record array.
				records.slice(0, 2); // Left data {Symbol, TransactTime, Text}
				updateHistory("Orders", orderKeys, records, ""); // Update data.	
			}		
		}
			
	}		
}

Remove Orders from DB

The script does the following:

  1. Prepares a record for update
  2. Uses RefSeqNum (45) tag value for getting data from the rejected message
  3. Checks if the rejected message has required tags
  4. Removes records related to the rejected messages from DB
removeOrders.js
//getting rejected message from the storage.
rejectedSeqNum = getNumField(45); 
sender = getStringField(56); // Get SenderCompId as TragetCompId in Reject message
target = getStringField(49); // Get TragetCompId as SenderCompId in Reject message
//print("[DEBUG] removeOrders.js: getting message with " + rejectedSeqNum + " sequence number from storage for session " + sender + "-" + target + ".");
getMsgBySeqNum(sender, target, Number(rejectedSeqNum));

listID = getStringField(66);
noOrders = getNumField(73);
ordersGrp = getGroup(73)
if ( (listID != null) && (noOrders > 0) && isGroupValid(ordersGrp) )  // check if message is valid 
{
	for (i=0; i < noOrders; ++i) 
	{
		clOrdId = getStringField(ordersGrp, i, 11);	
		// ListID, ClOrdId are key fields. clOrdId should be not null.
		if (clOrdId != null) 
		{
			orderKeys = new Array( noOrders, clOrdId ); 
			//print("[DEBUG] Removing record with listID=" + listID + " and clOrdId=" + clOrdId + " from 'Orders' table.");
			// get data from history
			records = removeRecordFromHistory ("Orders", orderKeys);
		}
	}		
}

Message examples

NewOrderList.txt
8=FIXT.1.1|9=254|35=E|1128=6|49=SENDER|56=FIXEDGE|34=2|52=20170922-06:42:15.240|66=TestList1|394=1|68=2|73=2|11=Order1|67=1|55=EPAM|54=1|60=20170922-22:33:44.999|38=1|40=1|44=20.5|58=TEST|11=Order2|67=2|55=EPAM|54=2|60=20170922-22:33:44.999|38=1|40=1|44=20.5|58=TESTTEST|10=244|