Overview

IBM MQ Transport Adaptor Adapter (hereafter MQ TA) is an add-in for FIXServer that allows clients to communicate with FIXServer over IBM MQSeries middleware. The document contains full instructions how to install MQ TA for testing.

Description

MQ TA is responsible for:

• Establishing and maintaining sessions with MQ Series Server to communicate with Clients
• Transmitting messages to its clients
• Delivering messages from clients
• Facilitating monitoring and administration

Image Modified

Monitoring and Administration

...

• Status
• Number of messages sent
• Number of messages received
• Time of the last successful message sending
• Time of the last successful message receiving
• List of clients

Integration Guide

Installation MQ

...

Adapter to FixEdge

IBM MQ Transport Adaptor Adapter is present on FIXEdge package on bin directory.

MQ adaptor Adapter requires IBM MQ client libraries installed in the host with FIXEdge, the libraries can be found on the official IBM MQ site.

Configuration

IBM MQ Transport Adaptor Adapter can be configured using FIXICC or directly by editing the FIXEdge.properties file of the correspondent FIX Edge instance. All properties to be set up are described in the table below. You can also use FIXEdge.properties file from the package as an example.

Make sure that configuration of the adaptor adapter correlates with the MQ Series Server parameters.

Configuration Sample to FIX MQ conversion flow

The following configuration describes the basic FIX to MQ conversion message flow.

...

1. Configure sessions in FIXEdge.properties

   Code Block
   title FIXEdge.properties
   #------------------------------------------------------------ # FIX Layer Section #------------------------------------------------------------ FixLayer.FixEngine.Sessions = SENDER, RECEIVER FixLayer.FixEngine.Session.SENDER.Version = FIX44 FixLayer.FixEngine.Session.SENDER.StorageType = persistentMM FixLayer.FixEngine.Session.SENDER.Role = Acceptor FixLayer.FixEngine.Session.SENDER.SenderCompID = FIXEDGE FixLayer.FixEngine.Session.SENDER.TargetCompID = SENDER FixLayer.FixEngine.Session.SENDER.EncryptMethod = 0 FixLayer.FixEngine.Session.SENDER.InSeqNum = 0 FixLayer.FixEngine.Session.SENDER.OutSeqNum = 0 FixLayer.FixEngine.Session.SENDER.Description = FIX session for test mq sender FixLayer.FixEngine.Session.SENDER.StartTime = 08:00 FixLayer.FixEngine.Session.SENDER.TerminateTime = 23:59 FixLayer.FixEngine.Session.SENDER.RecreateOnLogout = true FixLayer.FixEngine.Session.SENDER.IntradayLogoutTolerance = true FixLayer.FixEngine.Session.SENDER.ForceSeqNumReset = false FixLayer.FixEngine.Session.SENDER.SocketPriority = EVEN FixLayer.FixEngine.Session.SENDER.HandleSeqNumAtLogon = false FixLayer.FixEngine.Session.RECEIVER.Version = FIX44 FixLayer.FixEngine.Session.RECEIVER.StorageType = persistentMM FixLayer.FixEngine.Session.RECEIVER.Role = Acceptor FixLayer.FixEngine.Session.RECEIVER.SenderCompID = FIXEDGE FixLayer.FixEngine.Session.RECEIVER.TargetCompID = RECEIVER FixLayer.FixEngine.Session.RECEIVER.EncryptMethod = 0 FixLayer.FixEngine.Session.RECEIVER.InSeqNum = 0 FixLayer.FixEngine.Session.RECEIVER.OutSeqNum = 0 FixLayer.FixEngine.Session.RECEIVER.Description = FIX session for test mq receiver FixLayer.FixEngine.Session.RECEIVER.StartTime = 08:00 FixLayer.FixEngine.Session.RECEIVER.TerminateTime = 23:59 FixLayer.FixEngine.Session.RECEIVER.RecreateOnLogout = true FixLayer.FixEngine.Session.RECEIVER.IntradayLogoutTolerance = true FixLayer.FixEngine.Session.RECEIVER.ForceSeqNumReset = false FixLayer.FixEngine.Session.RECEIVER.SocketPriority = EVEN FixLayer.FixEngine.Session.RECEIVER.HandleSeqNumAtLogon = false

2. Configure MQ AdaptorAdapter

   Code Block
   title FIXEdge.properties
   #------------------------------------------------------------ # Transport Layer Section #------------------------------------------------------------ #Comma separated list of identifiers of Transport Adapters should be loaded. TransportLayer.TransportAdapters = TransportLayer.MQAdaptor #------------------------------------------------------------------------- # MQ Adaptor configuration #------------------------------------------------------------------------- TransportLayer.MQAdaptor.Description = MQ Transport Adaptor DLL TransportLayer.MQAdaptor.Type = DLL TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-MD-x64.dll TransportLayer.MQAdaptor.TimeIntervalBeforeReconnect = 1000 TransportLayer.MQAdaptor.BufferSizeForMessage = 65536 TransportLayer.MQAdaptor.MQ_HOSTNAME = ecsa00400539.epam.com TransportLayer.MQAdaptor.MQ_MANAGER_NAME = FIXEdge TransportLayer.MQAdaptor.MQ_PORT = 1414 TransportLayer.MQAdaptor.MQ_CCSID = 437 TransportLayer.MQAdaptor.MQ_CHANNEL_NAME = TestConnection TransportLayer.MQAdaptor.WaitIntervalForGet = 1000 TransportLayer.MQAdaptor.NumAttemptReconnect = 10 TransportLayer.MQAdaptor.StorageDirectory = FIXEdge1/log TransportLayer.MQAdaptor.UseTransactions = true TransportLayer.MQAdaptor.SessionNumber = 2 # MQ QA session TransportLayer.MQAdaptor.SessionNumber = 1 TransportLayer.MQAdaptor.Session.1.ClientID = MQHub1 TransportLayer.MQAdaptor.Session.1.ToClientQueue = MQClient_test TransportLayer.MQAdaptor.Session.1.FromClientQueue = TransportLayer.MQAdaptor.Session.1.ErrorQueue = MQClient_error TransportLayer.MQAdaptor.Session.1.SmartXMLProcessor = true TransportLayer.MQAdaptor.Session.1.SendTransactionSize = 300 TransportLayer.MQAdaptor.Session.1.ReceiveTransactionSize = 1 TransportLayer.MQAdaptor.SessionNumber = 2 TransportLayer.MQAdaptor.Session.2.ClientID = MQHub2 TransportLayer.MQAdaptor.Session.2.ToClientQueue = TransportLayer.MQAdaptor.Session.2.FromClientQueue = MQClient_test TransportLayer.MQAdaptor.Session.2.ErrorQueue = MQClient_error TransportLayer.MQAdaptor.Session.2.SmartXMLProcessor = true TransportLayer.MQAdaptor.Session.2.SendTransactionSize = 300 TransportLayer.MQAdaptor.Session.2.ReceiveTransactionSize = 1

3. Configure BL 

   Code Block
   language xml
   <?xml version="1.0" encoding="UTF-8"?> <FIXEdge> <BusinessLayer> <Rule Description="Convert ExecutionReport from FIX.4.4 to FIXML.4.4"> <Source Name="SENDER" /> <Action> <Convert SourceProtocol="FIX.4.4" TargetProtocol="FIXML.4.4"/> <Send Name="MQHub1"/> </Action> </Rule> <Rule Description="Route MQ messages back to receiver and convert from FIXML.4.4 to FIX.4.4"> <Source Name="MQHub2" /> <Action> <Convert SourceProtocol="FIXML.4.4" TargetProtocol="FIX.4.4"/> <Send Name="RECEIVER"/> </Action> </Rule> <DefaultRule> <Action> <DoNothing/> </Action> </DefaultRule> </BusinessLayer> </FIXEdge>

   Note
   Full list of possible versions to convert can be found in Convert

4. Configure Queues Managers on IBM Websphere server

Error queue

An error queue is a local queue to which error messages can be sent by the WebSphere MQ connector when a message operation fails. For target queues, the error queue acts as a backup option for the target queue when messages cannot be written to the target queue

...

1. FIXEdge can't produce data to MQ. (e.g. due to there is no connection to MQ server.)
2. FIXEdge can't consume data from MQ server.
3. Other various MQ errors. (E.g. due to configuration issues, or message content issues)

FIXEdge can't produce data to MQ

The reason for getting messages to ErrorQueue is diverse, for each is necessary individual analysis of the logs.

...

• Use different Error Queues per session. It would be easier to troubleshoot issues if the session would have own error queue.
  It can be configured with FIXEdge property:

  Code Block
  language perl
  TransportLayer.MQAdaptor.Session.1.ErrorQueue = MQClient_error_1 TransportLayer.MQAdaptor.Session.2.ErrorQueue = MQClient_error_2

• Use dmpmqmsg tool for moving messages from the error queue to producer on. So all messages from the error queue should be replayed.

  Code Block
  language perl
  dmpmqmsg -m MQClient -I MQClient_error -o MQClient_error_2 -s FIXEdge

  -m QueueManagerName
  -I Input queue name
  -o Output queue name
  -s Message content, as all messages have senderCompId and TargetCompId it can be used for filtering.

FIXEdge can't consume data from MQ server and other various MQ errors

These cases are rare ones if there were no configuration changes. 

Issues not causing messages in the error queue

There are cases that do not lead to the messages in the Error queue.

...

For troubleshooting, this case it's recommended to upgrade FIXEdge to version 6.9.0 with a Managed Queue feature.
It was designed for cases when messages should be pushed to the counterparty when they come back online and do not support recovery with resend requests.

Transactions in MQ

FIXEdge MQ Transport Adapter has a transaction mechanism. Before sending it to MQ the messages are persisted in the files.

...

Start and Stop operations for IBM MQ TA

...

Please refer to the Transport adaptor operations for FIXEdge C++ page for additional details about Start and Stop operations.

Start operation

Start operation can be applied to IBM MQ TA sessions in the Disconnected state.

...

"Session '<session_name>': The session <session_name> can't be started: <description> Reason: Operation cannot be applied"

Stop operation

Stop operation can be applied to IBM MQ TA sessions in the Running and Connecting states.

...

In case when IBM MQ TA session is in a Disconnected state and IBM MQ TA receives the message, this message is persisted in storage to be sent on IBM MQ TA session start.

Troubleshooting

The message is not sent to the client with multiple MQ TA configuration

The message is not delivered to MQ.

The following error appears in FixEdge.log when you use one MQ .dll for several adaptorsseveraladapters:

ERROR   [MQQueueWriter]  <Thread_ID>  Removed message doesn't equal to sent message! Storage: 'MQTA_ID' Message removed from storage

...

1. Use a single MQ TA library with several sessions 

   Code Block
   language perl
   TransportLayer.TransportAdapters = TransportLayer.MQAdaptor TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-MD-x64.dll #... TransportLayer.MQAdaptor.SessionNumber = 2 TransportLayer.MQAdaptor.Session.1.ClientID = MQTA_ID_1 TransportLayer.MQAdaptor.Session.1.ToClientQueue = MQClient_to_1 TransportLayer.MQAdaptor.Session.1.FromClientQueue = TransportLayer.MQAdaptor.Session.1.ErrorQueue = TransportLayer.MQAdaptor.Session.1.ReceiveTransactionSize=10 TransportLayer.MQAdaptor.Session.1.SendTransactionSize=10 TransportLayer.MQAdaptor.Session.2.ClientID = MQTA_ID_2 TransportLayer.MQAdaptor.Session.2.ToClientQueue = MQClient_to_2 TransportLayer.MQAdaptor.Session.2.FromClientQueue = TransportLayer.MQAdaptor.Session.2.ErrorQueue = TransportLayer.MQAdaptor.Session.2.ReceiveTransactionSize=10 TransportLayer.MQAdaptor.Session.2.SendTransactionSize=10

2. Use several MQ TA with own library

   Code Block
   language perl
   TransportAdapters = TransportLayer.MQAdaptor, TransportLayer.MQAdaptor2 TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-MD-x64.dll #... TransportLayer.MQAdaptor2.DllName = bin/MQTAAddin-MD-x64_2.dll #...

IBM MQ Error codes

The following error appears in FixEdge.log

...

Please find the API Error codes on the official site.

Reason code 2085

2021-12-15 17:27:31,361 GMT   ERROR   [MQQueueWrite]  140457136101120  ImqQueue::open ended with reason code 2085

...

• TransportLayer.MQAdaptor.MQ_MANAGER_NAME
• TransportLayer.MQAdaptor.MQ_CHANNEL_NAME
• TransportLayer.MQAdaptor.Session.1.ToClientQueue
• TransportLayer.MQAdaptor.Session.1.FromClientQueue
• TransportLayer.MQAdaptor.Session.1.ErrorQueue

MQ Adapter can't find all its dependencies

FIXEdge got the following ERROR records in the FixEdge.log during initialization that some libraries are not found on Windows:

...

1. Make sure that IBM MQ Client is installed on the host with FIXEdge
2. Make sure that the environment variable PATH contains the path to the IBM MQ Client bin folder.
3. Make sure MQTAAddin-xx.dll MQTAAddin-xx.so library exists in FIXEdge/bin folder.

The message is not processed due to incorrect rule found

Message was received from the client, but is not processed and forwarded by FIXEdge. The following error appears in FixEdge.log:

...

<Convert SourceProtocol="FIX.4.4" TargetProtocol="FIXML.4.4"/>

WebSphere MQ Setup Guide

Getting MQ Soft

If you need to set up your own MQ server, you can download software from IBM Site

Websphere MQ Configuration

Steps to configure access to Websphere MQ (running on Windows) from the remote host are described below. This configuration is performed on the MQ Server host.

1. Create User
   1. Create a non-domain user e.g. mquser in 'mqm' group.
   2. Change its domain to a local hostname e.g. EVUAKYISD0223.
   3. Grant this user permissions to log in remotely.
   4. Grant this user Administrative permissions.
2. Start MQ
   1. Start all MQ services ('IBM MQSeries' and 'IBM Websphere MQ (Installation N)') under user created on the previous step.
      In order to change the user for service, it is needed to go to Services. Select service and open its properties, go to the 'Log On' tab, select 'This user' and specify user credentials.
3. Configure Manager and Queues
   1. Start MQExplorer.
   2. Add new Queue Manager (e.g. TestMQ):
      
   3. Add Server-connection channel to created Queue Manager (e.g.  TestConnection ):
      
      
      
   4. Set channel MCA with a user from step 1.1 ( mquser ):
      
      
      
4. Add queues, (e.g. MQClient_error, MQClient_from, MQClient_to):
   
   

   
   

Check connection to Queue Manager from a remote host

Check connection using MQ Explorer

In order to use the described approach, you need to install IBM WebSphere MQ Explorer to the client box.

...

QM should be accessible via remote MQ Explorer.

Check connection using FE

In order to check the connection to Queue Manager from the remote (client) host using FE:

1. Configure FE using the following sample properties:

   Code Block
   title FIXEdge.properties
   TransportLayer.TransportAdapters = TransportLayer.MQAdaptor #------------------------------------------------------------------------- # MQ Adaptor Settings #------------------------------------------------------------------------- TransportLayer.MQAdaptor.Description = MQ Transport Adaptor DLL TransportLayer.MQAdaptor.TimeIntervalBeforeReconnect = 1000 TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-MD-x64.dll TransportLayer.MQAdaptor.StorageDirectory = FixEdge1/log TransportLayer.MQAdaptor.NumAttemptReconnect = -1 TransportLayer.MQAdaptor.WaitIntervalForGet = 10000 TransportLayer.MQAdaptor.BufferSizeForMessage = 65536 TransportLayer.MQAdaptor.MQ_HOSTNAME = EVUAKYISD0223.kyiv.epam.com TransportLayer.MQAdaptor.MQ_MANAGER_NAME = TestMQ TransportLayer.MQAdaptor.MQ_PORT = 1414 TransportLayer.MQAdaptor.MQ_CCSID = 437 TransportLayer.MQAdaptor.MQ_CHANNEL_NAME = TestConnection TransportLayer.MQAdaptor.SessionNumber = 1 TransportLayer.MQAdaptor.Session.1.ClientID = MQClient TransportLayer.MQAdaptor.Session.1.FromClientQueue = MQClient_from TransportLayer.MQAdaptor.Session.1.ToClientQueue = MQClient_to TransportLayer.MQAdaptor.Session.1.ErrorQueue = MQClient_error TransportLayer.MQAdaptor.Session.1.SmartXMLProcessor = false

2. Check if Queues are accessible:

   Code Block
   title FixEdge.log
   [DEBUG] 20120718-11:41:33.609 [2840] [MQEngine] - Was created queue 'MQClient_from' [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - createClientQueue() finished [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - ClientQueue created. [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - MQQueuesReader::readFromQueuesAndPassToTL(). [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - fromClientQueue->getFirst(). [DEBUG] 20120718-11:41:38.039 [12692] [MQEngine] - Was created queue 'MQClient_to' [DEBUG] 20120718-11:41:38.251 [12692] [MQEngine] - Was created queue 'MQClient_error' [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Queues were created for client 'MQClient' [NOTE] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Restore messages for out queue 'MQClient'. [NOTE] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Create storage for the: MQClient [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Load storage. [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Persistance storage use file: D:\Program Files\B2Bits\FIX Edge\v.5.4.1.40091\FixEdge1\log\MQ-MQClient_07180941382511.out [NOTE] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Restore messages for error queue 'MQClient'. [NOTE] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Create storage for the: MQClient-Errors [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Load storage. [DEBUG] 20120718-11:41:38.252 [12692] [MQQueueWriter] - Persistance storage use file: D:\Program Files\B2Bits\FIX Edge\v.5.4.1.40091\FixEdge1\log\MQ-MQClient -Errors_07180941382511.out [DEBUG] 20120718-11:41:43.678 [2840] [MQQueueRead] - Was performed get operation on queue 'MQClient_from' with reason code :2033 [DEBUG] 20120718-11:41:43.678 [2840] [MQQueueRead] - MQQueueRead::getFirst failed to get message '' [DEBUG] 20120718-11:41:43.678 [2840] [MQQueuesReader_Debug] - MQQueuesReader::readFromQueuesAndPassToTL(). [DEBUG] 20120718-11:41:43.678 [2840] [MQQueuesReader_Debug] - fromClientQueue->getFirst().

3. Route message to MQ and check FE logs:

   Code Block
   title FixEdge.log
   [DEBUG] 20120718-11:48:16.389 [7968] [FL_MsgTrace] - New message FIX was received from session with ID 'TROIKALSE'. Message: '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224'. [DEBUG] 20120718-11:48:16.389 [7968] [BL_Layer] - Process incoming message. [DEBUG] 20120718-11:48:16.389 [7968] [BL_RoutingTable] - There are no suitable rules were found for message with Source ID 'LSE', executing DefaultRule. [DEBUG] 20120718-11:48:16.389 [7968] [CC_Layer] - BL has processed a message. Number of client IDs for delivery :0. Number or FIX sessions for delivery :0.. Number or sources identifiers for delivery :1. Source IDs: 1. 'MQClient' [DEBUG] 20120718-11:48:16.389 [7968] [TransportLayer] - Sending message '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224' to client MQClient [DEBUG] 20120718-11:48:16.389 [7968] [MQQueueWriter] - Was put message to messageQueue. Message = (ToClientQueue, MQClient, 8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224) [DEBUG] 20120718-11:48:16.389 [7968] [MQQueueWriter] - Store into the Persistance storage. [DEBUG] 20120718-11:48:16.389 [12228] [MQQueueWriter] - Was get message from MessageQueue. Message: ( toClientQueue,MQClient, 8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224) [DEBUG] 20120718-11:48:16.460 [12228] [MQQueueWrite] - Was performed put operation on queue 'MQClient_to' with reason code :0. Message = '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224' [DEBUG] 20120718-11:48:16.460 [12228] [MQQueueWrite] - message was successfully sent to to queue 'MQClient_to' with reason code :0 [DEBUG] 20120718-11:48:16.460 [12228] [MQQueueWriter] - message was put to ToClient queue.Client 'MQClient'. MEssage '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224'

Configuring SSL in IBM MQ Server

This instruction assumes that non-SSL configuration works OK

...

TransportLayer.MQAdaptor.SslKeyRepository = C:\\B2BITS\\FIXEdge\\FixEdge1\\conf\\key
TransportLayer.MQAdaptor.SslCipherSpecification = TRIPLE_DES_SHA_US
TransportLayer.MQAdaptor.SslClientAuthentication = false
TransportLayer.MQAdaptor.SslPeerName =

Troubleshooting

Disabling IBM MQ security

MQ TA has not supported the authentication in IBM MQ Server. 

It can lead to failure in MQ TA initialization with the error records in the log:

...

If the following error message appears while accessing remote MQ manager using MQExplorer

Solution

The authentication should be disabled. You should perform the following steps:

1. Login to the remote server where MQ Instance is running

2. Open cmd

3. Run 'runmqsc'. Check if connected to correct QM:

   

4. Execute 'ALTER QMGR CHLAUTH(DISABLED)':

   

5. Try to reestablish connection by clicking 'No' in the dialog:

   

6. And Finish in the dialog:

   

7. If the error is still here, look at Channel authentication records and make sure that your account is not in the Block User List:
   
   
   
   
8. Try again steps 5-6.

Also you can find how to disable IBM MQ authentication on the official site.