Overview

IBM MQ Transport Adaptor (hereafter MQ TA) is an add-in for FIXServer that allows clients to communicate with FIXServer over IBM MQSeries middleware. The document contains full instructions how to install MQ TA for testing.

Description

MQ TA is responsible for:

• Establishing and maintaining sessions with MQ Series Server to communicate with Clients
• Transmitting messages to its clients
• Delivering messages from clients
• Facilitating monitoring and administration

Monitoring and Administration

MQ TA provides the following monitoring information:

• Number of messages sent
• Number of messages received
• Time the last message was sent/received for each client
• List of clients

Integration Guide

Installation MQ Adaptor to FixEdge

Normally MQ adaptor is distributed as a zip-archive with the name MQAdaptor-xx.zip, where xx stands for MQ adaptor version. The distribution package consists of:

To install MQ adaptor:

• IBM MQ Client libraries must be installed on the FIX Edge machine where MQ Adaptor is ran. Third-party libraries are not incuded into TA package. They can be downloaded from  here.
• Unpack the package and copy MQTAAddin-xx.dll/ MQTAAddin-xx.so  into the FIX Edge /bin folder. Configure adaptor according to the instructions below.

• Restart FIX Edge.

Configuration

IBM MQ Transport Adaptor can be configured using FIXICC or directly by editing the FIXEdge.properties file of the correspondent FIX Edge instance. All properties to be set up are described in the table below. You can also use FIXEdge.properties file from the package as an example.

Make sure that configuration of the adaptor correlates with the MQ Series Server parameters.

Error queue

An error queue is a local queue to which error messages can be sent by the WebSphere MQ connector when a message operation fails. For target queues, the error queue acts as a backup option for the target queue when messages cannot be written to the target queue

As per the solution diagram above, there are the following reasons for messages sent to the error queue:

1. FIXEdge can't produce data to MQ. (e.g. due to there is no connection to MQ server.)
2. FIXEdge can't consume data from MQ server.
3. Other various MQ errors. (E.g. due to configuration issues, or message content issues)

FIXEdge can't produce data to MQ

The reason for getting messages to ErrorQueue is diverse, for each is necessary individual analysis of the logs.

To process messages from the queue, it's recommended:

• Use different Error Queues per session. It would be easier to troubleshoot issues if the session would have own error queue.
  It can be configured with FIXEdge property:

  TransportLayer.MQAdaptor.Session.1.ErrorQueue = MQClient_error_1
  
  TransportLayer.MQAdaptor.Session.2.ErrorQueue = MQClient_error_2

• Use dmpmqmsg tool for moving messages from the error queue to producer on. So all messages from the error queue should be replayed.

  dmpmqmsg -m MQClient -I MQClient_error -o MQClient_error_2 -s FIXEdge

  -m QueueManagerName
  -I Input queue name
  -o Output queue name
  -s Message content, as all messages have senderCompId and TargetCompId it can be used for filtering.

FIXEdge can't consume data from MQ server and other various MQ errors

These cases are rare ones if there were no configuration changes. 

Issues not causing messages in the error queue

There are cases that do not lead to the messages in the Error queue.

E.g.: FIXEdge can't push the data to the FIX connection because the destination is down.

For troubleshooting, this case it's recommended to upgrade FIXEdge to version 6.9.0 with a Managed Queue feature.
It was designed for cases when messages should be pushed to the counterparty when they come back online and do not support recovery with resend requests.

Transactions in MQ

FIXEdge MQ Transport Adapter has a transaction mechanism. Before sending it to MQ the messages are persisted in the files.

A path to the files can be configured with a property:

TransportLayer.MQAdaptor.StorageDirectory = FIXEdge1/log

The file storages are needed for keeping undelivered messages that require resending after FIXEdge restart (e.g. in case of failure).

Each file represents an MQ Session and the name of the file contains ID of the session E.g.:

MQ Session logs have data and corresponding transaction flag:

1. • Status of the undelivered transaction: Record valid='Y'.
   • Status of the finished transaction. The message has been sent to MQ: Record valid='N'.

Examples of the log from MQ_<ClientID>_xxx.out

<Record valid='Y' size='Ó      ' data='8=FIX.4.4 9=3259 35=AE 49=FIXClient 56=FIXEdgeMQHub1 115=STRING 128=STRING 90=1 91=D 34=169 50=STRING 142=STRING 57=STRING...10=139'/>
...
<Record valid='N' size='Ó      ' data='8=FIX.4.4 9=3259 35=AE 49=FIXClient 56=FIXEdgeMQHub1 115=STRING 128=STRING 90=1 91=D 34=208 50=STRING 142=STRING 57=STRING...10=137'/>

Troubleshooting

The message is not sent to the client with multiple MQ TA configuration

The message is not delivered to MQ.

The following error appears in FixEdge.log when you use one MQ .dll for several adapters:

ERROR   [MQQueueWriter]  <Thread_ID>  Removed message doesn't equal to sent message! Storage: 'MQTA_ID' Message removed from storage

Root cause

To different threads use shared resources from a single library MQ TA.  For example, it may happen if several transport adapters are configured wrong.

Solution

A user should get rid of usage one shared a library by several threads. There are 2 ways: 

1. Use a single MQ TA library with several sessions 

   TransportLayer.TransportAdapters = TransportLayer.MQAdaptor
   
   TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-vc10-MD-x64.dll
   #...
   TransportLayer.MQAdaptor.SessionNumber = 2
   
    TransportLayer.MQAdaptor.Session.1.ClientID = MQTA_ID_1
    TransportLayer.MQAdaptor.Session.1.ToClientQueue = MQClient_to_1
    TransportLayer.MQAdaptor.Session.1.FromClientQueue = 
    TransportLayer.MQAdaptor.Session.1.ErrorQueue = 
    TransportLayer.MQAdaptor.Session.1.ReceiveTransactionSize=10
    TransportLayer.MQAdaptor.Session.1.SendTransactionSize=10 
   
    TransportLayer.MQAdaptor.Session.2.ClientID = MQTA_ID_2
    TransportLayer.MQAdaptor.Session.2.ToClientQueue = MQClient_to_2
    TransportLayer.MQAdaptor.Session.2.FromClientQueue = 
    TransportLayer.MQAdaptor.Session.2.ErrorQueue = 
    TransportLayer.MQAdaptor.Session.2.ReceiveTransactionSize=10
    TransportLayer.MQAdaptor.Session.2.SendTransactionSize=10 

2. Use several MQ TA with own library

   TransportAdapters = TransportLayer.MQAdaptor, TransportLayer.MQAdaptor2
   
   TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-vc10-MD-x64.dll
   #...
   
   TransportLayer.MQAdaptor2.DllName = bin/MQTAAddin-vc10-MD-x64_2.dll
   #...

IBM MQ Error codes

The following error appears in FixEdge.log

ERROR   [MQQueuesReader]  1284  MQQueuesReader unable to create queue for client'trgQ'. Reason: Failed to set character set. Code:2538
ERROR   [MQQueuesReader]  244  MQQueuesReader unable to create queue for client'srcQ'. Reason: Failed to set character set. Code:2538
ERROR   [MQQueueWriter]  3368  MQTA failed to execute operation for client 'MQHub1': Failed to set character set. Code:2538

Solution

Please find the API Error codes on the official site.

WebSphere MQ Setup Guide

Getting MQ Soft

If you need to set up your own MQ server, you can download software from IBM Site

Websphere MQ Configuration

Steps to configure access to Websphere MQ (running on Windows) from the remote host are described below. These configuration is performed on the MQ Server host.

1. Create User
   1. Create non-domain user e.g. mquser in 'mqm' group.
   2. Change its domain to local host name e.g. EVUAKYISD0223.
   3. Grant this user permissions to login remotely.
   4. Grant this user Administraive permissions.
2. Start MQ
   1. Start all MQ services ('IBM MQSeries' and 'IBM Websphere MQ (Installation N)') under user created on previous step.  .
      In order to change user for service go to Services. Select service and open its properties, go to 'Log On' tab, select 'This user' and specify user credentials.
3. Configure Manager and Queues
   1. Start MQExplorer.
   2. 
      Add new Queue Manager (e.g. TestMQ)
   3. 
      Add Server-connection channel to created Queue Manager (e.g.  TestConnection )
   4. 
      Set channel MCA with user from step 1.1 ( mquser ):
4. 
   

   Add queues.

Check connection to Queue Manager from remote host

Check connection using MQ Explorer

In order to use the described approach you need to install IBM WebSphere MQ Explorer to the client box.

In order to check connection to Queue Manager from the remote (client) host:

1. Open MQExplorer on remote host
2. Add Remote Queue Manager performing the following steps:
   1. 
      Specify Queue Manager name

   2. 

      Specify host and Server-connection channel

      
      

   3. Click Finish

QM should be accessible via remote MQ Explorer.

Check connection using FE

In order to check connection to Queue Manager from the remote (client) host using FE:

1. Configure FE using the following sample properties:

   FIXEdge.properties

   TransportLayer.TransportAdapters = TransportLayer.MQAdaptor
   
   #-------------------------------------------------------------------------
   # MQ Adaptor Settings
   #-------------------------------------------------------------------------
   
   TransportLayer.MQAdaptor.Description = MQ Transport Adaptor DLL
   TransportLayer.MQAdaptor.TimeIntervalBeforeReconnect = 1000
   
   TransportLayer.MQAdaptor.DllName = bin/MQTAAddin-vc10-MD-x64.dll
   TransportLayer.MQAdaptor.StorageDirectory = FixEdge1/log
   TransportLayer.MQAdaptor.NumAttemptReconnect = -1
   TransportLayer.MQAdaptor.WaitIntervalForGet = 10000
   TransportLayer.MQAdaptor.BufferSizeForMessage = 65536
   
   TransportLayer.MQAdaptor.MQ_HOSTNAME = EVUAKYISD0223.kyiv.epam.com
   TransportLayer.MQAdaptor.MQ_MANAGER_NAME = TestMQ
   TransportLayer.MQAdaptor.MQ_PORT = 1414
   TransportLayer.MQAdaptor.MQ_CCSID = 437
   TransportLayer.MQAdaptor.MQ_CHANNEL_NAME = TestConnection
   
   TransportLayer.MQAdaptor.SessionNumber = 1
   TransportLayer.MQAdaptor.Session.1.ClientID = MQClient
   TransportLayer.MQAdaptor.Session.1.FromClientQueue = MQClient_from
   TransportLayer.MQAdaptor.Session.1.ToClientQueue = MQClient_to
   TransportLayer.MQAdaptor.Session.1.ErrorQueue = MQClient_error
   TransportLayer.MQAdaptor.Session.1.SmartXMLProcessor = false

2. Check if Queues are accessible:

   FixEdge.log

   [DEBUG] 20120718-11:41:33.609 [2840] [MQEngine] - Was created queue 'MQClient_from'
   [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - createClientQueue() finished
   [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - ClientQueue created.
   [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - MQQueuesReader::readFromQueuesAndPassToTL().
   [DEBUG] 20120718-11:41:33.609 [2840] [MQQueuesReader_Debug] - fromClientQueue->getFirst().
   [DEBUG] 20120718-11:41:38.039 [12692] [MQEngine] - Was created queue 'MQClient_to'
   [DEBUG] 20120718-11:41:38.251 [12692] [MQEngine] - Was created queue 'MQClient_error'
   [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Queues were created for client 'MQClient'
   [NOTE]  20120718-11:41:38.251 [12692] [MQQueueWriter] - Restore messages for out queue 'MQClient'.
   [NOTE]  20120718-11:41:38.251 [12692] [MQQueueWriter] - Create storage for the: MQClient
   [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Load storage.
   [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Persistance storage use file: D:\Program Files\B2Bits\FIX Edge\v.5.4.1.40091\FixEdge1\log\MQ-MQClient_07180941382511.out
   [NOTE]  20120718-11:41:38.251 [12692] [MQQueueWriter] - Restore messages for error queue 'MQClient'.
   [NOTE]  20120718-11:41:38.251 [12692] [MQQueueWriter] - Create storage for the: MQClient-Errors
   [DEBUG] 20120718-11:41:38.251 [12692] [MQQueueWriter] - Load storage.
   [DEBUG] 20120718-11:41:38.252 [12692] [MQQueueWriter] - Persistance storage use file: D:\Program Files\B2Bits\FIX Edge\v.5.4.1.40091\FixEdge1\log\MQ-MQClient -Errors_07180941382511.out
   [DEBUG] 20120718-11:41:43.678 [2840] [MQQueueRead] - Was performed get operation on queue 'MQClient_from' with reason code :2033
   [DEBUG] 20120718-11:41:43.678 [2840] [MQQueueRead] - MQQueueRead::getFirst failed to get message ''
   [DEBUG] 20120718-11:41:43.678 [2840] [MQQueuesReader_Debug] - MQQueuesReader::readFromQueuesAndPassToTL().
   [DEBUG] 20120718-11:41:43.678 [2840] [MQQueuesReader_Debug] - fromClientQueue->getFirst().

3. Route message to MQ and check FE logs:

   FixEdge.log

   [DEBUG] 20120718-11:48:16.389 [7968] [FL_MsgTrace] - New message FIX was received from session with ID 'TROIKALSE'. Message: '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224'.
   [DEBUG] 20120718-11:48:16.389 [7968] [BL_Layer] - Process incoming message.
   [DEBUG] 20120718-11:48:16.389 [7968] [BL_RoutingTable] - There are no suitable rules were found for message with Source ID 'LSE', executing DefaultRule.
   [DEBUG] 20120718-11:48:16.389 [7968] [CC_Layer] - BL has processed a message. Number of client IDs for delivery :0. Number or FIX sessions for delivery :0.. Number or sources identifiers for delivery :1.
   
           Source IDs:
                 1. 'MQClient'
   
   [DEBUG] 20120718-11:48:16.389 [7968] [TransportLayer] - Sending message '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224' to client MQClient
   [DEBUG] 20120718-11:48:16.389 [7968] [MQQueueWriter] - Was put message to messageQueue. Message = (ToClientQueue, MQClient, 8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224)
   [DEBUG] 20120718-11:48:16.389 [7968] [MQQueueWriter] - Store into the Persistance storage.
   [DEBUG] 20120718-11:48:16.389 [12228] [MQQueueWriter] - Was get message from MessageQueue. Message: ( toClientQueue,MQClient, 8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224)
   [DEBUG] 20120718-11:48:16.460 [12228] [MQQueueWrite] - Was performed put operation on queue 'MQClient_to' with reason code :0. Message = '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224'
   [DEBUG] 20120718-11:48:16.460 [12228] [MQQueueWrite] - message was successfully sent to to queue 'MQClient_to' with reason code :0
   [DEBUG] 20120718-11:48:16.460 [12228] [MQQueueWriter] - message was put to ToClient queue.Client 'MQClient'. MEssage '8=FIX.4.49=16535=D49=LSE56=TROIKA34=250=3073757=ECL_TURQ97=Y52=20120718-09:48:16.38811=900010081=1003000321=255=TESTA54=160=20000809-18:20:3238=400040=244=3059=010=224'

Configuring SSL in IBM MQ Server

This instruction assumes that non-SSL configuration works OK

1. On IBM MQ server start IBM Key Management tool

2. Create new Key Database File
- Key database type: CMS
- File name: key.kdb
- Location: use same location as specified in "SSL Key Repository" parameter of your Queue Manager (for example, if SSL Key Repository is "C:\Program Files (x86)\IBM\WebSphere MQ\Qmgrs\FIXEdge\ssl\key", then set Location: C:\Program Files (x86)\IBM\WebSphere MQ\Qmgrs\FIXEdge\ssl\)

3. Create Password and set "Stash password to a file" checkbox

4. Click on "New Self-Signed..." button to create self-signed certificate

- Key Label: set it as "ibmwebspheremq<QM_name>", here QM_name - name of your Queue Manager in lower case
- Common Name: set any
- Organization: set any
- Organization Unit: set any

5. Close IBM Key Management using menu Key Database File -> Close and then Key Database File -> Exit

6. Go to SSL Key Repository folder and make sure that only SYSTEM and your MQ user have full access to key.kdb, key.rdb and key.sth files. All other user may have only "Read" permissions

7. In Websphere MQ Explorer go to Channels

8. Open Properties of your channel and on SSL tab select SSL Cipher Spec. For example, TRIPLE_DES_SHA_US.

    Select SSL Authentication = Optional if you don't plan to use SSL client authentication.

    Apply your changes

9. From SSL Key Repository folder (see step 2) copy three files: key.kdb, key.rdb and key.sth files somewhere on FIXEdge's server (for example, FixEdge1/conf)

10. Configure FIXEdge.properties to use SSL connection with MQ:

TransportLayer.MQAdaptor.SslKeyRepository = C:\\B2BITS\\FIXEdge\\FixEdge1\\conf\\key
TransportLayer.MQAdaptor.SslCipherSpecification = TRIPLE_DES_SHA_US
TransportLayer.MQAdaptor.SslClientAuthentication = false
TransportLayer.MQAdaptor.SslPeerName =

Troubleshooting

Disabling IBM MQ security

MQ TA has not supported the authentication in IBM MQ Server. 

It can lead to failure in MQ TA initialization with the error records in the log:

Make sure that QueueManager on the server-side is running and FIXEdge are trying to set the right code character set.

If the following error message appears while accessing remote MQ manager using MQExplorer

Solution

The authentication should be disabled. You should perform the following steps:

1. Login to the remote server where MQ Instance is running

2. Open cmd

3. Run 'runmqsc'. Check if connected to correct QM:

   

4. Execute 'ALTER QMGR CHLAUTH(DISABLED)':

   

5. Try to reestablish connection by clicking 'No' in the dialog:

   

6. And Finish in the dialog:

   

7. If the error is still here, look at Channel authentication records and make sure that your account is not in the Block User List:
   
   
   
   
8. Try again steps 5-6.

Also you can find how to disable IBM MQ authentication on the official site.