Page Comparison

Overview

FIX Grep is FIXGrep is a console command-line utility that provides extended mechanisms for FIX log analyses and message tracking. 

Tool This tool can process 20000 messages per second. FIXGrep can do context search searches by interpreting tags tag names and values value descriptions.

In addition, FIXGrep can extract the sequence of changes for orders or other FIX business objects and identify their final states.for orders or other FIX business objects and identify their final states.

Installation on Linux

Follow the instructions below to install FIXGrep on Linux servers or upgrade FIXGrep to a newer version.

1. Download the FIXGrep version for Linux from the Client Space or request access via b2bits.com website.

2. Extract the archive to the required directory.

   Code Block
   language text theme Confluence
   tar -xzvf FIXGrep-1.2.12.0-linux-x64.tar.gz -C /FIXEdge/

3. Request the license from the sales@btobits.com or use the one you already own.

4. Copy the license into the directory where the files are located.

   Code Block
   language text theme Confluence
   cp /tmp/download/fixgrep.license /FIXEdge/fixgrep-1.2.12.0/

5. Run the query.

   Code Block
   language text theme Confluence
   ./fixgrep 35="0" /log/FIXEdge/*.in -o result.log

Invoking FIXGrep

The general synopsis of the FIXGrep command FIXGrep command line is:

 fixgrep EXPRESSION [FILE...] [keys...] [command]
 fixgrep --config FILE 

 ./fixgrep EXPRESSION [FILE...] [keys...] [command]
 ./fixgrep --config FILE 

EXPRESSION is a parameter – which is used without a specific key in front and it is specified right after the command name.

The following format is supported: “<tag>=<regexp> <tag>=<regexp>”, conditions are separated by “ ” space (space" "), tag and regular expression are separated by “=” ( equality sign ("="), number if of conditions is unlimited, the whole  pattern pattern is quoted.

fixgrep "35=D 55=WMI|WJ.*" ./messages.in --list

FILE is a file path. It can be specified by mask. If multiple files are specified they must be separated by space (" "). If the file name contains space it must be enclosed in double quotes.

fixgrep "35=A" ./test.txt ./me* --list

Keys

Output: -o (--output)

Key The key is used to direct output data to a file. When -o is specified out put output data is saved to the specified file without being printed in stdout.

fixgrep "35=D|F" ./te* -o result.log

Output: -ecsv (--export-csv)

Key The key is used to direct output data to the file in CSV format. 

Header The header file is used to put messages. Each message is a line. Only fields , specified in the header , are to will be extracted from messages.

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=D" ./message* -ecsv ./report.csv -o ./report.csv

[fixmgr@localhost FIXGrep]$ ./fixgrep "35=D" ./test1.in -ecsv report.csv -o report_out.csv

report.csv before execution:

11,38,55

report.csv after execution:

,11,38,55
,00002-0511-2412-0-01092008,20,OTI

Sort: -sa (--sort-ascending), -sd (--sort-descending)

Key The key is used to show messages in a certain order by tags tag values. Messages can be sorted by more than one criterion however only one criterion can be specified for each tag. Sorting is applied in order of appearance. Tags involved in sorting criterion criteria are separated by space. If the tag is not found message is moved to the end of the list sorted by this criterion.

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=D 55=K.*" ./me* -sd 55 -sa 11 

[fixmgr@localhost FIXGrep]$ ./fixgrep "35=D 55=K.*" ./te* -sd 55 11 -sa 200

Filter: -m (--match), -nm (--not-match)

Key The key is used to get messages, which correspond to the certain criteria. Match The match criterion check checks the value of the specified tag against the specified regular expression. In case of The --match criterion is satisfied if when the field value matches the expression, in case of while the --not-match criterion is satisfied if when the field value does not match the expression.

Criteria are checked in the order of appearance (which gives user which provides users with the ability to write more efficient rule)rules. It is possible to specify more than one criterion for the same tag, and all expressions will be applied. Tag and regular expression are separated by space. Message The message is shown only if all conditions are satisfied (true) i.e. if at least one match condition fails message is not shown. If the tag, for which the expression is defined, is not present in the message then the condition is considered as failed.

fixgrep "35=.*" ./message.in -m "35" "D|F" "200" "200801" "55" "X.*" -nm "55" "XL|XEQ"

Filter: -p (--present), -np (--not-present)

Key The -p key is used to filter message messages by tag presence. All messages, which that do not contain specified tags, are not present in the result list.

Key The -np key is used to filter message messages by tag presence. Only those messages, which that do not contain specified tags, are present in the result list. 

fixgrep "35=.*" ./messages.in -p 11 -np 41 -o report.txt

Filter: -t (--time), -nt (--not-time)

Key The key is used to get messages by time. Only messages, which are in the specified time interval (- earlier than the first value and later than the second value ) - are shown in the result (or not in the specified interval if -nt is used). Condition

The condition can be applied only to the FIX fields , which have of the following data types: month-year, UTCTimestamp, UTCTimeOnly, UTCDateOnly, LocalMktDate, TZTimeOnly, TZTimestamp, UTCDate. Values must be in the format according to the data type. Key

The key can appear more than once; in . In this case, it is applied in the order of appearance.

fixgrep "35=D" ./messages.in -t 52 "20230109-13:33:00.000" "20230109-13:33:05.000"

Details: -D=<N> (--details=<N>)

Key The key is used to change the friendliness of the messages friendliness. Numbers are replaced by human-readable text depending on the level of the specified details leveldetail. Key The key is optional.

• -D=0 (--details=0) – simple raw data view i.e. tags and values are shown as is (default, the same as not specified).

[fixmgr@localhost FIXGrep]$ ./fixgrep "35=A" ./test1.in -nt 52 20080109-21:27:00.00098 20080109-21:27:00.00098 -sw "|" -D=0
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "test1.in"...
8=FIX.4.2|9=73|35=A|49=SNQX|56=DBSUSH|34=1|50=SNPY|52=20080109-13:32:04.000|98=0|108=60|10=242|

Completed.

• -D=1 (--details=1) – tags are shown by name.

[fixmgr@localhost FIXGrep]$ ./fixgrep "35=A" ./test1.in -nt 52 20080109-21:27:00.00098 20080109-21:27:00.00098 -sw "|" -D=1
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "test1.in"...
BeginString=FIX.4.2|BodyLength=73|MsgType=A|SenderCompID=SNQX|TargetCompID=DBSUSH|MsgSeqNum=1|SenderSubID=SNPY|SendingTime=20080109-13:32:04.000|EncryptMethod=0|HeartBtInt=60|CheckSum=242|

Completed.

• -D=2 (--details=2) – tags are shown by names, and values are shown by description (the message can become big and multi-string).

[fixmgr@localhost FIXGrep]$ ./fixgrep "35=A" ./test1.in -nt 52 20080109-21:27:00.00098 20080109-21:27:00.00098 -sw "|" -D=2
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "test1.in"...
BeginString=FIX42|BodyLength=73|MsgType=LOGON|SenderCompID=SNQX|TargetCompID=DBSUSH|MsgSeqNum=1|SenderSubID=SNPY|SendingTime=20080109-13:32:04.000|EncryptMethod=None / other|HeartBtInt=60|CheckSum=242|

Completed.

...

It is possible to set more human-friendly symbol symbols as field separatorseparators. Single A single value can be specified for -sw (soh-write) to override the separator used in the source by the one specified by the user.

Keys are optional.

[fixmgr@localhost FIXGrep]$ ./fixgrep "35=A" ./messages.in -sw " "
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "messages.in"...
8=FIX.4.2 9=73 35=A 49=SNQX 56=DBSUSH 34=1 50=SNPY 52=20080109-13:32:04.000 98=0 108=60 10=242
8=FIX.4.2 9=76 35=A 49=SNQX 56=DBSUSH 34=3782 50=SNPY 52=20080109-21:27:00.000 98=0 108=60 10=151

Completed.

Additional keys: -C, -L

“The -L” L key instructs to output file list where message messages that satisfy conditions are found, messages themselves are not written;.

“The -C” C key instructs to output the number (/count ) of found messages, messages themselves are not written;.

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=.*" ./me* -m 55 "WJJ" -L -C
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "message1.in"...
Processing file "message2.in"...
Processing file "message3.in"...
Found 5 message(s)
Found in file(s):
./message3.in


Completed.

...

Writes all found FIX messages (to the stdout taking into account all modifications implied by keys) to the stdout.

This is the default command i.e. if no command is specified the --list command is executed.

Command: --validate (-v=)

Applies validation to all found FIX messages (according to the specified validation level taking into account all modifications implied by keys) according to the specified validation level. Writes to the stdout all found errors. 

• -v=0 (--validate=0) – no validation (, default, the same as not specified).
• -v=1 (--validate=1) – check if the message is well-formed.
• -v=2 (--validate=2) – check if the message is well-formed, and all required fields presentsare present.
• -v=3 (--validate=3) – check if the message is well-formed, all required fields presentsare present, and all fields field values are correct.

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=.*" ./incorrect_messages.in -v=3
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "incorrect_messages.in"...
Field value 'DSTOK' does not meet ValBlock dictionary conditions in tag ExDestination (100) in message New Order - Single (D) with sequence number 2.
1  errors found.

...

The command is used to print the status of the orders for given matching conditions.

fixgrep "35=.*" ./message.in -osl

...

The command is used to build a life - cycle for the order.

For the given client order ID ClOrdID(FIX tag 11) extract back and forward all orders, replaces, cancels, executions, and rejects participated in the order life-cycle (CLOrdID .

...

...

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=.*" ./EP* -ho "E06"
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "EPAR7AFIX.in"...
Order E06:

EXC=>FIX 8=FIX.4.4☺9=184☺35=8☺49=EXC☺56=FIX☺34=36☺52=20230306-11:30:16.750☺369=36☺37=80☺11=E06☺17=403389☺150=0☺39=0☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺151=50☺14=0☺6=0☺60=20230306-11:30:16.738☺10=067☺
Status:
New 0/50 @0

EXC=>FIX 8=FIX.4.4☺9=201☺35=8☺49=EXC☺56=FIX☺34=37☺52=20230306-11:30:16.750☺369=36☺37=80☺11=E06☺17=38A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=7☺31=8922.5☺151=43☺14=7☺6=8922.5☺60=20230306-11:30:16.738☺10=243☺
Status:
Partially filled 7/50 @8922

EXC=>FIX 8=FIX.4.4☺9=205☺35=8☺49=EXC☺56=FIX☺34=38☺52=20230306-11:30:16.750☺369=36☺37=80☺11=E06☺17=39A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=8☺31=8922☺151=35☺14=15☺6=8922.233333☺60=20230306-11:30:16.738☺10=187☺
Status:
Partially filled 15/50 @8922

EXC=>FIX 8=FIX.4.4☺9=207☺35=8☺49=EXC☺56=FIX☺34=39☺52=20230306-11:30:16.751☺369=36☺37=80☺11=E06☺17=40A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=3☺31=8921.5☺151=32☺14=18☺6=8922.111111☺60=20230306-11:30:16.738☺10=001☺
Status:
Partially filled 18/50 @8922

EXC=>FIX 8=FIX.4.4☺9=205☺35=8☺49=EXC☺56=FIX☺34=40☺52=20230306-11:30:16.751☺369=36☺37=80☺11=E06☺17=41A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=8☺31=8921☺151=24☺14=26☺6=8921.769231☺60=20230306-11:30:16.738☺10=176☺
Status:
Partially filled 26/50 @8921

EXC=>FIX 8=FIX.4.4☺9=207☺35=8☺49=EXC☺56=FIX☺34=41☺52=20230306-11:30:16.751☺369=36☺37=80☺11=E06☺17=42A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=1☺31=8920.5☺151=23☺14=27☺6=8921.722222☺60=20230306-11:30:16.738☺10=005☺
Status:
Partially filled 27/50 @8921

EXC=>FIX 8=FIX.4.4☺9=200☺35=8☺49=EXC☺56=FIX☺34=42☺52=20230306-11:30:16.751☺369=36☺37=80☺11=E06☺17=43A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=4☺31=8920☺151=19☺14=31☺6=8921.5☺60=20230306-11:30:16.738☺10=174☺
Status:
Partially filled 31/50 @8921

EXC=>FIX 8=FIX.4.4☺9=205☺35=8☺49=EXC☺56=FIX☺34=43☺52=20230306-11:30:16.751☺369=36☺37=80☺11=E06☺17=44A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=1☺31=8919.5☺151=18☺14=32☺6=8921.4375☺60=20230306-11:30:16.738☺10=179☺
Status:
Partially filled 32/50 @8921

EXC=>FIX 8=FIX.4.4☺9=205☺35=8☺49=EXC☺56=FIX☺34=44☺52=20230306-11:30:16.752☺369=36☺37=80☺11=E06☺17=45A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=4☺31=8919☺151=14☺14=36☺6=8921.166667☺60=20230306-11:30:16.738☺10=196☺
Status:
Partially filled 36/50 @8921

EXC=>FIX 8=FIX.4.4☺9=206☺35=8☺49=EXC☺56=FIX☺34=45☺52=20230306-11:30:16.752☺369=36☺37=80☺11=E06☺17=46A☺150=F☺39=1☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=6☺31=8918.5☺151=8☺14=42☺6=8920.785714☺60=20230306-11:30:16.738☺10=251☺
Status:
Partially filled 42/50 @8920

EXC=>FIX 8=FIX.4.4☺9=200☺35=8☺49=EXC☺56=FIX☺34=46☺52=20230306-11:30:16.752☺369=36☺37=80☺11=E06☺17=47A☺150=F☺39=2☺581=2☺55=CA☺54=2☺38=50☺40=2☺44=8900☺59=0☺18=2☺529=D☺32=8☺31=8900☺151=0☺14=50☺6=8917.46☺60=20230306-11:30:16.738☺10=191☺
Status:
Filled 50/50 @8917

Command: --history-session (-hs)

For the given session ID ( – pair of SenderCompID(49) and TargetCompID(56) – build the history for the FIX session.

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=.*" ./messages.in -hs "SNQX" "DBSUSH"
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "messages.in"...
8=FIX.4.2☺9=73☺35=A☺49=SNQX☺56=DBSUSH☺34=1☺50=SNPY☺52=20080109-13:32:04.000☺98=0☺108=60☺10=242☺
State "SNQXDBSUSH" I: "Wait for confirm logon" 20080109-13:32:04.000

8=FIX.4.2☺9=87☺35=1☺49=SNQX☺56=DBSUSH☺34=2☺50=SNPY☺52=20080109-13:32:09.000☺112=Gap Fill Completed...☺10=169☺
State "SNQXDBSUSH" I: "Wait for confirm logon" 20080109-13:32:09.000

8=FIX.4.2☺9=56☺35=5☺49=SNQX☺56=DBSUSH☺34=3769☺52=20080109-21:25:59.000☺10=145☺
State "SNQXDBSUSH" I: "Wait for confirm logout" 20080109-21:25:59.000

8=FIX.4.2☺9=76☺35=A☺49=SNQX☺56=DBSUSH☺34=3782☺50=SNPY☺52=20080109-21:27:00.000☺98=0☺108=60☺10=151☺
State "SNQXDBSUSH" I: "Wait for confirm logout" 20080109-21:27:00.000

8=FIX.4.2☺9=68☺35=2☺49=SNQX☺56=DBSUSH☺34=3783☺52=20080109-21:27:00.000☺7=6248☺16=0☺10=159☺
State "SNQXDBSUSH" I: "Wait for confirm logout" 20080109-21:27:00.000

8=FIX.4.2☺9=97☺35=4☺49=SNQX☺56=DBSUSH☺34=3770☺43=Y☺52=20080109-21:27:00.000☺122=20080109-21:27:00☺123=Y☺36=3784☺10=093☺
State "SNQXDBSUSH" I: "Wait for confirm logout" 20080109-21:27:00.000

8=FIX.4.2☺9=90☺35=1☺49=SNQX☺56=DBSUSH☺34=3784☺50=SNPY☺52=20080109-21:27:05.000☺112=Gap Fill Completed...☺10=070☺
State "SNQXDBSUSH" I: "Wait for confirm logout" 20080109-21:27:05.000


Completed.

Command: --search (-s)

FIX Grep FIXGrep performs a context search in the loaded data by some plain text. Human readable text can be specified for search e.g. “39=0” or “OrdStatus=0” or “OrdStatus=New”.

Each message is converted to into 3 forms:

• The raw FIX message

...

• .
• The detailed FIX message

...

• where tags are replaced by

...

• field names.
• The wide form

...

• where tags are replaced by field names, and values when possible are replaced by descriptions

...

• .

Search is performed in all these 3 lists.

fixgrep "35=.*" ./message.in -s "Symbol=QHN"

...

All keys can be specified in the configuration file. All extra spaces and new lines are removed while the file is parsed. Key The key cannot be combined with other keys.

Configuration file content is interpreted as it was passed in the command line.

fixgrep --config cmd.txt

"35=.*" ./me* -m 55 "WJJ" -L -C

...

This command is used to establish a TCP connection and send a sequence of bytes from a file. E.g. replay messages from the session log.

c:\Program Files (x86)\B2BITS\FIXGrep>fixgrep "35=.*" ./messages.in --send-tcp localhost 9106
FIXGrep 1.5.0.135
Copyright (C) B2BITS EPAM Systems Company. All Rights Reserved.

Processing file "messages.in"...
Sent 908487 of 908487 bytes.