History implementation in DSL
- author
- 1 Overview
- 2 Configuration structure
- 2.1 InMemoryHistory
- 2.1.1 Structure template
- 2.1.2 Sample
- 2.2 JDBCHistory
- 2.2.1 Structure template
- 2.2.2 Main structure
- 2.2.3 Sample
- 2.1 InMemoryHistory
- 3 DSL Instructions
- 3.1 MessageRule
- 3.1.1 Condition section
- 3.1.1.1.1 Key exists
- 3.1.1.1.2 Key not exists
- 3.1.1.1.3 Value by key is empty
- 3.1.1.1.4 Value by key is not empty
- 3.1.1.1.5 Value exists
- 3.1.1.1.6 Value not exists
- 3.1.1.1.7 Check part of values by indexes
- 3.1.1.2 Common instructions
- 3.1.2 Action section
- 3.1.2.1
- 3.1.2.2
- 3.1.2.3
- 3.1.2.4
- 3.1.2.5
- 3.1.2.6
- 3.1.2.7
- 3.1.2.8 Main commands
- 3.1.2.8.1 Save to
- 3.1.2.8.2 Remove
- 3.1.2.8.3 RemoveFromHistory
- 3.1.2.8.4 Clear
- 3.1.2.8.5 Show
- 3.1.2.8.6 Update
- 3.1.2.8.7 Call procedure
- 3.1.2.8.8 Route by DBTable
- 3.1.2.9 History block
- 3.1.2.9.1 Template
- 3.1.2.9.2 Show
- 3.1.2.9.3 Get from history and set to message field
- 3.1.2.10 Set static value
- 3.1.2.11 Fields block
- 3.1.1 Condition section
- 3.2 More samples
- 3.1 MessageRule
- 4 Stored procedures
- 5
- 6 Variables storage
Overview
FIXEdge Java provides Solutions for handling and storing data on the business layer—Histories. Information can be handled in memory or through database queries, depending on the defined type. These types are available in the current version:
· Information can be stored in the memory storage with the help of InMemoryHistory.
· Information can be stored in the JDBC database via JDBCHistory.
Histories can be used in existing message/event rules builders using routingContext (see History implementation) or in routing rules through the suggested Groovy DSL design.
The DSL Layer's many history types are configured and handled according to the instructions in this manual.
Configuration structure
The configuration of the histories must be done through the histories block in rulesDSL depending on the type.
It’s possible to work with histories at different levels of DSL rules: condition, action, custom blocks.
InMemoryHistory
In order to configure InMemoryHistory, it’s required to declare a block memoryHistory in the histories section and describe the list of required parameters:
· name <history name> - type: String. Must be unique .
· keyTags <key tags: simple or compose> - type: Integer or List of integers. Tag(s) number(s) in received on BL message (FIXFieldList), which creates a key (simple or compose).
· valueTags <value tags: simple or compose> type: Integer or List of integers. Tag(s) number(s) in received on BL message (FIXFieldList), that will be stored by prepared key.
Structure template
Main structure
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | import com.epam.fej.routing.RoutingContext
import static dsl.CommonRulesDsl.rulesDSL
rulesDSL(routingContext as RoutingContext) { histories { memoryHistory { name <history name> keyTags <key tags: simple or compose> valueTags <value tags: simple or compose>
} } } |
Sample
Main structure
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | import com.epam.fej.routing.RoutingContext
import static dsl.CommonRulesDsl.rulesDSL
rulesDSL(routingContext as RoutingContext) { histories { memoryHistory { name "SenderTargetCompId History" keyTags 34 valueTags 49, 56
} memoryHistory { name "ClOrdID History" keyTags 34, 35 valueTags 11 } } } |
JDBCHistory
For configuring JDBCHistory, its required to declare a block db(<dataSource>) in the histories and describe the list of parameters:
· DataSource - type: String. Must be unique. Group identifier, that contains parameters for connecting to the database. Its set through the conf/history.properties.
o <dataSource>.url=<connectionUrl> - required.
o <dataSource>.username=<username> - required.
o <dataSource>.password=<password> - required.
o <dataSource>.driverClassName=<driverClassName> - required.
o <dataSource>.databaseName=<databaseName> - optional. The name of the catalogue must be specified (the same specified in the URL). Required to be used in case when two databases contain procedures with the same name.
o <dataSource>.schemaName=<schemaName> - optional. The name of the schema must be specified. For example, when using it with a MsSQL datasource and a specified catalog name, its required to specify the schema so that the procedure is recognized (dbo is the default).
o <dataSource>.maximumPoolSize=<maximumPoolSize> - optional, default - 10, type: Integer. This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections.
o <dataSource>.idleTimeout=<idleTimeout> - optional, default - 3000, type: Integer. The maximum duration that a connection may remain idle in the pool is managed by this attribute.
· Procedures - structure block, that describes the list of stored procedures or custom requests, that can be applied to declared dataSource.
Includes:
o procedure(<procedure name>) - block that describes procedure's configuration:
§ <procedure name> - required, type: String, name of stored procedure. Used as the identifier of the procedure/request being accessed if no <id> was specified. In this case, the uniqueness of the names for all databases is important. (Otherwise, the <id> is additionally specified). It is also used to generate a stored procedure call request in the "CALL <procedure name> (<InParams template>);" format if a custom template has not been previously specified.
§ <id> - optional, type: String, is used when it is necessary to set the uniqueness of the procedure configuration. For example, for two different databases, there are procedures with the same name. To distinguish them when calling, own id is assigned to each structure. It must be unique in comparison with the names/ids of other procedures , regardless of the database being accessed.
§ inParams - optional, block, that describes and after - prepare expected incoming params. The order in which the parameters are described is important:
· fixParam (<tag num>) - get from FIXFieldList (incoming message) value by <tag num> as String.
· fixParam (<tag num>, <default value>) - get from FIXFieldList (incoming message) value by <tag num> as String. If the value does not exist, then substitute the specified default <default value>
· param - general block that describes incoming param structure
o tag - optional, type: int, define tag number, that should take and save from incoming FIXFieldList
o defaultValue - optional, type: any serializable object. This value will be substituted if the specified tag is not found.
o dataType - optional, type: String, default: String. Possible values (case insensitive): INT, LONG, FLOAT, DOUBLE, BIGDECIMAL,BYTE, BYTES, BOOL, STRING, TIMESTAMP, LOCALMKTTIME, TENOR, TIMEONLY. Unknown type - get from FIXFieldList as bytes.
o saveFullMessage - optional, type: boolean. Declare to save full incoming message as byte array (db blob type)
§ outParams - optional, block, that describes out parameters/result set
· param - expect name of OUT param for stored procedure and its java.sql.Types
· rowMapper - type: RowMapper. Custom implementation of method to map each row of data in the ResultSet.
· dbHistory - structure block, that describes the configuration of the view on selected table. Describes by:
o name - required, type: String, must be unique over ALL declared histories. This parameter is used as an identifier for accessing the described structure.
o updateOnSave - optional, type: boolean, default: false. Determines whether to update database records with the same key or ignore them.
o table - required, structure block, that describes selected table (in whole or in part):
§ name - required, type: String. Declared table name
§ keys - required, structure block, that describes the list of key columns in the table. Consist of one or a list of described column structures:
· column - structure block, that describes the table column:
o tag - optional, type: int, define tag number, that should take and save from incoming FIXFieldList
o defaultValue - optional, type: any serializable object. This value will be substituted if the specified tag is not found.
o dataType - optional, type: String, default: String. Possible values (case insensitive): INT, LONG, FLOAT, DOUBLE, BIGDECIMAL,BYTE, BYTES, BOOL, STRING, TIMESTAMP, LOCALMKTTIME, TENOR, TIMEONLY. Unknown type - get from FIXFieldList as bytes.
o saveFullMessage - optional, type: boolean. Declare to save full incoming message as byte array (db blob type). It is not recommended to use the type as a key.
o customFormatter - optiona, type: Functional callback. Allows the user to define the format for saving a value to a column using a callback. Accepts FIXFieldList as input, returns HistoryObject (HistoryKey/HistoryValue) as output: HistoryObject apply(FIXFieldList list);
§ values - required, structure block, that describes the value columns in the table. Consist of one or a list of described column structures (see keys column description)
Structure template
Main structure
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
|
Sample
conf/rules.groovy
DB configuration
Collapse source
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | import com.epam.fej.routing.RoutingContext
import static dsl.CommonRulesDsl.rulesDSL import static java.sql.Types.INTEGER
class Product { String name BigDecimal price
public Product(String name, BigDecimal price) { this.name = name this.price = price } } rulesDSL(routingContext as RoutingContext) { histories { db("datasource1"){ procedures { procedure ("insert_default_value") procedure ("insert_ClOrdId"){ id "saveClOrdId" inParams { tag 11 defaultValue "default Order id" } } procedure ("select_by_price"){ id "selectProducts" inParams { tag 58 defaultValue "default value" } outParams { param "counter", Types.INTEGER rowMapper { rs, num -> return new Product(rs.getString("productName"), rc.getBigDecimal("price"))} } } } dbHistory { name "SenderTargetHistory" table { name "SenderTargetCompId" keys { column { name "SeqNum" tag 34 dataType "Int" } } values { column { name "SenderCompId" tag 49 } column { name "TargetCompId" tag 56 }
} } } dbHistory { name "ClOrdIdHistory" table { name "ClOrdID_table" updateOnSave true keys { column { name "SeqNum" tag 34 dataType "Int" } column { name "MsgType" tag 35 } } values { column { name "ClOrdID" tag 11 defaultValue "None" } } } } } } } |
conf/history.properties
history.properties
|
In the FIXEdge Java from the package, the following SQL databases are supported:
§ MySql
§ PostgreSQL
§ H2
§ HSQL
DSL Instructions
MessageRule
Condition section
To work with a History in a Condition section, its required to declare the corresponding block and pass the name of the history in which check is needed as a parameter.
Comparably to other conditional blocks, additional and/or blocks can be used inside so that to facilitate communication between tasks. By extension, and.
Template
messageRule {
condition {
Name | Description | Template | Example |
Key exists | Prepare a key based on a given structure and an incoming message. Check whether the corresponding one exists in the history. |
|
|
Prepare a key based on a given value list. Check whether the corresponding one does not exist in the history. |
|
| |
Key not exists | Prepare a key based on a given structure and an incoming message. Check whether the corresponding one not exists in the saved in the history. |
|
|
Prepare a key based on a given value list. Check whether the corresponding one does not exist in the history. |
|
| |
Value by key is empty | Checks if there is such a key, prepared on a given value list, and nothing is stored by it (value equal to null). |
|
|
Checks if there is such a key, on a given structure and an incoming message, and nothing is stored by it (value equal to null). |
|
| |
Value by key is not empty | Checks if there is a key, prepared on a given structure and an incoming message, and the value stored by the key is not empty (not equal to null) |
|
|
Checks if there is such a key, prepared on a given value list, and nothing is stored by it (value equal to null). |
|
| |
Value exists | Prepare a key based on a given structure and an incoming message. Check whether the corresponding one exists in the WHOLE history. |
|
|
Value not exists | Prepare a key based on a given structure and an incoming message. Check whether the corresponding one does not exist in the WHOLE history. |
|
|
Check part of values by indexes | Since the HistoryValue itself can be folded from several tags and stored as an array of objects, check of each of the fields of the folded value has been added. | ||
Checks if there is a value for a given key that stores some HistoryValue, it is necessary to check only some parts of this value by indexes (tag's ordinal number in the configuration). Not_empty - value is not null, empty - value null. |
|
| |
|
| ||
history(<history name>) {
// set of instructions
}
}
}
Common instructions
Action section
To work with history in actions, its required to either use the main commands or declare the corresponding block and pass the name of the history as a parameter for which the action is performed. Additional instructions for working with histories are provided in the previously presented DSL blocks header, fields.
Main commands
Name | Description | Template | Sample |
Save to | Stores the prepared key and values in the storage in accordance with the described structure. |
|
|
Remove | Deletes a record from the History by the generated key from the provided values |
|
|
RemoveFromHistory | Deletes a record from the History by the the prepared key by incoming message using described structure. |
|
|
Clear | Clean the entire histories |
|
|
Show | Use the logger to print all values |
|
|
Update | Update the value, if it existed, with the generated key and the new value |
|
|
Call procedure | Execute the procedure(update/insert) declared in the database history structure, passing parameters. As a result, it returns nothing. Use only for methods for updating data in a table. |
|
|
Execute the procedure(update/insert) declared in the database history structure, pass as parameters declared object list . As a result, it returns nothing. Use only for methods for updating data in a table. |
|
| |
Execute the procedure(update/insert) declared in the database history structure, pass as parameters declared closure object list. As a result, it returns nothing. Use only for methods for updating data in a table. |
|
| |
Route by DBTable | A narrowly targeted command to quickly routing the key formed by the structure and send a message according to the value in the destination column. |
|
destinationColumn "Destination" }
|
History block
If you need to work with a specific History, by analogy to Condition, you can use the history section.
Template
messageRule {
action {
history(<history name>) {
// set of instructions
}
}
}
List of available instructions:
Show
Get the value by the key generated pared from the specified parameters and show it in full or in part
get <key params> show fullget <key params> show <value tag (simple/list)>get <key params> showByIndex <value tag index (simple/list)>
Get the value by the key generated from the incoming message and the declared history structure and display it all together.
getValue and show full
Get from history and set to message field
Get the value by the key generated from the specified parameters and set it to message field fully or in partially.
get <key params> setToField <message tag num>
get <key params> items <value tag index (simple/list)> setToField <message tag num>
get <key params> tags <value tag (simple/list)> setToField <message tag num>
Get the value by the key generated from the incoming message and the declared history structure and set it to message field in fully or in partially.
getValue and setToField <message tag num>
getValue and byItems <value indexes> setToField <message tag num>
getValue and byTags <tags> setToField <message tag num>
Set static value
Fill a specific column with the specified static value.
setToColumn <column name> value <static value>
setToColumn <column name> value obtainedFrom tag <tag num of incoming message>
Fields block
Using in fields block (set to field indirect from history):
set <message tagNum> value from history <history name> by key
set <message tagNum> value from history <history name> key <key params>
set <message tagNum> value from history <history name> items <value tag index (simple/list)>
set <message tagNum> value from history <history name> tags <value tag (simple/list)>
More samples
Example. Save to history
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
|
Example. Remove from history
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
Example. Clear history
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
Example. Show history
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
Example. Message handling rules using DSL instructions and custom code in History
import com.epam.fej.routing.RoutingContext
import com.epam.fej.routing.history.History
import com.epam.fej.routing.history.HistoryKey
import com.epam.fej.routing.history.HistoryValue
import static dsl.CommonRulesDsl.rulesDSL
String historyName = "ClOrdIdRoutingTable"
rulesDSL(routingContext as RoutingContext) {
histories {
db("mysql") {
dbHistory {
name historyName
table {
name "ClOrdIdTable"
keys {
column {
name "MsgType"
tag 34
}
column {
name "SeqNum"
tag 35
dataType "Int"
}
}
values {
column {
name "ClOrdId"
tag 11
}
column {
name "SenderCompID"
tag 49
}
column {
name "TargetCompID"
tag 54
}
}
}
}
}
}
messageRules {
messageRule("Rule1. Save to the history") {
condition { msgType "D" }
action {
saveTo historyName
context exit
}
}
messageRule("Rule2. Replace the incoming message tag with a value from the history") {
condition { msgType "B" }
action {
history(historyName) {
getValue and byTags 11 setToField 58
}
sendTo "session1"
context exit
}
}
messageRule("Rule3. Retrieve value from the history and redirect an incoming message based on the received data") {
condition { msgType "AE" }
action {
custom { ctx ->
History history = routingContext.getHistory(historyName)
HistoryKey key = history.prepareKey(ctx.message)
HistoryValue value = history.get(key)
//Get saved ClOrdId value
String savedClOrdId = value.get().get(0)
//Send an incoming message through the destination endpoint with the same identifier as the received ClOrdId
routingContext.getDestinationById(savedClOrdId).send(ctx.message)
history.releaseResources(key, value)
}
context exit
}
}
}
}
Stored procedures
A comprehensive description of the procedure, its structural placement within the declared history, has already been provided in this section.
In this block, we will review the general structure, delve more deeply into the API and invocation methods, and discuss customization possibilities.
Structure template
In the code block below, variants of storage procedure configurations are provided again.
According to the given templates, every template will be discussed further, followed by practical implementation. Each type of parameter (IN/OUT, returned value) can be combined according to the procedure structure.
Procedure template
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
|
Name | Description | Procedure sample, MsSQL | Declaration | Call |
Procedure without parameter(-s)
|
| USE [stored_procedures] SET ANSI_NULLS ON
| // Template 1:
|
|
// Template 2
| // Import section
// Action block
| |||
| ||||
Procedure with incoming parameter(-s) |
| USE [stored_procedures] | // Template 3
// Template 3.3
// Template 3.1
| // Get from incoming FIXFieldList 34 and 56 tag, pass as procedure param
|
// Pass static value. Scope - current message rule's action
| ||||
//override template for incoming message. Scope - current message rule's // action
// Template 3.2
| ||||
Procedure with OUT parameter (-s) |
| USE [stored_procedures] | // Template 4
| //Import part
//action block
|
Procedure, that return result set |
| USE [stored_procedures] | // Template 5
| //import block
//row entity (can be declared before/after rulesDsl block in rules.groovy or imported)
//action block
|
Configuration and call procedures
Procedure class API
The object described in the corresponding configuration block can be obtained through the routingContext by calling it by the appropriate name or id.
Procedure procedure = routingContext.getProcedure("outHistory")
All methods are available to the user for selecting the one that meets the requirements.
The table below lists the methods existing in the Procedure class. They are used through customized blocks depending on the declared structure.
Method | Description | Procedure configuration example | Custom block example |
Procedure# executeUpdate() | Executes a procedure request for updates (e.g., UPDATE, INSERT, TRUNCATE, etc.). @return int - the quantity of affected rows |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12
|
Procedure# executeUpdate(Object[] objects) | Executes a procedure request for updates (e.g., UPDATE, INSERT, TRUNCATE, etc.). @param objects - an array of incoming parameters @return int - the quantity of affected rows |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12
|
Procedure# executeUpdate(FIXFieldList msg) | Executes a procedure request for updates (e.g., UPDATE, INSERT, TRUNCATE, etc.). According to the described configuration, it receives and transmits values from the input message. @param msg - an incoming FIX message @return int - the quantity of affected rows |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12
|
Procedure# executeRequest() | Executes a procedure request which returns some values (SELECT). @return a list of objects according to the mapper |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
|
Procedure# executeRequest(Object[] objects) | Executes a procedure request which returns some values (SELECT). @param objects - an array of incoming parameters @return a list of objects according to the mapper |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
|
Procedure# executeRequest(FIXFieldList msg) | Executes a procedure request which returns some values (SELECT). According to the described configuration, it receives and transmits values from the input message. @param msg - an incoming FIX message @return a list of objects according to the mapper |
|
Example
|
Procedure# executeRequestReturnOutParams() | Executes a procedure request that returns some values via OUT parameters.
@return a map of named objects |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
|
Procedure# executeRequestReturnOutParams (Object[] objects) | Executes a procedure request that returns some values via OUT parameters. @param objects - an array of incoming parameters @return a map of named objects |
|
Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
|
Procedure#executeRequestReturnOutParams (FIXFieldList msg ) | Executes a procedure request that returns some values via OUT parameters. @param msg - an incoming FIX message @return a map of named objects |
| example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
|
Full rules.groovy with some samples
rules.groovy Collapse source
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 |
|
Variables storage
DSL creation provides first description of the builder to create a set of instructions and then to execute them. At the same time, like in ordinary functions, these instructions do not have the classical work with variables.
Sometimes, it can be needed to use them, for example, for checking in several places or writing to several tags. For such cases, it is more convenient to save the value once obtained and then reuse it later.
Of course, for complex cases customizable blocks can always be used where its possible to work with dsl directly.
In cases of get/set - for the action block a variable storage is implemented with the scope of the same action execution.
History configuration for examples:
histories { memoryHistory { name "testHistory" keyTags 49, 38 valueTags 11, 55, 44 }}
Rule part | Description | Template | Example |
Set to varStorageYou can set values through the following instructions. | |||
History block | From the received list of values
|
| example 1 2 3 4 5
|
From the received list of values
| history (<history name>) {
| example 1 2 3 4 5
| |
From the received list of values
|
| example 1 2 3 4 5
| |
From the incoming message, prepare the corresponding key for the history. Retrieve the history value and save it to the variable storage under the specified object key
|
| example 1 2 3 4 5
| |
From the incoming message, prepare the corresponding key for the history. Retrieve the value?, get it's items by indexes and save them as list to the variable storage under the specified object key
|
| example 1 2 3 4 5
| |
From the incoming message, prepare the corresponding key for the history. Retrieve the value, get it's items by tags
|
| example 1 2 3 4 5
| |
|
|
|
|
Get from varStorageYou can get values through the following instructions | |||
Fields block | Retrieve the value from the variable storage, stored under the specified object key
|
| example 1 2 3 4 5
|
Root action block | Fill a specific table column with the specified static value. Set this value if declared by config was not founded. Only for JDBC history.
|
| example 1 2 3
|
Remove var from varStorage | |||
Root action block | Remove specified item from var storage
|
| example 1 2 3
|
Clear varStorage | |||
Root action block | Clear variable storage |
| example 1 2 3
|
Custom block | |||
Custom block | Get varStorage as map and save key-value pair. |
| example 1 2 3 4 5
|
Custom block | Get varStorage as map and get saved value by key. |
|
|