AlgoTraderAlgoTrader Documentation

Chapter 24. Adapters

24.1. Fix Interface
24.2. IB Native Interface
24.3. IB Fix Interface
24.4. Bloomberg
24.5. QuantHouse
24.6. EzeSoft / Real Tick
24.7. Trading Technologies (TT)
24.8. Currenex
24.9. JP Morgan
24.10. UBS
24.11. DukasCopy
24.12. FXCM
24.13. LMAX
24.14. Nexus Prime
24.15. Fortex
24.16. Coinigy
24.16.1. Setup Instructions
24.17. Quandl
24.18. Session life-cycle events
24.19. Drop-copy support
24.20. FIX configuration
24.20.1. FIX logging
24.20.2. FIX message persistence

The following sections give a detailed overview of the different adapters available for AlgoTrader.

AlgoTrader uses QuickFix/J for it's Fix connections and currently supports FIX 4.2 and 4.4. Because FIX messages are not compatible between different version, the two distinct services Fix42OrderService and Fix44OrderService exist. Incoming messages are handled by their corresponding Fix42MessageHandler and Fix44MessageHandler.

To configure a Fix trading connection the following steps have to be taken care of:

  • Add the corresponding fix trading profile to the VM argument spring.profiles.active (e.g. cNXFix):

    -Dspring.profiles.active=live,pooledDataSource,cNXFix,embeddedBroker,html5,InfluxDB 
  • Add the fix session to /algotrader/core/fix.cfg (an example file fix-template.cfg is provided in the same directory):

    [session]
    SessionQualifier=CNXT
    BeginString=FIX.4.4
    SenderCompID=xxx
    TargetCompID=CNX
    SocketConnectHost=dret-fix-ssl.currenex.com
    SocketConnectPort=443
    SocketUseSSL=Y
    Username=xxx
    Password=xxx
    ValidateIncomingMessage=N
    ResetOnLogon=Y
    Inactive=Y
  • Make sure there is an entry in the MySql account table where the column ORDER_SERVICE_TYPE matches the type of the fix interface (e.g. CNX_FIX) and the column SESSION_QUALIFIER matches the SessionQualifier specified in the file fix.cfg.

If market data is also received through a Fix interface the following items need to be added as well:

  • Add the corresponding fix market data profile to the VM argument spring.profiles.active (e.g. cNXFix):

    -Dspring.profiles.active=live,pooledDataSource,cNXMarketData,embeddedBroker,html5,InfluxDB 
  • Add the fix session to /algotrader/core/fix.cfg (an example file fix-template.cfg is provided in the same directory):

    [session]
    SessionQualifier=CNXMD
    BeginString=FIX.4.4
    SenderCompID=xxx
    TargetCompID=CNX
    SocketConnectHost=dret-fix-ssl.currenex.com
    SocketConnectPort=443
    SocketUseSSL=Y
    Username=xxx
    Password=xxx
    ValidateIncomingMessage=N
    ResetOnLogon=Y
    Inactive=Y
  • When making subscriptions add the FeedType corresponding to the Fix interface (e.g. CNX) or configure the default feed type via VM argument:

    -Dmisc.defaultFeedType=CNX 

For further information regarding QuickFix/J configuration please visit the QuickFix/J documentation

The Fix infrastructure consists of the following classes:

Table 24.1. Fix Infrastructure

Class / InterfaceDescription
SessionA Session represents a connection to a broker / exchange / market data provider
ApplicationFor each Session an Application object is created. It will forward incoming messages to the corresponding MessageHandlers
DefaultFixApplication
FixApplicationFactoryIs responsible for the creation of Applications
DefaultFixApplicationFactory
FixMultiApplicationSessionFactory

Creates a Session and Application using the specified FixApplicationFactory according to the following steps:

  • lookup the FixApplicationFactory by its name

  • create an Application

  • create a DefaultSessionFactory

  • create a Session

ExternalSessionStateHolder Represents the current state of a Session (i.e. DISCONNECTED, CONNECTED, LOGGED_ON and SUBSCRIBED)
DefaultFixSessionStateHolder
MarketDataFixSessionStateHolderA ExternalSessionStateHolder for market data sessions that will subscribe to securities as soon as the session is logged on.
FixOrderIdGeneratorGenerator for Fix Order Ids. The default implementation reads the last Order Ids from the Fix log on start-up.
DefaultFixOrderIdGenerator
FixAdapter Management Adapter for the Fix environment. Allows the creation of dynamic sessions, sending Messages and managing Order Ids.
DefaultFixAdapter
ManagedFixAdapterManageable implementation of a FixAdapter (based on JMX)
FixEventSchedulerQuickFix/J currently supports daily sessions (with a daily session 7 times a week) and weekly sessions (with one weekly session). However some brokers (e.g. JP Morgan) use daily sessions during workdays. To accomplish this scenario, AlgoTrader allows creation of a weekly logon/logoff event (e.g. Mo 08:00:00 and Fr 18:00:00) using Esper Statements
DefaultFixEventScheduler
EventPattern
FixSocketInitiatorFactoryBeanA Spring Factory Bean that creates the SocketInitiator necessary for all Fix Sessions.
Fix42MarketDataMessageHandlerMessage Handler for incoming Fix market data messages. These classes need to be sub classed by the corresponding market data interface. Messages are propagated into the Esper Engine.
Fix44MarketDataMessageHandler
Fix42OrderMessageHandlerMessage Handler for incoming Fix Execution Reports. These classes need to be sub classed by the corresponding order interface. Messages are propagated into the Esper Engine.
Fix44OrderMessageHandler

Per default Fix interfaces uses the following items to identify a particular instrument:

Options

Exchange IB_CODE

SecurityFamily CURRENCY

SecurityFamily SYMBOL_ROOT

Option STRIKE

Option TYPE

Option EXPIRATION

SecurityFamily CONTRACT_SIZE

Future

SecurityFamily CURRENCY

Exchange IB_CODE

SecurityFamily SYMBOL_ROOT

Future EXPIRATION

SecurityFamily CONTRACT_SIZE

Forex

SecurityFamily CURRENCY

Exchange IB_CODE

Forex BASE_CURRENCY

Stock

SecurityFamily CURRENCY

Exchange IB_CODE

Stock SYMBOL

Fund

SecurityFamily CURRENCY

Exchange IB_CODE

Index SYMBOL

The native IB Interface connects to the local Trader Workstation (TWS) or IB Gateway instance and uses methods supplied by the IB client. The interface is fully capable of handling IB's Financial Advisor functionality like Sub Accounts, Account Groups and Allocation Profiles.

The IB interface supports Market Data, Historical Data, Order Processing, Retrieval of account information as well as Reference Data.

To configure an IB connection the following steps have to be taken care of:

If market data is also received through the IB interface the following items need to be added as well:

The IB infrastructure consists of the following classes:


The IB interface has the following options to identify a particular instrument:

In addition the following items apply to the IB Native interface

  • The IB Native interface uses the RT_VOLUME events to process incoming trade events

  • The IB Native interface propagates daily OPEN and CLOSE prices to strategies in case the following VM argument is enabled

    -Db.emitOpenClose=true
  • The IB Native interface propagates VWAP prices to strategies in case the following VM argument is enabled

    -Db.emitVWAP=true
  • The IB Native interface expects orders to be sent with their order ids in ascending order. The Class IBOrderIdSynchronizer is responsible to make sure order ids are actually in ascending order. In case an order id is skipped the IBOrderIdSynchronizer will wait for up to maxOrderSyncTime milliseconds for the order with the correct order id to arrive.

  • The IB Native interface supports trading of tradeable / non-synthetic combinations by placing BAG orders through the IB interface.

  • The IB Native interface reports volBid, volAsk and vol in lots of 100 contracts for US equities. Please see the following page for further details on handling of Odd Lot Orders

For further details on the IB native interface please visit the IB API Reference Guide

The IB Fix Interface provides the same Order Management features as the IB Native Interface. However Market Data is not available through this interface.

The interface is fully capable of handling IB's Financial Advisor functionality like Sub Accounts, Account Groups and Allocation Profiles.

For further details on the IB Fix interface please visit the IB FIX/CTCI Users' Guide

The IB Fix interface uses standard Fix instrument definitions mentioned at the end of section Section 24.1, “Fix Interface”.

The Bloomberg interface supports Market Data, Historical Data as well as Reference Data.

The Bloomberg interface provides both synchronous connections and asynchronous connections. Asynchronous connections are generally used for live market data whereas synchronous connections are used for retrieval of historical data as well as retrieval of reference data.

If market data is received through the Bloomberg interface the following items need to be added:

The Bloomberg infrastructure consists of the following classes:


Bloomberg uses the BBGID field of the Security table to identify instruments.

For further details on the Bloomberg interface please visit the Bloomberg Open API

The QuantHouse adapter is based on the QuantHouse ultra low latency market data feed QuantFEED. The QuantHouse adapter supports live Market Data.

If market data is received through the QuantHouse interface the following items need to be added:

The QuantHouse infrastructure consists of the following classes:


QuantHouse uses the Exchange MIC and Security SYMBOL fields to identify instruments.

For further details on the QuantHouse interface please contact QuantHouse

EzeSoft / RealTick provides connectivity to about 30 institutional and 10 retail brokers.

The EzeSoft / RealTick Fix interface currently supports only Order Processing.

The Fix Implementation of EzeSoft / RealTick is well conforming with the Fix Standard no customizations had to be made.

The IB Fix interface uses standard Fix instrument definitions mentioned at the end of section Section 24.1, “Fix Interface”.

Supports a wide range of future and option contracts tradeable at multiple venues / exchanges.

The main features of the Currenex platform are

The Currenex implementation of the FIX/4.4 protocol has some peculiarities

Currenex uses the columns Forex BASE_CURRENCY and SecurityFamily CURRENCY to identify an instrument.

The JP Morgan Fix interface supports Order Processing only.

As the JP Morgan Fix Implementation is well conforming with the Fix Standard no customizations had to be made

The IB Fix interface uses standard Fix instrument definitions mentioned at the end of section Section 24.1, “Fix Interface”.

The UBS Fix interface supports Order Processing for futures and options only.

As the UBS Fix Implementation is well conforming with the Fix Standard no customizations had to be made

The IB Fix interface uses standard Fix instrument definitions mentioned at the end of section Section 24.1, “Fix Interface”.

The DukasCopy interface supports Market Data as well as Order Processing.

Since the DukasCopy Fix Implementation does not follow the Fix Standard very closely, a few customizations had to be made:

DukasCopy uses the columns Forex BASE_CURRENCY and SecurityFamily CURRENCY to identify an instrument.

The DukasCopy Fix interface uses an SSL encrypted connection which is supported by QuickFix/J using Mina. In addition the DukasCopy interface requires username and password to be send with the Logon message. For this purpose the class DCLogonMessageHandler is used as an outgoing MessageHandler.

FXCM interface FIX/4.4 protocol does not deviate much from the standard but has some peculiarities about the way FIX sessions are established

FXCM uses the columns Forex BASE_CURRENCY and SecurityFamily CURRENCY to identify an instrument.

Supports only a limited number of securities, mainly Forex

LMAX implementation of the FIX/4.4 protocol has some peculiarities

LMAX uses the column LMAXID of the security table to identify an instrument

Nexus Prime is a MetaTrader MT4 FIX interface provided by IS Risk Analytics. The Nexus Prime interface uses Fix 4.4 and it supports FX only. Due to the underlying MetaTrader MT4 a few limitations apply.

  • Market Data subscriptions cannot be cancelled

  • Orders cannot be modified, instead one needs to cancel the current order first and then resend a new one.

  • Buy limit orders need to be placed below the market price. Sell limit orders need to be placed above the market price

  • Buy stop orders need to be placed above the market price. Sell stop orders need to be placed below the market price.

  • Minimum trade size allowed on most currency pairs is .01 lots which is 1000 notional

Nexus Prime uses the columns Forex BASE_CURRENCY and SecurityFamily CURRENCY to identify an instrument.

Fortex uses almost vanilla Fix/4.4 protocol with a very customizations. It supports FX only.

Fortex uses the columns Forex BASE_CURRENCY and SecurityFamily CURRENCY to identify an instrument.

Coinigy provides connectivity to 45+ of most popular cryptocurrency exchanges allowing to trade hundreds of different crypto currencies. The Coinigy Interface connects to the Coinigy API endpoints via REST and Socket Cluster protocols.

The Coinigy interface supports Market Data, Order Processing, Retrieval of account information as well as Reference Data.

The Coinigy infrastructure consists of the following classes:


Coinigy uses the columns Security CNGID and Exchange CNGID to identify an instrument.

To assist setting the default_forex column with Coinigy AlgoTrader provides the setting cng.defaultExchangeCodes. When instances of the same currency pair are available on multiple exchanges AlgoTrader will set the default_forex column to true for the first exchange found in cng.defaultExchangeCodes.

For further details on the Coinigy interface please visit the Coinigy API Documentation

To setup a connection to Coinigy the following steps have to be taken:

  • Sign-up for a Coinigy account on Coinigy Sign up

  • Enable two factor authentication (2FA) on the account following the 2FA Instructions

  • In the API accounts settings add the API keys from all of the exchanges where an account is setup according to these Instructions

  • In the account preferences generate a new Coinigy API key and Secret Key set it inside conf-cng.properties

  • In the account preferences click the button 'Click to reveal my Private Channel ID (Websocket API)' and set the Private Channel ID inside conf-cng.properties

  • Update the following settings inside conf.properties, conf-core.properties and conf-cng.properties: as needed

conf.properties:

# the currency all portfolio balances will be calculated in
#{"type":"ch.algotrader.enumeration.Currency","label":"Portfolio Base Currency"}
misc.portfolioBaseCurrency=BTC

# the number of digits all portfolio balances will be displayed with
#{"type":"Integer","label":"Portfolio Digits"}
misc.portfolioDigits=8

conf-core.properties:

# default feed type to use for market data (InteractiveBrokers)
#{"type":"String","label":"Default Feed Type"}
misc.defaultFeedType=CNG

conf-cng.properties:

#{"type":"String","label":"REST Url"}
cng.restUrl = https://api.coinigy.com/api/v1/

#{"type":"String","label":"Web Socket Url"}
cng.wssUrl = wss://sc-02.coinigy.com/socketcluster/

#{"type":"String","label":"WSS Private Channel"}
cng.wssPrivateChannel = XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

#{"type":"String","label":"API Key"}
cng.apiKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

#{"type":"String","label":"API Secret"}
cng.apiSecret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

#{"type":"String","label":"CNG reverse exchanges"}
cng.reverseExchanges = BITS,BTCC,PLNX

#{"type":"String","label":"CNG Default Exchange Codes"}
cng.defaultExchangeCodes = PLNX,BITF,KRKN,GDAX,BTCE,OK,BTRX,BT38,BITS,HUOB

#{"type":"String","label":"CNG session qualifier"}
cng.sessionQualifier = CNG

All of these settings change be added through VM arguments as well, e.g.:

-Dcng.wssPrivateChannel=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
-Dcng.apiKey=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

In order to populate the database with Coinigy Accounts, Exchanges, Security Families and Securities run the ReferenceDataStarter with cNGReferenceData spring profile enabled and program argument: all. For further details please visit Chapter 23, Reference Data.

Quandl is a public service that provides a wide range of financial, economic and alternative data. AlgoTrader allows downloading historical data from Quandl. For more information about Quandl please have a look at the Quandl Docs/Help.

Data on Quandl is divided into databases. Each database contains multiple datasets. For instance EOD database contains end-of-day data for all publicly-traded US stocks. Each database/dataset pair is uniquely identified by database_code/dataset_code pair. For instance EOD/AAPL is the globally unique code for the AAPL stock dataset within the EOD database. The Quandl database browser can be used to find suitable databases for desired instrument type, region and data type.

The QdlHistoricalDataService is integrated with the AlgoTrader Historical Data Download and needs to be enabled by specifying the qdlHistoricalData Spring profile (see section Section 22.3, “Historical Data Download”). The QdlHistoricalDataService transforms retrieved Quandl data into AlgoTrader historical bars that are stored into InfluxDB. Transformation rules between the Quandl data format and AlgoTrader Bar format are defined in the file quandl.yml. By default the file quandl.yml already contains the transformation rules for most commonly used Quandl databases. Additional transformation rules can be added to the file as needed:

EOD: 1
    barSize: DAY_1 2
    columnMapping: 3
        dateTime: Date
        open: Open
        high: High
        low: Low
        close: Close
        vol: Volume

1

The Quandl database code

2

barSize supported by the Quandl database (e.g. DAY_1 or MIN_1)

3

Column mappings between Quandl data fields and AlgoTrader BarVO fields

The QdlHistoricalDataService specific properties are defined inside the file conf-qdl.properties:

#{"type":"String","label":"Base URL"}
qdl.baseUrl = https://www.quandl.com/api/v3/datasets

#{"type":"String","label":"API Key"}
qdl.apiKey = ATVxxxxxxxxxxxxx

To use the QdlHistoricalDataService please replace the property qdl.apiKey with the API Key that can be retrieved through the Quandl Account Settings.

In terms of historical data download a mapping between the Quandl database and the SecurityFamily entity is defined by the quandl_database field in the security_family table. Similarly a mapping between the Quandl dataset and the Security entity is defined by the quandl_dataset field in the security table. AlgoTrader sample data files (samples/db/mysql/mysql-data.sql and samples/db/mysql/h2-data.sql) already contain quandl_database/quandl_dataset values for all sample security families and most sample securities.

All trading interface adapters generate session events, which enable the server engine as well as individual strategies to listen for and react to session events such as session being fully established or temporary loss of connectivity.

@Override

public void onChange(final SessionEventVO event) {
    switch (event.getState()) {
        case CONNECTED:
            // session connected but not yet authenticated
            break;
        case LOGGED_ON:
            // session connected and authenticated
            break;
        case DISCONNECTED:
            // session disconnected
            break;
    }
}

LMAX and Trading Technologies interfaces provide support for so called drop-copy mode wherein the adapter can receive order status and fill messages from orders initiated externally (usually by external applications such as native trading front-ends). By default external fills get recorded as transactions of the SERVER strategy and allocated to the external account specified in the original execution report message. One, however, can provide a custom implementation of DropCopyAllocator interface in order to apply custom transaction allocation logic.

FIX configuration is stored in fix.cfg file by default.

These are default parameters suggested by AlgoTrader for all FIX sessions:

[default]
ConnectionType=initiator
HeartBtInt=30
ReconnectInterval=5
FileStorePath=files/fix
FileLogPath=log
FileLogHeartbeats=N
FileIncludeMilliseconds=Y
FileIncludeTimeStampForMessages=Y
SLF4JLogHeartbeats=N

Details of individual FIX sessions are expected to be provided by the brokerages.

For further information regarding QuickFix/J configuration please visit the QuickFix/J documentation