AlgoTraderAlgoTrader Documentation

Chapter 21. Market Data

21.1. Creation of Bars based on Ticks
21.2. Market Data File Format
21.2.1. Tick Data Files
21.2.2. Bar Data Files
21.3. Numeric Precision
21.4. Price normalization
21.5. Market Data Gap Checking
21.6. Generic Events

AlgoTrader provides Market Data Interfaces with the following market data providers:

AlgoTrader allows having multiple market data interfaces active at the same time so market data received from different market data providers can be compared in real-time.

To enable either of those Market Data Interfaces the following two steps have to be executed:

  1. The correct Spring Profile has to be activated according to Section 26.3, “Spring Profiles”

  2. For Bloomberg market-data-bb and for InteractiveBrokers market-data-ib has to be added to the VM argument server-engine.init when running the AlgoTrader server.

All Market Data Interfaces have a set of unique artifacts:

Market Data Events itself are available in three different types:

As the following diagram shows, market data providers deliver Price Events (Trade, Bid & Ask) or individual Tick Fields.

In Simulation Ticks or Bars can be provided through CSV-Files.

Market Data Event Types

Figure 21.1. Market Data Event Types

Processing of MarketData is handled through the MarketDataService, which calls the the market data provider specific ExternalMarketDataService implementations. Every broker / exchange specific service has to provide implementation of this interface e.g (e.g. IBMarketDataServiceImpl or BBMarketDataServiceImpl).

The most important methods provided by the MarketDataService are subscribe and unsubscribe. Through the use of these methods new MarketData can be subscribed and unsubscribed. Subscribed securities are persisted within the DB-table subscription. The actual subscription of securities to the external broker is done through the market data provider specific MarketDataService.

Market data provider interfaces are responsible for receiving market data provider specific Market Data and sending them into the Esper Service Instance of the AlgoTrader Server. The Esper Service Instance will then convert these Events into generic MarketDataEvents (i.e. Ticks or Bars) which will be propagated to subscribed Strategies.

Inside each strategy the MarketDataCacheService keeps a copy of the last MarketDataEvent for each Security. For further details see Section 5.3.7, “MarketDataCacheService”.

Market Data Propagation

Figure 21.2. Market Data Propagation

In both Simulation and Live Trading, Bars can be generated from Ticks or Price Events through the use of Esper Statements: use the time_batch to create bars:

  first(last) as open,
  max(last) as high,
  min(last) as low,
  last(last) as close
from min)
group by

and expr_batch for constant volume bars:

  max(last) as high,
  min(last) as low,
  first(last) as open,
  last(last) as close
from > 1000)
group by

All market data files are placed inside the following directory structure:

An alternative approach is to feed market data for multiple securities using one file. E.g. it is possible to feed market data for futures using market data from the corresponding generic future. In this approach an additional column security has to be added to the market data file which will be used to identify the actual Security.

The first line within the file is the header row.

The file name for Section 21.6, “Generic Events” follows a different logic.

In General different Securities are traded with different numeric precision (e.g. S&P Futures prices are two digits, whereas FX prices are usually 5-6 digits). To accommodate different numeric precisions, AlgoTrader provides the following two Fields inside the class SecurityFamily:

Price normalization comes into play when multiple market data and/or trading adapters are in use that use different price multipliers / contract sizes for the same instrument. For example one adapter might quote prices in dollars with a contract size of 10 where as another one might quote them in cents with a contract size of 1000. The default contract size will be stored in the security_family table with all other cases (i.e. exceptions) stored using the broker_parameters table.

Since a continuous data feed of market data is essential for most trading strategies, AlgoTrader contains a feature that automatically warns if no market data has been received for a prolonged period of time. For this purpose the class SecurityFamily has a property maxGap, that defines the maximum number of minutes allowed without any market data updates. This is enforced by the CHECK_TICK_GAPS statement.

In addition to MarketDataEvents (i.e. Ticks and Bars) there are general purpose Generic Events that can contain any type of information (e.g. virtual market data, signals, exposure values, etc.). In contrary to MarketDataEvents, which are sent from the AlgoTrader Server to subscribed strategies, these Generic Events are sent from one strategy to one or more receiving strategies. A Generic Event class has to subclass GenericEventVO. Subscription to Generic Events is based on the class of the Generic Event.

Example: To subscribe to a Generic Event of type Signal (which is a subclass of GenericEventVO), the following needs to be done:

Class<?>[] genericEvents = new Class[] { Signal.class };


To send a GenericEvent, use the following method:


It is also possible to feed Generic Events via CSV Files in Simulation Mode. To enable this, the following VM argument has to be set:

-DdataSource.feedGenericEvents = true

The filename of the CSV File has to be according to this schema: