AlgoTraderAlgoTrader Documentation

Chapter 5. Strategy Development

5.1. Creating a Trading Strategy
5.1.1. AlgoTrader Strategy Wizard
5.1.2. AlgoTrader Maven Archetype
5.1.3. Generated Artifacts Simple Archetype
5.1.4. Generated Artifacts Esper Archetype
5.2. Building a Trading Strategy
5.3. Hints for Strategy Development
5.3.1. Java based Strategies
5.3.2. Esper based Strategies
5.4. Strategy Groups

Warning

It is recommended to perform thorough Simulation / Back Testing of newly developed strategies. After that the strategy should be tested with a Paper Trading Account. At the end of a thorough test procedure, the new strategy can be put into production. At the beginning of live trading it is recommended to use a small trading account only.

The following diagram shows the general procedure for developing new strategies:

Strategy Development Process

Figure 5.1. Strategy Development Process


The following paragraph will give a short example based on a simple moving average strategy (with the Short Name MOV).

The following 2 options can assist in creating a new trading strategy, the AlgoTrader Strategy Wizard and the AlgoTrader Maven Archetype.

The Strategy Wizard provides an easy way to automatically create all artifacts necessary for an AlgoTrader based trading strategy. Internally the Strategy Wizards makes use of the AlgoTrader Archetype, see Section 5.1, “Creating a Trading Strategy ”. The Strategy Wizard provides options for two different types of Trading Strategies:

  • Esper based Strategies

  • Simple Strategies (without Esper)

The Strategy Wizard can be started via the File / New / Other which will bring up the following screen where the Maven Project wizard can be selected:


On the next step the location for the newly created project (trading strategy) as well as the projects working set can be selected.


On the next screen please select the Default Local Catalog and algotrader-archetype-esper for Esper based strategies or algotrader-archetype-simple for simple strategies (without Esper).


Group Id

The maven group id (e.g. algotrader), all lower-case, can contain periods

Artifact Id

The maven artifact id (e.g. algotrader-ema), all lower-case, can contain dashes

Version

The maven version (e.g. 1.0.0-SNAPSHOT), x.y.z, plus optionally -SNAPSHOT

Package

The java package name (ch.algotrader.strategy), all lower-case, can contain periods.

name

The name of the strategy (e.g. ema), all lower-case, no periods, no dashes

serviceName

The name of the strategy service (e.g. EMA), first letter upper-case or all upper-case, do not include Service at the end (e.g. do not specify EMAService)

Note

For Spring Auto-Wiring to work the package name needs to be ch.algotrader.strategy. If a different package is assigned services (e.g. OrderService and LookupService) will not be available.

For all of these items previously entered values can be reused by clicking the combo-box to the right of the field.


When clicking finish the Strategy Wizard will create a new eclipse project using the AlgoTrader Artifact.

The AlgoTrader Maven Archetype is a project template that can be used to create a new AlgoTrader trading Strategy. To use the Maven Archetype execute the following command from the command line in a new empty directory.

To create Esper bases strategies execute (replace <version> with the corresponding AlgoTrader version):

mvn archetype:generate -DarchetypeGroupId=algotrader -DarchetypeArtifactId=algotrader-archetype-esper -DarchetypeVersion=<version>

To create simple strategies execute (replace <version> with the corresponding AlgoTrader version):

mvn archetype:generate -DarchetypeGroupId=algotrader -DarchetypeArtifactId=algotrader-archetype-simple -DarchetypeVersion=<version>

The Maven Archetype will ask for the following input parameters:

The Simple Archetype will generate the following artifacts:

This is the main Java-class containing the Business Logic.

The references to the Services provided by the AlgoTrader Server (e.g. OrderService, PositionService, etc.) will be auto injected on startup by the Spring Framework

public class EMAService extends StrategyService {


  private final long accountId = 1;
1
  private final long securityId = 25;
  private final long orderQuantity = 10000;
  public EMAService() {
    setStrategyName("EMA");
2
  }
  @Override
  public void onStart(final LifecycleEventVO event) {
3
    getSubscriptionService().subscribeMarketDataEvent(getStrategyName(), this.securityId);
  }
  public void sendOrder(Side side) {
    MarketOrderVO order = MarketOrderVOBuilder.create()
4
      .setStrategyId(getStrategy().getId())
      .setAccountId(this.accountId)
      .setSecurityId(this.securityId)
      .setQuantity(this.orderQuantity)
      .setSide(bar.getClose().compareTo(bar.getOpen()) > 0 ? Side.BUY : Side.SELL)
      .build();
    getOrderService().sendOrder(order); 
5
  }
}

The class EMAService method contains the following items:

1

Gets references to settings defined in conf-ema.properties.

2

Sets the name of the strategy

3

Once the strategy has reached the START live cycle phase subscribe to the security needed for this strategy

4

Construct an Order Value Object using the MarketOrderVOBuilder. The OrderVO contains a reference to the strategy the security, the account as well as the quantity and the order side.

5

Send the Order to the market via the OrderService

The Esper Archetype will generate the following artifacts:

This is the main Java-class containing the Business Logic.

The references to the Services provided by the AlgoTrader Server (e.g. OrderService, PositionService, etc.) will be auto injected on startup by the Spring Framework

public class EMAService extends StrategyService {


  private @Value("#{@emaConfigParams.accountId}") long accountId;
1
  private @Value("#{@emaConfigParams.securityId}") long securityId;
  private @Value("#{@emaConfigParams.orderQuantity}") long orderQuantity;
  @Override
  public void onStart(final LifecycleEventVO event) {
2
    getSubscriptionService().subscribeMarketDataEvent(getStrategyName(), this.securityId);
  }
  public void sendOrder(Side side) {
    MarketOrderVO order = MarketOrderVOBuilder.create()
3
      .setStrategyId(getStrategy().getId())
      .setAccountId(this.accountId)
      .setSecurityId(this.securityId)
      .setQuantity(this.orderQuantity)
      .setSide(side)
      .build();
    getOrderService().sendOrder(order); 
4
  }
}

The class EMAService method contains the following items:

1

Gets references to settings defined in conf-ema.properties.

2

Once the strategy has reached the START live cycle phase subscribe to the security needed for this strategy

3

Construct an Order Value Object using the MarketOrderVOBuilder. The OrderVO contains a reference to the strategy the security, the account as well as the quantity and the order side.

4

Send the Order to the market via the OrderService

A typical Spring Configuration File looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:p="http://www.springframework.org/schema/p"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="emaConfigParams" 
1
        class="ch.algotrader.config.spring.CustomConfigParamsFactoryBean" >
        <property name="global" ref="configParams"/>
        <property name="resource">
            <value>classpath:/conf-ema.properties</value>
        </property>
    </bean>

    <bean id="emaEngine" 
2
        class="ch.algotrader.esper.EngineFactoryBean">
        <property name="strategyName" value="EMA"/>
        <property name="configResource" value="esper-ema.cfg.xml"/>
        <property name="configParams" ref="emaConfigParams"/>
        <property name="initModules" value="test"/>
    </bean>

    <bean id="emaService" 
3
        class="ch.algotrader.EMAService" autowire="byName">
        <property name="strategyName" value="EMA"/>
        <property name="engine" ref="emaEngine"/>
     </bean>

</beans>

This file contains the following Spring Bean Definitions:

1

contains a Map of all properties based on settings defined in conf-ema.properties

2

Creates the Esper Engine based on strategyName, configResource, configParams and initModules and optional runModules definitions

3

Creates the Strategy Service based on strategyName definition and engine reference. All dependencies of the ch.algotrader.service.StrategyService will be injected automatically through auto wiring

Execute the following Maven command to start a Maven build of the trading strategy:

mvn install

The Maven modules can now be deployed to a Maven repository (e.g. Sonatype Nexus) using:

mvn deploy

For further details regarding a maven deploy please visit the Maven deploy plug-in page

Execute the following Docker command to create a Docker image of the trading strategy that can be used for productive deployments:

docker build -t xyz .

Please replace xyz with the name of the trading strategy.

The Docker image can now be pushed to a Docker repository (e.g. Docker Hub, Sonatype Nexus or Amazon ECR). For further details on pushing to a Docker repository please visit:

It is possible to develop trading strategies purely in Java for simplicity. Very often it is though helpful to use Esper for Market Data Analysis and Signal Generation in addition to Java code. The following sections will provide hints on developing strategies both in Java and Esper.

Java bases Strategies typically consist of a single Java class where all logic is implemented inside event handler methods (e.g. onInit, onBar, onTick, onOrderStatus, etc.)

Due to the expiring nature of Futures and Options, corresponding Positions have to be rolled prior to the Expiration Date. Typically, this would involve the following steps:

When dealing with futures one has to decide on when to roll from the Front-Month future into the Back-Month future. For this different philosophies exist:

  • Roll on a fixed day prior to the expiration date or the first notice date

  • Roll when the Back-Month future starts having a higher traded volume than the Front-Month future

  • Use the constant maturity method as described in the following section

Since Futures have an expiration date and are therefore not continuous, it is often not possible to base indicators on them. There are several method for dealing with this situation:

  • use the raw data and ignore the fact that price time series will have jumps

  • On the rollover day, add the difference between yesterdays closing price of the Back-Month future and yesterdays closing price of the Front-Month future to the combined time series. Alternatively one can use the difference between today's opening price of the Back-Month future and yesterdays closing price of the Front-Month future if only the time series for the generic 1st future is available. This method can be used for P&L calculation it might however lead to negative prices on long time series.

  • Instead of using addition as mentioned in the previous item use multiplication. This method however cannot be used for P&L calculations. Depending on the indicator in use either addition or multiplication will be adequate.

  • Use the constant maturity method as described in the following section

Please see Section 6.7, “Multi Security Simulations” for options on how to back test futures and options based strategies that require multiple securities to be subscribed and unsubscribed during the back test.

Sometimes one needs to compare output data files to verify a trading strategy or to find a problem during strategy development. CSV files can be imported into Microsoft Excel for the purpose of comparing them. Comparing two files however remains tricky and repeated exporting/importing and comparing in Excel can be cumbersome.

AlgoTrader provides a Java utility to compare ("diff") CSV files and assert the contents of the two files. For example the following statement compares a backtesting output file actual.csv with an expected result file expected.csv:

CsvDiff.diffAndLogAssertionErrors(fileDiffer, 

        new File("expected.csv"), new File("actual.csv")); 

The fileDiffer argument instructs the tool how to perform the diff operation. It can be constructed as follows:

FileDiffer fileDiffer = new FileDiffer(expectedDef, actualDef, differ);

In the above statement, expectedDef and actualDef define the columns of the two files; the differ gives exact instructions on how to perform the comparison of the files.

The columns of a CSV file are provided as implementations of CsvColumn which can be done with an enum:

public enum OrderReport implements CsvColumn {


    Date(new DateConverter("dd.MM.yyyy HH:mm:ss")),
    Instrument(SymbolConverter.INSTANCE),
    Side(StringConverter.INSTANCE),
    OrderType(StringConverter.INSTANCE),
    Size(LongConverter.INSTANCE),
    Limit(DoubleConverter.INSTANCE);
    
    private OrderReport(ValueConverter<?> converter) {
        this.converter = converter;
    }
    private final ValueConverter<?> converter;
    
    @Override
    public int index() {
        return ordinal();
    }
    
    @Override
    public ValueConverter<?> converter() {
        return converter;
    }
}

An example CSV file definition is then created as follows:

boolean hasHeaderLine = true;

CsvDefinition orderReportDef = new CsvDefinition(hasHeaderLine, OrderReport.values()); 

After defining the columns of both CSV files via CsvDefinition as indicated above we need to give instructions on how to perform the actual diff operation itself. In the simplest case we just compare the CSV files line by line and assert all or selected columns of the two files:

SimpleDiffer differ = new SimpleDiffer.Builder()

    .assertEqual(OrderReport.Date, OrderReport.Date)
    .assertEqual(OrderReport.Instrument, OrderReport.Instrument)
    .assertEqual(OrderReport.Side, OrderReport.Side)
    .assertEqual(OrderReport.Size, OrderReport.Size)
    .build();

The diff can also contain more complex instructions. For instance assume that we have exactly one BUY/SELL order per instrument and day but the ordering of instrument and side within a given date may be random. To align the correct rows for comparison, we must provide grouping and sorting hints for this case:

SimpleDiffer simpleDiffer = new SimpleDiffer.Builder()

    .assertEqual(OrderReport.Date, OrderReport.Date)
    .assertEqual(OrderReport.Instrument, OrderReport.Instrument)
    .assertEqual(OrderReport.Side, OrderReport.Side)
    .assertEqual(OrderReport.Size, OrderReport.Size)
    .build();
    
SortingDiffer sortingDiffer = new SortingDiffer.Builder()
    .add(OrderReport.Instrument, OrderReport.Instrument)
    .add(OrderReport.Side, OrderReport.Side)
    .build(simpleDiffer);
    
GroupDiffer differ = new GroupDiffer.Builder()
    .add(OrderReport.Date, OrderReport.Date)
    .build(sortingDiffer);

Now we are ready to run the CsvDiff.diffAndLogAssertionErrors(..) statement from above. The result may look similar to the following:

[..FAILED] 2 diffs at lines[exp/act]=45/51 in columns[exp/act]=[Size/Size] {exp-file=...}
[..FAILED] files: exp=OrderReport.csv, act=OrderReport_20141112_034538.csv
[..FAILED] lines: exp=45, act=51
[..FAILED] group: [13.01.2014]
[..FAILED]  ===========================================================================
[..FAILED]  NUM |   COLUMN(S)  |   EXPECTED   |    ACTUAL    | MESSAGE
[..FAILED]  ----|--------------|--------------|--------------|-------------------------
[..FAILED]  (1) | Size         |      2023769 |      2023762 | Values don't match ...
[..FAILED]  ----|----------------------------------------------------------------------
[..FAILED]  (1) | exp-line: [45] 13.01.2014 22:00:00,NYSE:PEP,BUY,Market,2023769,22.38
[..FAILED]      | act-line: [51] 13.01.2014 22:00:00,NYSE:PEP,BUY,Market,2023762,22.382 
[..FAILED]  ===========================================================================

When developing strategies using Esper in addition to Java code it is generally recommended to do Time-based Market Data Analysis and Signal Generation inside Esper Statements. Procedural actions like placing an order or subscribing to a Security are predominantly done inside Java Code.

A common use case is to wait for the full execution or cancellation of an order and then take some additional action.

Section 11.4.8.2, “Trade callback” explains how to use a TradeCallback for this purpose.

In this context it is also important to remember that when trying to close a position there might still be open orders associated with the corresponding security and strategy. It is suggested to cancel all corresponding orders, attach a TradeCallback to the cancellation and only close the position once all cancels have been confirmed.

Also keep in mind that an order might receive multiple fills in live trading. For example if one wants to send a Stop Order for each executed Order it is important to use the filled quantity and not on the original order quantity.

In handling partial fills the TargetPositionOrder can be useful, please see: Chapter 25, Execution Algos

Upon system startup strategies run through the life cycle phases as defined in Section 4.2, “Live Trading Mode”. At the same time market data connections are established. Due to the asynchronous nature of these two processes it is not predetermined which one will complete first. Typically within the START life cycle phase market data is subscribed. This however will fail if the market data adapter has not reached its SUBSCRIBED state by that time. To circumvent this issue the following Esper patter can be used, which waits for both the first LifecycePhaseVO and SessionEventVO to arrive before it fires:

select * 
from pattern[LifecycleEventVO(phase=LifecyclePhase.`START`) 
  and SessionEventVO(state=ConnectionState.SUBSCRIBED)];

With the constant maturity approach positions both in the Front-Month future as well as in the Back-Month future are established at all times. The weighting between the two positions is adjusted on a daily basis so that the combined time-to-expiration stays constant (e.g. 30 days)

With the following statements it is possible to calculate a constant maturity market value:

@Name('INSERT_INTO_FRONT_FUTURE_TICK')
insert into
  FrontFutureTick
select
  tick.*
from
  TickVO as tick unidirectional,
  method:lookupService.getSecurity(tick.securityId) as security
where
  cast(security.duration?, int) = 1;

@Name('INSERT_INTO_BACK_FUTURE_TICK')
insert into
  BackFutureTick
select
  tick.*
from
  TickVO as tick unidirectional,
  method:lookupService.getSecurity(tick.securityId) as security
where
  cast(security.duration?, int) = 2;

@Name('CONSTANT_MATURITY')
select
  ConstantMaturityUtil.getConstantMaturityValue(front, back) as value
from
  pattern [every(
    (front=FrontFutureTick -> (back=BackFutureTick where timer:within(1 hour)))
      or
    (back=BackFutureTick -> (front=FrontFutureTick where timer:within(1 hour)))
  )];

The actual static method to calculate the constant maturity value looks like this:

public static double getConstantMaturityValue(TickVO frontTick, TickVO backTick) {


  Future frontFuture = (Future) frontTick.getSecurity();
  Future backFuture = (Future) backTick.getSecurity();
  double weight = (double) (backFuture.getExpiration().getTime()
      - DateUtil.getCurrentEPTime().getTime() - Duration.MONTH_1.getValue()))
      / ((double) (backFuture.getExpiration().getTime()
      - frontFuture.getExpiration().getTime());
  return weight * frontTick.getCurrentValueDouble()
    + (1 - weight) * backTick.getCurrentValueDouble();
}

Often a strategy has several states that it runs through during execution (e.g. FLAT, PENDING_LONG, PENDING_SHORT, LONG, SHORT, etc.). For these situations it is advisable to use a Java Enum, e.g.:

package ch.algotrader.strategy;


public enum State {
  FLAT, PENDING_LONG, PENDING_SHORT, LONG, SHORT;
}

In addition a corresponding Esper Variable has to be configured:


<variable name="state" type="ch.algotrader.strategy.State"/>

If the variable needs to have an initial value, the variable has to be declared with a statement.

@Name('CREATE_VAR_STATE')
create variable ch.algotrader.enumeration.State state = State.FLAT;        

It is now possible to query current state inside Esper statements like this:

select * from ... where state = State.FLAT;

In case a strategy trades multiple instruments and each instrument has its own state an Esper Named Window can be used instead of an Esper Variable.

@Name('METRICS_WINDOW')
create window 
	MetricsWindow.std:lastevent() 
as
	Metrics;

@Name('INSERT_INTO_METRICS_WINDOW')
insert into
	MetricsWindow
select
	*
from 
	Metrics;

The first statement creates the Named Window and the second statements inserts all Metrics events into the Named Window.

The Metrics object is a simple Java POJO that holds the state per instrument (and potential other information regarding the instrument):

public class Metrics implements Serializable {

    private static final long serialVersionUID = 5972079135237671512L;

    private long securityId;
    private State state;

    public Metrics(long securityId, State state) {
        this.securityId = security;
        this.state = state;
    }

    // getters and setters
}

To initialize the Named Window one Metrics event per instrument has to be sent into the Esper Engine upon startup of the strategy

public void onInit(final LifecycleEventVO event) {

    getEngine().sendEvent(new Metrics(securityId, State.FLAT));
}

It is now possible to query current state inside Esper statements like this:

select state from MetricsWindow where securityId = ...;

Often a Java Action is triggered multiple times by a certain situation, because the underlying cause takes a finite amount of time to be resolved.

Example: The current market level exceeds a defined stop, which triggers a closing order. However during the time the order is being executed at the market, additional market data events are received. Because the position is not yet closed by that time, another undesired closing order might get placed.

To prevent actions from happening multiple times a state object mentioned in the previous section is again very helpful .

Whenever the predefined signal condition is met the state will be set to PENDING

@Name('LONG_TRIGGER')
on
	//trigger event (can also be the MetricsWindow itself)
update 
	MetricsWindow as metricsWindow
set
	state = State.PENDING_LONG
where
	// condition
and
    metricsWindow.state = State.SHORT or metricsWindow.state = State.FLAT

@Name('SEND_ORDER')
@Subscriber(className='xyzService#sendOrder')
select
    state
from
    MetricsWindow
where
    state = State.PENDING_LONG 
or 
    state = State.PENDING_SHORT;

Once the state has been changed to PENDING_LONG the statement SEND_ORDER will trigger an order to be sent. Since the LONG_TRIGGER statement only triggers if the state is either SHORT or FLAT it will not trigger again once the state has been set to PENDING_LONG.

Once the order has been fully executed (potentially using a Section 11.4.8.2, “Trade callback”) the state needs to be changed to LONG

getEngine().executeQuery("update MetricsWindow set state = State.LONG where securityId = " + securityId);

AlgoTrader provides extensive support for strategy groups. In simulation mode multiple strategies can be run simultaneously as part of a strategy group. It is even possible to run multiple instances of the same strategy (with different parameters).

For this purpose AlgoTrader provides support for strategy and engine templates as well as strategy groups based on Spring XML configuration and abstract Spring beans. This enables strategy developers to define abstract templates for strategy engines and strategy services and then define concrete instances of those strategies with custom configuration.

Definition of strategy templates and engine templates typically look like this..


<bean id="boxServiceTemplate" class="ch.algotrader.strategy.box.BoxService" abstract="true">
    <property name="lookupService" ref="lookupService"/>
    <property name="marketDataCacheService" ref="marketDataCacheService"/>
    <property name="portfolioService" ref="portfolioService"/>
    <property name="positionService" ref="positionService"/>
    <property name="measurementService" ref="measurementService"/>
    <property name="orderService" ref="orderService"/>
    <property name="subscriptionService" ref="subscriptionService"/>
    <property name="commonConfig" ref="commonConfig"/>
</bean>

<bean id="boxEngineTemplate" class="ch.algotrader.esper.EngineFactoryBean" abstract="true">
    <property name="configResource" value="esper-box.cfg.xml"/>
    <property name="initModules" value="box-init"/>
    <property name="runModules" value="current-values, box-run"/>
</bean>

Based on these templates concrete strategy instances can be configured.


<at:strategy name="box-narrow"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate"
             resourceName="box-narrow.properties" />

<at:strategy name="box-wide"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate"
             resourceName="box-wide.properties" />

Only parameters configClass, engineTemplate, and serviceTemplate attributes are mandatory. Configuration can be further simplified if configuration resources follow the convention <strategy-name>.properties.


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:at="http://www.algotrader.ch/schema/spring/config"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
    http://www.algotrader.ch/schema/spring/config http://www.algotrader.ch/schema/spring/config/algotrader.xsd">

<at:strategy name="box-narrow"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate" />

<at:strategy name="box-wide"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate" />

</beans>

Multiple strategy instances can be grouped together and assigned individual weights in the group. Strategy groups can executed as one unit using provided Spring profile.


<beans profile="simpleGroup">
    <at:strategyGroup id="simpleGroup">
        <at:strategyItem name="box-narrow" weight="0.2"/>
        <at:strategyItem name="box-wide" weight="0.8"/>
    </at:strategyGroup>
</beans>

To start the simpleGroup activate the Spring profile through the following VM argument:

-Dspring.profiles.active=...,simple

The AlgoTrader Eclipse IDE provides a visual editor for strategy groups, for further details please see Section 13.2.3, “AlgoTrader Configuration Editor”