AlgoTraderAlgoTrader Documentation

Chapter 5. Strategy Development

5.1. Creating a Trading Strategy
5.1.1. AlgoTrader Strategy Wizard
5.1.2. AlgoTrader Maven Archetype
5.1.3. Generated Artifacts
5.1.4. MovService.java
5.1.5. module-mov.epl
5.1.6. conf-mov.properties
5.1.7. esper-mov.cfg.xml
5.1.8. applicationContext-client-mov.xml
5.1.9. h2-mov.sql
5.1.10. mysql-mov.sql
5.1.11. Dockerfile
5.2. Building a Trading Strategy
5.3. Hints for Strategy Development
5.3.1. Strategy starters
5.3.2. Print Statement Selects
5.3.3. Logging values of an Indicator to a log file
5.3.4. Access to Esper Variables
5.3.5. Esper Utility classes
5.3.6. Storing the latest Market Data Events
5.3.7. MarketDataCacheService
5.3.8. Prioritizing Statements
5.3.9. Market Data Event Prefeeding
5.3.10. Creation of Bars based on Ticks
5.3.11. Reacting upon a newly subscribed security
5.3.12. Reacting upon an order
5.3.13. Waiting on market data session upon strategy startup
5.3.14. Tagging of orders
5.3.15. Exposing a strategy specific method through the client
5.3.16. Prevent LazyInitializationException
5.3.17. Prevent an action from happening multiple times
5.3.18. Execute an action once a day at a certain time
5.3.19. Rolling of Futures and Options
5.3.20. Handle Constant Maturity Futures
5.3.21. State based Strategy
5.3.22. Prevent Memory leaks
5.3.23. Comparing CSV files
5.4. Strategy Groups

Warning

It is recommended to do a thorough Simulation / Back Testing of newly developed strategies. After that the strategy should be tested with a Paper Trading Account. At the end of a thorough test procedure, the new strategy can be put into production. At the beginning of live trading it is recommended to use a small trading account only.

The following diagram shows the general procedure for developing new strategies:

Strategy Development Process

Figure 5.1. Strategy Development Process


The following paragraph will give a short example based on a simple moving average strategy (with the Short Name MOV).

Currently the following two tools can assist in creating a new trading strategy, the AlgoTrader Strategy Wizard and the AlgoTrader Maven Archetype.

The Strategy Wizard provides an easy way to automatically create all artifacts necessary for an AlgoTrader based trading strategy. Internally the Strategy Wizards makes use of the AlgoTrader Archetype, see Section 5.1, “Creating a Trading Strategy ”.

The Strategy Wizard can be started via the File / New / Other which will bring up the following screen where the Maven Project wizard can be selected:


On the next step the location for the newly created project (trading strategy) as well as the projects working set can be selected.


On the next screen please select the AlgoTrader Catalog and the algotrader archetype.


On the next screen the following items have to be entered:

  • Group Id: the maven group id (e.g. ch.algotrader), all lower-case, can contain periods

  • Artifact Id: the maven artifact id (e.g. algotrader-test), all lower-case, can contain dashes

  • Version: the maven version (e.g. 1.0.0-SNAPSHOT), e.g. x.y.z, plus optionally -SNAPSHOT

  • Package: the java package name (e.g. ch.algotrader), all lower-case, can contain periods

  • name: name of the strategy (e.g. test), all lower-case, no periods, no dashes

  • serviceName: name of the strategy service (e.g. Test), first letter upper-case, do not include Service at the end (e.g. do not specify BreakoutService)

For all of these items previously entered values can be reused by clicking the combo-box to the right of the field.


When clicking finish the Strategy Wizard will create a new eclipse project using the AlgoTrader Artifact.

This is the main Java-class containing the Business Logic.

The references to the Services provided by the AlgoTrader Server (e.g. OrderService, PositionService, etc.) will be auto injected on startup by the Spring Framework

public class MovService extends StrategyService {


  private @Value("#{@movConfigParams.accountId}") long accountId;
1
  private @Value("#{@movConfigParams.securityId}") long securityId;
  private @Value("#{@movConfigParams.orderQuantity}") long orderQuantity;
  @Override
  public void onStart(final LifecycleEventVO event) {
2
    getSubscriptionService().subscribeMarketDataEvent(getStrategyName(), this.securityId);
  }
  public void sendOrder(Side side) {
    MarketOrderVO order = MarketOrderVOBuilder.create()
3
      .setStrategyId(getStrategy().getId())
      .setAccountId(this.accountId)
      .setSecurityId(this.securityId)
      .setQuantity(this.orderQuantity)
      .setSide(side)
      .build();
    getOrderService().sendOrder(order); 
4
  }
}

The class MovService method contains the following items:

1

Gets references to settings defined in conf-mov.properties.

2

Once the strategy has reached the START live cycle phase subscribe to the security needed for this strategy

3

Construct an Order Value Object using the MarketOrderVOBuilder. The OrderVO contains a reference to the strategy the security, the account as well as the quantity and the order side.

4

Send the Order to the market via the OrderService

A typical Spring Configuration File looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:p="http://www.springframework.org/schema/p"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="movConfigParams" 
1
        class="ch.algotrader.config.spring.CustomConfigParamsFactoryBean" >
        <property name="global" ref="configParams"/>
        <property name="resource">
            <value>classpath:/conf-mov.properties</value>
        </property>
    </bean>

    <bean id="movEngine" 
2
        class="ch.algotrader.esper.EngineFactoryBean">
        <property name="strategyName" value="MOV"/>
        <property name="configResource" value="esper-mov.cfg.xml"/>
        <property name="configParams" ref="movConfigParams"/>
        <property name="initModules" value="test"/>
    </bean>

    <bean id="movService" 
3
        class="ch.algotrader.MovService" autowire="byName">
        <property name="strategyName" value="MOV"/>
        <property name="engine" ref="movEngine"/>
     </bean>
     
    <bean id="strategyQueue" 
4
        class="org.apache.activemq.command.ActiveMQQueue">
        <constructor-arg value="MOV.QUEUE" />
    </bean>

</beans>

This file contains the following Spring Bean Definitions:

1

contains a Map of all properties based on settings defined in conf-mov.properties

2

Creates the Esper Engine based on strategyName, configResource, configParams and initModules and optional runModules definitions

3

Creates the Strategy Service based on strategyName definition and engine reference. All dependencies of the ch.algotrader.service.StrategyService will be injected automatically through autowiring

4

used to receive strategy specific events in distributed mode, see Chapter 27, Events and Messaging

Execute the following Maven command to start a Maven build of the trading strategy:

mvn install

The Maven modules can now be deployed to a Maven repository (e.g. Sonatype Nexus) using:

mvn deploy

For further details regarding a maven deploy please visit the Maven deploy plugin page

Execute the following Docker command to create a Docker image of the trading strategy that can be used for productive deployments:

docker build -t xyz .

Please replace xyz with the name of your trading strategy.

The Docker image can now be pushed to a Docker repository (e.g. Docker Hub, Sonatype Nexus or Amazon ECR). For further details on pushing to a Docker repository please visit:

It is generally recommended to do Time-based Market Data Analysis and Signal Generation inside Esper Statements. Procedural actions like placing an order or subscribing to a Security are predominantly done inside Java Code.

The following two Starters are available to start a trading strategy in embedded and in distributed mode.

Strategy classes can provide listeners for life-cycle events in order to receive notifications about strategy life-cycle phase transitions and execute custom life-cycle dependent logic

@Override

public void onInit(final LifecycleEventVO event) {
    switch (event.getOperationMode()) {
        case SIMULATION:
            ...
            break;
        case REAL_TIME:
            ...
            break;
    }
}
@Override

public void onPrefeed(final LifecycleEventVO event) {
    switch (event.getOperationMode()) {
        case REAL_TIME:
            ...
            break;
    }
}

For further information on life-cycle events please visit Section 23.13, “Session life-cycle events”

A common use case is to wait for the full execution or cancellation of an order and then take some additional action.

Section 11.4.8.2, “Trade callback” explains how to use a TradeCallback for this purpose.

In this context it is also important to remember that when trying to close a position there might still be open orders associated with the corresponding security and strategy. It is suggested to cancel all corresponding orders, attach a TradeCallback to the cancellation and only close the position once all cancels have been confirmed.

Also keep in mind that an order might receive multiple fills in live trading. For example if one wants to send a Stop Order for each executed Order it is important to use the filled quantity and not on the original order quantity.

In handling partial fills the TargetPosition Algo can be useful, please see: Chapter 24, Execution Algos

Upon system startup strategies run through the life cycle phases as defined in Section 4.2, “Live Trading Mode”. At the same time market data connections are established. Due to the asynchronous nature of these two processes it is not predetermined which one will complete first. Typically within the START life cycle phase market data is subscribed. This however will fail if the market data adapter has not reached its SUBSCRIBED state by that time. To circumvent this issue the following Esper patter can be used, which waits for both the first LifecycePhaseVO and SessionEventVO to arrive before it fires:

select * 
from pattern[LifecycleEventVO(phase=LifecyclePhase.`START`) 
  and SessionEventVO(state=ConnectionState.SUBSCRIBED)];

Due to the expiring nature of Futures and Options, corresponding Positions have to be rolled prior to the Expiration Date. Typically, this would involve the following steps:

When dealing with futures on has to decide on when to roll from the Front-Month future into the Back-Month future. For this different philosophies exist:

  • Roll on a fixed day prior to the expiration date or the first notice date

  • Roll when the Back-Month future starts having a higher volatility than the Front-Month future

  • Use the constant maturity method as described in the following section

Since Futures have an expiration date and are therefore not continuous, it is often not possible to base indicators on them. There are several method for dealing with this situation:

  • use the raw data and ignore the fact that price time series will have jumps

  • On the rollover day, add the difference between yesterdays closing price of the Back-Month future and yesterdays closing price of the Front-Month future to the combined time series. Alternatively one can use the difference between today's opening price of the Back-Month future and yesterdays closing price of the Front-Month future if only the time series for the generic 1st future is available. This method can be used for P&L calculation it might however lead to negative prices on long time series.

  • Instead of using addition as mentioned in the previous item use multiplication. This method however cannot be used for P&L calculations. Depending on the indicator in use either addition or multiplication will be adequat.

  • Use the constant maturity method as described in the following section

Please see Section 6.6, “Multi Security Simulations” for options on how to back test futures and options based strategies that require multiple securities to be subscribed and unsubscribed during the back test.

With the constant maturity approach positions both in the Front-Month future as well as in the Back-Month future are established at all times. The weighening between the two positions is adjusted on a daily basis so that the combined time-to-expiration stays constant (e.g. 30 days)

With the following statements it is possible to calculate a constant maturity market value:

@Name('INSERT_INTO_FRONT_FUTURE_TICK')
insert into
  FrontFutureTick
select
  tick.*
from
  TickVO as tick unidirectional,
  method:lookupService.getSecurity(tick.securityId) as security
where
  cast(security.duration?, int) = 1;

@Name('INSERT_INTO_BACK_FUTURE_TICK')
insert into
  BackFutureTick
select
  tick.*
from
  TickVO as tick unidirectional,
  method:lookupService.getSecurity(tick.securityId) as security
where
  cast(security.duration?, int) = 2;

@Name('CONSTANT_MATURITY')
select
  ConstantMaturityUtil.getConstantMaturityValue(front, back) as value
from
  pattern [every(
    (front=FrontFutureTick -> (back=BackFutureTick where timer:within(1 hour)))
      or
    (back=BackFutureTick -> (front=FrontFutureTick where timer:within(1 hour)))
  )];

The actual static method to calculate the constant maturity value looks like this:

public static double getConstantMaturityValue(Tick frontTick, Tick backTick) {


  Future frontFuture = (Future) frontTick.getSecurity();
  Future backFuture = (Future) backTick.getSecurity();
  double weight = (double) (backFuture.getExpiration().getTime()
      - DateUtil.getCurrentEPTime().getTime() - Duration.MONTH_1.getValue()))
      / ((double) (backFuture.getExpiration().getTime()
      - frontFuture.getExpiration().getTime());
  return weight * frontTick.getCurrentValueDouble()
    + (1 - weight) * backTick.getCurrentValueDouble();
}

Sometimes one needs to compare output data files to verify a trading strategy or to find a problem during strategy development. CSV files can be imported into Microsoft Excel for the purpose of comparing them. Comparing two files however remains tricky and repeated exporting/importing and comparing in Excel can be cumbersome.

AlgoTrader provides a Java utility to compare ("diff") CSV files and assert the contents of the two files. For example the following statement compares a backtesting output file actual.csv with an expected result file expected.csv:

CsvDiff.diffAndLogAssertionErrors(fileDiffer, 

        new File("expected.csv"), new File("actual.csv")); 

The fileDiffer argument instructs the tool how to perform the diff operation. It can be constructed as follows:

FileDiffer fileDiffer = new FileDiffer(expectedDef, actualDef, differ);

In the above statement, expectedDef and actualDef define the columns of the two files; the differ gives exact instructions on how to perform the comparison of the files.

The columns of a CSV file are provided as implementations of CsvColumn which can be done with an enum:

public enum OrderReport implements CsvColumn {


    Date(new DateConverter("dd.MM.yyyy HH:mm:ss")),
    Instrument(SymbolConverter.INSTANCE),
    Side(StringConverter.INSTANCE),
    OrderType(StringConverter.INSTANCE),
    Size(LongConverter.INSTANCE),
    Limit(DoubleConverter.INSTANCE);
    
    private OrderReport(ValueConverter<?> converter) {
        this.converter = converter;
    }
    private final ValueConverter<?> converter;
    
    @Override
    public int index() {
        return ordinal();
    }
    
    @Override
    public ValueConverter<?> converter() {
        return converter;
    }
}

An example CSV file definition is then created as follows:

boolean hasHeaderLine = true;

CsvDefinition orderReportDef = new CsvDefinition(hasHeaderLine, OrderReport.values()); 

After defining the columns of both CSV files via CsvDefinition as indicated above we need to give instructions on how to perform the actual diff operation itself. In the simplest case we just compare the CSV files line by line and assert all or selected columns of the two files:

SimpleDiffer differ = new SimpleDiffer.Builder()

    .assertEqual(OrderReport.Date, OrderReport.Date)
    .assertEqual(OrderReport.Instrument, OrderReport.Instrument)
    .assertEqual(OrderReport.Side, OrderReport.Side)
    .assertEqual(OrderReport.Size, OrderReport.Size)
    .build();

The diff can also contain more complex instructions. For instance assume that we have exactly one BUY/SELL order per instrument and day but the ordering of instrument and side within a given date may be random. To align the correct rows for comparison, we must provide grouping and sorting hints for this case:

SimpleDiffer simpleDiffer = new SimpleDiffer.Builder()

    .assertEqual(OrderReport.Date, OrderReport.Date)
    .assertEqual(OrderReport.Instrument, OrderReport.Instrument)
    .assertEqual(OrderReport.Side, OrderReport.Side)
    .assertEqual(OrderReport.Size, OrderReport.Size)
    .build();
    
SortingDiffer sortingDiffer = new SortingDiffer.Builder()
    .add(OrderReport.Instrument, OrderReport.Instrument)
    .add(OrderReport.Side, OrderReport.Side)
    .build(simpleDiffer);
    
GroupDiffer differ = new GroupDiffer.Builder()
    .add(OrderReport.Date, OrderReport.Date)
    .build(sortingDiffer);

Now we are ready to run the CsvDiff.diffAndLogAssertionErrors(..) statement from above. The result may look similar to the following:

[..FAILED] 2 diffs at lines[exp/act]=45/51 in columns[exp/act]=[Size/Size] {exp-file=...}
[..FAILED] files: exp=OrderReport.csv, act=OrderReport_20141112_034538.csv
[..FAILED] lines: exp=45, act=51
[..FAILED] group: [13.01.2014]
[..FAILED]  ===========================================================================
[..FAILED]  NUM |   COLUMN(S)  |   EXPECTED   |    ACTUAL    | MESSAGE
[..FAILED]  ----|--------------|--------------|--------------|-------------------------
[..FAILED]  (1) | Size         |      2023769 |      2023762 | Values don't match ...
[..FAILED]  ----|----------------------------------------------------------------------
[..FAILED]  (1) | exp-line: [45] 13.01.2014 22:00:00,NYSE:PEP,BUY,Market,2023769,22.38
[..FAILED]      | act-line: [51] 13.01.2014 22:00:00,NYSE:PEP,BUY,Market,2023762,22.382 
[..FAILED]  ===========================================================================

AlgoTrader provides extensive support for strategy groups. In simulation mode multiple strategies can be run simultaneously as part of a strategy group. It is even possible to run multiple instances of the same strategy (with different parameters).

For this purpose AlgoTrader provides support for strategy and engine templates as well as strategy groups based on Spring XML configuration and abstract Spring beans. This enables strategy developers to define abstract templates for strategy engines and strategy services and then define concrete instances of those strategies with custom configuration.

Definition of strategy templates and engine templates typically look like this..


<bean id="boxServiceTemplate" class="ch.algotrader.strategy.box.BoxService" abstract="true">
    <property name="lookupService" ref="lookupService"/>
    <property name="marketDataCacheService" ref="marketDataCacheService"/>
    <property name="portfolioService" ref="portfolioService"/>
    <property name="positionService" ref="positionService"/>
    <property name="measurementService" ref="measurementService"/>
    <property name="orderService" ref="orderService"/>
    <property name="subscriptionService" ref="subscriptionService"/>
    <property name="commonConfig" ref="commonConfig"/>
</bean>

<bean id="boxEngineTemplate" class="ch.algotrader.esper.EngineFactoryBean" abstract="true">
    <property name="configResource" value="esper-box.cfg.xml"/>
    <property name="initModules" value="box-init"/>
    <property name="runModules" value="current-values, box-run"/>
</bean>

Based on these templates concrete strategy instances can be configured.


<at:strategy name="box-narrow"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate"
             resourceName="box-narrow.properties" />

<at:strategy name="box-wide"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate"
             resourceName="box-wide.properties" />

Only parameters configClass, engineTemplate, and serviceTemplate attributes are mandatory. Configuration can be further simplified if configuration resources follow the convention <strategy-name>.properties.


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:at="http://www.algotrader.ch/schema/spring/config"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
        http://www.algotrader.ch/schema/spring/config http://www.algotrader.ch/schema/spring/config/algotrader.xsd">

<at:strategy name="box-narrow"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate" />

<at:strategy name="box-wide"
             configClass="ch.algotrader.strategy.box.BoxConfig"
             engineTemplate="boxEngineTemplate"
             serviceTemplate="boxServiceTemplate" />

</beans>

Multiple strategy instances can be grouped together and assigned individual weights in the group. Strategy groups can executed as one unit using provided Spring profile.


<beans profile="simpleGroup">
    <at:strategyGroup id="simpleGroup">
        <at:strategyItem name="box-narrow" weight="0.2"/>
        <at:strategyItem name="box-wide" weight="0.8"/>
    </at:strategyGroup>
</beans>

To start the simpleGroup activate the Spring profile as follows:

-Dspring.profiles.active=...,simple

The AlgoTrader Eclipse client provides a visual editor for strategy groups, for further details please see Section 13.3.3, “AlgoTrader Configuration Editor”