ch-future-trader

Info

Simple trading

• Algorithms:
  • Only 1 operation per day
  • Just few operations per month
  • Low benefit but crescent
• Several assets
• There isn't real-time orders, real-time data monitoring, ...
• All orders are sent together just when market opens:
  • Enter, stop-loss, exit

Groups

• Schedule:
  • All Assets in a group, open/close at same time
  • Close before next-group opening, with some minutes of distance
  • Uses city as timezone
  • Need to be calculated each session (using day, month and year), to handle Daylight Saving Time

Orders

• Sending:
  • At market opening
• Filling:
  • Enter:
    • At market opening
  • Exit:
    • At market closing
    • Stop-loss
• Enter is always with exact price:
  • Always at Current-market-price
  • Always with a Limit order
  • Because Ibkr doesn't allow Stop-loss with price relative to filled Enter-price:
    • So a Stop-loss with fixed price is used
  • Prevent sending orders that will enter and exit at same time

Algorithms

• Bars are "1 day" size
• Can't be used because can't be backtested:
  • Several enter/exit per day
  • Enter some time after market opens
  • Exit some time before market closes
  • Dynamic/trailing stop-loss
  • Take-profit:
    • If both take-profit and stop-loss can fill, can't know what happens first
• Algorithm development:
  • A Combi is:
    • Parametric combination of an Algorithm:
      • For example: Stop-loss: 20, Ema-period: 30, ...
    • An instance with local variables, to be own to each Asset
    • Slug safety identifies a Combi (Algorithm-slug + Mods)
  • Enter-price is specified in a range: higher/lower/any
    • If Current-market-price is not inside range, trading is ignored
  • Stop-loss can be specified:
    • Relative to filled Enter-price
    • Fixed price
  • Prices don't need to be in tick-size
  • "processToday" is called just at order-sending: just at market-opening
• Assets have different size (price, multiplier, ...):
  • An algorithm must work in all Assets

Regular Trading Hours (RTH)

• Regular = Liquid; VS total
• Both bars and orders are outside-and-inside-RTH
• Benefits:
  • We got better results in Backtest
  • Same for all Assets (Ibkr doesn't offer RTH in DTB-exchange Assets)

Session

• 22:00 UTC:
  • Change of session: 07Mar2019 -> 08Mar2019
  • Used in "util.getCurrentSession" and "cpp.getSessionBars"

Bars

• "17:00" (bar.date): 17:00-17:29 (real)
• Bar starts to exist just after market-opening
• Types of bar:
  • Finalized:
    • All data is definitive
    • When bar is not from current-session
  • Live:
    • Data is being updated
    • When bar is from current-session
• Current-market-price:
  • Finalized bars: "open" field
  • Live bar: "close" field
    • Sometimes Api takes 1 minute to respond, but "close" field is the one at time of responding (it's a correct value)
• If an Asset started to exist (in Exchange/Broker market-data) for less than 2 years ago from now:
  • A null-bar must be manually created in database:
    • Being first bar in TWS: 6 May 2019
    • The created bar must for session: 5 May 2019

Assets

• Nominal = Price (quotation) * Multiplier
• Tick-size:
  • Minimum price variation:
    • For example "0.25":
      • 1408.673 -> 1408.75
      • Possible values: 1.00, 1.25, 1.50, 1.75, 2.00, ...
• Continuous assets are used:
  • To know current expiry:
    • Continuous points internally to the current expiry
    • Change to next expiry is done 3 days before last day
  • Avoid knowing details about each expiry: start-ending dates, holidays, symbol, ...
  • Avoid joining data of several expiries in order to fill indicators with large periods
  • Continuous-prices don't exactly match with expiry-prices, but it's not a problem:
    • Some algorithms are already modifying this value ("price-margin")
• Trading-hours are requested in order to know special schedules of an expiry:
  • For example, in Christmas MEFF closes at 14h
• Compatible with any kind of future asset:
  • Index, forex, material, ...
  • For example: Eur/Usd, Gas, ...
  • Large prices are supported, such as: 0.009190
• DB content requirements:
  • No volatility assets
  • Only indexes:
    • At least 20 component companies
    • From at least 4 different sectors
  • With volume:
    • Last 3 months
    • Outside-and-inside-RTH
    • If Asset has monthly expiries: Check a regular (non-quarterly) expiry, such as: Jan, Feb, Apr, ...
  • Tick-size: Additional 0's matching TWS number of decimals
  • Several sizes of same underlying (plus, mini, ...)
  • Several exchanges of same underlying:
    • For example: "Nikkei Ose", "Nikkei Globex"
  • Has volume in non-quarterly expiries
  • All products listed at Ibkr site has been studied, as 29-Jan-2020

Float precision

• Javascript has precision errors working with decimal numbers
• This error can affect App calculations and roundings
• "decimal.js" library is used to solve this
• Uses precision "20":
  • Assets price needs up to "7"
  • Calculation of Ptt needs up to "12":
    • "Per ten thousand" of Asset price can increase it by "5"
    • 7 + 5 = 12
• Decimal objets:
  • Created with string parameter, to avoid unprecision:
    • x = new Decimal('123.456')
  • Inmutable: Not changed by its methods
  • Methods can be chained:
    • x.dividedBy('2').plus('7')

Theoric filling

• Ignored in Covid Sessions: 24Feb2020-24May2020
• Theoric-filling doesn't exactly match with real-filling:
  • But difference is tiny
• Limit order (Enter):
  • Theoric:
    • Order is filled when market-price is equal to limit-price
  • Real:
    • Order may (or not) be filled if ask/bid-price is equal to limit-price
    • Order is guaranteed to be filled if ask/bid-price surpasses limit-price
• Market order (Exit-at-closing, Stop-loss):
  • Theoric:
    • Filled at market-price
  • Real:
    • Filled at ask/bid-price
• Market movement:
  • Theoric:
    • Market-price doesn't move
  • Real:
    • Ask/bid-price moves
    • Order can never fill ("losing the market")
    • Order can fill at different price
• Filling time is different (price is different):
  • Enter:
    • Theoric: Just at opening
    • Real: Some seconds after opening
  • Exit-at-closing:
    • Theoric: When settlement-price already applied
    • Real: Some minutes before market closes

Blocking

• Groups schedule don't overlap
• "Non-blocking": All Assets can order-sending in same Session
• "Blocking" (per group):
  • Only 1 Asset can order-sending per Session
  • N-positions to use all bank money

Reports

• Combi-parameters with same Slug (e.g. "breakMargin") in different Algorithms, are considered the same parameter
• Reports:
  • List (Non-blocking)
  • List (Blocking)
  • Full-charts (Blocking)
  • Simple-charts (Non-blocking): To see if Assets do strange things, like big changes in small amount of time
  • Simple-charts (Blocking): To see how Assets are blocked
  • Trace (Non-blocking): To see how Algorithm works
  • Trace (Blocking): To see what happens each day
• Phases generation:
  • 1:
    • List (Non-blocking)
  • 2:
    • List (Non-blocking)
    • List (Blocking)
  • 3:
    • List (Non-blocking)
    • List (Blocking)
    • Full-charts (Blocking)
  • 4:
    • List (Non-blocking)
    • List (Blocking)
    • Full-charts (Blocking)
    • Simple-charts (Non-blocking)
    • Simple-charts (Blocking)
  • 5:
    • List (Non-blocking)
    • List (Blocking)
    • Full-charts (Blocking)
    • Simple-charts (Non-blocking)
    • Simple-charts (Blocking)
    • Trace (Non-blocking)
    • Trace (Blocking)
  • 6: Sent by Notification
    • Full-charts (Blocking)
    • Trace (Blocking)

Libs

• Broker:
  • Handles order of arguments, expected responses, filter responses
  • Allowed types for input/output parameters:
    • Strings (numeric parameters must be sent as strings)
    • Boolean
  • All prices as input parameters (that will be sent to Api) must be tick-sized:
    • For example prices in orders
• Broker-execute:
  • Handles communication with C++ program
  • But Ibkr Api adaptation, for example when several Api calls are needed, is handled by C++ itself
  • A C++ program execution can only perform 1 Api request:
    • This means normally 1 Api call
    • But sometimes can be 2 Api calls, for example "reqHistoricalData" and "cancelHistoricalData"
  • Handles timeout
  • Ibkr Api used is "no-SSL":
    • Gateway is in same machine, so it's a local connection
• Broker-queue:
  • Handles:
    • Api Client-id
    • Gateway timming limitations
  • Parallel executions, but not started at same time
  • Uses all Client-ids, except "0"
• Backtest:
  • Several algorithms
  • All historical data
  • Output results
• Operate:
  • 1 algorithm
  • Today
  • Send orders
• Algorithm:
  • Creates Combis
  • Handles communication with Combis
  • Strategy price is a range
  • Planning generation (processToday):
    • Data from Algorithm (Strategy) is checked to be valid
    • Returned data (Planning) is sanitized/definitive:
      • Can be used directly, without checking/modifying content
    • "Range" is translated to "exact":
      • Price is moved to Current-market-price, to achieve an instant filling
      • If Current-market-price is not inside range, trading is ignored
    • Relative Stop-loss is converted to fixed
    • Prices are converted to tick-size
• Indicator:
  • Wraps "technicalindicators" package
  • Validates and sanitizes input/output data
• Bar:
  • Build day bars with custom schedule
• Util:
  • Miscellaneous functions useful for rest of App

Ibkr Gateway configuration

• Login screen:
  • Select API type: IB API
  • Select language: English
  • Region: Europe
  • Trading Mode: Live Trading
  • ??? Settings directory
  • Time Zone: UCT
  • Use SSL: Checked
  • Verbose logging: Unchecked
  • Host: Empty
  • Port: Empty
  • No internet connectivity: Unchecked
  • Connect through private WAN: Unchecked
• Configure menu:
  • Automatic Encryption: Checked
  • Settings:
    • General:
      • ??? Memory Allocation
    • API:
      • Settings:
        • Enable DDE clientes: Unchecked
        • Socket port:
          • Live: 7496 or 4001
          • Simulated: 7497 or 4002
        • Allow connections from localhost only: Checked

Development

Notes

• DB error handling:
  • Use just 1 "method", "then" and "catch" per line:
    • Model.findAll.then().catch();
    • Model.save.then().catch();
  • If error, code inside "then" is not called:
    • For example, a callback is not called
  • Catch "err" may be null, so ensure callback is called with an error
• Async:
  • Loops ("async.each", ...):
    • A cb('error') will stop remaining iterations
  • Prevent stack overflows and App-event-loop blocking:
    • each(coll, async.ensureAsync(function(item, cb) { cb(); }), done)
    • series([async.ensureAsync(function(cb) { cb(); })], done)
• Puppeteer:
  • Is configured with "--no-sandbox"
  • So only trust content must be used

Maintenance

• Year 2038 problem: timestamp 32-bit integer

Installation

• Install NodeJS v12.17.0
• Install PostgreSQL 11 and create database
• Install C++ compiler ("Visual Studio Build Tools" on Windows)
• Install Interactive-brokers "IB Gateway 972" / "Trader Workstation 972"
• Global dependencies:

  npm install -g foreman
  

• Project dependencies:

  npm install
  

Configuration

Env file "/.env"

NODE_ENV=desarrollito
NODE_OPTIONS=--max-old-space-size=4096

DATABASE_URL=postgres://xxx:xxx@localhost:5432/ch-future-trader
NOTIFICATION_TOKEN=
NOTIFICATION_CHATID=
QUANTY_HOST=
QUANTY_SSH_PRIVKEY=
METY_TOKEN=
BACKTEST_PHASE=1

LOG_BROKER=1
LOG_BROKER_EXECUTE=1
LOG_BROKER_QUEUE=1
LOG_BACKTEST=1
LOG_ALGORITHM=1
LOG_OPERATE=1
LOG_INDICATOR=1
LOG_BAR=1
LOG_QUANTY=1
LOG_QUANTY_EXECUTE=1
LOG_METY=1
LOG_METY_EXECUTE=1
LOG_APP_EXECUTE=1

• NODE_OPTIONS:
  • Unset in Production
• BACKTEST_PHASE:
  • See "Reports.Phases"
  • 1, 2, 3, 4, 5, 6

Execution

nf run node app.js backtest
nf run node app.js operate <group>
nf run node app.js status
nf run node app.js clear

• backtest: Execute a Backtest
• operate: Send Order to Gateway
• status: Send Notification with Gateway Status
• clear: Send Clear to Gateway

Broker Compilation

Windows (using "Visual Studio Developer Command Prompt" in project root directory):

nmake /F makefile.win > tmp\broker-compile.log

Deploy to Production

Hosting

• git push master

Broker

• Set version in "makefile.win" and "makefile"
• Set version in "config/config.js"
• Delete old version in "bin/"
• Execute in project root directory:

Windows (using "Visual Studio Developer Command Prompt"):

nmake /F makefile.win clean
nmake /F makefile.win pro > tmp\broker-compile.log

Linux:

make pro > tmp/broker-compile.log 2>&1