ch-future-monitor

TODO

• Coger Cartera
• Operar en source
  • Si cambian marcado de Cartera, se pensaría que no la ha hecho y abriría infinitos contratos

Maintenance

• Create Symbols:
  • activeFrom: First day, 03:00:00 GMT (+1 second after previous Symbol).
  • activeTo: Last day, 02:59:59 GMT.
  • INSERT INTO "Symbols" ("name", "slug", "key", "type", "activeFrom", "activeTo", "createdAt", "updatedAt") VALUES
• Create Holydays.
  • Day, 0:00:00 GMT
  • http://www.meff.es/esp/Calendario
  • http://www.bolsamadrid.es/esp/aspx/Inversores/Agenda/Calendario.aspx
  • node dev/var/get-year-weekens.js 2018
  • INSERT INTO "Holidays" ("date", "createdAt", "updatedAt") VALUES
  • (1515196800, '2018-10-03 01:39:00+02', '2018-10-03 01:39:00+02'),
  • (1515283200, '2018-10-03 01:39:00+02', '2018-10-03 01:39:00+02');
  • Also special holidays (in week).

Info

The "Etter" protocol

• Mo: { "command": "start", "sid": 1 }

• Ap: { "command": "start", "sid": 1, "id": xxx, "key": xxx, "source": "gvc", "version": "1.0.0", "type": "getter", "category": "meff", "subcategory": "all" }

• Mo: { "command": "stocks", "sid": 2 }

• Ap: { "command": "stocks", "sid": 2, "items": [] }

• Mo: { "command": "start", "sid": 1 }

• Ap: { "command": "start", "sid": 1, "id": xxx, "key": xxx, "source": "gvc", "version": "1.0.0", "type": "better", "money": 77.64 }

• Error responses for "start", "events" and "results":

• Ap: { "command": "error", "sid": 2, "code": 1, "value": "Error" }

• Ap: { "command": "error", "sid": 2, "code": 5, "value": "Busy" }

Ticks docbase

• Only is stored data that is different from previous one.
• It's a Capped collection.
  • Get all ordered by Date, from older to newer:
    • All: "Tick.find()".
    • Symbol: "Tick.find({ si: 21 })".
  • Get all ordered by Date, from newer to older:
    • All: "Tick.find().sort({ $natural: -1 })".
    • Symbol: Not efficient.
  • Get last by Date:
    • All: "Tick.findOne().sort({ $natural: -1 })".
    • Symbol:
      • "Tick.countDocuments({ si: 21 })".
      • "Tick.findOne({ si: 21 }).skip( - 1)".

Indicators

• "Date":
  • From server.
  • To limit and sort data.
• "Time"
  • From data.
  • To exactly define date of data.
• "Date" and "Time" are joined, so it can't work near the change of day, due to differences in clock and timezone.
• "Bar-date" is the start of the bar: "17:24" 17:24:00 - 17:24:59
  • VChart names same period as next one: "17:25" 17:24:00 - 17:24:59
• Assumes that all non-holiday days have same open market hours: 08:00-20:00.
• It allows bars of up to 1 seconds.

Algorithms

• Each algorithm has isolated Events, Orders and Contracts.
• All do simulated operative.
• Real one also does real operative, isolated from it's own simulation.
• Both, live and past, executes with delay of some seconds to ensure data is valid.
  • First seconds data are lost (except Price).
  • Occasionally, valid data comes still later than delay, and this creates some minor time-data desynchronization.
  • After a delay in live, all previous bars are executed at same time, with data of last one (the more recent).
• Uninformed timeout marks operation-data as invalid when lack of update.
  • Communication problem in App, Source, ...
  • Assumes data is constantly updated in Source.
  • Price remains to minimize effect on Indicators, but it's also invalid.
• Scene: { date: {Integer} Seconds, Source clock. isLive: {Boolean} True: Data just getted from Source. False: Don't do real operation. indiSymbol/operSymbol: { Indicators Symbol data. indi: {Object} Indicators value. data: { price: {Float} Quotation price. buyTitles: {Integer} Stock buy titles. buyPrice: {Float} Stock buy price. sellTitles: {Integer} Stock sell titles. sellPrice: {Float} Stock sell price. } isUninformed: {Boolean} True: 'indi' and 'data' are Null. } }

Symbols

• There are two types of symbols: To get indicators from, and to operate with.
• Symbols are auto changed by date (requires restart application).
  • "Ibex July 2018" -> "Ibex September 2018".
• Indicators are created for each symbol.
  • Several for Simulation.
  • One for Live.

Operative

• Events, Orders and Contracts are used to determine the state.
• Orders and Contracts data (in database or cache) are updated each cron-second with Source Portfolio data.
• If Portfolio is suddenly empty, for example due to an Etter error, database data is not affected.
• Changes not controlled by the App are compatible and updated in Orders/Contracts.
  • Operate manually in Source.
  • Market automatic order-cancellation or contract-closement.

Development

Installation

• Install NodeJS v8.6.0
• Install PostgreSQL 9.5 and create database
• Install MongoDB
• Global dependencies:

  npm install -g foreman
  

• npm install.

Configuration

Env file "/.env"

NODE_ENV=desarrollito
PORT=5010

DEBUG=1
CRON_FREQUENCY=1
DATABASE_URL=postgres://xxx:xxx@localhost:5433/ch-future-monitor
MONGODB_URI=mongodb://xxx:xxx@localhost:27017/ch-future-monitor
MONGOLAB_COPPER_URI=mongodb://xxx:xxx@localhost:27017/ch-future-ticks
TICKS_DB_URI=mongodb://xxx:xxx@localhost:27017/tmp
OUT_PATH=C:\tmp\future

REAL_ALGORITHM=v3

• DEBUG
  • 1: -
  • 2: - and verbose Get-stocks.
• REAL_ALGORITHM
  • To specify the algorithm that does real operative.

Execution

nf start

Ticks docbase

Backup

• mongoexport -h <host>:<port> -d <name> -c ticks --sort '{$natural: 1}' -u <user> -o <path>/ticks.json.
• mongoimport -h <host>:<port> -d <name> -c ticks --maintainInsertionOrder -u <user> --file <path>/ticks.json.

Build

1. Get VChart file, generated when loading "ticks" symbol inside VChart app.
2. Get existing data from production db:
   • mongoexport -h <host>:<port> -d <name-pro> -c ticks -u <user> -o <path>/<file-existing>.json.
3. Empty temporal db.
4. Add existing data to temporal db (different symbol that VChart):
   • mongoimport -h localhost:27017 -d <name-tmp> -c ticks -u <user> --file <path>/<file-existing>.json.
5. Add VChart data to temporal db, for each symbol (no matters the order. Symbol-key must exist in Postgre):
   • node dev/ticks-db/add-vchart.js <path>/<symbol-key>.tck <symbol-key>.
6. Create index on temporal db:
   • Name: d
   • Keys: { "d" : 1 }
7. Get new data, ordered by date:
   • mongoexport -h localhost:27017 -d <name-tmp> -c ticks --sort '{d: 1}' -u <user> -o <path>/<file-new>.json.
8. Empty production db (must be already auto-created by the app).
9. Add new data to production db, ordered by date:
   • mongoimport -h <host>:<port> -d <name-pro> -c ticks --maintainInsertionOrder -u <user> --file <path>/<file-new>.json.