Proof Of Concept Recap and Game Plan

With Proof Of Concept postings completed, I now have a better understanding of what it will take to develop my trading robot. I have demonstrated that it’s possible to perform the following:


- Use JRuby to interface with TWS API

- Get market quote

- Get historical quote

- Check account information (balance and open position info)

- Check order status

- Submit stock order (market and limit orders)

- Submit options order

- Submit complex options order




Along the way, I have also identified some limitations and glitches within TWS API and JRuby. Most of these can be overcome, but they make development harder. Here are just a few:


- JRuby error messages are not that readable and it can be difficult to determine where the error is coming from. So during development, it’s important to test early and often before too much coding is done.

- TWS API or JRuby upgrades may cause the application to break. Any decision to perform an upgrade should not be taken lightly and complete regression testing will be needed.

- If TWS pops up a dialog window, any API calls will be interrupted until the window is closed. The finished product will need a way to detect or prevent this from happening.

- openOrder() callback method sometimes get triggered more than once for unknown reasons.

- Retrieved data (such as market price) from TWS isn’t perfect. So it’s a good idea to build in some sort of redundant requests to make sure no trading decision is made because of bad pricing data.

- There is a need to be able to recover in the case where order is submitted and TWS crashes. Part of the recovery process would be to detect what orders have been submitted, and executed or not.


It is my intent to develop the program in three phases, with the actual trading robot phase being the last, and most difficult to create. The first phase of development will be a watch list of my investment portfolio that will issue buy/sell signals according to my plan. There is no trading automation in this phase, as these investments not in my Interactive Brokers account. The second phase will be the backbone of the Trading Robot. It will keep track of which positions I have in my Interactive Brokers account and provide a user interface to manage data. The following is a more detailed description of each phase:

Phase 1) Watcher - Ruby on Rails application which allows me to watch my investment portfolio (composed of various ETFs and mutual funds). It will get security prices at regular time intervals or on demand and will issue buy/sell signals according to my rules (mostly based on moving averages). Data will be stored in a database and pricing information will be gathered from various sources on the web. It will provide a web interface from which to edit watch list data.




Phase 2) Account Monitor - Extension of the above Rails application. It will store open positions and trading rules in the database. It will provide a web interface to control application parameters and display activities performed by Trading Robot.




Phase 3) Trading Robot - Separate JRuby application with connection to TWS and database. It will populate the database with data from TWS, apply trading rules, and perform trades when needed. It will run in a batch mode and not have a user interface. All of the Trading Robot's activities will be monitored through the Account Monitor.

Proof Of Concept 8 of 8g: Complex Options Order

Complex (or combination) option orders are orders composed of more than one leg for the same underlying security. They are submitted and executed as a single transaction and price. TWS complex orders allow for a maximum of up to four legs. Examples of different options strategies and corresponding legs are:

# of Legs	Strategies
2	vertical, diagonal, calendar, straddle, strangle
3	butterfly
4	iron condor, iron butterfly, double diagonal

In this coding example, I will create a market order to buy a vertical call spread which is composed of:
BUY 1 SPY AUG 50 CALL
SELL 1 SPY AUG 55 CALL

The steps of creating a complex order are:
1) Use EClientSocket.reqContractDetails() method to get contract id for each option leg.
2) Create a ComboLeg object for each option leg using contract id from step 1).
3) Create a Contract object and add ComboLeg objects created from step 2).
4) Create an Order object and submit it using Contract object from step 3).

# To change this template, choose Tools | Templates
# and open the template in the editor.
require 'java'
require 'TwsApi.jar'

include_class 'ib.client.EClientSocket'
include_class 'ib.client.Contract'
include_class 'ib.client.Order'
include_class 'ib.client.AnyWrapper'
include_class 'ib.client.EWrapper'
include_class 'ib.client.ComboLeg'

class TestComboOptionsOrder
  include EWrapper  #note "class TestMktData < EWrapper" syntax is no longer supported in JRuby 1.0

  attr_accessor :order_id, :contractArray
  @mySocket


  def initialize
    puts 'init'
    @mySocket = EClientSocket.new(self)
    @contractArray = Array.new  # array of hash
  end

  def eConnect
    @mySocket.eConnect("127.0.0.1", 7496, 0)
  end

  # Request all open orders that were placed from all API clients linked to one TWS,
  #  and also from the TWS
  #   This is a one time shot call. When its done you DO NOT continue to receive
  #   information about new placed orders
  def  reqAllOpenOrders
    @mySocket.reqAllOpenOrders
  end

  def submitOptionsOrder(order_id, symbol, call_put, expiry, strike,
                         buy_sell, qty, order_type, limit_price, tif)
    # --------------------------------
    myContract = Contract.new
    myContract.m_symbol    = symbol
    myContract.m_secType   = 'OPT'
    myContract.m_exchange  = 'SMART'
    myContract.m_currency  = 'USD'
    # options specific below ---
    myContract.m_expiry    = expiry   # string format YYYYMM
    myContract.m_right     = call_put # CALL, PUT
    myContract.m_strike    = strike   # number

    myOrder = Order.new
    myOrder.m_action         = buy_sell    # BUY, SELL
    myOrder.m_totalQuantity  = qty
    myOrder.m_orderType      = order_type # MKT, LMT, STP, SPLMT, TRAIL, etc
    myOrder.m_lmtPrice       = limit_price
    myOrder.m_tif            = tif        # DAY, GTC
    @mySocket.placeOrder(order_id, myContract, myOrder)
  end

  # new method to submit order
  def submitOrder(order_id, myContract, myOrder)
    @mySocket.placeOrder(order_id, myContract, myOrder)
  end

  def cancelOrder(order_id)
    @mySocket.cancelOrder(order_id)
  end

  # contract details will be received via the contractDetails() function on the EWrapper
  def reqContractDetails (req_id, contract)
    @mySocket.reqContractDetails(req_id, contract)
  end

  def eDisconnect
    @mySocket.eDisconnect
  end


  def contract_msg(contract)
    puts '    contract_msg:'
    puts '      conid        = ' + contract.m_conId.to_s
    puts '      symbol       = ' + contract.m_symbol
    puts '      secType      = ' + contract.m_secType
    puts '      exchange     = ' + contract.m_exchange.to_s
    puts '      currency     = ' + contract.m_currency
    puts '      localSymbol  = ' + contract.m_localSymbol.to_s
    puts '      multiplier   = ' + contract.m_multiplier.to_s
    puts '      expiry       = ' + contract.m_expiry.to_s
    puts '      strike       = ' + contract.m_strike.to_s
    puts '      right        = ' + contract.m_right.to_s
  end

  def contract_details_msg(contract_detail)
    puts '    contract_details_msg:'
    puts '      marketName   = ' + contract_detail.m_marketName.to_s
    puts '      tradingClass = ' + contract_detail.m_tradingClass.to_s
    puts '      minTick      = ' + contract_detail.m_minTick.to_s
    puts '      price magnifier = ' + contract_detail.m_priceMagnifier.to_s
    puts '      orderTypes      = ' + contract_detail.m_orderTypes.to_s
    puts '      validExchanges  = ' + contract_detail.m_validExchanges.to_s
    puts '      underConId      = ' + contract_detail.m_underConId.to_s
  end

  def order_msg(order)
    puts '    order_msg:'
    puts '      order id     =' + order.m_orderId.to_s
    puts '      client id    =' + order.m_clientId.to_s
    puts '      action       =' + order.m_action.to_s
    puts '      quantity     =' + order.m_totalQuantity.to_s
    puts '      type         =' + order.m_orderType.to_s
    puts '      lmtPrice     =' + order.m_lmtPrice.to_s
    puts '      TIF          =' + order.m_tif.to_s
    puts '      parent Id    =' + order.m_parentId.to_s
    puts '      permId       =' + order.m_permId.to_s


  end

  def order_state_msg(order_state)
    puts '    order_state_msg:'
    puts '      status        =' + order_state.m_status.to_s
    puts '      initMargin    =' + order_state.m_initMargin.to_s
    puts '      maintMargin   =' + order_state.m_maintMargin.to_s
    puts '      commission    =' + order_state.m_commission.to_s

  end

  #///////////////////////////////////////////////////////////////////////
  #// Interface methods
  #///////////////////////////////////////////////////////////////////////

  def tickPrice( ticker_id, field, price, can_auto_execute)
  end

  def tickSize(  ticker_id,  field,  size)
  end

  def tickOptionComputation( ticker_id,  field,  implied_vol,
                             delta,  model_price,  pv_dividend)
  end

  def tickGeneric( ticker_id,  tick_type,  value)
  end

  def tickString( ticker_id,  tick_type,  value)
  end

  def tickEFP( ticker_id,  tick_type,  basis_points,
               formatted_basis_points,  implied_future,  hold_days,
               future_expiry,  dividend_impact,  dividends_to_expiry)
  end

  def orderStatus(  order_id,  status,  filled,  remaining,
                    avg_fill_price,  perm_id,  parent_id,  last_fill_price,
                    client_id,  why_held)
    puts '~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'
    puts 'orderStatus>: orderId=' + order_id.to_s + ', clientId=' + client_id.to_s + ', permId=' + perm_id.to_s +
         ', status=' + status + ' filled=' + filled.to_s + ', remaining=' + remaining.to_s + ', '
    puts '              avgFillPrice=' + avg_fill_price.to_s + ', lastFillPrice=' + last_fill_price.to_s +
        ', parent Id=' + parent_id.to_s  + ', whyHeld=' + why_held.to_s
  end

  def openOrder(  order_id,  contract,  order,  order_state)
    puts '--------------------------------------------------------'
    puts 'openOrder> open order: orderId=' + order_id.to_s + ' --- '
    contract_msg(contract)
    order_msg(order)
    order_state_msg(order_state)
  end

  def updateAccountValue( key,  value,  currency,  account_name)
  end

  def updatePortfolio( contract,  position,  market_price,  market_value,
                         average_cost,  unrealized_pnl,  realized_pnl,  account_name)
  end

  def updateAccountTime( time_stamp)
  end

  # triggered right after logging into TWS
  def nextValidId(  order_id)
    @order_id = order_id   #storing order id to be used when placing order
    puts 'nextValidId>  Next Valid Order Id = ' + @order_id.to_s
  end

  # call back with details.  determine which request it was and store details
  #  data with the associated request.
  def contractDetails( req_id,  contract_details)
    aContract = contract_details.m_summary
    # find the hash with matching :req_id and set :conid
    @contractArray.find{ |contract| contract[:req_id] == req_id } [:conid] = aContract.m_conId

    puts  'contractDetails()>  reqId = ' + req_id.to_s + ' -------------------'
    contract_msg(aContract)
    contract_details_msg(contract_details)
    puts  ' --------'
  end

  def bondContractDetails( req_id, contract_details)
  end

  def contractDetailsEnd( req_id)
    puts 'contractDetailsEnd() ' + req_id.to_s + '>  Ended ------------------------'
  end

  def execDetails(  order_id,  contract,  execution)
  end

  def updateMktDepth(  ticker_id,  position,  operation,  side,  price,  size)
  end

  def updateMktDepthL2(  ticker_id,  position,  market_maker,  operation,
                         side,  price,  size)
  end

  def updateNewsBulletin(  msg_id,  msgType,  message,  orig_exchange)
  end

  def managedAccounts(  accounts_list)
  end

  def receiveFA( fa_data_type,  xml)
  end

  def historicalData( req_id,  date,  open,  high,  low,
                 close,  volume,  count,  eWAP,  has_gaps)
  end

  def scannerParameters( xml)
  end

  def scannerData( req_id,  rank, contract_details,  distance,
                   benchmark,  projection,  legs_str)
  end

  def scannerDataEnd( req_id)
  end

  def realtimeBar( req_id, time,  open,  high,  low,  close,  volume,  wap,  count)
  end

  def currentTime( time)
  end

  def fundamentalData( req_id,  data)
  end

  # ---------------------------------------------------
  #  new callback methods added in API 9.60
  # ---------------------------------------------------
  def openOrderEnd()  # invoked when all orders are sent to a client as a response to reqOpenOrders(), 
                      #   reqAllOpenOrders  and right after connected to TWS
    puts 'openOrderEnd>  Ended ------------------------'
  end

  def accountDownloadEnd(account_name) #invoked when a complete snapshot of an account state is sent to a client as a response to reqAccountUpdates()
  end

  def execDetailsEnd( req_id) # invoked when all executions are sent to a client as a response to reqExecutions()
  end

  def deltaNeutralValidation(req_id, under_comp)  # for Delta-Neutral DN RFQ
  end

  # ---------------------------------------------------
  # -- methods from AnyWrapper  -- must be declared
  # ---------------------------------------------------

  def connectionClosed()
    puts ' [API.connectionClosed] Closed connection with TWS'
  end

  def error(err_obj_str)
     puts ' [API.msg1] ' + err_obj_str               if err_obj_str.is_a?(String)
     puts ' [API.msg3] ' + err_obj_str.getMessage()  if err_obj_str.is_a?(Exception)
  end

  def error(one, two, str)
     puts ' [API.msg2] ' + str + ' {' + one.to_s + ', ' + two.to_s + '}'
  end

end  # class

# *********************************************************************************************
#  Running the code
# *********************************************************************************************
  x = TestComboOptionsOrder.new()
  begin
      x.eConnect
      puts 'connected ...'
      sleep(2)   # need this here to give time for nextValidId() to be called
      puts '******************************'
      puts '**  STEP 1)  ******************'
      puts '.. Defining contract for Aug 90 C '
      spyAug90c             = Contract.new
      spyAug90c.m_symbol    = 'SPY'
      spyAug90c.m_secType   = 'OPT'
      spyAug90c.m_exchange  = 'SMART'
      spyAug90c.m_currency  = 'USD'
      spyAug90c.m_expiry    = '20090821'   # string format YYYYMM or YYYYMMDD -- if YYYYMM used and normal and quarterly
                                           #  options exist, will return contract details for both
      spyAug90c.m_right     = 'CALL' # CALL, PUT
      spyAug90c.m_strike    = 90   # number

      puts '.. Adding Aug 90 C to contract array'
      request_id   = 1
      x.contractArray << {:req_id => request_id, :contract => spyAug90c, :conid => '' }

      puts '.. Requesting contract detail and id info'
      x.reqContractDetails(request_id, spyAug90c)
      sleep(5);
      puts ' '
      puts '.. Defining contract for Aug 95 C '
      spyAug95c             = Contract.new
      spyAug95c.m_symbol    = 'SPY'
      spyAug95c.m_secType   = 'OPT'
      spyAug95c.m_exchange  = 'SMART'
      spyAug95c.m_currency  = 'USD'
      spyAug95c.m_expiry    = '20090821'   # string format YYYYMM or YYYYMMDD -- if YYYYMM used and normal and quarterly
                                           #  options exist, will return contract details for both
      spyAug95c.m_right     = 'CALL' # CALL, PUT
      spyAug95c.m_strike    = 95   # number
      puts '.. Adding Jul 95 C to contract array'
      request_id   = 2
      x.contractArray << {:req_id => request_id, :contract => spyAug95c, :conid => '' }
      puts 'Requesting contract detail and id info'
      x.reqContractDetails(request_id, spyAug95c)
      sleep(5);

      puts '******************************'
      puts '**  STEP 2  ******************'
      puts '.. Now have info on both contracts, create combo leg for each'
      puts '.. Create leg #1'
      leg1      = ComboLeg.new(x.contractArray.find{ |id| id[:req_id] == 1 } [:conid], # contract id
                               1,          # ratio
                               'BUY',      # BUY or SELL
                               'SMART',    # p_exchange
                               0           # p_openClose- always 0 for retail customers
                               )
      puts '.. Create leg #2'
      leg2      = ComboLeg.new(x.contractArray.find{ |id| id[:req_id] == 2 } [:conid], # contract id
                               1,          # ratio
                               'SELL',      # BUY or SELL
                               'SMART',    # p_exchange
                               0           # p_openClose- always 0 for retail customers
                               )
      puts '******************************'
      puts '**  STEP 3  ******************'
      puts '.. Create Contract'

      myContract  = Contract.new
      myContract.m_symbol    = 'SPY'
      myContract.m_secType   = 'BAG'   # BAG is a combination type
      myContract.m_exchange  = 'SMART'
      myContract.m_currency  = 'USD'

      puts '.. Add leg 1 to combo contract'
      myContract.m_comboLegs.add(leg1)
      puts '.. Add leg 2 to combo contract'
      myContract.m_comboLegs.add(leg2)
      puts '******************************'
      puts '**  STEP 4  ******************'
      puts '.. Create combo Order'
      myOrder = Order.new
      myOrder.m_action         = 'BUY'    # BUY, SELL
      myOrder.m_totalQuantity  = 1
      myOrder.m_orderType      = 'MKT' # MKT, LMT, STP, SPLMT, TRAIL, etc
      myOrder.m_tif            = 'DAY'        # DAY, GTC

      puts '.. submitting order'
      x.submitOrder(x.order_id += 1, myContract, myOrder)
      sleep(5);
      puts '==========================='
      puts '*** requesting open orders ***'
      x.reqAllOpenOrders
      sleep(5)
      x.eDisconnect
      puts 'disconnected'
  rescue  Exception => e
      puts 'Exception ' + e.message
      x.eDisconnect
  end

Running the code produces the following output:


init
Server Version:44TWS Time at connection:20090712 16:51:47 ESTconnected ...
nextValidId> Next Valid Order Id = 81
openOrderEnd> Ended ------------------------
[API.msg2] Market data farm connection is OK:usopt {-1, 2104}
[API.msg2] Market data farm connection is OK:usfarm {-1, 2104}
******************************
** STEP 1) ******************
.. Defining contract for Aug 90 C
.. Adding Aug 90 C to contract array
.. Requesting contract detail and id info
contractDetails()> reqId = 1 -------------------
contract_msg:
conid = 60388362
symbol = SPY
secType = OPT
exchange = SMART
currency = USD
localSymbol = SWGHL
multiplier = 100
expiry = 20090821
strike = 90.0
right = C
contract_details_msg:
marketName = SWG
tradingClass = SWG
minTick = 0.01
price magnifier = 1
orderTypes = ADJUST,ALERT,ALGO,ALLOC,AON,AVGCOST,BASKET,COND,CONDORDER,DAY,DEACT,DEACTEOD,FOK,GAT,GTC,GTD,GTT,HID,HPENNY,ICE,IOC,LIT,LMT,MIT,MKT,MTL,NONALGO,OCA,PAON,RELSTK,SCALE,SCALERST,STP,STPLMT,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,VOLAT,WHATIF,
validExchanges = SMART,AMEX,BOX,CBOE,IBSX,ISE,MIBSX,NASDAQOM,PHLX,PSE
underConId = 756733
--------
contractDetailsEnd() 1> Ended ------------------------

.. Defining contract for Aug 95 C
.. Adding Jul 95 C to contract array
Requesting contract detail and id info
contractDetails()> reqId = 2 -------------------
contract_msg:
conid = 60388382
symbol = SPY
secType = OPT
exchange = SMART
currency = USD
localSymbol = SWGHQ
multiplier = 100
expiry = 20090821
strike = 95.0
right = C
contract_details_msg:
marketName = SWG
tradingClass = SWG
minTick = 0.01
price magnifier = 1
orderTypes = ADJUST,ALERT,ALGO,ALLOC,AON,AVGCOST,BASKET,COND,CONDORDER,DAY,DEACT,DEACTEOD,FOK,GAT,GTC,GTD,GTT,HID,HPENNY,ICE,IOC,LIT,LMT,MIT,MKT,MTL,NONALGO,OCA,PAON,RELSTK,SCALE,SCALERST,STP,STPLMT,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,VOLAT,WHATIF,
validExchanges = SMART,AMEX,BOX,CBOE,IBSX,ISE,MIBSX,NASDAQOM,PHLX,PSE
underConId = 756733
--------
contractDetailsEnd() 2> Ended ------------------------
******************************
** STEP 2 ******************
.. Now have info on both contracts, create combo leg for each
.. Create leg #1
.. Create leg #2
******************************
** STEP 3 ******************
.. Create Contract
.. Add leg 1 to combo contract
.. Add leg 2 to combo contract
******************************
** STEP 4 ******************
.. Create combo Order
.. submitting order
[API.msg2] Order Message:
Warning: your order will not be placed at the exchange until 2009-07-13 09:30:00 US/Eastern {82, 399}
--------------------------------------------------------
openOrder> open order: orderId=82 ---
contract_msg:
conid = 28812380
symbol = SPY
secType = BAG
exchange = SMART
currency = USD
localSymbol = SPY
multiplier =
expiry =
strike = 0.0
right = ?
order_msg:
order id =82
client id =0
action =BUY
quantity =1
type =MKT
lmtPrice =0.0
TIF =DAY
parent Id =0
permId =1009111242
order_state_msg:
status =PreSubmitted
initMargin =
maintMargin =
commission =1.7976931348623157e+308
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
orderStatus>: orderId=82, clientId=0, permId=1009111242, status=PreSubmitted filled=0, remaining=1,
avgFillPrice=0.0, lastFillPrice=0.0, parent Id=0, whyHeld=
===========================
*** requesting open orders ***
--------------------------------------------------------
openOrder> open order: orderId=82 ---
contract_msg:
conid = 28812380
symbol = SPY
secType = BAG
exchange = SMART
currency = USD
localSymbol = SPY
multiplier =
expiry =
strike = 0.0
right = ?
order_msg:
order id =82
client id =0
action =BUY
quantity =1
type =MKT
lmtPrice =0.0
TIF =DAY
parent Id =0
permId =1009111242
order_state_msg:
status =PreSubmitted
initMargin =
maintMargin =
commission =1.7976931348623157e+308
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
orderStatus>: orderId=82, clientId=0, permId=1009111242, status=PreSubmitted filled=0, remaining=1,
avgFillPrice=0.0, lastFillPrice=0.0, parent Id=0, whyHeld=
openOrderEnd> Ended ------------------------
disconnected

I ran the code after hours and was able to confirm that the order had been placed in TWS.


To see the individual legs of the order, just right click on the order and select 'Show Legs'



Individual legs will be displayed.



When the market opens the next day, my vertical call spread order executed at market price.



That's all there is to it. Complex option orders are not all that complex after all.

Progress Update...

I finally have time to get back to working on my coding. I got a complex options order working today (a bull call spread). It'll take me a few days to clean up and document the code though.

Stay tuned....