Thursday, July 23, 2009

Proof Of Concept Recap and Game Plan

With Proof Of Concept postings completed, I now have a better understanding of what it will take to develop my trading robot. I have demonstrated that it’s possible to perform the following:


- Use JRuby to interface with TWS API

- Get market quote

- Get historical quote

- Check account information (balance and open position info)

- Check order status

- Submit stock order (market and limit orders)

- Submit options order

- Submit complex options order




Along the way, I have also identified some limitations and glitches within TWS API and JRuby. Most of these can be overcome, but they make development harder. Here are just a few:


- JRuby error messages are not that readable and it can be difficult to determine where the error is coming from. So during development, it’s important to test early and often before too much coding is done.

- TWS API or JRuby upgrades may cause the application to break. Any decision to perform an upgrade should not be taken lightly and complete regression testing will be needed.

- If TWS pops up a dialog window, any API calls will be interrupted until the window is closed. The finished product will need a way to detect or prevent this from happening.

- openOrder() callback method sometimes get triggered more than once for unknown reasons.

- Retrieved data (such as market price) from TWS isn’t perfect. So it’s a good idea to build in some sort of redundant requests to make sure no trading decision is made because of bad pricing data.

- There is a need to be able to recover in the case where order is submitted and TWS crashes. Part of the recovery process would be to detect what orders have been submitted, and executed or not.


It is my intent to develop the program in three phases, with the actual trading robot phase being the last, and most difficult to create. The first phase of development will be a watch list of my investment portfolio that will issue buy/sell signals according to my plan. There is no trading automation in this phase, as these investments not in my Interactive Brokers account. The second phase will be the backbone of the Trading Robot. It will keep track of which positions I have in my Interactive Brokers account and provide a user interface to manage data. The following is a more detailed description of each phase:

Phase 1) Watcher - Ruby on Rails application which allows me to watch my investment portfolio (composed of various ETFs and mutual funds). It will get security prices at regular time intervals or on demand and will issue buy/sell signals according to my rules (mostly based on moving averages). Data will be stored in a database and pricing information will be gathered from various sources on the web. It will provide a web interface from which to edit watch list data.




Phase 2) Account Monitor - Extension of the above Rails application. It will store open positions and trading rules in the database. It will provide a web interface to control application parameters and display activities performed by Trading Robot.




Phase 3) Trading Robot - Separate JRuby application with connection to TWS and database. It will populate the database with data from TWS, apply trading rules, and perform trades when needed. It will run in a batch mode and not have a user interface. All of the Trading Robot's activities will be monitored through the Account Monitor.

Wednesday, July 15, 2009

Proof Of Concept 8 of 8g: Complex Options Order

Complex (or combination) option orders are orders composed of more than one leg for the same underlying security. They are submitted and executed as a single transaction and price. TWS complex orders allow for a maximum of up to four legs. Examples of different options strategies and corresponding legs are:

# of Legs Strategies
2 vertical, diagonal, calendar, straddle, strangle
3 butterfly
4 iron condor, iron butterfly, double diagonal


In this coding example, I will create a market order to buy a vertical call spread which is composed of:
BUY 1 SPY AUG 50 CALL
SELL 1 SPY AUG 55 CALL

The steps of creating a complex order are:
1) Use EClientSocket.reqContractDetails() method to get contract id for each option leg.
2) Create a ComboLeg object for each option leg using contract id from step 1).
3) Create a Contract object and add ComboLeg objects created from step 2).
4) Create an Order object and submit it using Contract object from step 3).



# To change this template, choose Tools | Templates
# and open the template in the editor.
require 'java'
require 'TwsApi.jar'

include_class 'ib.client.EClientSocket'
include_class 'ib.client.Contract'
include_class 'ib.client.Order'
include_class 'ib.client.AnyWrapper'
include_class 'ib.client.EWrapper'
include_class 'ib.client.ComboLeg'

class TestComboOptionsOrder
include EWrapper #note "class TestMktData < EWrapper" syntax is no longer supported in JRuby 1.0

attr_accessor :order_id, :contractArray
@mySocket


def initialize
puts 'init'
@mySocket = EClientSocket.new(self)
@contractArray = Array.new # array of hash
end

def eConnect
@mySocket.eConnect("127.0.0.1", 7496, 0)
end

# Request all open orders that were placed from all API clients linked to one TWS,
# and also from the TWS
# This is a one time shot call. When its done you DO NOT continue to receive
# information about new placed orders
def reqAllOpenOrders
@mySocket.reqAllOpenOrders
end

def submitOptionsOrder(order_id, symbol, call_put, expiry, strike,
buy_sell, qty, order_type, limit_price, tif)
# --------------------------------
myContract = Contract.new
myContract.m_symbol = symbol
myContract.m_secType = 'OPT'
myContract.m_exchange = 'SMART'
myContract.m_currency = 'USD'
# options specific below ---
myContract.m_expiry = expiry # string format YYYYMM
myContract.m_right = call_put # CALL, PUT
myContract.m_strike = strike # number

myOrder = Order.new
myOrder.m_action = buy_sell # BUY, SELL
myOrder.m_totalQuantity = qty
myOrder.m_orderType = order_type # MKT, LMT, STP, SPLMT, TRAIL, etc
myOrder.m_lmtPrice = limit_price
myOrder.m_tif = tif # DAY, GTC
@mySocket.placeOrder(order_id, myContract, myOrder)
end

# new method to submit order
def submitOrder(order_id, myContract, myOrder)
@mySocket.placeOrder(order_id, myContract, myOrder)
end

def cancelOrder(order_id)
@mySocket.cancelOrder(order_id)
end

# contract details will be received via the contractDetails() function on the EWrapper
def reqContractDetails (req_id, contract)
@mySocket.reqContractDetails(req_id, contract)
end

def eDisconnect
@mySocket.eDisconnect
end


def contract_msg(contract)
puts ' contract_msg:'
puts ' conid = ' + contract.m_conId.to_s
puts ' symbol = ' + contract.m_symbol
puts ' secType = ' + contract.m_secType
puts ' exchange = ' + contract.m_exchange.to_s
puts ' currency = ' + contract.m_currency
puts ' localSymbol = ' + contract.m_localSymbol.to_s
puts ' multiplier = ' + contract.m_multiplier.to_s
puts ' expiry = ' + contract.m_expiry.to_s
puts ' strike = ' + contract.m_strike.to_s
puts ' right = ' + contract.m_right.to_s
end

def contract_details_msg(contract_detail)
puts ' contract_details_msg:'
puts ' marketName = ' + contract_detail.m_marketName.to_s
puts ' tradingClass = ' + contract_detail.m_tradingClass.to_s
puts ' minTick = ' + contract_detail.m_minTick.to_s
puts ' price magnifier = ' + contract_detail.m_priceMagnifier.to_s
puts ' orderTypes = ' + contract_detail.m_orderTypes.to_s
puts ' validExchanges = ' + contract_detail.m_validExchanges.to_s
puts ' underConId = ' + contract_detail.m_underConId.to_s
end

def order_msg(order)
puts ' order_msg:'
puts ' order id =' + order.m_orderId.to_s
puts ' client id =' + order.m_clientId.to_s
puts ' action =' + order.m_action.to_s
puts ' quantity =' + order.m_totalQuantity.to_s
puts ' type =' + order.m_orderType.to_s
puts ' lmtPrice =' + order.m_lmtPrice.to_s
puts ' TIF =' + order.m_tif.to_s
puts ' parent Id =' + order.m_parentId.to_s
puts ' permId =' + order.m_permId.to_s


end

def order_state_msg(order_state)
puts ' order_state_msg:'
puts ' status =' + order_state.m_status.to_s
puts ' initMargin =' + order_state.m_initMargin.to_s
puts ' maintMargin =' + order_state.m_maintMargin.to_s
puts ' commission =' + order_state.m_commission.to_s

end

#///////////////////////////////////////////////////////////////////////
#// Interface methods
#///////////////////////////////////////////////////////////////////////

def tickPrice( ticker_id, field, price, can_auto_execute)
end

def tickSize( ticker_id, field, size)
end

def tickOptionComputation( ticker_id, field, implied_vol,
delta, model_price, pv_dividend)
end

def tickGeneric( ticker_id, tick_type, value)
end

def tickString( ticker_id, tick_type, value)
end

def tickEFP( ticker_id, tick_type, basis_points,
formatted_basis_points, implied_future, hold_days,
future_expiry, dividend_impact, dividends_to_expiry)
end

def orderStatus( order_id, status, filled, remaining,
avg_fill_price, perm_id, parent_id, last_fill_price,
client_id, why_held)
puts '~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'
puts 'orderStatus>: orderId=' + order_id.to_s + ', clientId=' + client_id.to_s + ', permId=' + perm_id.to_s +
', status=' + status + ' filled=' + filled.to_s + ', remaining=' + remaining.to_s + ', '
puts ' avgFillPrice=' + avg_fill_price.to_s + ', lastFillPrice=' + last_fill_price.to_s +
', parent Id=' + parent_id.to_s + ', whyHeld=' + why_held.to_s
end

def openOrder( order_id, contract, order, order_state)
puts '--------------------------------------------------------'
puts 'openOrder> open order: orderId=' + order_id.to_s + ' --- '
contract_msg(contract)
order_msg(order)
order_state_msg(order_state)
end

def updateAccountValue( key, value, currency, account_name)
end

def updatePortfolio( contract, position, market_price, market_value,
average_cost, unrealized_pnl, realized_pnl, account_name)
end

def updateAccountTime( time_stamp)
end

# triggered right after logging into TWS
def nextValidId( order_id)
@order_id = order_id #storing order id to be used when placing order
puts 'nextValidId> Next Valid Order Id = ' + @order_id.to_s
end

# call back with details. determine which request it was and store details
# data with the associated request.
def contractDetails( req_id, contract_details)
aContract = contract_details.m_summary
# find the hash with matching :req_id and set :conid
@contractArray.find{ |contract| contract[:req_id] == req_id } [:conid] = aContract.m_conId

puts 'contractDetails()> reqId = ' + req_id.to_s + ' -------------------'
contract_msg(aContract)
contract_details_msg(contract_details)
puts ' --------'
end

def bondContractDetails( req_id, contract_details)
end

def contractDetailsEnd( req_id)
puts 'contractDetailsEnd() ' + req_id.to_s + '> Ended ------------------------'
end

def execDetails( order_id, contract, execution)
end

def updateMktDepth( ticker_id, position, operation, side, price, size)
end

def updateMktDepthL2( ticker_id, position, market_maker, operation,
side, price, size)
end

def updateNewsBulletin( msg_id, msgType, message, orig_exchange)
end

def managedAccounts( accounts_list)
end

def receiveFA( fa_data_type, xml)
end

def historicalData( req_id, date, open, high, low,
close, volume, count, eWAP, has_gaps)
end

def scannerParameters( xml)
end

def scannerData( req_id, rank, contract_details, distance,
benchmark, projection, legs_str)
end

def scannerDataEnd( req_id)
end

def realtimeBar( req_id, time, open, high, low, close, volume, wap, count)
end

def currentTime( time)
end

def fundamentalData( req_id, data)
end

# ---------------------------------------------------
# new callback methods added in API 9.60
# ---------------------------------------------------
def openOrderEnd() # invoked when all orders are sent to a client as a response to reqOpenOrders(),
# reqAllOpenOrders and right after connected to TWS
puts 'openOrderEnd> Ended ------------------------'
end

def accountDownloadEnd(account_name) #invoked when a complete snapshot of an account state is sent to a client as a response to reqAccountUpdates()
end

def execDetailsEnd( req_id) # invoked when all executions are sent to a client as a response to reqExecutions()
end

def deltaNeutralValidation(req_id, under_comp) # for Delta-Neutral DN RFQ
end

# ---------------------------------------------------
# -- methods from AnyWrapper -- must be declared
# ---------------------------------------------------

def connectionClosed()
puts ' [API.connectionClosed] Closed connection with TWS'
end

def error(err_obj_str)
puts ' [API.msg1] ' + err_obj_str if err_obj_str.is_a?(String)
puts ' [API.msg3] ' + err_obj_str.getMessage() if err_obj_str.is_a?(Exception)
end

def error(one, two, str)
puts ' [API.msg2] ' + str + ' {' + one.to_s + ', ' + two.to_s + '}'
end

end # class

# *********************************************************************************************
# Running the code
# *********************************************************************************************
x = TestComboOptionsOrder.new()
begin
x.eConnect
puts 'connected ...'
sleep(2) # need this here to give time for nextValidId() to be called
puts '******************************'
puts '** STEP 1) ******************'
puts '.. Defining contract for Aug 90 C '
spyAug90c = Contract.new
spyAug90c.m_symbol = 'SPY'
spyAug90c.m_secType = 'OPT'
spyAug90c.m_exchange = 'SMART'
spyAug90c.m_currency = 'USD'
spyAug90c.m_expiry = '20090821' # string format YYYYMM or YYYYMMDD -- if YYYYMM used and normal and quarterly
# options exist, will return contract details for both
spyAug90c.m_right = 'CALL' # CALL, PUT
spyAug90c.m_strike = 90 # number

puts '.. Adding Aug 90 C to contract array'
request_id = 1
x.contractArray << {:req_id => request_id, :contract => spyAug90c, :conid => '' }

puts '.. Requesting contract detail and id info'
x.reqContractDetails(request_id, spyAug90c)
sleep(5);
puts ' '
puts '.. Defining contract for Aug 95 C '
spyAug95c = Contract.new
spyAug95c.m_symbol = 'SPY'
spyAug95c.m_secType = 'OPT'
spyAug95c.m_exchange = 'SMART'
spyAug95c.m_currency = 'USD'
spyAug95c.m_expiry = '20090821' # string format YYYYMM or YYYYMMDD -- if YYYYMM used and normal and quarterly
# options exist, will return contract details for both
spyAug95c.m_right = 'CALL' # CALL, PUT
spyAug95c.m_strike = 95 # number
puts '.. Adding Jul 95 C to contract array'
request_id = 2
x.contractArray << {:req_id => request_id, :contract => spyAug95c, :conid => '' }
puts 'Requesting contract detail and id info'
x.reqContractDetails(request_id, spyAug95c)
sleep(5);

puts '******************************'
puts '** STEP 2 ******************'
puts '.. Now have info on both contracts, create combo leg for each'
puts '.. Create leg #1'
leg1 = ComboLeg.new(x.contractArray.find{ |id| id[:req_id] == 1 } [:conid], # contract id
1, # ratio
'BUY', # BUY or SELL
'SMART', # p_exchange
0 # p_openClose- always 0 for retail customers
)
puts '.. Create leg #2'
leg2 = ComboLeg.new(x.contractArray.find{ |id| id[:req_id] == 2 } [:conid], # contract id
1, # ratio
'SELL', # BUY or SELL
'SMART', # p_exchange
0 # p_openClose- always 0 for retail customers
)
puts '******************************'
puts '** STEP 3 ******************'
puts '.. Create Contract'

myContract = Contract.new
myContract.m_symbol = 'SPY'
myContract.m_secType = 'BAG' # BAG is a combination type
myContract.m_exchange = 'SMART'
myContract.m_currency = 'USD'

puts '.. Add leg 1 to combo contract'
myContract.m_comboLegs.add(leg1)
puts '.. Add leg 2 to combo contract'
myContract.m_comboLegs.add(leg2)
puts '******************************'
puts '** STEP 4 ******************'
puts '.. Create combo Order'
myOrder = Order.new
myOrder.m_action = 'BUY' # BUY, SELL
myOrder.m_totalQuantity = 1
myOrder.m_orderType = 'MKT' # MKT, LMT, STP, SPLMT, TRAIL, etc
myOrder.m_tif = 'DAY' # DAY, GTC

puts '.. submitting order'
x.submitOrder(x.order_id += 1, myContract, myOrder)
sleep(5);
puts '==========================='
puts '*** requesting open orders ***'
x.reqAllOpenOrders
sleep(5)
x.eDisconnect
puts 'disconnected'
rescue Exception => e
puts 'Exception ' + e.message
x.eDisconnect
end





Running the code produces the following output:


init
Server Version:44TWS Time at connection:20090712 16:51:47 ESTconnected ...
nextValidId> Next Valid Order Id = 81
openOrderEnd> Ended ------------------------
[API.msg2] Market data farm connection is OK:usopt {-1, 2104}
[API.msg2] Market data farm connection is OK:usfarm {-1, 2104}
******************************
** STEP 1) ******************

.. Defining contract for Aug 90 C
.. Adding Aug 90 C to contract array
.. Requesting contract detail and id info
contractDetails()> reqId = 1 -------------------
contract_msg:
conid = 60388362
symbol = SPY
secType = OPT
exchange = SMART
currency = USD
localSymbol = SWGHL
multiplier = 100
expiry = 20090821
strike = 90.0
right = C
contract_details_msg:
marketName = SWG
tradingClass = SWG
minTick = 0.01
price magnifier = 1
orderTypes = ADJUST,ALERT,ALGO,ALLOC,AON,AVGCOST,BASKET,COND,CONDORDER,DAY,DEACT,DEACTEOD,FOK,GAT,GTC,GTD,GTT,HID,HPENNY,ICE,IOC,LIT,LMT,MIT,MKT,MTL,NONALGO,OCA,PAON,RELSTK,SCALE,SCALERST,STP,STPLMT,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,VOLAT,WHATIF,
validExchanges = SMART,AMEX,BOX,CBOE,IBSX,ISE,MIBSX,NASDAQOM,PHLX,PSE
underConId = 756733
--------
contractDetailsEnd() 1> Ended ------------------------

.. Defining contract for Aug 95 C
.. Adding Jul 95 C to contract array
Requesting contract detail and id info
contractDetails()> reqId = 2 -------------------
contract_msg:
conid = 60388382
symbol = SPY
secType = OPT
exchange = SMART
currency = USD
localSymbol = SWGHQ
multiplier = 100
expiry = 20090821
strike = 95.0
right = C
contract_details_msg:
marketName = SWG
tradingClass = SWG
minTick = 0.01
price magnifier = 1
orderTypes = ADJUST,ALERT,ALGO,ALLOC,AON,AVGCOST,BASKET,COND,CONDORDER,DAY,DEACT,DEACTEOD,FOK,GAT,GTC,GTD,GTT,HID,HPENNY,ICE,IOC,LIT,LMT,MIT,MKT,MTL,NONALGO,OCA,PAON,RELSTK,SCALE,SCALERST,STP,STPLMT,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,VOLAT,WHATIF,
validExchanges = SMART,AMEX,BOX,CBOE,IBSX,ISE,MIBSX,NASDAQOM,PHLX,PSE
underConId = 756733
--------
contractDetailsEnd() 2> Ended ------------------------
******************************
** STEP 2 ******************

.. Now have info on both contracts, create combo leg for each
.. Create leg #1
.. Create leg #2
******************************
** STEP 3 ******************

.. Create Contract
.. Add leg 1 to combo contract
.. Add leg 2 to combo contract
******************************
** STEP 4 ******************

.. Create combo Order
.. submitting order
[API.msg2] Order Message:
Warning: your order will not be placed at the exchange until 2009-07-13 09:30:00 US/Eastern {82, 399}
--------------------------------------------------------
openOrder> open order: orderId=82 ---
contract_msg:
conid = 28812380
symbol = SPY
secType = BAG
exchange = SMART
currency = USD
localSymbol = SPY
multiplier =
expiry =
strike = 0.0
right = ?
order_msg:
order id =82
client id =0
action =BUY
quantity =1
type =MKT
lmtPrice =0.0
TIF =DAY
parent Id =0
permId =1009111242
order_state_msg:
status =PreSubmitted
initMargin =
maintMargin =
commission =1.7976931348623157e+308
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
orderStatus>: orderId=82, clientId=0, permId=1009111242, status=PreSubmitted filled=0, remaining=1,
avgFillPrice=0.0, lastFillPrice=0.0, parent Id=0, whyHeld=
===========================
*** requesting open orders ***
--------------------------------------------------------
openOrder> open order: orderId=82 ---
contract_msg:
conid = 28812380
symbol = SPY
secType = BAG
exchange = SMART
currency = USD
localSymbol = SPY
multiplier =
expiry =
strike = 0.0
right = ?
order_msg:
order id =82
client id =0
action =BUY
quantity =1
type =MKT
lmtPrice =0.0
TIF =DAY
parent Id =0
permId =1009111242
order_state_msg:
status =PreSubmitted
initMargin =
maintMargin =
commission =1.7976931348623157e+308
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
orderStatus>: orderId=82, clientId=0, permId=1009111242, status=PreSubmitted filled=0, remaining=1,
avgFillPrice=0.0, lastFillPrice=0.0, parent Id=0, whyHeld=
openOrderEnd> Ended ------------------------
disconnected


I ran the code after hours and was able to confirm that the order had been placed in TWS.


To see the individual legs of the order, just right click on the order and select 'Show Legs'



Individual legs will be displayed.



When the market opens the next day, my vertical call spread order executed at market price.



That's all there is to it. Complex option orders are not all that complex after all.

Sunday, July 12, 2009

Progress Update...

I finally have time to get back to working on my coding. I got a complex options order working today (a bull call spread). It'll take me a few days to clean up and document the code though.

Stay tuned....