Proof Of Concept - 8 of 8 : Going Native

After playing around with JRuby and TWS API trying to resolve the problems described in my prior post, there has been no real progress. I have decided to go native and try out the sample code in Java to ascertain if the problem is with Java or JRuby. In NetBeans, I created a new Java Application called IBJava, created a new class and copied TWS API classes under the IBJava\Src folder. As with everything in this project, nothing is as easy as it seems. The final code to just connect and disconnect from TWS is as follows:

package ibjava;

import com.ib.client.*;


public class StockBot  implements EWrapper {

   public StockBot()   {
        EClientSocket m_client = new EClientSocket(this);
        m_client.eConnect("127.0.0.1",7496, 0);
        System.out.println("Connected.");
         while (i<990000) i++;
        m_client.eDisconnect();
   }

    public static void main(String[] args)   {

        System.out.println("Hello");
        StockBot myBot= new StockBot();
    }
    // ---------------------------------------------------  
    // -- methods from EWrapper  -- must be declared  
    // ---------------------------------------------------  
    public void tickPrice( int tickerId, int field, double price, int canAutoExecute){
        System.out.println("Price Update: Id: "+tickerId+" Type: "+field+" Price: "+price);
    }
    public void tickSize( int tickerId, int field, int size){
        System.out.println("Volume Update: Id: "+tickerId+" Type: "+field+" Size: "+size);
    }
    public void tickOptionComputation( int tickerId, int field, double impliedVol,
            double delta, double modelPrice, double pvDividend){
    }
    public void tickGeneric(int tickerId, int tickType, double value){
    }
    public void tickString(int tickerId, int tickType, String value){
    }
    public void tickEFP(int tickerId, int tickType, double basisPoints,
            String formattedBasisPoints, double impliedFuture, int holdDays,
            String futureExpiry, double dividendImpact, double dividendsToExpiry){}
    public void orderStatus( int orderId, String status, int filled, int remaining,
            double avgFillPrice, int permId, int parentId, double lastFillPrice,
            int clientId, String whyHeld){
    }
    public void openOrder( int orderId, Contract contract, Order order, OrderState orderState){
    }

    public void updateAccountValue(String key, String value, String currency, String accountName){}
    public void updatePortfolio(Contract contract, int position, double marketPrice, double marketValue,
            double averageCost, double unrealizedPNL, double realizedPNL, String accountName){}
    public void updateAccountTime(String timeStamp){}
    public void nextValidId( int orderId){}
    public void contractDetails(int reqId, ContractDetails contractDetails){}
    public void bondContractDetails(int reqId, ContractDetails contractDetails){}
    public void contractDetailsEnd(int reqId){}
    public void execDetails( int orderId, Contract contract, Execution execution){}
    public void updateMktDepth( int tickerId, int position, int operation, int side, double price, int size){}
    public void updateMktDepthL2( int tickerId, int position, String marketMaker, int operation,
            int side, double price, int size){}
    public void updateNewsBulletin( int msgId, int msgType, String message, String origExchange){}
    public void managedAccounts( String accountsList){}
    public void receiveFA(int faDataType, String xml){}
    public void historicalData(int reqId, String date, double open, double high, double low,
                      double close, int volume, int count, double WAP, boolean hasGaps){}
    public void scannerParameters(String xml){}
    public void scannerData(int reqId, int rank, ContractDetails contractDetails, String distance,
            String benchmark, String projection, String legsStr){}
    public void scannerDataEnd(int reqId){}
    public void realtimeBar(int reqId, long time, double open, double high, double low, double close, long volume, double wap, int count){}
    public void currentTime(long time){}
    public void fundamentalData(int reqId, String data){}

    // ---------------------------------------------------
    // -- methods from AnyWrapper  -- must be declared
    // ---------------------------------------------------
   
    public void connectionClosed() {
        System.out.println(" [API.connectionClosed] Closed connection with TWS");
    }
    public void error(String str) {
        System.out.println(" [API.msg1] " + str);
    }

    public void error(int one, int two, String str) {
        System.out.println(" [API.msg2] " + str + " {" + one + ", " + two + "}");
    }

    public void error(Exception e) {
        System.out.println(" [API.msg3] " + e.getMessage());
    }
    
}

Some interesting things to point out here:

1) If lines 76-90 are omitted, the following error is generated:


java.lang.ExceptionInInitializerError
Caused by: java.lang.RuntimeException: Uncompilable source code - ibjava.StockBot is not abstract and does not override abstract method connectionClosed() in com.ib.client.AnyWrapper
at ibjava.StockBot.(StockBot.java:11)
Exception in thread "main"
Exception in thread "main" Java Result: 1


This is due to the fact that EWrapper interface is a subclass of AnyWrapper interface, which means that I have to create AnyWrapper interface methods in addition to EWrapper interface methods. I did not do this in my JRuby code due to my limited Java knowledge. This error needs to be fixed in the JRuby version.


2) Running the code above with 'Incoming Connection' dialog ON seems to work. The output is:


run:
Hello
Server Version:43
TWS Time at connection:20090208 21:45:55 EST
Connected.
[API.connectionClosed] Closed connection with TWS
BUILD SUCCESSFUL (total time: 2 seconds)



3) Running the code above with 'Incoming Connection' dialog OFF generates the error:

run:
Hello
Server Version:43
TWS Time at connection:20090208 21:49:48 EST
Connected.
[API.msg2] Market data farm connection is OK:usopt {-1, 2104}
[API.msg2] Market data farm connection is OK:usfarm {-1, 2104}
[API.msg3] socket closed
[API.connectionClosed] Closed connection with TWS
BUILD SUCCESSFUL (total time: 0 seconds)


It's clear to me now that there are issues in Java that I need to understand and resolve before even attempting to get JRuby to work. By working at Java level, I at least get some useful error messages instead of JRuby's cryptic ones.

C'est la vie! My initial approach to this project was to be free from working with Java, but that turns out not to be the case. I obviously need to get my hands 'dirty' and this process will take a little more time. The bright side to this is that Java resources are everywhere and I'm sure I'll be able to get this fixed. Stay tuned...

---

02/11/09 Update
It occurred to me that the 'errors' I experienced may not actually be errors at all. After some research, it appears that just because the error() methods were triggered, doesn't actually mean errors occurred. They may just be messages that I can ignore. This is good news. With this in mind, I will proceed with Java coding.

Proof Of Concept - 8 of 8 : Stumbling Block

Here's where things get interesting. Now that I'm able to successfully connect to TWS, the next step is to do something useful such as getting current market quote. I found a nice site called StockBotProgramming.com which has sample code to do this in Java. I find that this code is more direct and easier to follow than the one in JavaAPIGettingStarted.pdf.

Some notes about this Java code:

• Main class implements EWrapper interface
• A class must implement all of the methods described in the interface
• The methods in the example is specific to that TWS API, and will not be the same as my version of API (9.51)
• Methods TickPrice() and TickSize() are callback methods that automatically get called when the quote become available
  

My task would be to create a class in Ruby which implements this EWrapper interface. More specifically, my version of the EWrapper. So the code will be a little different. Per Java interface definition, all the functions of EWrapper class needs to be declared. The easiest way to do this is to go directly the the EWrapper Java class file, copy, paste into Ruby class and convert to Ruby syntax.

This isn't quite as easy as it looks and I'm getting some strange behavior in the coding. Before handling this issue, I noticed there's a problem with prior post concerning Skipping Incoming connection. So I'm not defining a trusted IP anymore.

In order for readers to follow along, I have to post my code here. This is going to be hard to see. But I've found Google's syntax highlighter tool, so I'll be incorporating that into this blog from now on. The instructions are not too clear. I found this blog article to be much better.

A) This is my test code to request market data:

require 'java'
require 'TwsApi.jar'

include_class 'ib.client.EClientSocket'
include_class 'ib.client.Contract'
include_class 'ib.client.AnyWrapper'
include_class 'ib.client.EWrapper'

# classes for interface method parameters.  Not sure if necessary.  Can't hurt.
include_class 'ib.client.ContractDetails'
include_class 'ib.client.Execution'
include_class 'ib.client.OrderState'

class TestMktData  include EWrapper  #note class TestMktData < EWrapper syntax is no longer supported in JRuby 1.0

  @mySocket  

  def initialize 
    puts 'init'  
    @mySocket = EClientSocket.new(self)
  end

  def eConnect   
    @mySocket.eConnect("127.0.0.1", 7496, 0)
  end

  def requestQuote  
    puts 'running requestQuote'  
    # --------------------------------  
    myContract = Contract.new  
    myContract.m_symbol    = 'SPY'      
    myContract.m_secType   = 'STK'  
    myContract.m_exchange  = 'SMART'  
    myContract.m_currency  = 'USD'  
    # options specific below ---  
    myContract.m_expiry    = ''  
    myContract.m_right     = ''  
    request_id      = 1  
    ticklist        = ''  
    puts 'created new contract, calling reqMktData'  
    @mySocket.reqMktData(request_id, myContract, ticklist, true)  
    puts 'sleeping for 10 sec...'  
    sleep(10)
  end

  def eDisconnect
    @mySocket.eDisconnect
  end

  #///////////////////////////////////////////////////////////////////////
  #// Interface methods
  #///////////////////////////////////////////////////////////////////////

  def tickPrice( ticker_id, field, price, can_auto_execute)
    puts 'in tickPrice'  
    #puts 'tickPrice() id=' + ticker_id.to_s + ', price = ' + price.to_s
  end

  def tickSize(  tickerId,  field,  size)
    puts 'in tickSize'  
    #puts 'tickSize()  Id: ' + tickerId.to_s + ' Type: ' + field.to_s + ' Size: ' + size.to_s
  end

  def tickOptionComputation( ticker_id,  field,  implied_vol,  delta,  model_price, pv_dividend)
  end

  def tickGeneric( ticker_id,  tick_type,  value)
  end

  def tickString( ticker_id,  tick_type,  value)
  end

  def tickEFP( ticker_id,  tick_type,  basis_points, formatted_basis_points,  implied_future,  hold_days, future_expiry, dividend_impact,  dividends_to_expiry)
  end

  def orderStatus(  order_id,  status,  filled,  remaining, avg_fill_price,  perm_id,  parent_id,  last_fill_price, client_id,  why_held)
  end 

  def openOrder(  order_id,  contract,  order,  order_state)
  end

  def updateAccountValue( key,  value,  currency,  account_name)
  end

  def updatePortfolio( contract,  position,  market_price,  market_value, average_cost,  unrealized_pnl,  realized_pnl,  account_name)
  end

  def updateAccountTime( time_stamp)
  end

  def nextValidId(  order_id)
  end

  def contractDetails( req_id,  contract_details)
  end

  def bondContractDetails( req_id, contract_details)
  end

  def contractDetailsEnd( req_id)
  end

  def execDetails(  order_id,  contract,  execution)
  end

  def updateMktDepth(  ticker_id,  position,  operation,  side,  price,  size)
  end

  def updateMktDepthL2(  ticker_id,  position,  market_maker,  operation,  side,  price,  size)
  end

  def updateNewsBulletin(  msg_id,  msgType,  message,  orig_exchange)
  end

  def managedAccounts(  accounts_list)
  end

  def receiveFA( fa_data_type,  xml)
  end

  def historicalData( req_id,  date,  open,  high,  low, close,  volume,  count,  eWAP,  has_gaps)
  end

  def scannerParameters( xml)
  end

  def scannerData( req_id,  rank, contract_details,  distance, benchmark,  projection,  legs_str)
  end

  def scannerDataEnd( req_id)
  end

  def realtimeBar( req_id, time,  open,  high,  low,  close,  volume,  wap,  count)
  end

  def currentTime( time)
  end

  def fundamentalData( req_id,  data)
  end

end  #  class TestMktData

# *********************************************************************************************
#  Running the code
# *********************************************************************************************

 x = TestMktData.new()
 begin
     x.eConnect
     puts 'connected ...'
     x.requestQuote
     puts 'wait 10 sec'
     sleep(10)
     x.eDisconnect
     puts 'disconnected'
 rescue  Exception => e
     puts 'can not connect'
     puts 'Exception' + e.message
     x.eDisconnect
 end

Line 14: specifies that I'm using a Java interface from TWS API called EWrapper
Line 41: submits market quote for ticker SPY
Lines 54-62: methods that automatically gets called when market data is received
Lines 64-140: the rest of method specifications as required by EWrapper interface
Lines 148-161: runs the code by connecting, requesting market quote, and disconnecting

Running the code produces error:

init
Server Version:43

TWS Time at connection:20090201 22:48:24 EST

connected ...
running requestQuote
created new contract, calling reqMktData
sleeping for 10 sec...
Exception in thread "EReader" :1: #< nameerror::message:0x1de0c09 > (NoMethodError)
...internal jruby stack elided...
from (unknown).(unknown)(:1)

disconnected


What causes this 'stack elided' error? It seems like it has something to do with requesting market data, waiting (sleep), and receiving market data. So I went about commenting out lines of code to see what the behavior is and here's what I've found:

B) Commenting out sleep():

puts 'created new contract, calling reqMktData'    
@mySocket.reqMktData(request_id, myContract, ticklist, true)    
puts 'sleeping for 10 sec...' 
#   sleep(10)

produces no errors, but then again, the program didn't wait around to receive quote either:


init
Server Version:43

TWS Time at connection:20090201 23:09:01 EST

connected ...
running requestQuote
created new contract, calling reqMktData
sleeping for 10 sec...
disconnected


C) Can it be that sleep() is the actual problem and has nothing to do with reqMktData()? I tried just connecting, sleeping, and disconnecting:

x = TestMktData.new()  
begin
  x.eConnect
  puts 'connected ...'
  #      x.requestQuote       
  puts 'wait 10 sec'     
  sleep(10)     
  x.eDisconnect      
  puts 'disconnected'  
rescue  Exception => e      
  puts 'can not connect'      
  puts 'Exception' + e.message      
  x.eDisconnect  
end

Produces the same 'stack elided' error:


init
Server Version:43

TWS Time at connection:20090201 23:24:53 EST

connected ...
wait 10 sec
Exception in thread "EReader" :1: #< nameerror::message:0xd510e8> (NoMethodError) ...internal jruby stack elided... from (unknown).(unknown)(:1)

disconnected


D) So going back to my previous post on TWS Note - Skipping 'Incoming Connection'. I modify the connect and disconnect code to add the sleep():

require 'java'
require 'TwsApi.jar'
include_class 'ib.client.EClientSocket'   

mySocket = EClientSocket.new(self)  
begin    
  mySocket.eConnect("127.0.0.1", 7496, 0)  # api    
  puts 'connected ...'    
  sleep(10)  # this produces error    
  mySocket.eDisconnect    
  puts 'disconnected'  

rescue  Exception => e    
   puts 'can not connect'    
  puts e.message
end

Produces 'EReader' error, which is different from 'stack elided' error from before (although consistent with error in post on TWS Note - Skipping 'Incoming Connection'):

Server Version:43

TWS Time at connection:20090201 23:31:00 EST

connected ...
wait 10 sec
Exception in thread "EReader" java.lang.ClassCastException: InterfaceImpl326315435 at ib.client.EReader.eWrapper(EReader.java:47) at ib.client.EReader.run(EReader.java:66)

was connected, now disconnected


So what gives? Why do A and C give 'stack elided' errors and D give an 'EReader' error? Both have sleep() but the difference is that A and C define a new class which implements EWrapper. D doesn't do that. It could be that there are two different error conditions going on. I'm not sure what the answer is yet, but I will look into it some more.

Stay tuned ..