Ruby - Rails - PostgreSQL Issues

I encountered a couple of issues with the development stack. Luckily, I was able to solve them since they were documented on the web. By way of review, my development environment is as follows:
Operating System: Windows XP
Ruby: 1.8.6
Rails: 2.3.3
PostgreSQL: 8.4.0

A) The first error occured when starting WEBrick server. A pop up message displayed:
ruby.exe Ordinal Not Found
The ordinal 284 could not be located in the dynamic link library SSLEAY32.DLL

Searching the web led me to this post at Stackoverflow. My solution was to:

1) Rename libeay32.dll and ssleay32.dll in both ruby\bin folder and posgresql\lib folder to old_libeay32.dll and old_ssleay32.dll respectively. This way I don't lose these dlls in case my fix doesn't work.

2) Install Visual C++ 2008 Redistributables which can be obtained from Microsoft site. This is needed prior to installing OpenSSL below.

3) Install the latest OpenSSL from Shining Light Productions. I used "Win32 OpenSSL v0.9.8k Light".

4) Copy libeay32.dll and ssleay32.dll from OpenSSL folder to h ruby\bin and posgresql\lib folder.

5) Restart WEBrick server and the error should be gone.


B) The second error occured in the browser when Rails tried to connect to PostgreSQL. I tried to migrate a model using rake db:migrate. An error displayed:
Rake aborted!
undefined method 'quote_indent' for PGconn:class

Searching the web led me to this post at HighDot Forums. My solution was to:

1) Open Rails file config/initializers/new_rails_defaults.rb

2) Add the following code at the end of the file:


# Postgres "quote_indent for PGconn:class" fix from http://www.highdots.com/forums/ruby-rails-talk/quote_ident-283323.html
def PGconn.quote_ident(name)
%("#{name}")
end


After that, I did not experience any other issues with the development environment.

Ruby & Ruby On Rails Installation Complete

While there are numerous write ups on this process (just Google "install ruby on rails"), I thought I'd include a brief description for blog completeness

1) Go to Ruby on Rails download page to Download and install Ruby. Enable RubyGems option. I used Windows 1.8.6-27 Release Candidate 2.

2) When finish, open a command prompt and install rails 2.3.3 :
>gem install rails

3) Install PostgrSQL driver:
>gem install postgres-pr

4) Create sample Rails application:
>rails c:\temp\railstest

That's all there is to it. For a quick tutorial, I suggest going to http://guides.rubyonrails.org/ .

PostgreSQL 8.4.0 Setup Complete

The Installation process was very straightforward and provided no surprises. Simply go to www.postgreSQL.org and download release 8.4. Run the installer and it should be very easy. A windows service called "PostgreSQL Server 8.4" will be created and can be verified by checking Control Panel, Administrative Tools, Services. If it does not start automatically, simply right click on it and select Start.



Once the database is running, connect to it and create a 'tradingbot' database.

C:\temp>psql -U postgres
Password for user postgres:
psql (8.4.0)
WARNING: Console code page (437) differs from Windows code page (1252)
         8-bit characters might not work correctly. See psql reference
         page "Notes for Windows users" for details.
Type "help" for help.

postgres=# create database tradebot;
CREATE DATABASE
postgres=# \l
                                                 List of databases
   Name    |  Owner   | Encoding |         Collation          |           Ctype
           |   Access privileges
-----------+----------+----------+----------------------------+-----------------
-----------+-----------------------
 postgres  | postgres | UTF8     | English_United States.1252 | English_United States.1252 |
 template0 | postgres | UTF8     | English_United States.1252 | English_United States.1252 | =c/postgres

           : postgres=CTc/postgres
 template1 | postgres | UTF8     | English_United States.1252 | English_United States.1252 | =c/postgres

           : postgres=CTc/postgres
 tradebot  | postgres | UTF8     | English_United States.1252 | English_United States.1252 |
(4 rows)

postgres=# create user master with password 'blaster';
CREATE ROLE
postgres=# grant all on database tradebot to master;
GRANT
postgres=# \q

Line 1: Connect to default database
Line 9: Create database "tradebot"
Line 27: Create new user called "master"
Line 29: Grant privileges to "master"

Next step is to log into the new database and create a test table.

C:\temp>psql -d tradebot -U master
Password for user master:
psql (8.4.0)
WARNING: Console code page (437) differs from Windows code page (1252)
         8-bit characters might not work correctly. See psql reference
         page "Notes for Windows users" for details.
Type "help" for help.

tradebot=>\d
No relations found.
tradebot=> create table tester (id varchar(10) not null);
CREATE TABLE
tradebot=> \d
        List of relations
 Schema |  Name  | Type  | Owner
--------+--------+-------+--------
 public | tester | table | master
(1 row)


tradebot=> \d tester
           Table "public.tester"
 Column |         Type          | Modifiers
--------+-----------------------+-----------
 id     | character varying(10) | not null


tradebot=>

Line 1: Connect to tradebot database
Line 9: Get listing of all tables in the database. None returned because database is empty.
Line 11: Create "tester" table
Line 13: Get listing of all tables in the database. "Tester" table appears.
Line 21: Get "tester" table information

Upgrade To API v. 9.60

It has come to my attention that IB released their newest API version 9.60 on Jan 20, 2009. As mentioned in the last post, one drawback of the API's asynchronous request and callback model is that there's no way to tell if all the information has been returned. In the case of reqAllOpenOrders() method, a workaround is to intentionally issue a reqMktData() with an invalid ticker to intentionally generate an error. The developers at IB must have agreed that this is a flaw in their model. So in the latest API release, new methods have been added to the EWrapper interface to indicate that callback requests have ended.

I have decided to use this API version for my project. The process of replacing API version 9.51 with 9.60 is relatively simple. I just had to edit Java source code (Proof Of Concept - 4 of 8) and create a new version of TwsApi.jar file (Proof of Concept - 5 of 8). Finally I had to incorporate the new callback EWrapper methods in each of the code examples in Proof of Concepts - 8 of entries. The code to be added is as follows:

  # ---------------------------------------------------
  #  new callback methods added in API 9.60
  # ---------------------------------------------------
  def openOrderEnd()  # invoked when all orders are sent to a client as a response to reqOpenOrders()
  end

  def accountDownloadEnd(account_name) #invoked when a complete snapshot of an account state is sent to a client as a response to reqAccountUpdates()
  end

  def execDetailsEnd( req_id) # invoked when all executions are sent to a client as a response to reqExecutions()
  end

  def deltaNeutralValidation(req_id, under_comp)  # for Delta-Neutral DN RFQ
  end

I will go through each of the Proof Of Concept entries and update them to comply with API v. 9.60.

Proof Of Concept - 7 of 8 : NetBeans Ruby Plugins

Let's review what we have so far for my development environment. Java SDK is installed and IB's Java API successfully compiles in NetBeans. I was able to compile and run IB's test client program and create a JAR file of IB's API. JRuby is installed and runs on top of Java SDK. So now I'm ready to write Ruby code and call IB's Java API.

Before I can do that though, I need to have a development environment for Ruby. Fortunately, NetBeans also supports Ruby development along with Java. To enable Ruby support, go to Tools | Plugins | Available Plugins tab. Choose Ruby and JRuby plugins and click install. It may take a while to fr NetBeans to download all the libraries.


Test out Ruby plugin by creating a Hello World program. Go to File | New Project | Ruby Application. Name it HelloWorld.



Run the application by pressing F6 and 'Hello World' should display in the output window.


Success!! Ruby development is enabled in NetBeans.

---

Next step is to see if Java classes are accessible. This is a sample Ruby code that calls Java class TreeSet. Note how seamless it is to call Java classes using JRuby.

require 'java'
include_class 'java.util.TreeSet' 
 
set = TreeSet.new  
set.add "foo"  
set.add "Bar"  
set.add "baz"  
set.each do |v|    
  puts "value: #{v}"
end

Press F6 will produce the following in the output window.




Another way of testing this is (just plain JRuby, not NetBeans) to run it via command line. It produces the same output:

---

02/22/09 Update

JRuby plugin automatically installs version 1.1.4. This is an old version and NetBeans must be manually configured to use version 1.1.6 (installed in Proof Of Concept - 6 of 8). Please see Proof Of Concept 8 of 8b : Historical Quote for how to do this.

---

Last step is to see if IB's Java API archived file is accessible. Copy file TwsApi.jar (created in Proof Of Concept - 5 of 8) and put it in HelloWorld\lib folder. Run TWS in paper trading mode as this is necessary for for the API to connect to.

Code the following in main.rb. This is really simple code to connect and disconnect from TWS using IB's API.

 
require 'java'
require 'TwsApi.jar'
include_class 'ib.client.EClientSocket'  

mySocket = EClientSocket.new(self)  
begin 
   mySocket.eConnect("127.0.0.1", 7496, 0)  # api 
   puts 'connected ...' 
   mySocket.eDisconnect 
   puts 'disconnected' 
rescue
  Exception => e 
    puts 'can not connect' 
    puts e.message
end

Press F6 will produce the following output:

Server Version:43TWS Time at connection:20090113 15:27:27 ESTconnected ...disconnected

Running it via command line produces the same output:



At this point, the development environment is set up and Java and API classes are accessible. It is now time to start using IB's API to see its full power.

Proof Of Concept - 6 of 8 : Install JRuby

Go to JRuby home page and download the zip file. As of this writing, the current version is 1.1.6.

1. I extracted the zip file into C:\jruby\jruby-1.1.6RC1
2. Added the above path on to the end of PATH environment variable
3. Add new environment variable JAVA_HOME with value C:\Program Files\Java\jdk1.5.0_17 (this directory was created with installation of Java SDK in Proof Of Concept 3 of 8)
4. To test whether it installed correctly, run:

   
   jruby -v 
   

   
   

The above command will return the current version.

Proof Of Concept – 4 of 8 : IB's Java Test Client

Next step is to use NetBeans to compile and run IB's Java test client. The process is described in documentation JavaAPIGettingStarted.pdf (Chapter 5) on IB's web site. It's a good idea to read this doc and follow the directions. Running the Java test client basically consists of:

1. Create a new Java application project in NetBeans
2. Attach packages in Java\com and Java\TestJavaClient folders (see Proof Of Concept - 2 of 8) to the project
3. Press F6 to run
4. Test client magically runs and ready to connect to TWS

Well, this didn't quite work out for me. NetBeans was unable to compile the Java program. Note the red exclamation marks in the upper left corner.



Looking a little deeper at the API code, looks like there's some sort of problem with loading the classes in IB's packages. NetBeans just doesn't like the package names on all the Java class files.


I was stuck at this step for a little while but was able to resolve it (with a little bit of hacking). The issue is that NetBeans is unable to load IB's package names com.ib.client and TestJavaClient defined in Java classes in Java\com and Java\TestJavaClient folders. So some editing of the Java class files are required.


1. In Explorer, copy the two folders in Java\ folder and paste them into a new folder. This is where edited versions will be. I just don't like to lose the original version in case something goes wrong. I named this new folder twsapi\.

2. In NetBeans, for each .java file in ib.client package:

• At the top of the file, change text package com.ib.client; to package ib.client;
• Save, and there shouldn't be any errors

3. In Explorer, rename folder hierarchy from twsapi\TestJavaClient to twsapi\x\y\TestJavaClient



now becomes






4. In NetBeans, for each .java file in TestJavaClient package:

• At the top of the file, change text package TestJavaClient; to package y.TestJavaClient;
• In the rest of the file, change any occurrence of text import com.ib.client. to import ib.client
• Save, and there shouldn't be any errors

5. Create a new Java application project.

6. Attach packages in twsapi\com and twsapi\x folders to the project. Note no more red exclamation marks in the upper left corner.



7. Press F6 to run.

8. Test client runs and ready to connect to TWS. Try out its functionalities as specified in documentation JavaAPIGettingStarted.pdf.

Proof Of Concept - 3 of 8: Install Java SDK and Netbeans

Java Standard Edition Software Development Kit ( SE SDK) and NetBeans IDE must be installed in order to compile and run Java programs. Go to Sun's Java SE Downloads page and locate JDK 5.0 Update 17 with NetBeans IDE 6.5. As of this writing, Java latest release is JDK 6 Update 11. I was going to use this latest release, but after reading through some postings on IB's TWS API forum, some users experienced performance problems with JDK 6 and IB's API. This was not the case with JDK 5. So I've decided to use this the older.

The installation process installs Java SE SDK, Java SE Runtime Environment, and NetBeans IDE in separate folders. Mine are in c:\Program Files\Java\jdk1.5.0_17, c:\Program Files\Java\jre1.5.0_17, and c:\Program Files\NetBeans 6.5. Make a note of these as they will be needed later.

To check installation, run NetBeans and run sample projects that came with it. Just select the project and Run the project (F6).

Proof of Concept - 2 of 8 : IB's API

To install IB's TWS API, go to IB's Programming API page and download the software. I'm using Windows API version 9.51 dated Apr 25 2008. Note that this will install all the APIs: Excel DDE, Java, ActiveX, and C++.

The install is straight forward. Just run the install executable and it does the rest. I've chosen to install mine in the same folder as the TWS client (c:\jts). When done, make note of folders Java\com\ib\client and Java\TestJavaClient.



Java\com\ib\client folder contains all the API classes which will be needed in my trading robot.

Java\TestJavaClient folder contains IB's test application and is used to make sure that basic functionalities of the API works. I will use this code to make sure my development environment is set up properly, but will not need it for the trading robot.

Proof of Concept - Overview

The first thing to do is a proof of concept to make sure that IB API - Java - JRuby will all work together. So a number of installs and sample programs need to be written.

Here is the task list (details of each step to follow in separate posts):

1. Obtain paper trading user name and password for Trader Work Station. This account will be used for testing.
2. Install IB's Application Program Interface Software (v 9.51) - This is IB's proprietary API. The trading robot will use these APIs to interact with Trader Work Station.
   
3. Install Java SE SDK (v 5.0 Update 17) and Netbeans - Java SDK will enable Java programs and create Java Archive (.jar) files. Netbeans integrated development environment will be used to work with Java and Ruby code.
   
4. Compile and run IB's Java test client.
5. Create Java Archive file (.jar) with IB API classes.
6. Install JRuby (v 1.1.6) .
7. Enable Ruby plugins in NetBeans.
   
8. Write Ruby programs to test IB API running in JRuby.