rails on ruby 1.9.1 on ec2

amazon ec2 currently offers fedora servers preinstalled with ruby 1.8.6 which is used by its ami tools. while this is a good basic configuration, one may want to upgrade to ruby 1.9.1 which offers many improvements, performance being the key one.

upgrading to ruby 1.9.1 on an ec2 fedora server, and running a rails application on top of it is a multi step process but a straight forward one. below are the steps I took to get my amazon ec2 hosted production environment migrated to ruby 1.9.1

step I: launch your ec2 instance
this post assumes you know how to launch an ec2 instance, in my case i used ec2's 64bit basic fedora ami, i found the 32bit unusable due to stability issues i experienced with the original kernel. more details on this are logged here

step II: install ruby 1.9.1
first we want to get the dependancies out of the way, in my case i needed to install several libraries that will be later used by my rails app, you may have a smaller or larger dependencies list depending on the gems you need to run.

yum update
yum install httpd httpd-devel mysql mysql-devel libxml2 libxml2-devel gcc-c++

now lets install ruby 1.9.1, I choose to install in in my /opt directory, but you may choose elsewhere. we will be suffixing ruby with the "19" suffix so that we can run ruby 1.9.1 side by side with the existing ec2's ruby 1.8.6

cd /opt
wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p376.tar.gz
tar xvfz ruby-1.9.1-p376.tar.gz
cd ruby-1.9.1-p376
./configure --enable-shared --program-suffix=191
make
make install

now that we have ruby 1.9 compiled and installed, it's time to do some renaming and symbolic linking so that the system uses ruby 1.9.1 by default while overcoming ec2's hard-wired ruby 1.8.6 installation

mv /usr/bin/ruby /usr/bin/ruby186
mv /usr/bin/irb /usr/bin/irb186
mv /usr/bin/erb /usr/bin/erb186
mv /usr/bin/testrb /usr/bin/testrb186
mv /usr/bin/gem /usr/bin/gem186
mv /usr/bin/rake /usr/bin/rake186
mv /usr/bin/ri /usr/bin/ri186
mv /usr/bin/rdoc /usr/bin/rdoc186

ln -s /usr/local/bin/ruby191 /usr/bin/ruby
ln -s /usr/local/bin/rake191 /usr/bin/rake  
ln -s /usr/local/bin/gem191 /usr/bin/gem  
ln -s /usr/local/bin/irb191 /usr/bin/irb  
ln -s /usr/local/bin/erb191 /usr/bin/erb  
ln -s /usr/local/bin/ri191 /usr/bin/ri  
ln -s /usr/local/bin/rdoc191 /usr/bin/rdoc  
ln -s /usr/local/bin/testrb191 /usr/bin/testrb  

To confirm you are running the correct ruby version, run
>> ruby -v
you should see (assuming you download same revision as i did)
ruby 1.9.1p376 (2009-12-07 revision 26041) [i686-linux]

step III: installing rails
this step can prove to be more involved than others, depending on the gems you need to run. I have a detailed post on this here. the original post was written for OSX, but with the minor differences in ruby's location is perfectly correct any *nix.

step IV: bundling your custom AMI
now that we are done setting up the server, the next natural step is to bundle it as a custom ami so we can reuse it.

an issue we run into is that ec2's ruby based ami tools are not ruby 1.9 compatible, more specifically the ec2_upload_bundle tool which we need to use in order to bundle. luckily, we installed ruby 1.9.1 side by side to the original 1.8.6, so we have an easy way out: simply use ruby 1.8.6 to run the tools. to do so, change ec2 wrapper scripts in /usr/local to use ruby18 instead of ruby. for example change /usr/local/bin/ec2-bundle-vol from

#!/bin/bash
ruby /usr/lib/site_ruby/ec2/amitools/bundlevol.rb $*

to

#!/bin/bash
ruby186 /usr/lib/site_ruby/ec2/amitools/bundlevol.rb $*

apply the same principle to the rest of the ec2 wrapper scripts in /usr/local

Fixing soap4r for ruby 1.9

soap4r 1.5.8 is broken on ruby 1.9, below are the steps I took to fix it. Please note I have not attempted to fix soap4r completely, and that this fix does not go beyond the purposes of my usage, there are likely to be areas that are still broken under 1.9.

lib/soap/baseData.rb

replace

TypeParseRegexp = Regexp.new('^(.+)\[([\d,]*)\]$', nil, 'NONE')

with

TypeParseRegexp = Regexp.new('^(.+)\[([\d,]*)\]$')

lib/soap/generator.rb

replace

Regexp.new("[#{EncodeMap.keys.join}]", nil, XSD::Charset.encoding)

with

Regexp.new("[#{EncodeMap.keys.join}]")

lib/soap/property.rb

replace

COMMENT_REGEXP = Regexp.new('^(?:#.*|)$', nil, 'u')
CATDEF_REGEXP = Regexp.new("^\\[\\s*#{KEY_REGSRC}\\s*\\]$", nil, 'u')
LINE_REGEXP = Regexp.new("^#{DEF_REGSRC}$", nil, 'u')

with

COMMENT_REGEXP = Regexp.new("^(?:#.*|)$")
CATDEF_REGEXP = Regexp.new("^\\[\\s*#{KEY_REGSRC}\\s*\\]$")
LINE_REGEXP = Regexp.new("^#{DEF_REGSRC}$")

remove / comment out

# for ruby/1.6.
unless Enumerable.instance_methods.include?('inject')
  module Enumerable
    def inject(init)
      result = init
      each do |item|
       result = yield(result, item)
      end
      result
    end
  end
end

lib/soap/rpc/element.rb

replace

type_qname = TypeMap.index(mapped_class)

with

type_qname = TypeMap.key(mapped_class)

lib/xsd/charset.rb

replace

@internal_encoding = $KCODE

with

@internal_encoding = "UTF8"

replace

USASCIIRegexp = Regexp.new("\\A#{us_ascii}*\\z", nil, 'NONE')

with

USASCIIRegexp = Regexp.new("\\A#{us_ascii}*\\z")

replace

EUCRegexp = Regexp.new("\\A#{character_euc}*\\z", nil, 'NONE')

with

EUCRegexp = Regexp.new("\\A#{character_euc}*\\z", nil, 'n')

replace

SJISRegexp = Regexp.new("\\A#{character_sjis}*\\z", nil, 'NONE')

with

SJISRegexp = Regexp.new("\\A#{character_sjis}*\\z", nil, 'n')

replace

character_utf8 =
"(?:#{us_ascii}|#{twobytes_utf8}|#{threebytes_utf8}|#{fourbytes_utf8})"
UTF8Regexp = Regexp.new("\\A#{character_utf8}*\\z", nil, 'NONE')

with

character_utf8 = "(?:#{us_ascii}|#{twobytes_utf8}|#{threebytes_utf8}|#{fourbytes_utf8})"
UTF8Regexp = Regexp.new("\\A#{character_utf8}*\\z", nil, 'n')

lib/xsd/ns.rb

replace

ParseRegexp = Regexp.new('\A([^:]+)(?::(.+))?\z', nil, 'NONE')

with

ParseRegexp = Regexp.new('\A([^:]+)(?::(.+))?\z')

lib/xsd/xmlparser.rb

replace

NSParseRegexp = Regexp.new('^xmlns:?(.*)$', nil, 'NONE')

with

NSParseRegexp = Regexp.new("^xmlns:?(.*)$")

replace

raise LoadError unless XSD::XMLParser.constants.find { |c| c.downcase == name }

with

raise LoadError unless XSD::XMLParser.constants.find { |c| c.to_s.downcase == name.downcase }

Running a Ruby on Rails Application on OSX with Ruby 1.9

General

OSX comes preinstalled with ruby 1.87, however, ruby 1.9 has out there for quite a while now and does offer improvements in many areas, including performance, which is always a good trigger for an upgrade. A deeper overview can be found here.

Ruby 1.9

You can use MacPorts, Fink or compile from source. Hivelogic has a good post on how to install ruby on OSX here. I have chosen to use MacPorts as I use it for other ports:

$ sudo port install ruby1.9

If you are doing a side by side installation (i.e keeping you original ruby installation in place), you may want to do some symbolic linking to make sure the ruby, irb, gems, rake, ri and rdoc are pointing the 1.9 version. If you choose to skip this step, remember to use the ruby1.9, irb1.9, etc command instead.

Disclaimer: Please be careful with the below, the order and locations matter and one installation (yours) can be different than other (mine), if you are not familiar with OSX, bash and symbolic linking I suggest you do some readying before continuing with this step or just stick to using the 1.9 command namespace (See above).

The example below assume you ruby 1.9 is installed at /opt/local/bin/ruby1.9 (MacPort default location) and that your original ruby installation was a /System/Library/Frameworks/Ruby.framework/Versions/1.8 (OSX default location).

$ cd /System/Library/Frameworks/Ruby.framework/Versions
$ mkdir 1.9.1
$ mkdir 1.9.1/usr
$ mkdir 1.9.1/usr/bin

$ cd 1.9.1/usr/bin
$ ln -s /opt/local/bin/ruby1.9 ruby
$ ln -s /opt/local/bin/rake1.9 rake
$ ln -s /opt/local/bin/gem1.9 gem
$ ln -s /opt/local/bin/irb1.9 irb
$ ln -s /opt/local/bin/erb1.9 erb
$ ln -s /opt/local/bin/ri1.9 ri
$ ln -s /opt/local/bin/rdoc1.9 rdoc
$ ln -s /opt/local/bin/testrb1.9 testrb

$ cd /System/Library/Frameworks/Ruby.framework/Versions
$ rm Current
$ ln -s 1.9.1 Current

$ cd /usr/bin
#for some reason apple is linking directly to the v. 1.8 folder instead of the current directory, we will "fix" that.
$ mv gem gem1.8
$ ln -s /System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/gem
#for some reason apple has put the rake executable in /usr/bin instead of in the current; directory, we will "fix" that.
$ mv rake rake1.8
$ ln -s /System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/rake

To confirm you are running the correct ruby version:
$ ruby -v
$ gem env
$ irb
>> RUBY_VERSION

Rails

As you are installing a new ruby, you also need to reinstall all of you gems, as this artical is focused on ruby on rails, we will start with the rails gem. There is nothing exciting here, simply run the standard rails installation command

$ gem update --system
$ gem install rails

Other Gems

For the purposes of my application, I have installed the follwing gems:
actionmailer (2.3.4, 2.3.3)
actionpack (2.3.4, 2.3.3)
activerecord (2.3.4, 2.3.3)
activeresource (2.3.4, 2.3.3)
activesupport (2.3.4, 2.3.3)
builder (2.1.2)
cgi_multipart_eof_fix (2.5.0)
daemons (1.0.10)
fast_xs (0.7.3)
fastthread (1.0.7)
gem_plugin (0.2.3)
httpclient (2.1.5.2)
libxml-ruby (1.1.3)
macaddr (1.0.0)
mongrel (1.1.6)
mongrel_cluster (1.0.5)
mysql (2.8.1)
rack (1.0.0)
rails (2.3.4, 2.3.3)
rake (0.8.7)
rubygems-update (1.3.5)
rubyzip (0.9.1)
soap4r (1.5.8)
uuid (2.0.2)

Hacks

This is where the fun begins, some of the gems and rails modules are not fully baked for ruby 1.9, below are a list of hacks that I had to plug in to make my rails app work. Naturally, your case may be different, a good reference to gems under 1.9 can be found here.

mongrel gem
The 1.1.5 mongrel gem does not install over ruby 1.9. You may choose to fix it manually or install 1.1.6 from gems.rubyinstaller.org:

$ gem install mongrel --source http://gems.rubyinstaller.org

More info on this can be found here.

mongrel cluster
Needs a small syntax patch, fix the case statement at the end of the mongrel_cluster_ctl by replacing ":" with "then".

restful_authentication plugin
If you are using restful_authentication plugin you are likely to crash as soon as the plugin attempts to read data from cookies. This is due to a bug in active_support module. There is a documented patch here.

uploading
Most applications use some sort of uploading, mine does too. Rack 1.0.0 on 1.9 crashes does to unicode issues. There is a documented patch here.

ftools
ftools has been deprecated in 1.9, if your code is dependent on it, move to fileutils.

rubyzip gem
Uses ftools, you will need to remove the dependency. There is a documented patch here.

libxml-ruby gem
I found this gem to be unstable on 1.9, throwing many memory allocation errors. The solution was to update the libxml libs and then install the gem against the updated sources:

$ port install libxml2
$ gem install libxml-ruby -- --with-xml2-include=/opt/local/include/libxml2

More info on this can be found here.

soap4r
Broken, I monkey patched locally which was enough for my usage, but not a compelte solution. More info here.

These steps got my app on its feet, however, I am still testing stability and performance under the new setup. I will update this post if and when new problems or solutions come up.

Compiling mod_xsendfile on OSX

mod_xsendfile is an apache2 module which come handy when you need to serve static files through a secondary application layer or a proxied web server, both tend to serve static files slower and less efficiently than apache does. In such cases you want to delegate the request (file path) to apache so that the secondary layer/proxied web server are freed to move on and process other requests. A common example is a php download handler that needs to go to the db to fetch the file path, check permissions, etc but don't want to deal with the actual file transfer, another common example is a ruby on rails proxied web server such as mongrel which is slow in serving static files under stress due it's concurrency model and can quickly become a bottle neck. For more information on mod_xsendfile and how to configure it go here.

The above information well known, however mod_xsendfile installation documentation states the following:

1. Grab the source.
2. Compile and install:
>> apxs -cia mod_xsendfile.c
3. Restart apache
4. That's all.

If you try this on OSX the compilation step will fail due to missing architecture flags, hence, use the following instead:

>> apxs -cia -Wc,"-arch x86_64 -arch ppc -arch i386 -arch ppc64" -Wl,"-arch x86_64 -arch ppc -arch i386 -arch ppc64" mod_xsendfile.c

This will compile mod_xsendfile in all 4 OSX flavors, which should cover your case. Naturally, you can change the architecture flag to just the one that is relevant to your hardware. More info here.

Configuring CruiseControl as an OSX Startup Daemon

General

The purpose of this post is to describe how to configure a startup daemon for cruisecontrol on OSX. While CruiseControl ships with a startup script (cruisecontrol.sh), one would like CruiseControl to automatically start after system startup to avoid the manual step of starting it.

This post assumes previous knowledge with OSX, bash, and CruiseControl and does not attempt to cover how to install or generally configure CruiseControl on OSX. There are several articles covering the latter, while the topic of configuring CruiseControl startup daemon is only partially covered.

Creating the Daemon

• Navigate to the startup daemons folder

$ cd /Library/LaunchDaemons

• Create a new plist file named org.sourceforge.cruisecontrol.plist, the plist file will look something along these lines:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
 <dict>
         <key>Label</key>
         <string>org.sourceforge.cruisecontrol</string>
         <key>WorkingDirectory</key>
         <string>/usr/local/cruisecontrol</string>
         <key>ProgramArguments</key>
         <array>
                 <string>/System/Library/Frameworks/JavaVM.framework/Home/bin/java</string>
                 <string>-Xms128m</string>
                 <string>-Xmx256m</string>
                 <string>-Djavax.management.builder.initial=mx4j.server.MX4JMBeanServerBuilder</string>
                 <string>-Dcc.library.dir=/usr/local/cruisecontrol/lib</string>
                 <string>-Djetty.logs=/usr/local/cruisecontrol/logs</string>
                 <string>-jar</string>
                 <string>/usr/local/cruisecontrol/lib/cruisecontrol-launcher.jar</string>
                 <string>-jmxport</string>
                 <string>8000</string>
                 <string>-webport</string>
                 <string>8080</string>
                 <string>-rmiport</string>
                 <string>1099</string>
         </array>
         <key>EnvironmentVariables</key>
         <dict>
         </dict>
         <key>RunAtLoad</key>
         <true/>
         <key>Uses</key>
         <array>
                 <string>Network</string>
                 <string>Resolver</string>
         </array>
 </dict>
</plist>

Whats in the plist file?

• Label => Name of daemon, set to org.sourceforge.cruisecontrol

• WorkingDirectory => Location of the CruiseControl installation

• ProgramArguments => Java calls to cruisecontrol-launcher jar including configuration for CruiseControl port numbers, libs and log location. Add additional parameters as you see fit

• RunAtLoad => Run at startup, set to true

• Uses => Dependency on other services, in our case networking and resolver

Testing the Daemon

• Loading:

$ launchctl load org.sourceforge.cruisecontrol.plist

Use the above command to load the daemon. The system will do the same upon reboot, so this is only required when you are testing the daemon.

• Unloading:

$ launchctl unload org.sourceforge.cruisecontrol.plist

Use the above command to unload the daemon if and when it fails and you need to reload it again.

• Testing to see that the daemon is actually loaded

$ launchctl list

If everything went well, you should see you daemon listed (org.sourceforge.cruisecontrol) with a valid pid.

You can also monitor the startup process on cruiscontrol.log, or if you access to he UI, through the system's console.