Sharagoz' blog

Rails + SQL Anywhere 10 on Linux

2011-10-20T13:57:42+02:00

I recently had to make a rails app talk to an sqlanywhere database. The app was to be deployed on a linux server that runs nginx + passenger. This is a fairly simple process, but there arent a lot of information on it on the internet so I'm writing a quick article.

Setting up a development environment on (arch) linux:

Here's how to make a rails app talk to a local database server. If you wanna talk to an external DB then see the section on deployment.

(The paths used here assumes that you install sql anywhere to /opt/sqlanywhere10 and that you use the 64-bit version)

The first step is to download and install the sql anywhere 10 developer edition for linux and get it patched to the latest version. I cant link to the tarball as it requires registration.
Extract the files and run the setup file. The installation is straight forward. I installed all the modules to be able to set up a local database server and use the Sybase Central. This requires a developer key, which my company already had. I think you can get one by registering on the sybase site.

Right out of the gates you will have a couple of issues. Try to check which version you have:
/opt/sqlanywhere10/bin64/dbsrv10 -v
You will probably get this error:
error while loading shared libraries: libdbserv10_r.so: cannot open shared object file: No such file or directory
This is because sql anywhere requires some environment variables to be set up
You can set these up by sourcing a .sh script like so:
source /opt/sqlanywhere10/bin64/sa_config.sh
Try checking the version again. It should show something like:
10.0.1.4239

If your version is below 10.0.1.4000 you'll most likely run into issues down the road. Get a hold of the update patch (again I cant link, as it requires login). Do not continue until you get this resolved.

Having to source that sa_config.sh file every time you open a new shell gets annoying, so I recommend you add the command to /etc/profile or ~/.bashrc

Btw, you may want to check out Sybase Central. It can be started with this command (if you have sourced that config file):
scjview

Sybase central enables you to do administer databases through a GUI, which is helpfull.

Right, once you have a patched version of sqlanywhere 10 it is time to start a database server. Most likely you have an existing database you can use, otherwise you can create one through Sybase Central.

This command launches the server:
/opt/sqlanywhere10/bin64/dbsrv10 -n your_sa_server_name /path/to/database/file



If everything is set up correctly you should get a stream of output that ends with these lines:



Starting on port 2638

TCPIP link started successfully

Now accepting requests



Now we need to configuring Rails to talk to this database server and get the active record adapter installed.

Add the activerecord-sqlanywhere-adapter to the gemfile:



gem 'activerecord-sqlanywhere-adapter'



And run bundle install

Set up database.yml



development:

  adapter: sqlanywhere

  server: your_sa_server_name # This must match what you set as the -n directive when you started the database server

  database: filename # This must match the filename of your database file. E.G: If the filename is my_database.db then use "my_database"

  host: localhost

  encoding: utf-8



Start a console with rails c and execute a command to check that everything is OK:



ActiveRecord::Base.connection.execute("SELECT * FROM my_table")



At the point of writing there are some issues with the sqlanywhere adapter for active record when used with Rails 3.0 and 3.1. I have a fork on Github that fixes some of these issues. Hopefully the maintainer will look into it soon and get it fixed properly in the main repo.

Setting up the production environment on (arch) linux:


When installing on the server you only need the module called "Network Client" (no licence key required) if you are going to talk to a database on a different server.

Make sure you apply the patch on the server too.

When you are talking to an external database server you will have to specify its IP in database.yml



production:

  adapter: sqlanywhere

  database: external_db_name

  server: external_server_name

  commlinks: TCPIP(IP=80.80.80.80:2638)

  encoding: utf-8



When you deploy the app you'll most likely get this error "Could not load SQLAnywhere DBCAPI library". That is because those environment variables are missing again. On the server you cant simply source the script through /etc/profile or ~/.bashrc because the web server is not launched through a shell. Find a way to make the application server source the file.

I am using Nginx + passenger, and the way to do it here is to create a custom ruby wrapper that launches passenger. Here's how:

In nginx.conf there is a line that looks like this:



passenger_ruby /usr/local/rvm/wrappers/ruby-1.9.2-p180/ruby



Edit that command to use a custom wrapper that you'll define. E.G:



passenger_ruby /usr/local/rvm/wrappers/ruby-1.9.2-p180/custom_ruby



Copy the script from the original wrapper into the new custom wrapper and add the source command where appropriate. My wrapper looks like this:



#!/usr/bin/env bash


if [[ -s "/usr/local/rvm/environments/ruby-1.9.2-p180" ]]

then

  source "/opt/sqlanywhere10/bin64/sa_config.sh" # This is the custom line

  source "/usr/local/rvm/environments/ruby-1.9.2-p180"

  exec ruby "$@"

else

  echo "ERROR: Missing RVM environment file: '/usr/local/rvm/environments/ruby-1.9.2-p180'" >&2

  exit 1

fi



Restart nginx and that missing dbcapi error should be gone.

If you run into any issues, feel free to message me on Github and I'll try to assist.



Charles Max Wood is putting out great screencasts
2011-07-24T00:38:13+02:00
I think this guy deserves more attention than he seem to be getting.  

Keep up the good work Max!


User stamping in Rails 3
2011-07-21T22:39:36+02:00
If you used the user_stamp gem in Rails 2 to maintain creator_id and updater_id columns in every table, you'll find that it doesn't work in Rails 3. I have a patched version on my Github account that works in production mode in Rails 3, but falls victim to class reloading in development mode. Instead of fixing that by breaking the old API, I'm just gonna make a quick post here on how you can implement the functionality yourself without depending on a gem. It's so simple that it's not really worth an external dependency.

Add a new file to app/models called user_stamper.rb:



class UserStamper < ActionController::Caching::Sweeper

  observe Post, Comment, User #replace with your own models


  def before_validation(record)

    return unless current_user

    record.send(:creator_id=, current_user.id) if(record.new_record? && record.respond_to?(:creator_id=))

    record.send(:updater_id=, current_user.id) if(record.changed? && record.respond_to?(:updater_id=))

  end

  

  private  

  def current_user

    controller.send(:current_user) if(controller.respond_to?(:current_user))

  end

end



You'll also need this line in the ApplicationController:



cache_sweeper :user_stamper



That's all there is to it.


Interacting with Javascript prompts with Capybara
2011-07-21T21:50:35+02:00
I tried switching from Webrat to Capybara about 4 or 5 months ago, but aborted it due to some reason I can't recall. I think it was because the "have_selector" matcher wasn't implemented yet. I decided to give it another try today hoping to be able to implement some of those pending Javascript related specs Webrat was unable to handle.

And I was not disappointed, Capybara has excellent support for the selenium-webdriver. Look at this:



find('span', :text => 'Register your email address').click

prompt = page.driver.browser.switch_to.alert

prompt.send_keys('me@example.com')

prompt.accept

Email.first.address.should eq('me@example.com')



I am clicking a non-button/link that triggers a javascript prompt. I fill out the text field in this prompt and then click the "OK" button, and it works!

I'm in love with Capybara now.

Btw, converting the existing specs created for Webrat was a very easy job.

There was some syntax changes to the "have_selector" matcher, calls to "response" had to be replaced by "page", and the "contains" matcher had to be replaced by "have_content". Easy.

Right now I can't think of a single reason to choose Webrat over Capybara. Webrat isnt even maintained anymore as far as I can tell.


Rails exception handler gem released
2011-07-17T18:58:45+02:00
Quick announcement:

I just released a gem called "Rails Exception Handler", which is based on an earlier blog post of mine on rails exception handling. Check out the project on Github


Changing the repository location for gitosis
2011-05-24T17:16:35+02:00
This is how I did it on my company's git server, which resides on an Arch Linux box. Obviously, it's a very good idea to make a backup before starting a procedure like this.

To change the location of the repo you need to do 3 things:

1. Add a line to gitosis.conf telling it that the repo is not in the default location

2. Move the repository folder

3. Update a symlink to gitosis.conf

In the example below I am moving the repo from /srv/gitosis/repositories to /data/gitosis/repositories. Inside /srv/gitosis there is a symlink that looks like this, which has to be updated afterwards:

.gitosis.conf -> /srv/gitosis/repositories/gitosis-admin.git/gitosis.conf

gitosis.conf

I assume you are familiar with gitosis.conf, where you configure users, repos etc. Add this line:

[gitosis]

repositories = /data/gitosis/repositories

Important: Push these changes before you move the repository folder

Moving the repository folder

mv /srv/gitosis/repositories /data/gitosis/repositories

Update the symlink

ln -s -f /data/gitosis/repositories/gitosis-admin.git/gitosis.conf /srv/gitosis/.gitosis.conf

Couldn't find any info on this online, so I wrote this post. Hope it helps.


Doctype TV needs more love
2011-03-15T20:29:51+01:00
I can't belive these guys doesn't have more subscribers. They produce high quality screencasts on web UI related topics. Check them out and spread the word!

http://www.youtube.com/user/doctypetv


Rolling your own exception handler in Rails 3
2011-03-15T18:07:36+01:00
In Rails 2 you could override ActionController's "rescue_action_in_public" to do your own exception handling.

For Rails 3 the request handling was re-written to take full advantage of Rack, so now you need to use Rack middleware to catch and handle the exceptions manually.

For an introduction to Rack middleware you can checkout Railscasts ep. 151.

I will show you how to create a middleware app that serves as a backend for tracking exceptions in your rails apps. Catch the exception, store the relevant data in the database, and display a custom error message to the user. That's what I will cover. The front end is not the scope of this post.

The middleware skeletonCreate a file in the lib folder and call it "exception_handler.rb". Here's the minimal code that will get us started:



class ExceptionHandler

  def initialize(app)

    @app = app

  end

  def call(env)

    @app.call(env)

  rescue

    ["200", {"content-type" => "text/html"}, "An error has occured"]

  end

end



To hook the error handler into the application, add these lines to config/application.rb



require 'exception_handler'

config.middleware.use ExceptionHandler



For production usage you'd move these into production.rb, as you want regular error messages in development. If you trigger an error now you should receive a response that says nothing more than "An error has occured".

Note: What we have so far will not catch routing errors, I'll get to that in a minute.

Let's start by gathering the information we want to save.

Create a new class called "Parser" and define it within the ExceptionHandler class.



class Parser

  def initialize(exception, request)

    @exception = exception

    @request  = request

  end


  def relevant_info

    info = {}

    info[:class_name] = @exception.class.to_s # The exception class, "ActiveRecord::RecordNotFound"

    info[:message] = @exception.to_s # The actual error message, "Couldn't find Post with id=42"

    info[:trace] = @exception.backtrace.join("\n") # I bet you have seen a backtrace before

    info[:target_url] = @request.url # Which URL did the client try to access

    info[:referer_url] = @request.referer # Which URL did he come from

    info[:params] = @request.params.inspect # The request parameters

    info[:user_agent] = @request.user_agent # The user agent string

    return info

  end

end



To store this in the database we need a database table and a model:



rails g model ErrorMessage class_name:text message:text trace:text target_url:text referer_url:text params:text user_agent:text

rake db:migrate



To use the class we just made, change the code of the rescue block in ExceptionHandler:



rescue Exception => e

  request = ActionDispatch::Request.new(env)

  parsed_error = Parser.new(e, request)

  ErrorMessage.create!(parsed_error.relevant_info)

  ["200", {"content-type" => "text/html"}, "An error has occured"]

end



I am creating a new request object here. It is a bit easier to extract information from it than the env object itself.

Give this a test. What we have made so far is the gist of it. We catch the exception and store the info we want in the database. There are some important details to take care of though, before it can be put into production.

FilteringCrawlers and bots are going to generate a bunch of errors, regardless of what you put in robot.txt. Let's add some basic filtering. Change the create statement to this:



ErrorMessage.create(parsed_error.relevant_info) unless(parsed_error.ignore?)



Then define the ignore? function on the Parser class like so:



def ignore?

  # Ignore routing errors in requests without a referer as they are going to be bots in 99.99235% of the cases

  routing_errors = [ActionController::RoutingError, AbstractController::ActionNotFound, ActiveRecord::RecordNotFound]

  if(routing_errors.include?(@exception.class) && @request.referer.blank?)

    return true

  end

  # Ignore requests with user agent string matching this regxp as they are surely made by bots

  if(@request.user_agent =~ /\b(Baidu|Gigabot|Googlebot|libwww-perl|lwp-trivial|msnbot|SiteUptime|Slurp|WordPress|ZIBB|ZyBorg|Yandex|Jyxobot|Huaweisymantecspider|ApptusBot)\b/i)

    return true

  end

  return false

end



That's all the filtering I've needed. Some chose to ignore all routing errors, I prefer not to. Mortals like myself sometimes put broken links into production. If you start to get a large amount of error messages, it might be better to group them on the front end than apply a lot of filtering on the back end.

Catching routing errorsAs I said before, we are not catching routing errors yet. To catch those you need to override a method in ActionDispatch:

Put this at the end of error_handler.rb



module ActionDispatch

  class ShowExceptions

    private

      def render_exception_with_template(env, exception)

        # Handle error and return response

      end

      alias_method_chain :render_exception, :template

  end

end



Credit: I got that snipped from a blog post by AccUser.

A bit of refactoringInstead of copy pasting the code from the ExceptionHandler's rescue block into the ActionDispatch method, I'm gonna be DRY and extract the code into its own class, so that maintaining and expanding the code is easier. Define a new class within the ExceptionHandler class called "Handler":



class Handler

  def initialize(env, exception)

    @exception = exception

    @env = env

  end

  def handle_exception

    request = ActionDispatch::Request.new(@env)

    parsed_error = Parser.new(@exception, request)

    ErrorMessage.create(parsed_error.relevant_info) unless(parsed_error.ignore?)

    return response

  end

  def response

    ["200", {"content-type" => "text/html"}, "An error has occured"]

  end

end



Replace the rescue block with this line:

Handler.new(env, e).handle_exception

Replace the comment in the ActionDispatch function with this line:

ExceptionHandler::Handler.new(env, exception).handle_exception

With that you should be catching the routing errors as well.

Displaying a nicer error messageI prefer to display the error messages within the regular layout of the page. It looks nicer and feels less like "MAYDAY! MAYDAY!" for the user.

Trying to send responses with layouts without using a controller in Rails takes a bit of work, so I'm gonna define an error response controller in the exception catcher to take care of this. Put this at the bottom of exception_catcher.rb:



class ErrorResponseController < ActionController::Base

  def index

    render(:text => "An error has occured
The error message has been logged", :layout => 'layouts/application')

  end

end



Redefine the response function in the Handler class so that it calls upon this controller to generate the response:

ErrorResponseController.action(:index).call(@env)

The error message should now be rendered within the default layout of the site.

Regular loggingEven with your own error handler you may want to continue having the errors logged to the default log file.

If you want this, add this function to the Handler class:



def log_error(info)

  message = "#{info[:class_name]} (#{info[:message]}):\n  "

  message += Rails.backtrace_cleaner.clean(info[:trace].split("\n")).join("\n")

  Rails.logger.fatal(message)

end



Add this line to the handle_exception function:

log_error(parsed_error.relevant_info)

That will log an error to the log file that looks identical to the one Rails creates by default, without any filtering applied.

Keeping track of who triggered the request
In applications that has user accounts it is usefull to keep track of who experienced the error. If a critical error occurs it can enable you to contact the user directly. I once had an incident where users got an error page displayed after pressing "confirm order", which is about as bad as it gets, and knowing the login name of the ones who got this error enabled me to contact each one and tell them that the order went through OK.

Add a new function to the Parser class:



def user_info

  if(@controller.respond_to?(:current_user))

    current_user = @controller.current_user

    [:login, :username, :email].each do |field|

      return current_user.send(field) if(current_user.respond_to?(field))

    end

  end

  return nil

end



I made the function probe for fields on the user object, as I dont universally use the same login field in all my apps. If you dont always use "current_user" you can easily expand the function to probe for the helper method as well.

Add the line below to the relevant_info function and add the new field to the database table.

info[:user_info] = user_info

The initializer on the Parser class must be expanded to take the controller instance:



def initialize(exception, request, controller)

  @exception = exception

  @request  = request

  @controller = controller

end



and finally, the handle_exception function must fetch the controller instance and pass it to the handler



controller = @env['action_controller.instance']

parsed_error = Parser.new(@exception, request, controller)



That's it. You can get the entire code of exception_handler.rb from here. I defined all the code in one file here to keep the handler easily contained. It's a lot cleaner to split this up into several files and keep it contained as a gem/plugin instead.

You can use this code as a basis for making your own error tracking system. It wouldnt take a lot of work to extract this into a gem and have all your applications store error messages in the same database. With a simple web front on top of that you have a pretty decent system in place already that you can modify and expand as needed.

Happy bug hunting!