Coreguardian

Rails security resources

2010-08-30T23:57:40Z

As we make the push toward releasing the platform that I'm working on we've installed exception_notification in our Rails app. With the increased visibility of all the exceptions it became apparent quite quickly that there were numerous hits against the server from automated vulnerability scanners. These attempts were causing routing errors as they looked for paths like '/user/soapCaller.bs' – thankfully not targeting Rails applications.

The arrival of this sort of scan was not particularly surprising to me as I've seen similar scans in the past. I've even actively dabbled in some security research by running a few honeypot projects, I digress.

Even though these scans usually go after large installations such as WordPress, Drupal, Joomla and phpMyAdmin, it isn't stupid to take it as a reminder to keep up to date on security vulnerabilities. In the case of Ruby on Rails the starting point would be the rubyonrails-security google group and the Ruby on Rails blog.

Another great resource is Rails Inside. Rails Inside usually picks up any serious flaws and relays them to the community. In addition to this, they follow new releases of popular plugins/gems that may form part of your app. The site provides an important service because keeping up-to-date is a good way of reducing the risk of being caught by a vulnerability that has been dutifully patched by the maintainer.

The above is certainly not a complete list, so I'd like to hear if there are any other rails/ruby security resources that you find useful.

Security wake up call I don’t think that I’ve had this much adrenalin pumping...

Related posts brought to you by Yet Another Related Posts Plugin.

This week I learned – that bundler needs work

2010-08-01T12:41:10Z

One of the challenges over the last few weeks has been switching development environments to phase in the new Macbook Pros (sweet!). One of the problems has been the fact that both our rails projects use bundler.

Before I go on I should point out that @wycats was very helpful in sorting this problem out.

It's not that bundler is bad directly, but when you're using bundler on Mac but deploying to Ubuntu there are a few eccentricities – particularly with native gems.

If you've run 'bundle package' at any point in the past then bundler will have started keeping a copy of all the gems in vendor/cache. When you're using a gem that needs certain compile flags on Ubuntu but not MacOS X that poses a problem.

The way to do this with bundler pre 1.0 is/was to install the gem to the system using the requisite build flags, and then when bundler installs the gem it takes the already installed gem. If the gem is already in vendor/cache however that doesn't work.

Suffice it to say that after some confusion and removal of the gem from BOTH caches (vendor/cache and ~/.bundle) and then running bundle install will fixed the problem.

Incidentally bundler 1.0.0rc2 takes care of this problem by introducing the ability to add build flags to the .bundle/config file.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Ruby non-greedy matching (OR how interro saved the day)

2010-03-24T06:06:36Z

I've spend most of the afternoon working on a complex regex in order to parse command line argument forms (for lack of a better term). If you've ever run the man command you'll know what I'm talking about. Take the tar command as an example:

tar  [ - ] A --catenate --concatenate | c --create | d --diff --compare
       | --delete | r --append | t --list | u --update | x --extract  --get  [
       options ] pathname [ pathname ... ]

If you're already familiar with regular expressions you'll know that doing something like:

[\[].*[\]]

trying to match:

[ - ]

won't accomplish what you think. Instead of getting just the first set of braces you'll end up with the whole remainder of the string. This is because of a feature, we'll give it that title, called greedy matching. Greedy matching means that it takes the largest possible chunk that your regex will match, which in this case is the ']' on the end of pathname.

I was aware of what was going on, but not being a particular master of regular expressions, I wasn't sure how to get it to stop being greedy. As it turns out its quite easy.

.* - Greedy matching
.+ - Greedy matching
.*? – Non-greedy matching
.+? – Non-greedy matching

It could not be easier, once you know about it of course.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Why “Mad Libs” style signup forms are just a gimmick

2010-03-07T00:57:55Z

The last few days I’ve been seeing a few references to “Mad Libs”. Confused, I quickly punched the term into Google and discovered that it was a reference to something we’ve all done at school, filling in the blanks in a sentence. My google reader(this and this) shared the fact HuffDuffer started using a Mad Libs style form as their sign up form. The question was raised by a few people as to whether this was good design.

Luke Wroblewski (in collaboration with Vast.com) did some research and came up with astounding results that the conversion ratio rose some 25-40% when a Mad Libs style form was used. That’s an impressive, but meaningless number. Don’t get me wrong, I’m very happy that HuffDuffer has found something that works for them. But lets step back for a second and analyse why the results that Vast.com got might be this way.

Here’s a thought; maybe it’s just because its a completely novel approach to a signup form. But can you really see this becoming the standard way to do signup forms? How long do you think it will be before this becomes more annoying than amusing? So quit jibber jabbering about how good it is and take a look at the long term picture. Mad Libs style forms are not something you can take and apply elsewhere with guaranteed results.

To be fair, I’m biased against filling in the blanks. I hated the premise at school and I will probably continue to hate the practice until it, or I, ceases to exist.

Ruby non-greedy matching (OR how interro saved the day) I've spend most of the afternoon working on a complex...
Getting started with Google Maps My first few searches turned up some fairly unpolished methods...

Related posts brought to you by Yet Another Related Posts Plugin.

Security wake up call

2010-03-05T22:51:36Z

I don’t think that I’ve had this much adrenalin pumping through my system on a Saturday morning in quite a while. My girlfriend signed into her email (as she does most mornings), to discover an email from eBay Live Support asking for a code. She called me over and a few seconds later the email vanished. What happened after that point is a blur of password resets, both her and the would be hacker trying to gain control of the hotmail and through it the eBay account.

There were paranoid moments when passwords wouldn’t work, but in the end she’s still in control of the accounts. Just about every account she owns has now had the password and details changed to help protect it.

It’s intrusive though. How exactly did the attacker break into the account? Not phishing, Cherie is well aware of those type of malicious emails. Guessing the security questions? Maybe.

That raises other concerns though, email addresses become repositories of knowledge for our online lives. Just about every account you sign up for online has to have an email account linked to it, that means many details about your online life are there in fragments. We’ll never know exactly what the hacker had access to (albeit briefly).

I’m personally going to be reviewing all my accounts to make sure that they’re secure, and I’d advise you to do the same.

RockYou gets rocked by hackers (And I’m hilarious) Seems that simple lessons don’t get learned....

Related posts brought to you by Yet Another Related Posts Plugin.

vertis pushed to master at vertis/active_documentum

2010-02-24T04:59:00Z

vertis pushed to master at vertis/active_documentum

ActiveDocumentum I’ve finally decided to release ActiveDocumentum. ActiveDocumentum is a Ruby...

Related posts brought to you by Yet Another Related Posts Plugin.

Sinatra on Java

2010-02-21T11:43:41Z

With JRuby and Warbler it’s possible to get Sinatra, or any WebApp based on Rack, running on a myriad of different Java application servers. There are of course gotchas when it comes to using Warbler with the many different app servers, so this is a definitive guide to everything you have to do to get a simple Sinatra app running on the various application servers.

Why Sinatra?

There are examples of how to get Rails running on Tomcat and Websphere floating around the web, but I find Rails overkill for small projects. With that in mind, it’s worth looking at how to get Sinatra running on java application servers. Besides the weight of rails, Sinatra is a nice, easy to learn framework.

Start by installing Sinatra and Warbler. You don’t have to be using JRuby to install Warbler, the install will download a gem of the jruby jars.

Install Sinatra and Warbler

Lets start by installing the required gems.

$ sudo gem install sinatra warbler haml

Haml isn’t strictly required, but the template I’m going to use has views generated in haml, so if you’re following the tutorial closely you’ll want to install it.

Create a project folder (and structure)

I usually keep a sinatra project template handy. So I’m going to clone my existing template off github. You can create a more minimal example than the one I’ll download, this will get the job done though.

$ git clone git://github.com/vertis/sinatra-example.git deploy_test

TODO: Details about the project

Check that our page is displayed

$ rackup

Go to http://localhost:9292/ and you should see our default page.

Generate the warble config

$ mkdir config && warble config

Lets look at the config file that was generated:

If you tried packaging and installing this now, it would fail miserably because, the ‘init.rb’ file would not be included. The generated warble.rb only includes the following

config.dirs = %w(app config lib log vendor tmp)

In addition to this the gems that we installed above would not be install. Here is same config with the lines we need (and the other cruft removed).

Package up the War file

$ warble

From here on in we’ll be looking at any gotchas, when deploying to the different Application Servers.

Deploy to Glassfish and test (effort: moderate – working: yes)

You can get Suns open source application server from http://glassfish.org. The current stable version of Glassfish 2.1.1, though Glassfish 3 is in active development. The installer comes packaged as a jar file. You can run the installer with

$ java -Xmx256m -jar glassfish-installer-v2.1.1-b31g-linux.jar

After accepting the license it should put all the files in a folder called ‘glassfish’ in the current directory.

$ cd glassfish
You need to run the following commands to finish the setup:

$ chmod -R +x lib/ant/bin
$ lib/ant/bin/ant -f setup.xml

Once the software is installed you can start the domain with:

$ bin/asaadmin start-domain domain1

And use either the admin console or the autodeploy directory to deploy the war file.

Glassfish now has a working copy of our application.

Deploy to JBoss and test (effort: n/a – working: no)

JBoss has a community and an enterprise edition. For the purposes of this test we’ll be using the community edition. The current stable version of JBoss AS is 5.1.0 GA. You can get a copy of JBoss from http://www.jboss.org.

Getting started is as simple as unzipping the archive and running:

$ cd jboss-5.1.0.GA

$ bin/run.sh

You can then use the admin console to deploy the application. One gotcha here, the first time I deployed the application using the console I got the following nasty message:

Application initialization failed: no such file to load — rack
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/vendor/rack.rb:1
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/vendor/rack.rb:22:in `require’
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/jruby/rack/booter.rb:22:in `boot!’
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/jruby/rack/boot/rack.rb:9
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/jruby/rack/boot/rack.rb:1:in `load’
from <script>:1

Turns out that after some digging there is an open jruby bug about the issue - http://jira.codehaus.org/browse/JRUBY-3935

I also did a bit of digging through the logs and found:

16:27:50,703 ERROR [STDERR] Warning: JRuby home “/opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-stdlib-1.4.0.jar/META-INF/jruby.home” does not exist, using /tmp

I’ve not managed to find a solution to this problem. I will revisit this at some point in the future. After googling a little it may be possible to just revert to a few older versions that seemed to work.

Deploy to Jetty and test (effort: easy – working: yes)

The current version of Jetty is 7.0.1.v20091125, though the version that comes as part of your Linux distro may not be so up to date. You can either install it using your favorite package manager, or if you’re on Windows get it from the homepage at http://www.mortbay.org

Once you’ve installed Jetty copy the generated war file to the webapps folder, and run:

$ bin/jetty.sh

You should be able to go to http://localhost:8080/deploy_test

Congratulations you now have a working copy of your sinatra app on Jetty.
Deploy to Tomcat and test (effort: easy – working: yes)

The current stable version of Tomcat is 6.0.20. You can either install it using your favorite package manager, or if you’re on Windows get it from the homepage at http://tomcat.apache.org

You shouldn’t have to make any changes to get our web app to work on Tomcat. Once you’ve installed Tomcat copy the generated war file to the webapps folder, and run:

$ bin/startup.sh

You should be able to go to http://localhost:8080/deploy_test

Tomcat really is the bread and butter of Java Application Servers, especially outside the Enterprise.

Deploy to Websphere and test (effort: hard – working: yes)

NB: Websphere 6.1.0.11 was the first application server I ever deployed Sinatra too, it failed miserably. I spent a long time debugging and playing with it to make it work properly. The biggest problem stems from the fact that the default way of using rack as configured by warbler doesn’t work.

Start by logging into the administration console, it should be something like – http://localhost:9043/ibm/console

Click on ‘Servers’ and when it expands select ‘Application Servers’. From here you can setup a new server instance that we can use for our testing. Call the instance something like ‘deploy01′. You can follow the default creation steps all the way through.

Once you have a server instance to test on, you can deploy a new application. The big gotcha as mentioned above is that you can’t use filters, the good news is that it’s quite easy to switch out the rack filters for a rack servlet. Rather than duplicate information that already exists, I’ll link to the place I learned to deploy warbler to websphere, http://clint-hill.com/2008/11/26/jruby-on-rails-and-websphere/.

Websphere is not the easiest application server to setup in general, but once you get it all configured it is fairly robust. Worth the effort if you want an application server you won’t have to restart constantly (as can be the case with Documentum on Tomcat).

Deploy to Weblogic and test (effort: n/a – working: no)

Setting up Oracle Weblogic 10.3.2 was nothing short of awesome. The install process is intuitive and speedy, though the size is quite large, at ~600Mb, compared to smaller cousins such as Tomcat. There is a wizard that walks you through the process of setting up your first domain, what Tomcat would call an instance and Websphere would call a profile, once the software is installed. I chose the default options for everything and had a running Weblogic server in about 20mins (including download).

My previous experience with Weblogic, was the version bundled with Documentum D6SP1. I’ve found both that version and the current fully fledged Oracle version to be a joy to work with.

Once the installation process is finished you can find the administration console at http://localhost:7001/console, you can then login using the username/password you picked during installation.

From the admin console it is a simple matter of clicking on ‘Deployments’ on the left and then when the screen loads clicking ‘install’, browse to the directory with the deploy_test.war file in it and start the install.

You should now be able to access the deployed application at http://localhost:7001/deploy_test…

Application initialization failed: no such file to load — rack from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/vendor/rack.rb:1 from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/vendor/rack.rb:22:in `require’ from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/jruby/rack/booter.rb:22:in `boot!’ from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/jruby/rack/boot/rack.rb:9 from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/jruby/rack/boot/rack.rb:1:in `load’ from

Clearly thats not going to be the case though. This issue is very similar to the error message received for JBoss.

Conclusion

Not a bad scorecard really. Of all the Java application servers that I tested, only JBoss and Weblogic proved to be a problem. I’ll be retesting these two periodically to see if support has been improved (there are open tickets with JRuby). Until then I hope that this has been useful.

Rack – NoMethodError: undefined method `call’ for nil:NilClass Turns out Rack doesn’t read minds. I just spent the...
Lighter weight deployment with git-deploy Up until now I’ve been using a fairly standard capistrano...
ActiveDocumentum I’ve finally decided to release ActiveDocumentum. ActiveDocumentum is a Ruby...

Related posts brought to you by Yet Another Related Posts Plugin.

Exploring Documentum RESTful Services – Part 2

2010-02-21T07:42:47Z

I originally published this post to the Early Access Area for Documentum RESTful Services.

In the second part of my exploration of Documentum RESTful Services I promised that we’d delve into browsing around the docbase. Rather do that with ruby I thought I’d grab a copy of JQuery and have a look at what it takes to access the services using what is an increasingly popular javascript library. The most important part of delivering on this is the JQuery call:

$.getJSON(path, ...);

The important thing here is that because of concerns about cross-site scripting we can only call local paths. While getJSON allows remote paths now with the help of JSONP style callbacks, those require server side cooperation to work. I don’t know that they’re not implemented in Documentum RESTful Services, but flipping through the documentation I couldn’t find anything of that nature.

Rather than creating a War for one html file, I decided to show off nginx. Nginx is a superb webserver, and more importantly in this case, a reverse proxy. It has gotten a lot of attention in the Ruby on Rails community, which is where I fell in love with it. After grabbing a copy of nginx it’s simply a matter of doing a minor adjustment to the nginx.conf file.

location /resources {
  proxy_pass   http://127.0.0.1:8080/resources;
}

This will mean that anything below the /resources directory on our webserver is passed off to the tomcat instance on 8080. We can start nginx, and discover it works perfectly. Now that we can use just a path to reference the services, lets get started. We start with a fairly blank html file:

 <html>                                                                 
 <head>                                                                 
 <script type="text/javascript" src="jquery-1.3.2.js"></script>         
 <script type="text/javascript">                                        
   // we will add our javascript code here
   $(document).ready(function() {
        // do stuff when DOM is ready
   });  
 </script>                                                              
 </head>                                                                
 <body>                                                                 
   <div id="results"></div>                                         
 </body>                                                                
 </html>

Notice the included jquery.js, the call the $(document).ready, and the results div. We’re going to use the call I mentioned earlier ‘getJSON’ to populate the results div with the results of calling the ‘folders’ resource with no arguments. Lets look at the code.



$.getJSON("/resources/core/repositories/test_repo/folders.json",
       function(data){
       $.each(data.dataPackage.dataObject, function(i,item){
           $("#results").append(item.properties.object_name+"<br/>");
       });
});

If you put the above html file in the right place and run the example you’ll be prompted for the password, just like you would if you actually went to the resource endpoint itself, then you’ll see a nice list of the root cabinets. Suppose we want to more than just display the root cabinets though, that we want to use the returned results to allow us to click around the docbase. Here is an example that does just that:

It’s very rough, and doesn’t take into account many variables, such as relationships that are returned aren’t necessarily folders. But it does illustrate the ability to access Documentum using libraries like jQuery now.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Copy the structure but not the data.

2010-02-12T03:42:13Z

I’m working on a prototype at the moment that requires me to insert data into offline tables (offline as far as Documentum is concerned). The examples that I’ve found all resort to specifying the exact structure of the table.

create table DMI_OBJECT_TYPEx (

R_OBJECT_ID VARCHAR2(16) NOT NULL,

I_TYPE NUMBER(10,0) NOT NULL,

I_PARTITION NUMBER(10,0) NULL);

The example above is smaller than most of the tables I have to create. The weakness with this is that you have to look up the table structure. The writers probably chose this method because the much simpler syntax shown below also brings any data that is in the table.

CREATE TABLE DMI_OBJECT_TYPEx AS SELECT * FROM DMI_OBJECT_TYPE;

My initial thought was why not copy the table and then just truncate it, but after a bit of searching I stumbled upon the solution. Essentially adding a WHERE clause to the end of the query that never evaluates to TRUE enables us to take the structure without the data.

CREATE TABLE DMI_OBJECT_TYPEx AS SELECT * FROM DMI_OBJECT_TYPE WHERE 1=2;

No related posts. Related posts brought to you by Yet Another Related Posts Plugin.

Lighter weight deployment with git-deploy

2010-02-01T05:34:16Z

Up until now I’ve been using a fairly standard capistrano deploy.rb. The problem is that as we speak I’m trying to deploy a patch from my Windows work machine and it’s not working. The ethics of fixing one of my project from home aside, this is a problem. When I try and deploy the following error comes up

can’t convert Net::SSH::Authentication::Pageant::Socket into IO (TypeError)

The change I was trying to deploy stops in its tracks. After quite a bit of searching I found a thread about the error that dates back to Capistrano 2.5.3… from 2008. What a shining example of open source.

To be fair the error isn’t necessarily in Capistrano, it may in fact be in Net::SSH. The sad thing is that we’re over a year later, and nothing has been done to fix the problem. Is the number of people that use Ruby from Windows machines so low that no-one has managed to fix it in a year?

I’m not expecting Jamis to fix it. I get that he has too much to do, and doesn’t have time to give out a bunch of freebie support, but I’m now faced with the choice of either trying to fix the problem, a task I don’t currently have time for, or ditching capistrano.

I’ve considered switching to heroku in the past and just never made the leap. For a start the app in question has to many moving parts for heroku. One thing I did like though was the notion that to deploy all I had to do was ‘git push target master’ and the app would be updated and deployed.

After toying around with rolling my own solution, I stumbled upon mislav’s gem. It lacks some of the features that I’m looking for, but its a good deal closer to the level that I need. It lacks the bloat of capistrano, which is important, because the biggest barrier to me getting in and fixing capistrano would be the size of the library and knowing where to start.

I very quickly migrated my existing application to use git-deploy. It’s not perfect for every problem, particularly if you’re doing multi stage deployments, etc, but at least I’ll be able to do a deployment everywhere I can get access to git now.

Rack – NoMethodError: undefined method `call’ for nil:NilClass Turns out Rack doesn’t read minds. I just spent the...
Sinatra on Java With JRuby and Warbler it’s possible to get Sinatra, or...
The API Challenge One of my favorite finds this year is Programmable Web,...

Related posts brought to you by Yet Another Related Posts Plugin.