Here, there, wherever...

Drawing trees with Raphael.js and Ruby

2010-08-26T23:20:00.008+02:00

It's been quite some time since I last posted (I just realise I open every post like this - guess I'm one of the laziest bloggers out there :-).

I've been busy hacking Ruby and thought I share this. For a website I'm currently building, I needed to draw organisation and process hierarchies (ie. any type of tree or forest) in the browser. the drawings need to be interactive (ie. clickable for getting a node's details).

First challenge was how to do the drawing. Generate the image server side, but then what about clickable areas in the image? Draw the image client side, but then I really didn't want to implement a tree drawing algorithm in JavaScript...

I did some research about what's out there in the wild and came across Raphael.js (kudos Dmitry, great job!). Raphael is a clean, simple graphics API for JavaScript.

But I still didn't want to implement tree drawing in JavaScript, so came to the conclusion to implement the tree-drawing in Ruby so as to generate the JavaScript.

The result is further down in this post. TreeDraw is a class to which you pass a tree or forest (I'm using acts_as_tree to get methods like children, ancestors, parent) as well as some drawing options (optional, defaults are all defined).

You can also provide a base URL (e.g. "/mycontroller/myaction/") which will be augmented with a node's ID to become a link to viewing a node's details (e.g. "/mycontroller/myaction/5").

The code looks like this:


# The TreeDraw class draws trees using the Raphael Javascript drawing library.
class TreeDraw

  include ActionView::Helpers::TextHelper
  include ERB::Util

  # Construct a new Tree drawing instance following the Reingold-Tilford-Algorithm. Parameters are:
  # - roots => The forest to draw (array of roots of the forest)
  # - link => If specified, then each box will link to the specified link with the id of the specific node appended
  # - options => a hash allowing to override the various default drawing options
  def initialize(roots, link=nil, options = {})
 @roots = roots # the roots of the tree where each node responds to methods like "children", "parent", "ancestors"
 @link = link
 init_options(options)
 @x_pos = {} # hash to hold the x positions for each node
 @x_mod = {} # hash to hold the x position modifiers for each node
 @x_cumul_mod = {}
 @y_pos = {} # hash to hold the y positions for each node
 @max_width = {} # hash to hold the max width of each root's tree
 @connectors = [] # the svg path strings for all connectors in the tree 
 @completed = {}
 @total_width = 0 # total width of drawing canvas adopted to tree size
 @total_height = @options[:box_height] + @options[:v_padding] # total height of drawing canvas adopted to tree size, initialised to height of one box                               
  end

  # Return the Javascript required to draw the tree/forest at
  # the top left of the specified dom element id
  def draw(dom_element_id)
 calculate
 return generate_code(dom_element_id)
  end

  protected

  # Return the root for the specified node
  def root_for(node)
 if @roots.member?(node)==true
   return node
 end
 node.ancestors.each do |ancestor|
   if @roots.member?(ancestor)==true
  return ancestor
   end
 end
  end

  # Generate the SVG paths for connecting the specified node with its ancestor
  def connect(node)
 offset = @options[:box_width]/2
 if node.children.size > 0
   @connectors << "M#{@x_pos[node]+offset} #{@y_pos[node]+@options[:box_height]} L#{@x_pos[node]+offset} #{@y_pos[node]+@options[:box_height]+(@options[:v_padding]/2)}"
   @connectors << "M#{@x_pos[node.children.first]+offset} #{@y_pos[node]+@options[:box_height]+(@options[:v_padding]/2)} L#{@x_pos[node.children.last]+offset} #{@y_pos[node]+@options[:box_height]+(@options[:v_padding]/2)}"
 end
 node.children.each do |child|
   @connectors << "M#{@x_pos[child]+offset} #{@y_pos[child]-(@options[:v_padding]/2)} L#{@x_pos[child]+offset} #{@y_pos[child]}"
   connect(child)
 end
  end

  # Return the level of this node in the tree
  def level(node)
 root = root_for(node)
 return (node.ancestors().size() - root.ancestors().size()) + 1
  end

  # Layer all nodes in the appropriate y-coordinates
  def layer(node)
 l = level(node)
 @y_pos[node] = ((l - 1) * @options[:box_height]) + ((l - 1) * @options[:v_padding])
 if @y_pos[node] > @total_height
   @total_height = @y_pos[node]
 end
 node.children.each do |child|
   layer(child)
 end
  end

  # Return all nodes of the tree/forest on the specified level (regardless of whether these nodes are related)
  def nodes_for_level(level, node = nil)
 nodes = []
 if (level == 1)
   return @roots
 end
 if node == nil
   @roots.each do |node|
  nodes += nodes_for_level(level, node)
   end
 else
   if level==(level(node)+1)
  nodes += node.children
   else
  node.children.each do |child|
    nodes += nodes_for_level(level, child)
  end
   end
 end
 return nodes
  end

  # Return the left sibling of this node (note that "sibling" does not mean that the two nodes
  # belong to the same ancestor)
  def left_sibling(node)
 siblings = nodes_for_level(level(node))
 if siblings
   index = siblings.index(node)
   if (index > 0)
  return siblings[index-1]
   end
 end
 return nil
  end

  # Shift the specified node with respect to its left sibling.
  def shift_beyond(node)
 x = @x_pos[node]
 x += @x_mod[node]
 node.ancestors.each do |ancestor|
   if @x_mod[ancestor]==nil
  return x
   else
  x += @x_mod[ancestor]
   end
 end
 return x
  end

  # Position all nodes in the tree/forest
  def calculate()
 @roots.each do |root|
   first_pass(root)
   second_pass(root)
   layer(root)
   connect(root)
   if @total_width < @max_width[root]
  @total_width = @max_width[root]
   end
 end
  end

  # First pass placing the nodes horizontally.
  def first_pass(node)
 # first pass is post order, depth first
 node.children.each do |child|
   first_pass(child)
 end
 mod = 0
 x_pos = 0
 children = node.children
 if (children.size == 0) # leaf node
   left_sibling = left_sibling(node)
   if left_sibling # leaf node with left sibling
  x_pos = shift_beyond(left_sibling) + @options[:box_width] + @options[:h_padding]
   end
 else # node with children
   left_most_child = children.first
   right_most_child = children.last
   x_pos = (@x_pos[left_most_child] + @x_mod[left_most_child] + @x_pos[right_most_child] + @x_mod[right_most_child])/2
   left_sibling = left_sibling(node)
   if left_sibling # node with left sibling
  mod = [(@options[:box_width] + @options[:h_padding]) - (x_pos - shift_beyond(left_sibling)), 0].max
   end
 end
 @x_pos[node] = x_pos
 @x_mod[node] = mod
 @x_cumul_mod[node] = 0
 root = root_for(node)
 @max_width[root] ||= 0
 if (@x_pos[node]+@options[:box_width]) > @max_width[root]
   @max_width[root] = @x_pos[node]+@options[:box_width]
 end
  end

  # Second pass shifting the nodes horizontally according to the position of their left siblings.
  def second_pass(node)
 @x_pos[node] +=  @x_mod[node] + @x_cumul_mod[node]
 node.children.each do |child|
   @x_cumul_mod[child] = @x_mod[node] + @x_cumul_mod[node]
 end
 node.children.each do |child|
   second_pass(child)
 end
  end

  # Once the nodes have been positioned horizontally and vertically, generate the Raphael.js Javascript
  # to draw the tree in the browser
  def generate_code(dom_element_id)
 result = ""
 result += ""
  end

  # Generate the box and text for a node.
  def generate(node)
 result = generate_box(node)
 result += generate_text(node)
 node.children.each do |child|
   result += generate(child)
 end
 return result
  end

  # Draw the box and corresponding mouseover (for domTT popups).
  def generate_box(node)
 result = "t=paper.rect(#{@x_pos[node]}, #{@y_pos[node]}, #{@options[:box_width]}, #{@options[:box_height]}, #{@options[:box_round_edge]});"
 result += "t.attr({stroke:'#{@options[:box_color]}',fill:'#{@options[:box_color]}'});"
 result += "t.node.onmouseover=function(event){domTT_activate(this, event, 'content', '#{node.description.gsub(/\n/,"\\\n").gsub(/\r/,"\\\r")}', 'styleClass', 'tooltip')};"
 result += "t=paper.rect(#{@x_pos[node]+@options[:shadow_box_offset]}, #{@y_pos[node]+@options[:shadow_box_offset]}, #{@options[:box_width]}, #{@options[:box_height]}, #{@options[:box_round_edge]});"
 result += "t.attr({fill:'#{@options[:shadow_box_color]}',stroke:'#{@options[:shadow_box_color]}'});"
 result += "t.toBack();"
 return result
  end

  # Generate the node's text in the drawing as well as a link to displaying details about the node (if a link was specified upon construction) using node's id.
  def generate_text(node)
 length = node.name.size
 result = "t=paper.text(#{@x_pos[node]}, #{@y_pos[node]}, '#{truncate(node.name, :length => @options[:max_text_length], :ommission => "...")}\\n(#{truncate((node.reference ? node.reference : ""), :length => @options[:max_text_length], :ommission => "...")})\\n#{truncate((node.person ? node.person.full_name : ""), :length => @options[:max_text_length], :ommission => "...")}');"
 result += "t.attr({'text-anchor': 'middle', 'font-size': #{@options[:font_size]}, 'font-weight': '#{@options[:font_weight]}', 'font-family': '#{@options[:font_family]}', fill:'#{@options[:text_color]}'});"
 result += "t.translate(((#{@options[:box_width]} - t.getBBox().width)/2)+t.getBBox().width/2, #{@options[:box_height]}/2);"
 if (@link)
   result += "t.hover(function(){this.attr({fill: '#{@options[:highlight_color]}'});},function(){this.attr({fill: '#{@options[:text_color]}'});});"
   result += "t.click(function(){location='#{@link}/#{node.id}'});"
 end
 return result;
  end

  # Generate the connecting lines between nodes.
  def generate_connectors
 result ="t=paper.path('"
 @connectors.each do |path|
   result += " #{path}"
 end
 result += "');t.attr({stroke:'#{@options[:line_color]}'});"
 return result
  end

  # Initialise the options using defaults or the overrides specified upon construction of this object 
  def init_options(options)
 @options = options
 @options[:box_width]  ||= 200 # width of a box representing a node
 @options[:box_height]  ||= 50 # height of a box representing a node
 @options[:tree_padding]  ||= 50 # horizontal padding between trees of the forest
 @options[:v_padding]  ||= 50 # vertical padding between all nodes of the forest
 @options[:h_padding]  ||= 40 # horizontal padding between nodes of the same tree in the forest
 @options[:box_round_edge] ||= 5 # radius of round edges (0 for normal edges)
 @options[:box_color]  ||= "#9aafe5" # fill color of a box representing a node
 @options[:highlight_color] ||= "#2e6ab1" # highlight color of the text if there's a link
 @options[:shadow_box_color] ||= "#0e509e" # color of the shadow for each box representing a node
 @options[:shadow_box_offset]||= 3 # offset of the shadow for each box representing a node
 @options[:line_color]  ||= "#0e509e" # for connectors between boxes representing nodes
 @options[:text_color]  ||= "#000" # color of text in each box representing a node
 @options[:font_size]  ||= 10
 @options[:font_weight]  ||= "normal"
 @options[:font_family]  ||= "Arial"
 @options[:max_text_length] ||= 35 # truncate any text in the box after the specified characters (appending '...')
  end

end

The simplest way to use this is in the view of a Rails application like so:


      
        <%= TreeDraw.new([@organisation], url_for(:controller=>"organisation", :action=>"details", :id=>nil)).draw("org_chart_drawing") %>

Where @organisation is the root of a tree (using acts_as_tree), the specified URL provides the link to the details for a node and the string specified in the draw method tells TreeDraw to place the Raphael canvas in the element with ID org_chart_drawing.

When drawing the box for a node, I am using the domTT tooltip to display a description of the detail for a node...

Have fun!

2010-05-01T18:56:00.011+02:00

If you are using Ferret for Rails full text search, never forget to do the following:

gem install acts_as_ferret (duh!)

If this works with your Rails install - fine. Don't go on and change a winning team. If it doesn't work (e.g. when you do a rake db:migrate and you get

no such file to load - ferret

do the following

gem install ferret

and you should be fine.

Rendering XML with Rails

2010-02-18T09:24:00.003+01:00

I had to code a webservice with Rails yesterday. Its obvious purpose was to respond with XML formatted data to an HTTP GET request. I looked at some of the examples out there and ran into trouble. I lost some time searching for a solution, so here's the summary of the solution to save others the hassle.

Here's what people recommend doing to render XML from a HTTP request in Rails:


class WebServiceController < ApplicationController

 def people
  @people = Person.find :all
  respond do |format|
   format.xml
  end
 end

end

And the corresponding people.xml.builder in /views/web_service/:


xml.instruct! :xml, :version=>"1.0"

xml.people do
  @people.each do |person|
    xml.person do
        xml.firstname person.first_name
        xml.lastname person.last_name
        xml.age person.age
    end
  end
end

When testing this, I kept getting browser errors about invalid XML. When I looked at the data that was returned from my webservice call, I realised that the XML was generated like I wanted, but it was subsequently embedded into the application layout I am using for all of the views (views/layouts/application.rhtml).

The issue was resolved by changing my controller code to


class WebServiceController < ApplicationController

 def people
  @people = Person.find :all
  render :template => 'web_service/people.xml.builder', :layout => false
 end

end

Character Encoding UTF-8 with JPA/Hibernate, MySql and Tomcat

2009-10-21T16:32:00.011+02:00

I'm writing a little Java application using JPA (Hibernate implementation) and Spring. The application will run on Tomcat and uses MySql as the RDBMS.

The problem I had today was with the good old character encoding: I was able to store German Umlaut characters (üöä) properly in MySql, but whenever I retrieved them, they would be scrambled - regardless of whether I displayed the result on a web page or just printed it to the console.

So, the problem is: how to consistently set UTF-8 as the character encoding of choice throughout the whole stack:

for MySql as well as for any session coming through the JDBC driver in order to ensure that any entity created by Hibernate/JPA uses UTF-8
for Tomcat to make sure that any data served uses UTF-8

I know that you'll find a lot of material on the solution for each individual piece of software in my tech stack across the net. However, I still think it's worth to post this solution as I did not find all elements of it in one place (and don't want to search again the next time :-)

Here's what I did:

1. Ensure that MySql runs on UTF-8 as default: in the MySql configuration file my.cnf add the following in the section for mysqld:

[mysqld]
...
default-character-set=utf8
default-collation=utf8_general_ci
...

2. Configure your MySql JDBC driver connection as follows (obviously hostname, port and schema are probably different in your configuration :-):

jdbc:mysql://localhost:3306/test?useUnicode=true&connectionCollation=utf8_general_ci&characterSetResults=utf8

When configuring the above driver URL in your Spring XML context definition, don't forget to escape the Ampersand as you will get parsing errors otherwise.

3. Configure Tomcat for UTF-8 by adding the following line to your : catalina.bat or catalina.sh:

JAVA_OPTS="$JAVA_OPTS -Djavax.servlet.request.encoding=UTF-8 -Dfile.encoding=UTF-8"

Versions I am using: MySql 5.0, Tomcat 6.0.20, Spring 2.5.6, Java 6, MySql Connector 5.1.6

Happy hacking!

2009-01-12T23:29:00.002+01:00

Been a long time since I have posted anything. Got dragged into some, well - let's say, interesting work in my day job. Done some programming anyway.

GWT-Ext or SmartClient based GUI wiht a JPA/Spring/MySQL/JAX-B/JAX-RS driven backend. And I'm using Envers for auditing - I love it.

Got fed up writing all those REST requests, so basically just set up a service with a Groovy engine behind. This allows me to just send scripts using my plain POJO domain model from the GUI without having to expose every little piece of functionality as a REST service. All the while every script execution returns a piece of XML driving the AJAX GUI. On this front I love the store-based model of SmartClient over the store based model of GWT-Ext (because of the CRUD service hooks).

Need to play around with the Groovy bit some more - but I can almost hear the critics yell "SECURITY ISSUE"!

Protecting the whole thing with CAS and aspect-driven security on the server side for method execution as well as data retrievals/updates using JPA.

Need to play with this a bit more and will provide some code here. Watch this space...

Creating OSGi bundles from any JAR

2008-04-28T21:11:00.004+02:00

One of the major pains in OSGi is the creation of bundles. Peter Kriens' BND tool does a great job - it's powerful and can be used for creating the most complex OSGi bundles.

But haven't you ever wished for a simple tool at which you just throw an existing JAR file and it produces the corresponding OSGi bundle jar? Well, I have - and I have written the class below which does exactly that. And, yes, I have been found guilty of reinventing the wheel (see comment below) - a simple wrapper to BND would have achieved the same purpose except I learnt a few things along the way doing it myself :-)

This is a convenient way to quickly produce an OSGi bundle to be deployed in your OSGi runtime.

Clearly, this is not a replacement for BND or similar tools. It is a good 80% solution. Hopefully, we will soon see reasonably searchable OSGi bundle repositories come up (and I do not mean what is currently offered on the OSGi alliance site - you can't even search for bundles offering a certain class or package, to name just one use case). Hopefully, we will also see bundles offered for all of the major open source packages, not just a chosen few used in Eclipse. Ok, rant finished.

Here comes the code. [Deleted my comment about reinventing the weel from the original post - admittedly I have been found guilty].

And before I forget, this code depends on the following JARs:

Dependency: I had to adapt it to the current version of ASM (that was simple, the source code is available). If you send me an email, I can send you my version of the code.
ASM
Apache Commons IO

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import org.vafer.dependency.utils.DependencyUtils;
import org.apache.commons.io.FileUtils;

/**
* This class implements a simplistic approach to creating
* bundles from any JAR file.
*
* @author Mathias Richter
*/
public class Osgify
{

  /**
   * Return all packages imported by classes in the specified jar file.
   *
   * @param jarFileName The name of the JAR file to analyse.
   * @return A set of package names.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   */
  public static Set<String> getImportPackages( String jarFileName ) throws FileNotFoundException, IOException
  {
      HashMap<String, String> packages = new HashMap<String, String>();
      Set<String> dependencies = DependencyUtils.getDependenciesOfJar( new FileInputStream( jarFileName ) );
      for ( String name : dependencies )
      {
          if ( !name.startsWith( "[" ) )
          {
              int index = name.lastIndexOf( "." );
              if ( index > 0 )
                  name = name.substring( 0, index );
          } else
          {
              if ( name.length() > 2 )
              {
                  int index = name.lastIndexOf( "." );
                  if ( index > 2 )
                      name = name.substring( 2, index );
              } else
                  continue;
          }
          if ( !name.startsWith( "java." ) )
              packages.put( name, name );
      }
      return packages.keySet();
  }

  /**
   * Return a set of names of the packages declared in the specified JAR file.
   *
   * @param jarFileName The name of the JAR file to analyse.
   * @return The names of the packages declared by the classes in the JAR file.
   *
   * @throws java.io.IOException
   */
  public static Set<String> getExportPackages( String jarFileName ) throws IOException
  {
      HashMap<String, String> packages = new HashMap<String, String>();
      JarFile file = new JarFile( jarFileName );
      for ( Enumeration<JarEntry> e = file.entries(); e.hasMoreElements();)
      {
          String name = e.nextElement().getName();
          if ( name.endsWith( ".class" ) )
          {
              name = name.replaceAll( "/", "." );
              int index = name.lastIndexOf( "." );
              if ( index > 0 )
                  name = name.substring( 0, name.lastIndexOf( "." ) );
              index = name.lastIndexOf( "." );
              if ( index > 0 )
                  name = name.substring( 0, name.lastIndexOf( "." ) );
              packages.put( name, name );
          }
      }
      return packages.keySet();
  }

  /**
   * Analyse the specified JAR for its package imports and package declarations and
   * create a manifest file describing these in OSGi fashion.
   *
   * @param jarFileName The name of the JAR file for which to create an OSGi bundle manifest. May not be null.
   * @param bundleName The name of the bundle to delcrae in the manifest. May not be null.
   * @param bundleSymbolicName The symbolic name to declare in the manifest. If null, uses the bundle name.
   * @param bundleVersion The version to declare in the manifest. If null, uses "1.0.0".
   * @param bundleActivator The activator class to declare in the manifest. If null, does not specify an activator.
   * @param resolveOptional If true, all package imports will be marked with "resolution:=optional".
   *
   * @return The manifest contents as a string.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   */
  public static String getManifest( String jarFileName, String bundleName, String bundleSymbolicName, String bundleVersion, String bundleActivator, boolean resolveOptional ) throws FileNotFoundException, IOException
  {
      StringBuffer manifest = new StringBuffer();
      Set<String> im = getImportPackages( jarFileName );
      Set<String> ex = getExportPackages( jarFileName );
      im.removeAll( ex );
      manifest.append( "Bundle-Name: " );
      manifest.append( bundleName );
      manifest.append( "\n" );
      if ( bundleSymbolicName == null )
          bundleSymbolicName = bundleName;
      manifest.append( "Bundle-SymbolicName: " );
      manifest.append( bundleSymbolicName );
      manifest.append( "\n" );
      if ( bundleVersion == null )
          bundleVersion = "1.0.0";
      manifest.append( "Bundle-Version: " );
      manifest.append( bundleVersion );
      manifest.append( "\n" );
      if ( bundleActivator != null )
      {
          manifest.append( "Bundle-Activator: " );
          manifest.append( bundleActivator );
          manifest.append( "\n" );
      }
      manifest.append( "Import-Package: " );
      boolean previous = false;
      for ( String p : im )
      {
          if ( previous )
              manifest.append( "," );
          manifest.append( p );
          if ( resolveOptional )
              manifest.append( ";resolution:=optional" );
          previous = true;
      }
      manifest.append( "\n" );
      manifest.append( "Export-Package: " );
      previous = false;
      for ( String p : ex )
      {
          if ( previous )
              manifest.append( "," );
          manifest.append( p );
          previous = true;
      }
      manifest.append( "\n" );
      manifest.append( "\n" );
      return split( manifest.toString(), 70 );
  }

  /**
   * Format the specified source string to use a max line length as specified.
   * Lines will be broken in "manifest" fashion, ie. continuation lines start with
   * a single space.
   *
   * @param source the source String to format
   * @param maxLineLength The maximum line length in the result.
   * @return The formatted string.
   */
  protected static String split( String source, int maxLineLength )
  {
      StringBuffer result = new StringBuffer();
      StringTokenizer tokens = new StringTokenizer( source, "\n" );
      while ( tokens.hasMoreTokens() )
      {
          String line = tokens.nextToken();
          int length = line.length();
          for ( int index = 0; index < length;)
          {
              if ( index > 0 )
                  result.append( " " );
              int remainder = length - index;
              if ( remainder <= maxLineLength )
              {
                  result.append( line.substring( index, length ) );
                  index = length;
              } else
              {
                  result.append( line.substring( index, index + maxLineLength ) );
                  index += maxLineLength;
              }
              result.append( "\n" );
          }
      }
      return result.toString();
  }

  /**
   * If the process has output, return it as a String.
   *
   * @param process The process.
   * @return The process output as String.
   * @throws java.io.IOException
   */
  protected static String getOutput( Process process ) throws IOException
  {
      StringBuffer result = new StringBuffer();
      InputStream is = process.getInputStream();
      InputStreamReader isr = new InputStreamReader( is );
      BufferedReader br = new BufferedReader( isr );
      String line = null;
      while ( ( line = br.readLine() ) != null )
          result.append( line );
      is.close();
      isr.close();
      br.close();
      return result.toString();
  }

  /**
   * Create an OSGi jar from the specified JAR file using the specified bundle name.
   * The OSGi bundle jar will be created in the same directory as the original JAR.
   *
   * @param jarFileName The JAR file to analyse and produce an OSGi bundle from.
   * @param bundleName the name of the bundle.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   * @throws java.lang.InterruptedException
   */
  public static void osgify( String jarFileName, String bundleName ) throws FileNotFoundException, IOException, InterruptedException
  {
      osgify( jarFileName, bundleName, null, null, null, true );
  }

  /**
   * Create an OSGi jar from the specified JAR file using the specified bundle name and bundle symbolic name.
   * The OSGi bundle jar will be created in the same directory as the original JAR.
   *
   * @param jarFileName The JAR file to analyse and produce an OSGi bundle from.
   * @param bundleName the name of the bundle.
   * @param bundleSymbolicName The symbolic bundle name to use.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   * @throws java.lang.InterruptedException
   */
  public static void osgify( String jarFileName, String bundleName, String bundleSymbolicName ) throws FileNotFoundException, IOException, InterruptedException
  {
      osgify( jarFileName, bundleName, bundleSymbolicName, null, null, true );
  }

  /**
   * Create an OSGi jar from the specified JAR file using the specified bundle name and bundle symbolic name.
   * The OSGi bundle jar will be created in the same directory as the original JAR.     *
   *
   * @param jarFileName The name of the JAR file for which to create an OSGi bundle manifest. May not be null.
   * @param bundleName The name of the bundle to declare in the manifest. May not be null.
   * @param bundleSymbolicName The symbolic name to declare in the manifest. If null, uses the bundle name.
   * @param bundleVersion The version to declare in the manifest. If null, uses "1.0.0".
   * @param bundleActivator The activator class to declare in the manifest. If null, does not specify an activator.
   * @param resolveOptional If true, all package imports will be marked with "resolution:=optional".
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   * @throws java.lang.InterruptedException
   */
  public static void osgify( String jarFileName, String bundleName, String bundleSymbolicName, String bundleVersion, String bundleActivator, boolean resolveOptional ) throws FileNotFoundException, IOException, InterruptedException
  {
      int index = jarFileName.lastIndexOf( File.separator );
      String targetDir = "./";
      if ( index > 0 )
          targetDir = jarFileName.substring( 0, index + 1 );
      File tempDir = new File( targetDir + ".osgify" + System.currentTimeMillis() );
      FileUtils.forceMkdir( tempDir );
      try
      {
          Process p = Runtime.getRuntime().exec( "jar xf " + jarFileName, null, tempDir );
          if ( p.waitFor() != 0 )
              throw new Error( "Could not expand '" + jarFileName + "' in temporary directory: "  + getOutput( p ) );
          File manifest = new File( tempDir.getAbsolutePath() + "/META-INF/MANIFEST.MF" );
          if ( manifest.exists() )
              FileUtils.forceDelete( manifest );
          manifest = new File( tempDir.getAbsolutePath() + "/META-INF/MANIFEST.MF" );
          String manifestContent = getManifest( jarFileName, bundleName, bundleSymbolicName, bundleVersion, bundleActivator, resolveOptional );
          FileUtils.writeStringToFile( manifest, manifestContent );
          String netJarName = jarFileName.substring( jarFileName.lastIndexOf( File.separator ) + 1, jarFileName.lastIndexOf( "." ) );
          String targetJarName = targetDir + netJarName + "_" + bundleVersion + ".jar";
          String cmd = "jar cfm " + targetJarName + " META-INF/MANIFEST.MF ./*";
          p = Runtime.getRuntime().exec( cmd, null, tempDir );
          if ( p.waitFor() != 0 )
              throw new Error( "Could not create OSGi bundle jar '" + targetJarName + "' in temporary directory: " + getOutput( p )  );
      } finally
      {
          FileUtils.forceDelete( tempDir );
      }
  }

}

Standalone Java CAS Client

2008-04-15T21:28:00.004+02:00

There's a variety of clients for CAS. The Java-based clients (JA-SIG, Yale, see JA-SIG website) typically handle the browser-based client interaction with CAS very well through ServletFilter implementations.

Now what about programmatic authentication, i.e. achieving authentication through non-browser based applications? There exists a CAS .NET client but I did not manage to find the appropriate Java implementation. So here goes - it is based on the Apache HttpClient.

In case I missed any existing implementation achieving the same purpose, let's look at the bright side: at least now I understand the CAS protocol :-)

My CAS client works within any application. It uses the HttpClient and behaves like a browser client as CAS requires cookie support.

Here's the code:


import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.log4j.Logger;

/**
* The CasClient allows users to programmatically login
* to CAS protected services based on the CAS 2 protocol.
* This client behaves like a browser-client in terms of
* cookie handling.<br>
*
* @author Mathias Richter
*/
public class CasClient
{
  
   public static Logger LOG = Logger.getLogger( CasClient.class  );

   public static final String LOGIN_URL_PART = "login";
   public static final String SERVICE_VALIDATE_URL_PART = "serviceValidate";
   public static final String TICKET_BEGIN = "ticket=";
   private static final String LT_BEGIN = "name=\"lt\" value=\"";
   public static final String CAS_USER_BEGIN = "<cas:user>";
   public static final String CAS_USER_END = "</cas:user>";
  
   private HttpClient fClient;
   private String fCasUrl;
  
   /**
    * Construct a new CasClient.
    *
    * @param casUrl The base URL of the CAS service to be used.
    */
   public CasClient( String casBaseUrl )
   {
       this( new HttpClient(), casBaseUrl );
   }
  
   /**
    * Construct a new CasClient which uses the specified HttpClient
    * for its HTTP calls.
    *
    * @param client
    * @param casBaseUrl
    */
   public CasClient( HttpClient client, String casBaseUrl )
   {
       fClient = client;
       fCasUrl = casBaseUrl;
   }
  
   /**
    * Authenticate the specified username with the specified password.
    * This will not yield any ticket, as no service is authenticated
    * against. This wil just set the CAS cookie in this client upon
    * successful authentication.
    *
    * @param username
    * @param password
    */
   public void authenticate( String username, String password )
   {
       authenticate( null, username, password );
   }
  
   /**
    * Validate the specified service ticket against the specified service.
    * If the ticket is valid, this will yield the clear text user name
    * of the autenticated user.<br>
    * Note that each service ticket issued by CAS can be used exactly once
    * to validate.
    *
    * @param serviceUrl
    * @param serviceTicket
    *
    * @return Clear text username of the authenticated user.
    */
   public String validate( String serviceUrl, String serviceTicket )
   {
       String result = null;
       PostMethod method = new PostMethod( fCasUrl + SERVICE_VALIDATE_URL_PART );
       method.setParameter( "service", serviceUrl );
       method.setParameter( "ticket", serviceTicket );
       try
       {
           int statusCode = fClient.executeMethod(method);
           if (statusCode != HttpStatus.SC_OK)
           {
               LOG.error( "Could not validate: " + method.getStatusLine() );
               method.releaseConnection();
           } else
           {   
               result = extractUser( new String( method.getResponseBody() ) );
           }
       } catch ( Exception x )
       {
           LOG.error( "Could not validate: " + x.toString () );
           x.printStackTrace();
       }
       method.releaseConnection();
       return result;
   }
  
   /**
    * Authenticate the specified user with the specified password against the
    * specified service.
    *
    * @param serviceUrl May be null. If a url is specified, the authentication will happen against this service, yielding a service ticket which can be validated.
    * @param username
    * @param password
    * @return A valid service ticket, if and only if the specified service URL is not null.
    */
   public String authenticate( String serviceUrl, String username, String password )
   {
       String lt = getLt( serviceUrl );
       if ( lt == null )
       {
           LOG.error( "Cannot retrieve LT from CAS. Aborting authentication for '" + username + "'" );
           return null;
       }
       String result = null;
       PostMethod method = new PostMethod( fCasUrl + LOGIN_URL_PART );
       if ( serviceUrl != null ) // optional
           method.setParameter( "service", serviceUrl );
       method.setParameter( "_eventId", "submit" );
       method.setParameter("username", username );
       method.setParameter("password", password );
       method.setParameter("lt", lt );
       method.setParameter( "gateway", "true" );
       try
       {
           fClient.executeMethod(method);
           if ( serviceUrl == null )
           {
               if ( extractLt( new String( method.getResponseBody() ) ) != null ) // if CAS does not return a login page with an LT authentication was successful
               {
                   LOG.error( "Authentication for '" +  username + "' unsuccessful" );
                   if ( LOG.isDebugEnabled() )
                       LOG.debug( "Authentication for '" + username + "' unsuccessful." );
               } else
               {
                   if ( LOG.isDebugEnabled() )
                       LOG.debug( "Authentication for '" + username + "' unsuccessful." );
               }
           } else
           {
               Header h = method.getResponseHeader( "Location" );
               if ( h != null )
                   result = extractServiceTicket( h.getValue() );
               if ( result == null )
                   LOG.error( "Authentication for '" + username + "' unsuccessful." );
           }
       } catch ( Exception x )
       {
           LOG.error( "Could not authenticate'" + username + "':" + x.toString () );
       }
       method.releaseConnection();
       return result;
   }
  
   /**
    * Helper method to extract the user name from a "service validate" call to CAS.
    *
    * @param data Response data.
    * @return The clear text username, if it could be extracted, null otherwise.
    */
   protected String extractUser( String data )
   {
       String user = null;
       int start = data.indexOf( CAS_USER_BEGIN  );
       if ( start >= 0 )
       {
           start += CAS_USER_BEGIN.length();
           int end = data.indexOf( CAS_USER_END );
           if ( end > start )
               user = data.substring( start, end );
           else
               LOG.warn( "Could not extract username from CAS validation response. Raw data is: '" + data + "'" );
       } else
       {
           LOG.warn( "Could not extract username from CAS validation response. Raw data is: '" + data + "'" );
       }
       return user;
   }
  
   /**
    * Helper method to extract the service ticket from a login call to CAS.
    *
    * @param data Response data.
    * @return The service ticket, if it could be extracted, null otherwise.
    */
   protected String extractServiceTicket( String data )
   {
       String serviceTicket = null;
       int start = data.indexOf( TICKET_BEGIN  );
       if ( start > 0 )
       {
           start += TICKET_BEGIN.length();
           serviceTicket = data.substring( start );
       }
       return serviceTicket;
   }
  
   /**
    * Helper method to extract the LT from a login form from CAS.
    *
    * @param data Response data.
    * @return The LT, if it could be extracted, null otherwise.
    */
   protected String extractLt( String data )
   {
       String token = null;
       int start = data.indexOf( LT_BEGIN  );
       if ( start < 0 )
       {
           LOG.error( "Could not obtain LT token from CAS: LT Token not found in response." );
       } else
       {
           start += LT_BEGIN.length();
           int end = data.indexOf( "\"", start );
           token = data.substring( start, end );
       }       
       return token;
   }
  
   /**
    * This method requests the original login form from CAS.
    * This form contains an LT, an initial token that must be
    * presented to CAS upon sending it an authentication request
    * with credentials.<br>
    * If a service URL is provided (which is optional), this method
    * will post the URL such that CAS authenticates against the
    * specified service when a subsequent authentication request is
    * sent.
    *
    * @param serviceUrl
    * @return The LT token if it could be extracted from the CAS response.
    */
   protected String getLt( String serviceUrl )
   {
       String lt = null;
       HttpMethod method = null;
       if ( serviceUrl == null )
           method = new GetMethod( fCasUrl + LOGIN_URL_PART );
       else
       {
           method = new PostMethod( fCasUrl + LOGIN_URL_PART );
           ( ( PostMethod ) method ).setParameter( "service", serviceUrl );
       }
       try
       {
           int statusCode = fClient.executeMethod(method);
           if (statusCode != HttpStatus.SC_OK)
           {
               LOG.error( "Could not obtain LT token from CAS: " + method.getStatusLine() );
               method.releaseConnection();
           } else
           {
               Object o = method.getResponseHeaders() ;
               return extractLt( new String( method.getResponseBody() ) );
           }
       } catch ( Exception x )
       {
           LOG.error( "Could not obtain LT token from CAS: " + x.toString () );
       }
       method.releaseConnection();
       return lt;
   }
  
}

Making Spring Dynamic Modules work

2008-04-07T20:43:00.003+02:00

Spring Dynamic Modules (Spring support for OSGi) is a great thing. Just in case I am not the only one who struggled to get the required bundles straight, here goes. These are the bundles that I had to import to make Spring DM work on the Equinox OSGi implementation:

aopalliance.osgi-1.0-SNAPSHOT.jar
backport-util-concurrent.osgi-3.1-SNAPSHOT.jar
jcl104-over-slf4j-1.4.3.jar
log4j.osgi-1.2.15-SNAPSHOT.jar
slf4j-api-1.4.3.jar
slf4j-log4j12-1.4.3.jar
spring-aop-2.5.1.jar
spring-beans-2.5.1.jar
spring-context-2.5.1.jar
spring-core-2.5.1.jar
spring-osgi-core-1.0.jar
spring-osgi-extender-1.0.jar
spring-osgi-io-1.0.jar

Install these and then make sure that you install a bundle which contains the LOG4J configuration of your choice. This is described here, so I won't go over that again.

BTW, the version numbers on the above bundles might change, if you use later versions of Spring DM and their dependencies.

How to launch an OSGi runtime in your application with Equinox

2008-04-06T11:34:00.008+02:00

Here's a useful code snippet for launching an OSGi runtime from within your application. This is useful where you want to use OSGi in your own application as a plug-in mechanism.

I use Equinox, the Eclipse OSGi implementation.

import org.eclipse.osgi.baseadaptor.BaseAdaptor;
import org.eclipse.osgi.framework.adaptor.FrameworkAdaptor;
import org.eclipse.osgi.framework.internal.core.OSGi;

...

/**
* Launch the OSGi framework.
*
* @param installArea Directory path to where the OSGi bundles remain.
* @param configArea Direcotory path to where OSGi should store its configuration                data.
*/
public void launchOsgi( String installArea, String configArea )
{
    System.setProperty( "osgi.install.area", installArea );
    System.setProperty( "osgi.configuration.area", configArea );
    FrameworkAdaptor adaptor = new BaseAdaptor( null );
    OSGi osgiInstance = new OSGi( adaptor );
    osgiInstance.launch();
}

...

The installation area points to a location in the file system where you want to make your OSGi runtime believe it is "installed", i.e. where it would look for bundles if you used the install file:mybundle_1.0.0.jar command.

The config area points to the location where you want your OSGi runtime to store configuration information about bundles you install while it is running. As you probably know, OSGi stores information about every bundle you install such that upon the next restart, you will not have to install all bundles from scratch again.

Instead of setting system properties, which is admittedly a bit cumbersome, you could pass parameters to the BaseAdaptor constructor, but I havent figured out the parameter names yet :-)

From the OSGi instance you have created you can use osgiInstance.getBundleContext() to install and manage bundles.

That's basically all you need to build an OSGi-based plugin mechanism into your application.

Well, almost. Of course you will need to write some code to detect new bundles in the install area or other locations (at start time of your application or dynamically at runtime of your application). You could use functionality like Peter Kriens' FileInstall bundle, for example. Equinox has something similar but I have not looked at it yet :-)

I might cover this topic in one of my future posts.