Drogomir's blog - Home

Web development testing survey

2009-09-24T15:46:00Z

I would like to know what are testing habits of web developers and I’ve created a survey to find it out. It’s focused on testing, especially browser testing, but if do not test, your voice is still valueable, don’t give up and fill it anyway :)

Survey is on google docs: http://spreadsheets.google.com/viewform?formkey=dGxzNHQzT1hsUmtlVTRTVTcwMHR3aFE6MA

I will publish the results as soon as I will got sufficient number of votes or in 2-3 weeks.

Cucumber and Celerity - testing unobtrusive javascript

2009-06-16T11:25:00Z

In the world of javascript and ajax web apps I often find that writing applications supporting both javascript enabled and disabled is hard. For the sake of simplicity and due to deadlines I often write only javascript version of some of the features. But there are many situations where both scenarios should be supported.

The most important parts of app, especially the ones that must be crawled by google, should be written with unobtrusive style. And here comes the problem… I rarely see javascript testing and till today I haven’t done it myself. Celerity and cucumber (with help of culerity) can solve this problem. Here is short guide introducing this technique.

I will use Cucumber, celerity (which is HtmlUnit wrapper with API compatible with watir) and culerity. Culerity is a proxy between celerity and your app. Celerity requires JRuby and probably your app need MRI or REE – culerity resolves this problem.

Let’s get started.

You need to install JRuby in order to run celerity. After installing and adding jruby to your PATH install celerity gem (probably as a root):


jruby -S gem install celerity

Now you can create rails app and configure the environment:


rails culerity-example
sudo gem install cucumber rspec rspec-rails haml
# add config.gem "haml" to environment.rb
gem install langalex-culerity --source http://gems.github.com
cd culerity-example
./script/generate cucumber
./script/generate rspec
# now edit database.yml and set database options
rake db:migrate
rake features
rm features/step_definitions/webrat_steps.rb # cause we will be using celerity

At this point you should have cucumber configured and you should be able to run “rake features” with output similar to:


0 scenarios
0 steps
0m0.000s

Let’s add some tests! You will need step definitions and hooks. Culerity provides some basic step definitions and hooks which you can generate with ”./script/generate culerity” but I’ve changed them a bit for my needs, so you can find them on this example repository.

Copy those files to your app.

The first file is just rewrite of webrat steps and the second file adds hooks for firing celerity server and browser. Let me explain the hooks.rb file:


require 'culerity'
`mongrel_rails start -e cucumber -p 3001 -d`

Before do
  $server ||= Culerity::run_server
  $browser = Culerity::RemoteBrowserProxy.new $server, {:browser => :firefox}
  $browser.webclient.setJavaScriptEnabled(false) 
  @host = 'http://localhost:3001'
end

Before("@js") do |scenario|
  $browser.webclient.setJavaScriptEnabled(true)
end

at_exit do
  $browser.exit
  $server.close
  `mongrel_rails stop`
end

‘Before’ hooks are run before each scenerio. In first ‘before’ hook server and browser are set up and host is set to “http://localhost:3001” (change it if you want to run app on other address or port). Notice the line: $browser.webclient.setJavaScriptEnabled(false) – it disables javascript by default.

Second Before hook is fired only for scenarios tagged with @js tag. It will be useful for explicitly saying which scenarios should be tested with javascript.

I’ve also added lines to start mongrel before the tests and stop it at exit. It’s handy if you don’t want to run and restart mongrel manually

Now it is time to write some scenarios. File features/javascript.feature


Feature: Javascript
  In order to test javascript
  As a developer
  I need a way to run test scenarios with javascript enabled or disabled

  @js
  Scenario: With javascript
    Given I am on the homepage
    And I follow "Click me!" 
    Then I should see "Javascript rocks!" 

  Scenario: Without javascript
    Given I am on the homepage
    And I follow "Click me!" 
    Then I should see "I am also working without javascript!"

Both scenerios rely on “Click me!” link but have different expectations. ~~To run those tests start mongrel (or any other web server):~~


mongrel_rails start -e cucumber -p 3001 -d

~~This will fire mongrel in background on port 3001.~~ It is not necessary as mongrel control commands are in hooks.rb file

It’s best to operate on 2 console tabs while using celerity. One of the big drawbacks of using it, is lack of displaying rails exceptions. Because of that I run cucumber on one tab and “tail -f log/cucumber.log” on the other – I can see errors in the log without opening the browser.

Let’s run tests:

rake features

Of course both tests should fail with

Unable to locate Link, using :text and /Click me!/

and rails error in log file:

ActionController::RoutingError (No route matches "/" with {:method=>:get}):

Now we can fix it. We need some controller to show the link:

script/generate rspec_controller home

Add

map.root :action => "show", :controller => "home"

to routes file. Next copy app/views/layouts/main.html.haml (it just yields action and includes jquery.js and application.js) and jquery.js

You need to also set layout for home controller:

layout 'main'

And here comes html and javascript. app/views/home/show.html.haml


= link_to "Click me!", "?clicked=1", :id => "click_me" 

#text
  - if params[:clicked]
    I am also working without javascript!

It displays a link and a text if params[:clicked] is present. So after clicking on that link page will be reloaded with parameter clicked=1 and the text will be displayed.

Let’s check if it’s passing:

mongrel_rails restart; rake features

We need to restart mongrel before “rake features” in order to load changes because of “cache_classes = true”. It’s one of the drawbacks of using this method, but I’m sure that someone will find out better way to do that.

If you did everything properly the second scenario should pass now. We’re green! :D

Add javascript code to your application in order to make the second scenario pass:


jQuery(function() {
  $("#click_me").click(function() {
    $("#text").html("Javascript rocks!");
    return false;
  });
})

Now both scenarios should pass.

To sum up, now you should be able to:

configure rails app with cucumber and celerity
specify how your tests should be run by placing @js tag on top of javascript scenarios
test unobtrusive javascript with ease

TODOs:

~~figure out better way to reload rails app while testing~~
provide better error explanations in cucumber with celurity to test without tailing logs

Cheers!

Additional resources

Celerity ajax examples by Alvin Schur

Fixing safe_erb with memcached improved

2009-05-12T13:42:00Z

Ruby has many structures that are weird and dangerous to some programmers that works with Java or other more “traditional” OO languages. For me it’s great and metaprogramming capabilities makes ruby killer in case of DSLs, nice looking APIs and avoiding repeating yourself.

But many problems that are solved with alias_method_chain or method_missing can be solved easier without the code smell. I sometimes find myself designing some weird complex solutions and then, after looking at it saying: “Hey! Why not just use OO features!”. I encourage everybody to look critically at every piece of your code that seems to be overkill and can be replaced with standard OO design.

One of examples is fixing memcached and safe_erb duet with alias_method_chain I don’t know if this solution worked in older versions of rails, but in 2.3 I got stack level too deep error. After some thinking I’ve just created subclass of MemCacheStore:

And then you can simply set it as cache store: config.cache_store = :my_mem_cache_store

And that’s it, no alias method chain :)

Managing many app variants with git

2009-05-03T19:02:00Z

One of my friends works on application that have basically two variants with slight modifications between each of them. It can be horror to work with two similar applications without good SCM.

Happily we have git :)

How to deal with this problem? It’s based on git approach from one of my previous posts so read it first.

Main task is to find last commit with which applications were identical. Let’s say that it’s:


commit b576de0csadcscs846ef933edf75e449d12db39d
Author: FooBar
Date:   Thu Apr 16 13:07:01 2009 -0700

    Last touches before creating appliaction variant.

We must create a branch that will be ancestor for both variants:


git checkout -b both b576de0c

Now branch both keeps only things that are identical in every variant of application. If you want to make some changes and merge them with both applications just:


git checkout both
git checkout -b bugfix23
# fix this nasty bug
git checkout both
git merge bugfix23 # merge it back to both
git checkout variant1 
git merge both # merge the changes to variant 1
git checkout variant2
git merge both # merge the changes to variant 2

Why can’t I just do it without both?


# don't do it!
git checkout variant1
git checkout -b bugfix23 # now we are on bugfix23 which has variant's 1 commits in history
# fix this nasty bug
git checkout variant1 
git merge bugfix23 # everything is fine, mergeing changes from bugfix
git checkout variant2
git merge bugfix23 # now your repo is a mess - you've just merged all of the variant's 1 commits

So what to do if you accidentally commited changes to branch inheriting from one of the variants? You can just cherry-pick one of the commits:


git cherry-pick cs6dc876

It’s like a merge, but merging only one commit.

Reading docx files with ruby

2009-04-28T13:03:00Z

Just a quick post about handling docx (and other *x formats) with ruby.

These files are basically XML packed with zip, so reading them is as just extracting the zip and parsing xml files. Here is a snippet with getting pages count from docx file:

Reading the text is also easy – it’s located in word/document.xml.

I must say that this is a step at the right direction. Reading older microsoft formats without windows API is hard and painful… Although there is a possibility to run MS Office with wine and access it using Win32::Ole and Perl (I’ve tried it once, don’t ask…), there are license problems as far as I know.

My git workflow

2009-04-26T19:56:00Z

I know that this post may be obvious for experienced git users, but it may be useful for some of you.

Github has added default branch picking recently. It’s great, as you don’t have to use master branch as your main branch. I usually have three “main” branches in my repo – production, staging and development.

The main point of it is to have separate branch for each environment (sometimes I skip staging, but it’s often wise to have some point between development and production). My main problem with git (or maybe rather with my lack of knowledge about git) in the very beginning of using git was to learn how to use branches efficiently. After some time with git I’ve noticed that it’s best to keep each fix or idea in separate branch.

One crucial thing is to inherit from production when it’s possible. Let’s say you have those main branches I mentioned before. There is a bug #345 in bug tracker and you want to fix it and merge into production after showing results to the client. The best way is to begin with production:


git checkout -b bug345 production

It will create new branch bug345 with production as a parent branch. Now:


# fix the bug
git commit -m "Fixed the nasty bug #345"

Sometimes client doesn’t need reviewing the changes but when he does you should have ability to show it and merge it to the production branch later.


git checkout staging
git merge bug345
# deploy staging

After the client approval of your solution you can merge the bugfix to the production branch:


git checkout production
git merge bug345
# deploy production
# you can remove bug345 branch now
git branch -d bug345

The most important thing in this process is to always inherit from production branch, unless you need changes from other branches. Otherwise you could merge some needless junk from other branches or have to cherry-pick individual commits. With above solution everything is clean and you are sure that that you will merge only what you need.

Hope it will help :)

JSON-P support for Apache upload progress

2009-02-10T09:08:00Z

Just a quick note about new feature added to apache upload progress and jquery upload progress libs.

Ron Evans aka deadprogrammer has added support for JSON-P in this commit. What does it mean? Cross domain requests are now possible, so if you need such a functionality pull the newest changes.

Here is Ron’s description from README:

- JSON-P Support
You can also request progress updates by using JSON-P, if you are uploading the file from a different domain or subdomain than the web server that is handling your original request. Adding a “callback=yourCallbackFunction” parameter to your request to the progress server will activate this functionality.

For example, a request like: http://uploads.yourdomain.com/progress?callback=jsonp123&X-Progress-ID=1234

Would return the JSON-P function: jsonp123(new Object({ ‘state’ : ‘uploading’, ‘received’ : 35587, ‘size’ : 716595, ‘speed’ : 35587 }));

The normal JSON request: http://www.yourdomain.com/progress?X-Progress-ID=1234
Would return the JSON data: new Object({ ‘state’ : ‘uploading’, ‘received’ : 35587, ‘size’ : 716595, ‘speed’ : 35587 })

Remember to update jquery upload progress also, to use jsonp.

Enjoy :)

Tweaking Rails app with jQuery, part I

2008-07-03T16:40:00Z

Quick links:

completed demo (demo is reseted every few hours)
sources of application on github (or download tarball)

I’m in the train from Zgorzelec to Warsaw returning from my girlfriend’s place. Polish trains are like turtles, so I will have pretty much time for writing ;-)

I’ve wrote (or maybe it’s better to say copy&paste) little rails app like in Mike Clark’s tutorial for attachment_fu. A few months ago there was Mugshots exhibition in Yours Gallery in Warsaw based on work of Peter Doyle. I saw it with Kathleene, she took some pictures. Great! I have material to fill my new app, what else could I possibly dream of?! (yeah… macbook, but it’s obvious ;-).

Now you can admire my hard work: mugshots.drogomir.com/js/no-javascript/mugshots/

But wait… It’s not so cool… where are all those shiny javascript effects? Don’t worry. I will show you how to spice this dish.

We will need:

I’ve pushed application to github, so you can see entire code. Clone it or grab the tarball

There is one thing that is not straight forward. @main_js variable in app/views/layouts/main.rhtml:


<%= javascript_include_tag @main_js %>

It’s there for changing javascript file loaded. When url is app.com/js/some_javascript_file/mugshots, @main_js should be “some_javascript_file.js”. I’ve done this to have possibility to show you app with different javascript files without changing the code. See routes and mugshots_controller.rb to find out how it was done (or run “rake routes” in app dir to see routes).

Lets begin.

What to do first? It’s all about uploading files, so I would add upload progress bar to form in mugshots.drogomir.com/mugshots/new. To implement it you will need some kind of server module:

You have to install and enable one of the above modules to make progress bar work.

Then add some javascript to applications.js. This example is using “LightBoxFu”: – little script that I wrote to show progress bar as an overlay. It’s based on Riddle’s work – all positioning is in CSS (except expressions for IE) so it’s really light and fast. Ideal for such a task. If you don’t like lightBoxFu you can use any other form of displaying message (you can use some other lightbox with displaying code function or even blockUI plugin).


// handy trick, if we can't use $ beaceuse jQuery.noConflict
// was used, jQuery is passed as argument in document ready
// so we can name it $
jQuery(function($) {
  // add upload progress to our form
  $('form.progress').uploadProgress({
    start:function(){
      // after starting upload open lightBoxFu with our bar as html
      $.lightBoxFu.open({
        html: '<div id="uploading"><span id="received"></span><span id="size"></span><br/><div id="progress" class="bar"><div id="progressbar">&nbsp;</div></div><span id="percent"></span></div>',
        width: "250px",
        closeOnClick: false
      });
      jQuery('#received').html("Upload starting.");
      jQuery('#percent').html("0%");
    },
    uploading: function(upload) {
      // update upload info on each /progress response
      jQuery('#received').html("Uploading: "+parseInt(upload.received/1024)+"/");
      jQuery('#size').html(parseInt(upload.size/1024)+" kB");
      jQuery('#percent').html(upload.percents+"%");
    },
    interval: 2000,
    /* if we are using images it's good to preload them, safari has problems with
       downloading anything after hitting submit button. these are images for lightBoxFu
       and progress bar */
    preloadImages: ["/images/overlay.png", "/images/ajax-loader.gif"]
  });
});

And some styling for progress bar:


  #progress {
  margin: 8px;
  width: 220px;
  height: 19px;
}

#progressbar {
  background: url('/images/ajax-loader.gif') no-repeat;
  width: 0px;
  height: 19px;
}

That’s it, just add “progress” class to your form and progress bar is working:


<% form_for(:mugshot, :url => mugshots_path, 
                      :html => { :multipart => true, :class => "progress" }) do |f| -%>

Uploading files looks much better right now, check it here: http://mugshots.drogomir.com/js/progress/mugshots/new

So what now? I find the “add photo, click New mugshot, add photo” scenerio annoying. We could add more than one file on each submit. For that we will use jquery.MultiFile.js. This one is a bit tricky, cause we will need to tweak code handling uploads also.

Javascript enabling mutlifile:


jQuery(function($) {
  $('.multi-file').each(function() {
    // change name of element before applying MultiFile
    // so array of files can be send to server with mugshot[uploaded_data][]
    $(this).attr('name', $(this).attr('name') + '[]');
  }).MultiFile();
});

We must also add “multi-file” class to file field:


<%= f.file_field :uploaded_data, :class => 'multi-file' %>

From javascript point of view that’s all. Let’s see how uploaded photos are handled by rails app:


@mugshot = Mugshot.new(params[:mugshot])

So mugshot.uploaded_data is filled with data from params[:mugshot][:uploaded_data]. Good for one file. But with array of files we should create Mugshot for each file. I would add a method in model:


  def self.handle_upload(mugshot_params)
    # array for not saved mugshots
    mugshots = []
    if mugshot_params[:uploaded_data].kind_of?(Array)
      mugshot_params[:uploaded_data].each do |p| 
        unless p.blank?
          mugshot = Mugshot.new(:uploaded_data => p)
          mugshots << mugshot unless mugshot.save
        end
      end
    else
      mugshot = Mugshot.new(mugshot_params)
      mugshots << mugshot unless mugshot.save
    end
    mugshots
  end

and slightly change controller code:


  def create
    @mugshots = Mugshot.handle_upload(params[:mugshot])

    # if @mugshots is empty there are no errors
    if @mugshots.blank?
      flash[:notice] = 'Mugshot was successfully created.'
      redirect_to mugshots_url
    else
      render :action => :new
    end
  end

Only one problem left. Validation.

Easiest way is to change error_messages_for:


<%= error_messages_for :object => @mugshots %>

It works. But suppose you are uploading 3 files and 2 of them are too big. You will end with:

Size is not included in the list
Size is not included in the list

Which one was added? Some lottery here…

I would tweak attachment_fu error messages a bit. By default it uses validates_as_attachment method which simply adds:


  validates_presence_of :size, :content_type, :filename
  validate  :attachment_attributes_valid?

Instead validates_as_attachment we can isert our new code:


  validates_presence_of :size, :content_type, :filename, :message => Proc.new { |mugshot| "can't be blank (#{mugshot.filename})" }
  validate  :attachment_attributes_valid?

  def attachment_attributes_valid?
    [:size, :content_type].each do |attr_name|
      enum = attachment_options[attr_name]
      errors.add attr_name, "#{ActiveRecord::Errors.default_error_messages[:inclusion]} (#{self.filename})" unless enum.nil? || enum.include?(send(attr_name))
    end
  end

Now it’s a lot more readable:

Size is not included in the list (filename.jpg)
Size is not included in the list (filename1.jpg)

Submit form looks better now, but viewing files is still ugly. Maybe we could add some lightbox? No problem:


$('#mugshots li a').lightBox();

I used that lightbox cause I had it configured for my previous rails apps, but pick your favourite one, as there are gazilions of them.

This is first step of tweaking our app. Javascript is in step1.js file: mugshots.drogomir.com/js/step1/mugshots/new

What now? User can upload many files at one submit and see progress bar. What else do we need? Ajax! My preciousssss…

As all children know, XMLHttpRequest can’t upload files. What a shame… our new tweaked mugshots app is all about uploading files. Although you can’t do it with XHR, there is a way to imitate it. It is obtained by creating an iframe and uploading files to it.

Luckily Mike Malsup has done hard work for us writing jQuery form plugin.

First, we need our form. I would place it instead “New mugshot” link. Link has id=”new_mugshot_link”, so this piece of code will replace it with form:


  /* create upload form with multifile instead of new mugshot link */
  var form = $('<form method="post" enctype="multipart/form-data" class="progress ajax" action="/mugshots">');
  var label = $('<p><label for="mugshot_uploaded_data">Upload mugshot: </label></p>');
  var input = $('<input type="file" class="multi-file" id="mugshot_uploaded_data" size="30" name="mugshot[uploaded_data]"/>');
  label.append(input).appendTo(form);
  form.append('<p><input type="submit" value="Create" name="commit"/></p>');
  if (typeof(AUTH_TOKEN) != "undefined") form.append('<input type="hidden" value="'+AUTH_TOKEN+'" name="authenticity_token"/>'); 
  $('#new_mugshot_link').replaceWith(form);

Our form has to be send to an iframe, so we have to apply ajaxForm to it. After replacing link with form we can’t figure out when form is actually appended to DOM. To be sure that form is there, we can use livequery. It will fire callback function when ‘form.ajax’ will be available:


  $('form.ajax').livequery(function() {
    $(this).ajaxForm({iframe: true, success: function (responseText, statusText, form) {
      var url = $(form).attr('action');
      /* get new files */
      $.ajax({
          url: url,
          dataType: "script",
          beforeSend: function(xhr) {xhr.setRequestHeader("Accept", "text/javascript");},
      /* we need to update lightbox array to include new files */
          complete: function() { $('#mugshots li a').lightBox(); }
      });
    }});
  });

When new form tag with class “ajax” will be available callback function will be run. iframe option tells form plugin to add hidden iframe (it will handle file upload).

The above code has ajax call to ”/mugshots” url which will run index.js.erb (RJS), so we will need one:

app/views/mugshots/index.js.erb


  jQuery('#mugshots').html(<%= js render(:partial => 'mugshot', :collection => @mugshots) %>);

to handle it we need to use respond_to:


  respond_to do |format|
    format.html
    # layout => false is here beaceuse without it rails are looking
    # for layouts/index.js.erb
    format.js { render :layout => false }
  end

Normally I try not to use RJS to keep all my javascript (and ajax) logic in javascript files, but in case of images it isn’t so esay. I will write about it and about javascript templating systems in one of the next posts.

Take a look at: mugshots.drogomir.com/js/step2/mugshots Doesn’t it look nice?

There is only one problem :) No ajax validation. After submitting files, javascript can’t get any info about errors or uploaded files beaceuse it is treated like normal html request and response is loaded in an iframe. How to fix it? I’ll write about it in the next post. :)

Autotest and KNotify

2008-07-02T22:05:00Z

I’ve configured KNotify to work with ZenTest. To see knotify messages just drop below code to ~/.autotest


module KDENotify
 def self.span str, color
   "<span style=\"color: #{color}\">#{str}</span>" 
 end

 def self.notify title, msg, color
   system "dcop knotify default notify " +
          "eventname '#{span(title, color)}' '#{span(msg, color)}' '' '' 16 2" 
   end

 Autotest.add_hook :ran_command do |at|
   if at.results.split("\n").last.first =~ /([0-9]+\sexamples,\s([0-9]+)\sfailures?(,\s([0-9]+) pending)?)/
     message, failures, pending = $1, $2.to_i, $4.to_i
     if failures > 0
       notify "Tests failed", message, "darkred" 
     elsif pending > 0
       notify "Tests passed with some tests pending", message, "goldenrod" 
     else
       notify "Tests passed", message, "darkgreen" 
     end
   end
 end
end

Upload progress script with safari support

2008-06-30T12:14:00Z

Quick links:

Source on github for jquery version: http://github.com/drogus/jquery-upload-progress
Source on github for prototype version: http://github.com/drogus/prototype-upload-progress
jQuery demo: http://drogomir.com/files/blog/jquery-upload-progress/example
Prototype demo: http://drogomir.com/files/blog/prototype-upload-progress/example
Installing apache upload progress

Recently I’ve wrote about apache upload progress module. I work mainly on linux and I haven’t check my scripts on safari. It was working even on IE, what possibly could be harder to obtain? ;-) Some people reported that demo is not working on safari and Michele resolved the problem (thanks Michele :).

Solved! The only thing to do was to open WinXP on VirtualBox and check it on Safari 3. Michele’s solution worked well, but to make it work there must be html page with given structure and javascript. I like “one file” easy to use scripts without any issues with static files :) So what? Create an iframe dynamically, load scripts dynamically, one file, the only thing that user will have to set is path to scripts.

With Safari? No, not really. I’ve wrote it in a few minuttes and checked in firefox. It worked, great, now safari. Nope….

Although Safari have great CSS support, it is really terrible with Javascript. WYSIWYG, javascript history, ajax issues, now the upload progress and iframes. In edge case libraries I often see hacks for IE and safari mainly.

Luckily Apple released Windows version of safari and I can check my scripts and pages on safari. Good… developer tools don’t work, though… I had to do some alert-like IE style debugging ;-) After many hours of trying new more and more insane ways to create iframe and load scripts into it, code with document.write() worked!

Check the commit on github. Lines 18-22 especially. And 28-line issue with safari not waiting to load previous script.

It’s really sad that after working on firefox with firebug or opera with developer tools I have to fight with Safari which is supposed to be modern browser. I know that on Mac developing scripts for safari is easier, thanks to working debug tools, but hey! I work on Linux. Why do I have to run Safari on Wine or VirtualBox, lacking debug tools? It’s a pain. I feel like I’m working on explorer…

~~I will add prototype version~~ and possibly some usage page shortly (for know look at the demo code).

Upload progress bar with mod_passenger and apache

2008-06-18T21:18:00Z

UPDATE: I found 2 bugs in upload progress module. If you have already installed. update to at least 0.1 version: http://github.com/drogus/apache-upload-progress-module/commits/0.1

I’ve installed mod passenger on my server recently. It’s really great software. Now I don’t have to worry about monitoring, nginx proxy, load balancing, big file uploads… and it’s fast! With Ruby Enterprise Edition it’s even faster.

Personally I don’t care about people saying that phusion wants to promote themselves on REE as long as it gives faster ruby with lower memory use (but yes, I know, REE is not best choice for a name :).

After installing I’ve realised that my shiny upload progress bar (thanks to Upload Progress Module for nginx) was not working. Oh gods! What a tragedy!

But fear not. I’ve written apache upload progress module to have my lovely progress bar back again. As a lazy developer I’ve implemented reports in the same way as in nginx upload progress, so my applications are working without changing any signle line of code. If you were using nginx upload progress just drop the module, change your config and you’re good to go :)

I’m testing it in one of my production servers, but be carefull – it’s not well tested in other enviroments (i have gentoo with apache 2.2.8). Any feedback will be helpfull. Give me a note in comments if you encounter any problems.

So you want to be cool and have your own sexy progress bar in your app? Keep reading ;)

To install module you must download it using git:

git clone git://github.com/drogus/apache-upload-progress-module.git

or get the package: http://github.com/drogus/apache-upload-progress-module/tarball/master To compile/install/activate you have to use apxs2:

apxs2 -c -i -a mod_upload_progress.c

-c is for compiling
-i is for installing (copy mod_upload_progress.so to apache library dir)
-a is for activating (add LoadModule option into your apache conf file)

If you want to install and activate run this command as a root. Otherwise you can just compile and add LoadModule to apache conf:

LoadModule upload_progress_module path/to/apache-upload-progress-module/.libs/mod_upload_progress.so

Currently there is only one global option:

UploadProgressSharedMemorySize 1024000

This sets shared memory size to 1M. By default it’s 100kB.

To add tracking and reporting upload for a virtual host in apache you will need to add:


<Location />
    # enable tracking uploads in /
    TrackUploads On
</Location>

<Location /progress>
    # enable upload progress reports in /progress
    ReportUploads On
</Location>

Now all uploads will be tracked and reports are under /progress

Format of the report is JSON. From nginx wiki:

The returned document is a JSON text with the possible 4 results:

the upload request hasn’t been registered yet or is unknown:

new Object({ ‘state’ : ‘starting’ })

the upload request has ended:

new Object({ ‘state’ : ‘done’ })

the upload request generated an HTTP error:

new Object({ ‘state’ : ‘error’, ‘status’ : <error> })

One error code that is interesting to track for clients is HTTP error 413 (Request entity too large)

the upload request is in progress:

new Object({ ‘state’ : ‘uploading’, ‘received’ : <size_received>, ‘size’ : <total_size>})

The HTTP request to this location must have either an X-Progress-ID parameter or X-Progress-ID HTTP header containing the unique identifier as specified in your upload/POST request to the relevant tracked zone. If you are using the X-Progress-ID as a query-string parameter, ensure it is the LAST argument in the URL.

Now the last thing to do is to implement progress bar. I don’t like repeating others and there is great tutorial on setting up upload progress bar with nginx and merb

UPDATE: I released jquery upload progress library with Safari 3 support. More info here. UPDATE2: I’ve upgraded prototype version to work in Safari.

It’s for merb and nginx but if you drop the scripts in your rails app and with apache-upload-progress-module it will work. :) Basically if you have your own code handling uploads (for example using attachment_fu) you can just add javascript and css – it’s unobtrusive.

If you’re using prototype I’ve rewritten script and made a demo. You can also grab files

I hope you enjoy this article. Progress bar is in my opinion one of the most useful technics – there is nothing more annoying than large file uploading without any info on state of an upload.