Ibrahim Ahmed

Links for 2009-01-28 [del.icio.us]

2009-01-29T00:00:00-08:00

Links for 2009-01-08 [del.icio.us]

2009-01-09T00:00:00-08:00

rubypond's semantic_form_builder at master — GitHub
A custom builder (and some helper methods) for creating consistent and accessible forms in rails

Links for 2009-01-07 [del.icio.us]

2009-01-08T00:00:00-08:00

crafterm's sprinkle at master — GitHub
A software provisioning software! an unclear description to a very promising tool, to make it easy for us to deploy or install software locally or remotely, and the key feature is "dependencies management"

Links for 2009-01-04 [del.icio.us]

2009-01-05T00:00:00-08:00

OGRE Git Tutorial - excess.org
This is a recording of the Git tutorial given by Bart Trojanowski for the Ottawa Group of Ruby Enthusiasts.

Links for 2008-12-15 [del.icio.us]

2008-12-16T00:00:00-08:00

Simple Bug and Issue Tracking | Sifter
Sifter is a hosted bug and issue tracking application focused on making work less tedious.
Why Git is Better Than X
Ibrahim Ahmed: Rails ActionController facelift using ResourceController and named_scope
Make Rails controller more powerful yet simpler using ResourceController plugin and ActiveRecord named_scope.

Rails ActionController facelift using ResourceController and named_scope

2008-12-14T02:40:00.000-08:00

I am pretty sure that it is not only my Rails controllers which changed appearance more than once during the past 3 or 4 years. Concepts like RESTful controllers, nested resources, ActiveRecord scoping kept on changing the way we build our Rails controller. Nowadays, my Rails controllers are barely similar to what they looked like 4 years ago.

During pre REST days, my controllers were so ugly and fat. with too many actions to count and do a lot more than the controller meant to do in an MVC framework. Then came RESTful controller which was really, really a nice way to help application designer to organize and "design" the application. RESTful controllers concept is revolutionary in that it simplified or abstract away the complexity of the application plumbing and repetitive routines and made my poor little mind focus on the application design instead.

And then came resources scaffold
The scaffold generator now do us all the work to have a nice RESTful controller that can CRUD your model and responds to HTML, XML, JS. But I need to confess that I've never liked generators. Heck, it is the only reason why I don't use IDEs. They pollute my code with lines I didn't write, and that's plain ugly. It is lovely to have your controller speak REST in XML, but it was not a good feeling when I see all those respond_to lines scattered all over my controllers at the time when my application is not yet designed to be used as such.

And what made it really worse is that most of the time the generated controller needed almost 80% editing, to scope my models I use in the controllers, like @site.pages, @user.posts, and to paginate results, set model attributes assignments..etc. So it was obvious that generators wouldn't cut it. We needed some help to abstract all the repetitive codes in our controller and find a solution for problems like relations aware controller that respect scoping, can deal with nested resource and namespaces. I am sure that any rails developer (or any MVC web developer) knows the kind of problem I am talking about.

Resource Controller plugin
More than a year ago I found some Rails plugins to do just that. But I was too skeptical to use them in production. Time passed and there are now a really good, tested and tried solution for the kind of problem I am talking about. It is called "Resource Controller".

It is a fine plugin written by James Golick. Thank you James. The other plugins like resource_this and make_resourceful were steps in the right direction but Resource_Controller is the way to go.

After months of using it I feel pretty confident with resource controller. And as all tools that abstract away repetitive noise it allowed me to focus more on my application design and actually I developed more productive patterns in developing my application. Patterns that changed the face of my rails controllers beyond recognition. Here is a minimal Resource Controller.

12	class UsersController < ResourceController::Baseend

Suddenly, I had this ActiveRecord thrill again, but with Resource Controller this time. I bet this is the skinniest controller one could dream about. Come on Rails core team, when will we see Jamis' Resource controller merged!

Well the next thing to do is playing around and try to see how to solve the problems I mentioned above. But only after introducing my newest love, "Named scope". which prove very useful in DRYing my controller even more.

Named Scopes
I always thought that the best feature in ActiveRecord is scoping through associations. Like, '@user.posts.find(:all, :limit => 10)'. This feature alone is enough to use ActiveRecord. The next best ever feature in ActiveRecord is named_scope. I couldn't recommend enough using named_scope. Just use it. It is core since Rails 2.1.

Coupling named_scope with Resource Controllers really open new application design territory. Imagine the following requirement.

One pages controller that can be used to list all the pages in a portal
It could be used nested under users resource, so it should scope pages to user.
It should handle posts pending for approvals.
It should fetch the next and previous pages according to the context or the scope it works within, be it portal pages, users pages, or user's pending pages or portal's pending pages.

Let's build the controller
First we do the routes, a simple nested routes and then the controller.

map.resources :users do |user|
    user.resources :pages
end
map.resources :pages

class PagesController < ResourceController::Base
    belongs_to :user
end

This is the simplest form of controller to make it serve routes like /user/12/pages/new and do all the REST actions, be it scoped to user or not. But certainly life isn't that simple. What if you want to paginate results and get user by user_name instead of user_id like "/user/hany/pages".

class PagesController < ResourceController::Base  
  before_filter :set_scope 
  belongs_to :user 

  private 
  
  def parent_object 
    @parent ||= User.find_by_name(params[:user_id]) if params[:user_id] 
  end

  def set_scope 
    @scope = @portal.pages 
    @scope = @scope.by_user(parent_object.id) if parent_object 
  end 

  def collection 
    @pages ||= @scope.paginate(:all, :page => params[:page])
  end
end

Here I added some private methods. The parent_object method to get the parent object by name. I introduced the set_scope method which does set the scope (or context) of the collection set of pages we need to work with. Actually, parent_object and collection are both overridden methods of resource_controller. and set_scope method is mine, and I run it in a before_filter and use it in the controller to find the pages within the set scope. I could have just do scoping inside collection method, but later I will need to have a handle for the scope in different places in the controller other than "collection".

The set_scope method is interesting because I stack up the scope conditionally according to the context within which the controller is running. I mean if the controller is nested in users resource it will find params[:user_id] and the parent_object return a user object and then the @scope of users pages will be added. Which is a named scope in this case "Page.by_user()". I could have use parent_object.pages which is the natural association scoping of ActiveRecord I got with the "has_many :pages" but because my application serves several portals, and @portal got assigned according to subdomains on the application controller, I don't won't to loose the @portal scope '@portal.pages' so it was either make a named_scope for portals or users.

Notice here that @scope.find(:all, :conditions => {:user_id => parent_object.id}) will not do us any good because you will not be able then to narrow down those scopes by stacking them. But sure an inline or anonymous named_scope will suffice instead.

Now I need to make the controller serve pages pending for approvals and fetch the next and previous pages properly according to the scope. Some routes need to be set first. Changing the scope is only a matter of adding a new line to the set_scope method according to the passed params[], so all I need to do is passing non blank params[:pending]. Of course passing it hanged on the url like '/user/hany/pages?pending=1' is not an option. I need it to be clean like '/user/hany/pending_pages' and to say the truth it is not about cleanliness, it is about setting proper url automagically as I will show you later.

map.resources :users do |user|
  user.resources :pages, :requirements => {:pending => false }
  user.resources :pending_pages, :controller => 'pages', :requirements => {:pending => true  }
end
map.resources :pages, :requirements => {:pending => false }
map.resources :pending_pages, :controller => 'pages', :requirements => {:pending => true  }

Here I added pending_pages resources and pass explicitly the controller name because I will use my same pages controller. I made the resources properly set the pending parameter to true or false. I don't believe Rails 2.2 shallow routes option will spare me the pages resources repetitions. May be we need another way to make the parent resources optional?! but anyways, now that we have the routes properly set, let's see the controller.

class PagesController < ResourceController::Base
        
  before_filter :set_scope
  belongs_to :user

  private

  def parent_object
    @parent ||= User.find_by_name(params[:user_id]) if params[:user_id]
  end

  def set_scope
    @scope = @portal.pages
    @scope = @scope.by_user(parent_object.id) if parent_object
    @scope = @scope.pending(params[:pending])
  end

  def collection
    @pages ||= @scope.paginate(:all, :page => params[:page])
  end

  def route_name
    params[:pending] ? 'pending_post' : 'post'
  end
end

Here I added one line to set_scope to add the pending scope if the params[:pending] was true. But for a change I didn't check for params[:pending], instead I pass it to the named_scope which will check internally if it set or not and do the scope accordingly. Actually I did this in my real application to not be forced to make another not_pending named_scope. Because in real life you will need not to publish unpublished pages. errr.. ok, i mean the pending scope is unlike users scope, which is like, "if parent user is not set just show all the pages anyway", this will not work here, we need it to be, "if pending status is false just list only the published pages". And now it's a good chance to demonstrate how to pass arguments to named_scope.

1234	class Page < ActiveRecord::Base named_scope :by_user, lambda{\|u\| {:conditions => {:user_id => u} } } named_scope :pending, lambda{\|s\| {:conditions => {:published => s} } }end

where is the next and previous links?
It is a very healthy practice to provide a next and previous item links in your show item page. For many obvious reasons, for one is to spare the user going back to pages list which will stress your server as well. Just give her the links or previous and next page. But you have to prepare the links first in the show action. In Resource controller actions are abstracted so you don't have to do the repetitive code each time, just do what is specific to each controller action. In this case I need the show action to fetch the next and previous page according to the scope within which the pages controller works. And do that just before rendering the template or to be precise just before responding.

class PagesController < ResourceController::Base
        
  before_filter :set_scope
  belongs_to :user

  show do
    before do
      @next_item = @scope.next_to(object).first
      @previous_item = @scope.previous_to(object).first
    end 
  end

  private
  
  ...

end

This was easy, just assign your next and previous variables to the first item returned by the named scopes "next_to" and "previous_to". To be precise, named_scope are not results set (Array) so we need to do ".find(:first)" or ".first" as of Rails 2.1, to just hit the database and fetch the records. I passed 'object' to the named scopes which is a method refers to @page, you can override this method just to find your page the way you want it, say you want to find_by_premalink, for example. The following are how those named scopes were implemented.

class Page < ActiveRecord::Base
  named_scope :by_user, lambda {|u| {:conditions => {:user_id => u} } }
  named_scope :pending, lambda {|s| {:conditions => {:published => s} } }
  named_scope :next_to, lambda {|item|
    {
      :limit => 1, 
      :conditions => [ 'id > ?',item.id] ,
      :order => 'id asc'
    }
  }
  
  named_scope :previous_to, lambda {|item|
    {
      :limit => 1, 
      :conditions => ['id < ?', item.id] ,
      :order => 'id desc'
    }
  }
end

Well, that's all well a good, but pretty DRY controllers are not the end of all our sufferings, are they? the views are always the primary cause of migraine in web development, at least they are for me. This resource controller is doing all what I asked it for. But how the views will handle such complex set of nested and or conditional scoping? When I first asked myself this question and start to figure out what may the solutions be. I found my self saying, Hell, no no, I will not make all those conditions and the ever nested loops of if/else to just decide which url routes I shall use in the new and edit forms. But Alhamdulillah, thanks God, there is this most useful piece of code that ever existed in Rails view helpers, excuse the drama but it is really that good, it is the "Urligence" library which is a url helper that helps you generating urls! It is included in Resource Controller, so no need to download. This url helper is not only useful in views it is already used in the controller in redirect_to statements taken place deep inside resource controller, like after create for example it is by default redirect to show action. In this case which show route will it use? In our case we have at least three show routes:

/pages/:id
/user/:user_id/pages/:id
/user/:user_id/pending_pages/:id

The surprise is that Resource Controller will know exactly which route to use according to the context. Actually it does it using Urligence url helper. The helper themselves are too easy. You will have object_url and collection_url, that's it. In Rails resources native tongue: object_url to refer to any member action, and collection_url to refer to any collection action. And of course the HTTP verb or the request method will decide which action to address, like, post to collection_url to create and put to object_url to upate and such.

<% form_for :page, @object, :url => object_url, :html => {:method => :put} do |f| %>
   <%= f.text_field :title %>
   <%= submit_tag "Save" %> or <%= link_to "Cancel", object_url %> and go back to page.
<% end %>

<%= link_to("Next", object_url(@next_item)) if @next_item %>
<%= link_to("Previous", object_url(@previous_item)) if @previous_item %>

These snippets from the edit and show templates are pretty much clear and amazing at the same time.

What about controlling the flow of the application. What if you need to change the default flow of the resource controller and make the "delete" action redirect to the next item show page if it exists and to the pages index if there is no next page.

delete do
  before do
    @next_page = @scope.next_to(object).first
  end
  after do
    if @next_page
      redirect_to object_url(@next_page)
    else
      redirect_to collection_url
    end
  end
end

All we need to do is to explicitly add the delete action. I tried to fetch the next item in the current @scope "before" deleting. And "after" deletion I should redirect properly as required.

There are lot more in ResourceController, it lets you customize flash message for success and failure, add responds to methods, respect namespaces automagically and more.

Anyway, that's it regarding how ResourceController and Named Scopes has changed the way I write Rails ActionControllers. for more information refer to the following links:

Links for 2008-11-30 [del.icio.us]

2008-12-01T00:00:00-08:00

ratproxy - Google Code
A semi-automated, largely passive web application security audit tool, optimized for an accurate and sensitive detection, and automatic annotation, of potential problems and security-relevant design patterns based on the observation of existing, user-initiated traffic in complex web 2.0 environments. D

Links for 2008-11-28 [del.icio.us]

2008-11-29T00:00:00-08:00

Working with remote Git branches

2008-06-22T11:45:00.000-07:00

You are about to start a brand new module in your project. Your project is already in production. You cannot develop and test your new module while fixing minor bugs in your production line. You need branching.

So suppose we are on master branch and you have remote "origin" repository. And we need to create a new remote branch called "search" to track the development of the search module. the plan is to develop the search module on this branch "search" locally and push updates remotely for testing on a staging server or just to share the search module code with a colleague developer without polluting the master production branch with the unfinished yet search module.

First make sure your master branch are up to date.

$ git pull origin master

Then push your master branch to a new remote branch, notice that you enter the destination remote repository name first "origin" then the source which is local branch "master" then a column ":" and the path of the new remote branch we want to create "refs/heads/search"

$ git push origin master:refs/heads/search

After pushing and creating the remote "search" branch, we will examine the remote "origin.

$ git remote show origin
* remote origin
URL: ssh://ibrahim@kenana.org/git/portals.git
Remote branch(es) merged with 'git pull' while on branch master
master
New remote branches (next fetch will store in remotes/origin)
search
Tracked remote branches
master

Notice how there is a new remote branch and that next time we use "git fetch" the new remote branch "search" will be stored in remotes/origin.

$ git fetch
* refs/remotes/origin/search: storing branch 'search' of ssh://ibrahim-ahmed.com/git/portals
commit: 8bd83c4

Now we have all the remote origin branches stored locally in "refs/remotes/origin" including "search" branch, but there is no local branch yet to track it.

To make a branch out of the remote branch which is stored locally, we will use "git checkout", the "-b" search" is the argument we use to create a local branch. "--track" parameter is used to link the local branch "search" with the remote branch "origin/search"

$ git checkout --track -b search origin/search

Now you should be on "search" branch.

$ git branch
master
* search

Now you can edit, commit, pull and pull to your heart content. And when you need to fix something on remote/master which is used for production server, you just use the following to fix the bug, push to the remote master and then back again to your search module.

$ git checkout master
$ vim files_to_be_fixed
$ git commit -a -m "just fixed the bug"
$ git push origin master
$ git checkout search

Check SourceMage Git manual for further advanced usage examples.

Design an accessible Website using CSS

2008-04-06T15:16:00.000-07:00

Designing the web page layout is an essential step in the website development. It affects almost every aspect of the website development, from styles, content types, photos, search engine optimization SEO, and definitely the website user friendliness.

There were a deep misunderstanding to the Web as a capable media for content publishing and a potent marketing tool, the graphics designer were using the Internet shamelessly as a mere brochure. I mean they were designing a brochure (a real one) and convert it to a web page using one of the web graphics application like Fireworks, or Dreamweaver. The conversion was usually done using image maps and slicing. Graphics slicing were obscenely abused, I saw many crimes where developer create or import a brochure into Photoshop and start slicing the image into small chunks of gif images, and export the whole design as an HTML. At best they export part of the design from Photoshop and integrate the dynamic parts into the body area of the layout.

Leaving Photoshop and brothers to do the HTML is not a good idea. Really. It is like translating an English site to Arabic language using a machine translator. The end product is gibberish. The sliced layout with mixed Html and images, is harmful for many reasons I will try to highlight them in this post.

HTML is not a presentation layer. Using HTML to design the layout is not the job of HTML. HTML is an XML format to define the data structue of the web page document. I like to describe a document in terms of paragraphs, menu list, sidebar, blockquotes, ordered list, unordered list, those elements are unsurprisingly the elements of HTML. But how you want to render or represent the menu lists, the sidebar, the blockquote, representing those elements is entirely the job of CSS (Cascading Style Sheet) and not the HTML.

What made me a true believer of CSS layout, CSS styling and using HTML only in representing the document in its basic elements is that our eyes are not the only consumer of the web page we develop, there are others, like; search engines crawlers, blinds, mobile web browsers, printers..etc all those consumers will not see or use our graphics as we intended for it to be seen

Search engines will not understand that my website is concerned about farming and agriculture if I did export the titles of the agriculture sections as images from Photoshop. So People searching Google will not find your site only because Google didn't find thesearch keywords in your site. The same goes for blinds. There are using the Internet in a daily basis in Egypt, Arab countries and around the world. For blinds they use Text to speech software to convert "text" to speech, it will simply ignore images, so blinds will miss your website sections menu if it was made of a bunch of images racked into an HTML Table.

Another reason for using CSS is that blinds are needlessly forced to listen to a lot of repeatable content on the top of each of our pages, for no reason but that HTML is rendering first things first, so if we want to put the site menu, contact us, about us, login, and sidebars on top of the page, you could render it on top if you want to, but there is no need to fill the blind with boredom in order for him / her to reach the body of your page content, and force him / her to read through all those content again and again in each of your website pages. We can always use CSS to render different page elements in any position regalrdless of its order as an element in the HTML document.

To summarize the characteristics of a good Website design I can say;

No style graphics are embeded in HTML, all design and decoration images located in a separated CSS file. No menu list, titles, or any content is represented only by graphics, it should be a text first and then styled and decorated with any graphics the designer decide.
No Tables are used to define layout columns, Tables are only for tabled data, and not for layout.
Web page content comes first in the order of HTML document. The title and body of the article of the web page is the first thing you see if you open the source code of the page or see the page without CSS styles, that's good, cause that's how other blinds, search engines, mobile browser will see your page.

I am yet to see CSS developers in Egypt, job posts for CSS developer in Egypt are none. Employers do not know what they are missing, they bet on graphic designers to do the job, and usually graphic designer have no idea what CSS is all about, they will use CSS for styling at best, and not for layout. That's why I will do a small contribution and start a mini series of CSS lessons (hopefully in Arabic too), in order to implement the mentioned principles in designing a good website design.

Git the Version Control System - part one

2008-04-06T03:42:00.000-07:00

Git is a Version Control System, it is used to keep a history of all the changes you made into your code, manage releases, makes it easier for more than one programmer to share his/her code with the rest of the group.

There are tens of good VCS out there, the most famous was CVS Concurrent Version System, then came Subversion from the same team of CVS with the same concepts but cleaner than its ancestor. Both CVS and Subversion was free. There are other sophisticated VCS out there some of them are free and the other are proprietary. Linux was developed using a free license version of Mercurial which is pretty good CVS but for more some reason Linus Torvalds was challenged enough to start writing his own VCS, and Git was born.

There are two types of VCS, Central and Distributed. In central VCS like CVS and Subversion, there is always one central repository, you checkout a copy of whatever version you like from the repository, start working on it and then submit your changes into the central repository again.

The Distributed type like Git, there is one or more central repository, and you clone the repository into your local machine, and then you will have a standalone repository. Which is more like forking the development of the code at certain point in its development line into another repository. You then maintain your repository and change to your heart content, and it is up to you or the original (upstream) repository maintainer to merge your changes.

I will write a note on the basic operation of managing my code in GIT. For the sake of simplicity I will not write in this post about how to serve or access remote GIT repository using SSH, I will just assume that all repositories are on local machine.

$ mkdir myapp
$ cd myapp
$ git init
Initialized empty Git repository in .git/

I created a folder for my application, and inside the folder I make Git initialize a new repository.

At this point we should have a folder named .git inside the 'myapp' and that .git folder by itself is a repository, in this folder GIT will store all the transaction, changes, version, releases of your code. The beauty of GIT is that it does not pollute your code folder with other files like Subversion does. Subversion was creating an .svn folder inside each folder in my app folder.

As I said .git is the repository and the rest of your folder is now then called the working copy. the working copy is an important concept. It is your current version files, in Git if you switch to another version of your code, all the files in the folder will be changed to reflect the real version you switched to. We can create a repository without a working copy and that's what we do in case we need to make a central repository where programmer are pulling and pushing changes into it, in that case we do not need a working copy with the repository, this is called a 'bare' repository.

The .git repository folder is now empty, it has no files (or changes) in it. To start populating the repository we use the 'add' Git command. but we actually do not have any files yet, so let's make some files.

$ echo "hello" > dialog.txt

I created a file 'dialog.txt'. Now i will ask Git about the status, and it will answer me with the following information, which match files in three cases:

The modified files in my working copy that needs to be committed.
The modified files that are marked for committing.
The files that are not yet tracked by Git, and they need to be added first to the repository.

Now that we only have one file in the third case.

$ git status
# On branch master
#
# Initial commit
#
# Untracked files:
# (use "git add ..." to include in what will be committed)
#
# dialog.txt

Git suggest to use the add command to start tracking the file. What tracking means? Ok, in a working copy there is either tracked files that you want to keep track of, and untracked files which are either newly created and needs to be added to the repository or non source code files like log files, object and comoiled files and in this case we need to tell Git to ignore them and not notifying me about them when we use the 'status' command.

To be more specific the 'add' command does not add the files to the repository here or even convert the untracked files into tracked ones, the 'add' command is actually preparing the files to be committed or added to the repository in the next time you use the commit command. Actually 'add' is adding the files, tracked or not, to an area which is called the 'index', and after you add all the files you need and you feel satisfied that your all the changes you need to commit is in the stage, then you can use the commit command to commit your changes into the repository.

$ git add dialog.txt

Now that we added the dialog.txt it is now in the 'index' and will be commited by the next commit command.

$ git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
# (use "git rm --cached ..." to unstage)
#
# new file: dialog.txt
#

Well, it is time of our first commit. Git is storing the name and email of the committer with each commit, so we need to identify ourselves first to Git.

git config --global user.email "ibrahim@mymail.com"
git config --global user.name "Ibrahim Ahmed"
$ git commit -m "saying hello"

I committed the dialog.txt and notice the '-m', commit always needs a log message, try to write a message describing the changes you've done to the file.

$ echo "what's your name" >> dialog.txt
$ git status

I added one more line to dialog.txt, and git status to see the changes, and Git should giving me that dialog.txt is modified and will be committed in the next comit.

One more time, and excuse my boring repetition, you will need to add first the dialog.txt to the index area, before it is ready to be commit.

$ git add dialog.txt
$ git commit -m "asking about your name"

Ignoring files
Git status will always nag about untracked files, so we should either 'add' it or ignore it.

There are always some files in your app that you do not need to track, files like logs for example.

$ touch myapp.log
$ git status
# On branch master
#
# Untracked files:
# (use "git add ..." to include in what will be committed)
#
# myapp.log
$ echo "*.log" >> .git/info/exclude
$ git status
# On branch master
nothing to commit (working directory clean)

Notice how git status listed myapp.log in untracked files, and when we add a line containing '*.log' to '.git/info/exclude', git status ignored the myapp.log file and gives the message 'working directory clean'

Remove files
'git rm' removes files from the working directory and from the repository. But it also used to remove files from the 'index' using the following;

$ touch delete.me
$ git add delete.me
$ git commit -m "adding delete.me"
$ git rm delete.me
$ echo "how old are you?" >> dialog.txt
$ git add dialog.txt
$ git rm --cached dialog.txt

That's it for the first part of my note about Git, on the second note I will write about diff, clone, pull, adding remote repository, merge, and conflict resolution.

Login to Secure Shell Using Key Authentication

2008-04-02T12:43:00.000-07:00

Secure Shell (SSH) is the industry standard encryption protocal for accessing remote server and executing commands in standard shell but in a secure way. SSH is doing a better job protecting the login account from sniffers, way better than its ancestor 'Telnet' which was transfering login account username and password in clear text for all to see. In fact SSH is not only protecting login account by encryption but it is actually encrypting the whole session so that even if sniffers spy on your session they will not be able to decrypt it.

SSH login account is not the only way of authentication, there is SSH Keys authentication. Having an SSH server configured to accept key only authentication is better than server accepting passwords for authentication cause the later is vulnerable to dictionary attacks. So the most secure way to access a remote server is by using key authentication SSH. And that's what I am going to write a note about now.

First we need to make sure that our local system having ssh installed.

$ sudo apt-get install openssh-client

and make sure than ssh server is installed on the server

sudo apt-get install openssh-server
sudo /etc/init.d/sshd start

SSH Key generation
We need to generate a pair of keys, on public for the server to encrypt the data and a private key, which is the only key that could decipher the encrypted data, and this private key is by definition should be kept private. There is many secure algorithm for encryption with different degree of encryption strength. There is DSA and RSA, as far as I know DSA is the standard encryption for the USA government, DSA keys has a 1024 size limit, whereas RSA is unlimited .I chose to use RSA key with a 2048 length, here are the steps.

$ ssh-keygen -v -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ibrahim/.ssh/id_rsa): /home/ibrahim/.ssh/ibrahim_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/ibrahim/.ssh/ibrahim_rsa.
Your public key has been saved in /home/ibrahim/.ssh/ibrahim_rsa.pub.
The key fingerprint is:
66:d2:cc:7b:6a:62:f9:f5:c6:ef:69:fc:7b:87:0d:46 ibrahim@ibrahim
$ chmod 600 /home/ibrahim/.ssh/ibrahim_rsa
$ scp /home/ibrahim/.ssh/ibrahim_dsa.pub ibrahim@myremote.server.com:/home/ibrahim/.ssh

The latest scp command is named secure copy, which is a part of the ssh package, it is a secure remote copy command.

on remote server we should do the following.

$ cat /home/ibrahim/.ssh/ibrahim_rsa.pub > /home/ibrahim/.ssh/authorized_keys

You will be asked for the login password on remote before the copying commences. The file will be copied to login user home directory on remote (/home/ibrahim in that case).

Now, let's login to remote server using the password to configure the sshd server to disable password login and enable keys.

PermitRootLogin no
#Disable Login password
#PasswordAuthentication no
ChallengeResponseAuthentication no
#Allow forwarding yes
AllowTcpForwarding no

Uncomment 'PasswordAuthentication no' line only after making sure that the key authentication is working properly.
Disabling root login is recommended anyway, though not useful after disabling login password.
Allow forwarding is not recommended for multi user hosting envirnoment where keys could be exposed. Anyway, we should only allow it if we intend to forward keys from server to server but keep all our keys on the local machine, which is what I exactly want to do.

Now, let's try login using the keys.

$ ssh -i /home/ibrahim/.ssh/ibrahim_rsa ibrahim@myremote.server.com

you should be asked to approve connecting to the new server, and whither you trust it, answer yes, that's because we are connecting to the remote server to the first time.

Hopefully you should now have access to the remote server shell. Now, edit the sshd config file and disable login by password by uncommenting 'PasswordAuthentication no'.

It is recommended to protect the keys with a passphrase. it is straightforward to do so. In fact you will be asked to provide a passphrase to your private key during key generation and you can skip it if you want. In case you did skip it you can lock it again with a passphrase using the following.

$ ssh-keygen -p

Then it will prompt to put the key file path and you should enter then the password which must be more than 5 chrs.

Of course using keys is not only useful for security reasons, but also for not asking for password every time you use ssh. But thanks to ssh-agent we could save ourselves a few keystrokes, and more importantly use ssh in automated scripts without interrupting the script to prompt for passwords.

SSH Agent

$ eval `ssh-agent`
$ ssh-add /home/ibrahim/.ssh/ibrahim_rsa
$ ssh root@myremote.server.com

We first ran the ssh agent, which is actually a service. then use ssh-add to add the key, them ssh the remote server with only the user name and the remote server address, without providing the key, and if you have protected the key with passphrase you will be asked for the passphrase when you add it. The ssh-agent help in opening a session so we can use ssh to access remote server without giving any keys or password.

Switch proxy settings quickly in Firefox

2008-03-13T01:33:00.000-07:00

I keep on switching back and forth proxy setting in Firefox. I am behind a proxy at work and direct connection at home. For all poor souls who suffer the same hassle go install this little firefox addon QuickProxy.

Making GPRS Vodafone card work on Ubuntu

2008-03-12T17:17:00.000-07:00

I spent a couple of hours trying to make a GPRS Vodafone PCMCIA card to work in my Ubuntu laptop. And It's working, and here are my tips. But I believe the following instructions should be useful in other Linux distributions.

Make sure the following packages are installed:

sudo apt-get install pcmcia-cs
sudo apt-get install gnome-ppp

Configuring Gnome-ppp
There some Vodafone Egypt specific data needed for connection

APN: internet.vodafone.net
user: internet
password: internet

Type on a shell the following command and try to know the device assigned to the pcmcia GPRS card. In my laptop it was /dev/ttyS3. I noticed other blogs mentioning cards assigned to USB devices! So look carefully. You may need to eject and reinsert the card again and observe the messages on the syslog.

tail -f /var/log/syslog

After knowing the device assign to GPRS card, click "Setup" and choose the device from the list.

Select speed like 115200, there are bloggers mentioning 460800 so you may try it later. It doesn't work for me.

Select "Tone" as the phone line type.

Click on "Init Strings" and enter the following lines in Init2 and Init3 respectively.

Init2: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3: AT+CGDCONT=1,"IP","internet.vodafone.net"

.. and uncheck "wait for dialtone"

After I configured gnome-ppp I tried to connect, I used *99***1# as the dial number, clicked connect, and see the log. And I got the pppd died unexpectedly error message, that very message traveled me back 12 years ago, trying and failing to make ppp work. My frustration didn't last more than 10 minutes, I figured out that gnome-ppp was trying and failed to open wvdial files in /etc/ppp/peers, so i searched the net and get me a bunch of wvdial conf files, put them in place and here are them.

# File: /etc/ppp/peers/wvdial
# WvDial options
#
plugin passwordfd.so
#
noauth
name wvdial
defaultroute
replacedefaultroute
noipdefault
usepeerdns
novj

#File: /etc/wvdial.conf
[Dialer Defaults]
Modem = /dev/ttyS3
Baud = 115200
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2
Init3 = AT+CGDCONT=1,"IP","internet.vodafone.net"
Area Code =
Phone = *99***1#
Username = internet
Password = internet
Ask Password = 0
Dial Command = ATDT
Stupid Mode = 1
Compuserve = 0
Force Address =
Idle Seconds = 3000
DialMessage1 =
DialMessage2 =
ISDN = 0
Auto DNS = 1

[Dialer Another]
Init3 = AT+CGDCONT=1,"IP","internet.vodafone.net"

And then i tried again, and it worked this time, it connects, and fetch the DNS and I browsed to google.com successfully. But for only 120 seconds. Excactly 120 secs every time i connect and then ppd dies unexpectedly, as the error keeps saying. I noticed after some more google results that two options called lcp-echo-interval and lcp-echo-failure and that one of them was set to 30 that's the interval and the other was set to 4 and that was the number of failure and then pppd should die if it didn't receive response. But when i set those options to 0 the connection get stable and didn't disconnect. Those lcp values should be (or i add them) to '/etc/ppp/options'.

###################
# /etc/ppp/options
###################
asyncmap 0
noauth
crtscts
lock
hide-password
modem
proxyarp
lcp-echo-interval 0
lcp-echo-failure 0
noipx
novj

And that's it, now I can launch gnome-ppp and connect, but as a root. I created a gnome-ppp "gksu gnome-ppp" launcher on gnome panel.

And I can launch it from command line by `wvdial`.

The connection speed may not be the useful for real work, but it's faster

Related links:

Web2.0 applications presentation

2008-03-04T00:10:00.001-08:00

I made a simple presentation to enumerate few of the web2.0 application that encourage community content contributions. There are hundreds of community driven web2.0 applications, I just named a few. I selected one applciation per content type.

Blog services: www.blogger.com
Photos: www.flickr.com
Bookmarks: Del.icio.us
Videos: YouTube.com
Podcasts: Odeo.com
Presentations: www.slideshare.net
PDF documents: www.scribd.com

And of course the single most important web application is "Google Reader", which allows you to get the latest updates in one place.
RSS Reader: www.google.com/reader

Two most important features in all the mentioned applications; first they all support RSS feeds so you can subscribe and get the latest updates; Second, you can embed the content you uploaded whither it's photos, links, video, presentations, or even pdf, you can embed all these type of content right into your blog. You only have to copy and paste the code from the content page.

| View | Upload your own

Rails stack on Centos 5 with Nginx and memcached

2008-02-25T06:38:00.000-08:00

This is yet another rails stack installation how to. It is useful for installing a production rails hosting environment using Nginx as the reverse proxy server for a mongrel cluster, using memcached as the caching server and mysql as the database server. Of course I am assuming that all these services will be installed in the same server for the sake of simplicity.

Mysql
Make sure that Mysql is installed correctly and ensure it is running on startup.

sudo yum install mysql-server
sudo yum install mysql
sudo yum install mysql-devel
sudo checkconfig --level 345 mysqld on

Note: In case you run mysql on a dedicated server then you need not to install Mysql in the application servers, but make sure you install at least mysql-devel because it contains header files needed for Mysql gem installation.

Memcached
Memcache server needs livevent package as a dependency, so according to the deployment server system distribution we need to install the correct packages, in case of Centos 5 we are going to select the proper packages from Dag repositories from the following pages;

http://dag.wieers.com/rpm/packages/libevent/
http://dag.wieers.com/rpm/packages/memcached/

The following are the required steps to download and install memcached and libevent for centos 5.

wget http://dag.wieers.com/rpm/packages/memcached/memcached-1.1.13-4.el5.rf.i386.rpm
wget http://dag.wieers.com/rpm/packages/libevent/libevent-1.3b-1.el5.test.i386.rpm
sudo rpm -ivh libevent-1.3b-1.el5.test.i386.rpm
sudo rpm -ivh memcached-1.1.13-4.el5.rf.i386.rpm
sudo /sbin/chkconfig --level 345 memcached on

memcached configuration could be found on /etc/sysconfig/memcached set the CACHESIZE to desired memory cache size you want.

Nginx
Get the latest nginx package tar, unpack it and configure it as follows. We need to install nginx required library first before configuring Nginx though. After compiling we need to copy nginx init script to /etc/init.d/nginx and give it execute permission.

sudo yum install pcre
sudo yum install pcre-devel
sudo yum install zlib
sudo yum install zlib-devel
sudo yum install openssl
sudo yum install openssl-devel
sudo ./configure --sbin-path=/usr/local/sbin --with-http_ssl_module --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_stub_status_module --with-mail --with-mail_ssl_module --with-cc-opt="-I /usr/include/pcre" --pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx --conf-path=/etc/nginx/nginx.conf
make
make install
sudo cp nginx /etc/init.d/nginx
sudo chmod +x /etc/init.d/nginx

Note: you can use this conf file as a base for nginx configuration

Rubygems
If rubygems is installed but needs update run the following

sudo gem update --system

If not installed get the latest version from here, unpack and install.

ruby setup.rb

Rails, Ferret, Rake

sudo gem install rails -y
sudo gem install rake
sudo gem install ferret

Mongrel

sudo gem install mongrel

Mongrel Cluster
Install Mongrel Cluster gem, and copy the service startup script and make sure it starts on system startup.

sudo gem install mongrel_cluster
sudo cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/
sudo cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/default.yml /srv/webapp/config/mongrel_cluster.yml
sudo /sbin/chkconfig --level 345 mongrel_cluster on

You may need or should write your own mongrel_cluster configuration file, to define number of mongrel instances, user, group, port and other options. Here is an example:

---
cwd: /srv/webapp/
port: "8000"
environment: production
address: 127.0.0.1
pid_file: log/mongrel.pid
servers: 8
group: nginx
user: nginx

You may need to add nginx user and group to your server

/usr/sbin/useradd nginx

Mysql Ruby Driver
We have to do a little more than gem install here, cause it seems that Redhat based distros are putting Mysql in a place not expected by the gem installer. So we are going to either explicitly give ‘gem install’ the location of lib headers and include for mysql, or soft linking the /usr/include/mysql and /usr/lib/mysql folders before ‘gem install’. (The soft link solution works well for me in most servers)

sudo gem install mysql -- --with-mysql-include=/usr/include/mysql --with-mysql-lib=/usr/lib/mysql

update: New version of rubygems require double dash before the compiler configuration options.

sudo ln -s /usr/lib/mysql/ /usr/local/lib
sudo ln -s /usr/include/mysql/ /usr/local/include/
sudo gem install mysql

Memcached Ruby Driver

sudo gem install memcache-client -y

Using Google to send mails from Rails

2008-02-25T05:49:00.000-08:00

I was using 'sendmail' for sending mails from my web applications. The apps was sending mails for welcome messages, comment notifications, activation mails, send to a friend and all other features depending on sending emails to the application user. But recently, I was deploying a webapp on a server in which sendmail is not configured correctly. I didn't have time and wouldn't like to invest any time on system administration then, so I searched for another solution for my webapps to send mails through Google. I found ssmtp. It is a simple tool doing a simple task, sending mails. However it does not receive mail, expand aliases or manage a queue. It is just forwarding mails to the mailhost. SSMTP requires replacing 'sendmail'.

There is nothing required from Rails app to use ssmtp, actually it was transparent. Just make sure that the ActionMailer is configured to use sendmail.

ActionMailer::Base.delivery_method = :sendmail

SSMTP Installation
In Ubuntu I use apt-get to install the package. I am enabling all the sources on my ubuntu, so you may need to do so. In RPM based distribution like Centos in my case i used this link to download the required RPM.

on Centos

sudo wget ftp://mirror.switch.ch/mirror/epel/5/i386/ssmtp-2.61-11.4.el5.i386.rpm
sudo rpm -ivh ssmtp-2.61-11.4.el5.i386.rpm

on Ubuntu.

sudo apt-get install ssmtp

SSMTP Configuration
SSMTP configuration files lie in /etc/ssmtp so you need to edit the conf file and fill it with the your gmail account.

mailhub=smtp.gmail.com:587
AuthUser=my_gmail_account@gmail.com
AuthPass=my_gmail_password
UseSTARTTLS=YES

There is one feature you will get by using Google as your SMTP server, it's its sent mail folder which archive every mail sent by your application, this feature being a bless or damn is up to your needs.

Related links:

RPM package for ssmtp from RPM pbone.net
Igvita: ssmtp relay mail delivery in rails

Vim: Find and replace text across files

2008-01-06T10:29:00.000-08:00

Programmers almost always need the ability to find and replace certain text across multiple files in their projects. Most IDEs provide find and replace functionality. And there are many free tools online to just find and replace text across multiple files. Because I am using Vim as my favorite editor for over 15 years now, it came natural to choose Vim for such job, and not depend on another tool.

For a real life example, in one Rails project, I used caching in my view templates. I used a plugin to set an expiration time for the cache. I needed the cache function to look like:

<% cache("sidebox_latest_news", :expire => 15.minutes.from_now) do %>
...
<% end %>

I used this technique in about 25 of my templates. Then i decided not to use timed expiration of the cache. So I needed to remove the expire argument of the cache method. I needed to remove it from all of my 25 templates. I used Vim to find the needed piece of text to be removed.

In Vim, when we need to replace text in a file, we use the substitute command.

:s/old_text/new_text/

This should replace the first occurrence of "old_text" with "new_text" for the current cursor line. To substitute all matching occurrences in the line, just add the flag 'g' which refers to global. And to substitute all the file occurrences of the pattern, add the '%' before the substitute command.

:%s/old_text/new_text/g

Vim alerts if it didn't find the pattern, to disable the alert just append the flag 'e'.

:%s/old_text/new_text/ge

For a complete documentation for the substitute Vim command, use vim help.

:help substitute

Now, after we knew how to substitute text in Vim, we need to substitute the expire argument. we cannot do without using regular expressions. That's because the argument is not the same in all the files, it carries different time values, and may differ in spacing.

<% cache("sidebox_related_reports", :expire => 1.hour.from_now) do %>
...
<% end %>

So we need to match the whole argument from the comma at the start till the right parentheses and replace it with empty text. the regular expression we should use to match the argument is;

/, *:expire.+)/

I will quickly explain the pattern above, but please refer to any of the many tutorials online about Regular expression for more details. The pattern starts with a comma which is the start of the text we need to remove. Followed by a space and a '*' which means a comma followed by zero or more spaces then ':expire' followed by "one or more" "character". And finally the pattern ended with a right parentheses. So our pattern starts with a comma and ends with a right parentheses. Any text matching this pattern in all templates files should be replaced with empty text, or to be specific, with a right parentheses, as the parentheses is already part of the text to be removed.

To apply Vim command to all the templates files, I will use the 'args' command. I will pass all the templates files in the app/views folder to 'args' using wildcards, and because Rails store the templates files grouped in folders named after their controllers I will use double wildcard 'app/views/*/*' to list the files 2 levels deep under app/views folder.

After I passed the required files to 'args', I can apply whatever command I like to all these files using the command 'argdo'. First I will apply the substitute 's' command and then 'update' which will only save the modified files.

:args app/views/*/*
:argdo %s/, :expire.*)/)/ge | update

Finally, another neat option is adding the flag 'c' which will ask confirmation before each substitution.

:argdo %s/, :expire.*)/)/gec | update

What makes Ruby better than Java and PHP?

2007-09-02T03:40:00.000-07:00

Ruby by itself is not better than Java or PHP. Ruby is just a general purpose computer language. Each Language has its advantages and disadvantages, that's why it is according to the project you will undertake you should choose the better language that suits your needs.

For myself, I use Ruby in Web development. Cause the web development is very dynamic, every day you have new requirements that's why I prefer to use a language like Ruby that's make me develop more with simple code, and most important I like Ruby's elegantly clever syntax.

But again without a great framework like RubyOnRails I wouldn't use Ruby. Ruby on rails gives you everything you need from the begining, so you do not have to prepare all your tools just to start. I mean when you start a rails project you will have a Object Relational Mapping, Unit testing system, URL Mapping, MVC structure, templating system, caching, Ajax, webservices, emailer, tasks runner, command prompt console, and tens of other useful stuff that you otherwise spend a considerable amount of your project time trying to setup if you don't use a decent framework.

That said, there are decent frameworks in Java and PHP that can do the same as Rails, like PHP CackePHP, and Grails for java. But It's Rails who get all the hype because it is the one who really made it eaiser for developer to make professional websites.

Subversion, SSH and Capistrano for a fully automated deployment

2007-07-05T00:22:00.000-07:00

Deployment could be defined as; uploading an application to the hosting server and make any required modification or configuration to the file system, web server, database server, or whatever for the application to run properly.

For deployment, I use Capistrano, Subversion and Putty. Notice that I did not mention an FTP client. My minimal typical hosting environment consists of at least one instance of production web server and another instance as a staging web server, and each of them has its own file structure spanning to more than 3 levels and all the web servers are behind a reverse proxy server as the front end for web request, and I may run clusters of these instances. It could take me more than 10 minutes to do but the simplest tasks of deployment, like updating 3 files in 2 different folders, for example. As simple as it seems, but that kind of task is troublesome, especially that after uploading those files in their respective paths, and preview the updates on the staging server, and iterating a little bit in a preview, comments, fix, deploy loop, and finally redeploy in the production server, and restart the cluster instances. Well, at best, if half of your brain is actually a computer machine and you didn't make any silly overwrite or other nasty stuff to your application, your other brain half will get bored of these brain dead repetitive tasks, or you will simply screw yourself bad when you break your production hosting environment or worse destroy your production database.

We have to admit it; a programmer should always train himself to automate everything repetitive and boring he does, in order to spare his brain power on otherwise useful stuff that needs his attention like application architecture design and algorithms. Or simply to spare the world the kind of destruction our minds able to do when faced with boring repetitive tasks. This is a very popular concept which is called DRY, abbreviation of Don't Repeat Yourself.

Subversion
I believe that the centerpiece of any successful and decently automated deployment lies in the Version Control tool you use. Coupled with a decent SSH client and you're up to the deployment task. I use Subversion. It really makes my life easier. Imagine the following scenario;
• Write your application
• Import it to your online code repository.
• Continuously commit any changes you do in your application to your code repository.
• Need to deploy a development server; just check out a version of your application from the repository
• Changed anything in your application; just update your development server with any recent changes, and it will be in sync with your latest application code.
• Satisfied with your current development version, make a release tag in your code repository, and check out this version to a new production server.
• Need fixing, change the code, commit it, update your hosting server.
• Did a bad decision and decided to go back to a previous version; just update your application to the required version.

It is even simpler than it sounds. All these steps are done using simple commands like;
On local system:
$ mkdir myapp
$ cd myapp
$ svn import http://svn.myrepository.com/myapp
$ echo "puts 'hello'" > hello.rb
$ svn add hello.rb
$ svn commit hello.rb -m "Added Hello module that says hello"
On hosting server using secure shell:
$ svn checkout hello
$ svn update
$ svn update –r $required_revision_number

The beauty of this process of check in, check out, commit and update, is that you do not have to keep track which files are changed and need to be uploaded, or worse upload all your application folder each time you make an update.

Useful link: A Novices Tutorial on Subversion

Capistrano
Capistrano is the deployment tool i use for deploying my Rails applications on the hosting server. As quoted from Capistrano manual; Application deployment is one of those things that becomes more and more complicated as the scale of your application increases. With just a single box running your database and your application, it’s quite simple. But when you start putting your database on a different server, and then separating your web servers from your app servers, and eventually splitting your database into master and slave servers… It can get to where you almost don’t want to deploy your application any more.

Simply put, Capistrano makes it even easier to do tasks like fire up your ssh client, type some commands on the remote server to move, update, restart, or whatever task you would normally do by hand, you can write your commands in a script called "recipe" for Capistrano and as the next decent software, it will submit to your command and do every task you tell it to do.

Tasks like; login to remote server, update your application code, migrate your database, restart your server , are very likely will be repeated over and over, and that should notify you to start a new recipe for Capistrano to cook at your command.

Useful link: Capistrano: Automating Application Deployment

Putty
I like Putty, it is a secure shell client. Nothing bad with openssh client, but when i use Windows, it's pathetic command prompt console leave no choice but to use Putty with its own terminal window. By the way, I wonder why for God sake, there is no any alternatives for the bloody windows command prompt console, is there any out there? And I mean terminal or console not just another shell, cause i found many (Cygwin, bash, sh). [update: I found a great console for Windows, it is called Console. It is a great tool, it even support transparent background! when used with unix tools like sh, tar, cp, rm...etc and all you unix geeks will find yourself almost at home.]

Among the servers of the hosting data center we have at work, there is only one server that open a Secure Shell port to the internet. For a hosting data center, this is not a good idea. But we have to have a remote shell to login to, in case of emergencies or remote administration. My system admin collegue at work, had generated a pair key for my account on the hosting server. And that was a good idea. Because, it requires a private key for your account on the linux box for you to be able to login. No username/password login prompt anymore. This technique is both secure and easier than the regular login procedure.

Using key authorization is very useful if you intend to use any kind of automated deployment tools. Because such tools will almost always use key authenticated secure shell for login to the deployment server. And scattering your username/password all over your deployment scripts is not something you should do if you care about your hosting server security.

My first post

2007-02-17T10:14:00.000-08:00

Here is my first post. I hesitated many times before I started my blog, but it seems that you cannot avoid it, cause it will happen sooner or later. I worked as a web developer for so many years now. And I believe in specializing, so I chose to write about technologies used in web development as my specialty. But that will not prevent some posts about my personal life, work, habits, or whatever, to find its way into this blog. So, see you at my next post...