Luckily Dradis has a nice plugin system which is designed to ease the process of importing and exporting data from Dradis, so this isn't too tricky.

Creating the Plugin

As dradis has rails generators for import plugins, we can use that to create the basic structure. First off, obviously, we need a working Dradis installation to work from. There are instructions on the site for the latest svn version here and following those should give you a working version of the latest code.

Once that's done we can enter the dradis server directory an use this command to create the our import plugin.

rails generate import_plugin simple_vulndb

This creates a directory simple_vulndb_import under vendor/plugins and also creates a number of files for us to modify.

Configuring the Plugin

Here we'll just step through the bits that are necessary to get the plugin up and working. there's a number of files that we need to modify to get everything working ok. Most of this is just a modified version of the default vulndb_import plugin which is provided as part of Dradis.

First up is the configuration file in the plugin config directory.

Dradis uses YML config files which is a pretty easy syntax which is parameter : value

Here we can define the hostname, port and path for Dradis to access our vulnerability database. This also provides you the flexibility to change it (for instance if you've got a centralised version of the database as opposed to one hosted locally). The settings below are based on what we configured for the vulnerability database in the last post.

host: localhost
port: 3003
path: /vuln_search.json

with that done we can move on. Next up is the meta.rb file which can be found in lib/simple_vulndb_import/ . Here we just define the name of the plugin and the version information. So for example

  NAME = "Simple Vulnerability Database Import"
  # change this to the appropriate version
  module VERSION #:nodoc:
    MAJOR = 0
    MINOR = 1
    TINY = 0
     STRING = [MAJOR, MINOR, TINY].join('.')

would work fine. Next up the main piece we need to change, the filters.rb file. This is found in the same directory as the meta.rb file.

Creating the Filters

There's two main pieces to how I've set this up. The first is the filters. Essentially if we configure one of these for each of the search_types that we defined in the database (description, OWASP reference, Severity and Test Type) then we'll be able to search by those methods from within Dradis).

Dradis handles filters by creating a module within the filters module that you'll see pre-defined in the filters.rb template.

So for each of our searches we need to create a new module which looks a bit like this.

module TestTypeSearch
NAME = 'Search Database by Test Types'
  def self.run(params={})
    result = Filters::get_records('test_type',params['query'])
    records = Filters::prepare_results(result.body)
    return records
  end
end

what we're doing here is essentially setting up a NAME constant which contains (rather unsurprisingly) the filter name. then defining the behaviour when the filter is run. this is rather short as we're just calling two class methods and then returning the result.

When I was writing this file I realised that I was essentially just writing variations of the same logic four times, so in good ruby practice I tried to DRY up the code and moved most of the logic into the class methods get_records and prepare_results

get_records looks like this

def self.get_records(search_type,query)
  require 'cgi'
  conf_file = File.join(Rails.root, 'config', 'rvulndb_import.yml')
  conf = YAML::load( File.read CONF_FILE )
  http = Net::HTTP.new(conf['host'], conf['port'])      
  res = http.get(conf['path'] + '?search_type=' + search_type +'&query=' + CGI::escape(query))
end

So this method opens the configuration file that we defined earlier (you'll notice that it looks in the config directory under the rails root, so it's a good idea to put a copy in there). Once it's opened that it uses rubys' YAML class to read the file, sets up an http connection to the database mentioned in the config file and executes the query on the database. One thing to note here is the use of CGI::escape. This helps manage any use of characters that aren't allowed in URLs in our query string.

Ok, so after that method has completed we should have an array of 0 or more records that we can setup to be returned into dradis.

Next method up preps the records for input into Dradis

def self.prepare_results(json_data)
  recs = []
  jrec = ActiveSupport::JSON::decode(json_data)
  if jrec.length == 0
    error = Hash.new
    error['title'] = "No records found"
    error['description'] = "The search didn't return any records!"
    recs << error
    return recs
  end
  
  jrec.each do |jr|
    newrec = Hash.new
    newrec['title'] = jr['vulnerability']['title']
    newrec['description'] = Filters::build_description(jr['vulnerability'])
    recs << newrec
  end
  return recs
end

So this code just loads up the JSON data that our query should have returned, checks to make sure that we got some records (and returns an error if we didn't) then creates a hash for each record. There's one more bit of logic to explain in here which is the call to Filters::build_description. For neatness sake I broke that bit out. At the moment it's a pretty ugly text creation, but does the job :)

def self.build_description(note_data)
      <<-eos
Vulnerbility Title
------------------
#{note_data['title']}      
      
Vulnerability Description
------------------------
#{note_data['description']}
        
        
Vulnerability Remediation
-------------------------
#{note_data['remediation']}
        
        
Technical Notes
--------------
#{note_data['technical_notes']}
      eos
end

This just puts together the body of the note description for each finding, as one long string.

There's obviously a lot more that could be done with this (like better error handling and writing tests) but with those files complete, the module should work ok and you should be able to import vulns from your database directly into Dradis using the "import note" feature.

I've put a copy of the code for the plugin up here, in case it's helpful :)

One of the features available in Dradis is import plugins. This lets you create a link to an external information source, such as a OSVDB or a database of vulnerabilities.

Having a database of vulnerabilities or findings can be pretty useful in cutting down the time required for reporting on a test as you can keep standard wordings in place (who really wants to write the same section about preventing XSS more than once!).

So recently I knocked up a simple vulnerability database to link in to Dradis and I thought it might be of use, so here's the process.

Creating the App

We're going to use Ruby on Rails for this as it's nice and easy to develop for (as you'll see) and also that's what Dradis is based on, so makes sense to keep all the coding in the same underlying language. Also rails apps are very portable, they're basically contained within a single directory structure, so it's relatively easy to move them from place to place.

Before starting the application, there's the usual pre-req's. I'm using Ruby 1.9.2 and Rails 3 so having those installed is a good thing. If you're using Linux then it's helpful to have RVM working as some distros don't have ruby 1.9.2 packaged up as yet.

once you've got the pre-req's working, we can start by creating a rails app

rails new vulnlist

This creates a new application called vulnlist and adds all the standard rails files in.

Creating the Scaffold

Once we've got the app created, we can use rails scaffolding to quickly create the basic structure for our app. The web pages that scaffolding creates aren't the most pretty, but they'll do for now.

With the scaffold we can specify what fields we want to create in the database and also what data types those fields are.

rails generate scaffold Vulnerability title:string test_type:string description:text remediation:text technical_notes:text severity:string owasp_reference:string

Once we've completed this we can look at the basic app by setting up the database with

rake db:migrate

Ensuring that all our gems are installed ok with

bundle install

and launching the app

rails server

At this point browsing to http://127.0.0.1:3000/vulnerabilities should show a blank page with our fields in it. From this page we can create new vulnerabilities and edit or delete existing ones.

Now that we've got this basic structure setup it's worth using git to keep a handle on the source code. On Linux the procedure for this is pretty easy

If you've not already got it installed

sudo apt-get install git-core

then in the root of the application

git init
git add .
git commit -m "Initial Commit with Scaffold"

Having git running on the app will make it pretty easy to revert any mistakes made along the way, as long as we've done regular commits.

Setting up the Searches

So far we've got a basic structure in place and can do the basic Create, Read, Update, Delete cycle on our data. However for the dradis integration, what'd be useful is if we could search for vulnerabilities using various criteria and have the results returned to Dradis.

This turns out to be relatively straightforward. First what we need is a new action in our controller. Opening vulnlist/app/controllers/vulnerabilities_controller.rb we can see the existing actions that we've got for the application.

What we need to do now is add a new action to allow for vulnerability searches.

def vuln_search
    case params[:search_type]
    when "description"
      @vulnerabilities = Vulnerability.where("description like ?", "%"+params[:query]+"%")
    when "owasp"
      @vulnerabilities = Vulnerability.where("owasp_reference like ?",params[:query]+"%") 
    when "severity"
      @vulnerabilities = Vulnerability.find_all_by_severity(params[:query])
    when "test_type"
      @vulnerabilities = Vulnerability.find_all_by_test_type(params[:query])
    end
    respond_to do |format|
      format.xml {render :xml => @vulnerabilities}
      format.json {render :json => @vulnerabilities}
    end
  end

This defines a new method called "vuln_search" which takes two parameters, search_type and query. The search type parameter lets us pick from different finders. Rails provides access into the application database via ActiveRecord and this just uses a couple of the finder types for different parameters. Where the query is going to be one of a number of fixed values like "severity" which will be something like high, medium or low, we can just use a standard find_all_by_ approach, but where it's a more free text style search, we use Vulnerability.where and pass in the query parameter that way.

The respond_to block is a really nice feature of rails. By adding in the two lines for :xml and :json rails wires up responses so that we can get the data out in those format, no additional code required.

Now that we've got the basic code in place, we just need to modify the rails routes so that the application knows how to access our new method.

This is done by modifying the vulnlist/config/routes.rb file, and adding the following code

controller :vulnerabilities do
  get 'vuln_search' => :vuln_search
end

At this point, we've got the application basically working. If you put in a couple of test findings, then you should be able to go to http://127.0.0.1:3000/vuln_search.xml?search_type=Severity&Query=High for example and get some XML data back.

Tidying up

Now that we've got the basics working, there's a couple of additional steps that it's worth looking at to tidy some things up.

Selectors

First off, we'd like some of our fields (OWASP Reference, Severity and test type) to be one of a number of defined values. The "proper" way to do this would be to create additional models for these and link them in to the main vulnerabilities controller, but there's a quicker way which is probably going to work well enough for our purposes.

Opening up vulnlist/app/models/vulnerability.rb we can specify some Constant values for these settings

TEST_TYPES = ["Web Application","Windows Server","Unix Server","Wireless","Web Server","Oracle","MS SQL","MySQL","DB2"]
SEVERITY_LEVELS = ["Critical","High","Medium","Low","No Impact"]
OWASP_TOP_10 = ["A1 - Injection","A2 - Cross Site Scripting (XSS)","A3 - Broken Authentication and Session Management","A4 - Insecure Direct Object Reference","A5 - Cross-Site Request Forgery (CSRF)","A6 - Security Misconfiguration","A7 - Insecure Cryptographic Storage","A8 - Failure to Restrict URL Access","A9 - Insufficient Transport Layer Protection","A10 - Unvalidated Redirects and Forwards"]

Then we can modify the form that the scaffolding process created to use these arrays as a select list. The form is found in vulnlist/app/views/vulnerabilities/_form.html.erb. In that file we just need to replace the "text_field" lines for those three fields with the following select lines

<%= f.select :test_type, Vulnerability::TEST_TYPES, :prompt => "Select the test type" %>
<%= f.select :severity, Vulnerability::SEVERITY_LEVELS, :prompt => "Select the severity level" %>
<%= f.select :owasp_reference, Vulnerability::OWASP_TOP_10, :prompt => "Select the appropriate OWASP top 10 reference" %>

This picks up the Constants from our model and helps keep the data consistent.

Localhost Only

As you'll have noticed with this application, there's pretty much no security whatsoever. At the moment it's setup as a personal database only and isn't suitable to be on any kind of network. Adding that security isn't too difficult with rails, however it's not really a problem for the basic use case that we have here. Both the vulnerability list and the dradis installation only need to listen on the localhost.

Configuring rails to only listen on the localhost (as opposed to specifying it on the command line) is a bit hacky, but here's a way to do it based on this post and this dradis change . We need to modify the vulnlist/script/rails file and add the following lines

require 'rubygems'
require 'rails/commands/server'
require 'rack'
require 'webrick'

module Rails

  class Server < ::Rack::Server

    def default_options

      super.merge({

        :Port => 3003,

        :Host => "127.0.0.1",

        :environment => (ENV['RAILS_ENV'] || "development").dup,

        :daemonize => false,

        :debugger => false,

        :pid => File.expand_path("tmp/pids/server.pid"),

        :config => File.expand_path("config.ru")

      })

    end

  end

end

This also moves the application off the default port of 3000, to a new one of 3003 which hopefully shouldn't clash with other services.

Default Routes

At the moment if we visit the root page of our application, now at http://127.0.0.1:3003 we get the default rails welcome page. What'd be nicer is if we were re-directed to the vulnerability listing automatically.

That's easily done with two steps. First edit the vulnlist/config/routes.rb file and add the line

  root :to => "vulnerabilities#index"

then delete the file vulnlist/public/index.html file.

Summary

So at the end of this first part we've created a basic vulnerability database which we can search easily on a number of parameters.

The next step is to create the dradis plugin to hook the two together, which as I'll cover next time is a reasonably easy thing to do.

Web Application Security

For people looking to understand how to secure their web applications, in my opinion the best source of free information is the OWASP project . It's an umbrella for a number of web application security efforts.

They've got a wide variety of projects which target management, security testers and developers, but some of the best places to start for developers are the Owasp developers Guide which covers a wide range of topics about secure development in a fairly generic way, and there's also the Ruby on Rails Security Guide here or here which covers rails specific topics.

Both of those documents are relatively large but well worth delving into. For a quick few of the "most serious" web application security issues, OWASP also has the Top Ten project. The latest version, current in RC1 has some good summaries of the top risks, and is a good place to start getting an idea of the areas to be addressed...

There's also a newish book dedicated to the topic of security on rails. I picked up a copy at the conference (I'm a sucker for the O'Reilly stand!), so I've not had time to get all the way through it, but first impressions are that it'll be very good, with concrete examples of broken rails applications and how to fix them.

I was thinking of a way to sum up burp, so far the best I've got is "If you're doing web application testing and not using Burp, you're missing out !", it is that useful...

Burp has been around for a while now as one of a group of handy web application proxy tools along with the likes of Web Scarab and Paros. Over time it's developed a huge range of handy features, which make web app. testing a lot more productive.

There's the basics, in terms of intercepting and modifying requests, which all the tools in this category tend to do quite well, but on top of that there's just loads of other things that Burp does.

The Web app scanner is a good targetable way to test for several types of common web app vulnerabilities. As an add-on in the latest beta, there's a wizard which lets you reduce the number of requests that Burps going to make, by removing duplicates and similar requests. If you've used many commercial web app. scanners you'll know that volume of requests generated can be a real problem, so anything that can help bring down the load is useful.

Another stand-out feature of Burp for me, is intruder which lets you pick a specific parameter or set of parameters from a request and easily modify them. As an example pretty much every time I see a URL that looks like request.asp?id=123 , it's going to be worth running burp Intruder over that parameter to see what pops up when you change that value.

One of the things I have found about burp is that it's easy to miss functions that are available and not get the best out of it. Luckily there's a blog which covers a lot of the new features as they come out. Also I need to mention the Burp Tip of the Day series of posts on Cktricky's blog which has loads of good tips on getting more out of Burp.