The Cameron & Wilding Blog

Drupal Process Event Jan 27 - 29

Ben — Mon, 30 Jan 2012 11:52:27 +0000

I was fortunate enough to be one of the 60 attendees from 40 of Europe's key Drupal companies to descend on the Microsoft offices in Amsterdam. A long but fruitful weekend was spent discussing process, project management, marketing, sales, QA, testing and much more. Heading out on Friday I was exited by the prospect of working with some of the biggest and best Drupal companies in the world and I wasn't disappointed. The level of expertise and the open sharing of lessons learned, current issues and future plans was truly wonderful. I had hoped to come back with a few improvements we could implement to continue our constant desire for quality and best practice. I came back with that and much more.

With organisations from 2 to 55 people it allowed for everyone to be able to help with support and advice on matters central to companies at different growth stages. My favorite talk was around knowledge management and knowledge sharing so it seemed apt to do a short write up noting some of our practices. Within Cameron and Wilding we use a number of methods to allow the knowledge we are building up to be shared around the organisation. These include:
Our intranet and wiki where everything from project documents and write ups of lessons learned on challenging technical issues to links to interesting articles are stored. This is very much self managed by the whole team and each individual takes the time to update anything they feel may be valuable for other team members in the future.
Every Friday we carry out a 'show and tell' session at the end of the day with the whole team. This may be anything from tricks and tips with Google mail to Apache solr search within a project we have recently worked on. Following a short presentation it then naturally moves into Q and A and open discussion.
Code reviews do not just act as a quality control method but also help the team to share and develop their skills. In addition it means we also have multiple teams members that know each individual project and can work on them when required.
Project debriefs which we carry out after completion of all major projects. These allow us to ensure we are constantly improving from project to project and creating the best possible results. And finally for any external training providers we have come in, most recently for html5 and automated training processes, we carry out write ups, post training reviews of the training quality and practice the newly learned skills as soon as possible.
During the session Jan Depping from Microsoft told us a little about their knowledge management and knowledge sharing. Their knowledge sharing system monitors tweets, email discussions and other methods of communication to record key topics, projects and people. This then allows their team to not only search a database of knowledge developed and stored for that specific purpose but to carry out searches for information that traditionally would have been trapped in silos of email, twitter and other accounts.
It is truly wonderful that the level of community support within the developer community also transfers to business. The combined skills and knowledge of the community really shone through and confirmed what we already knew - Drupal is truly great.

tags:

Synchronising drupal users information with civicrm

José — Mon, 09 Jan 2012 15:34:02 +0000

Recently we have been working in a civicrm project and we thought it could be very useful to develop a module which helps us to reflect automatically any change made in an user account, in his related civicrm contact.

Civicrm offers an user's synchronisation functionality out of the box but it only checks drupal users without existing related civicrm contact and it creates the respective civicrm contacts. But we wanted something else: if for example we change the email address in an user account, the email address will have to change inmediately in the civicrm contact related to that drupal user.

For this purpose we need to use hook_user and hook_nodeapi because we don't wish to control changes in the basic account information, we want to control changes in some profile fields. . In our example we will show code inside a hook_user and we will do active use of the v3 civicrm API

The first step is to initialize civicrm to include the api:

<?php
if ( ! civicrm_initialize( ) ) {
            return;
}
global $civicrm_root;
require_once $civicrm_root . '/api/api.php';
?>

The second step is check if the drupal user (whose information we are trying to change) has a related civicrm contact, we use the civicrm api in relation with the UFMatch table, which stores the correspondence between drupal users uids and their related civicrm contacts ids:

<?php
$uf_params = array(
        'uf_id' => $account->uid,
        'version' => 3,
        'sequential' => 1,
        );
$UFMatch = civicrm_api( 'UFMatch','Get',$uf_params );
$contact_id = $UFMatch['values'][0]['contact_id'];
if($contact_id != NULL){ ...
?>

The way to use civicrm api calling to the civicrm_api function where the first argument is the table we can use for our query, the second argument is the kind of operation we wish (Get, Create, Delete...) and the third argument is an array containing information about the query (in our case, the user uid whose related civi contact id we want to get, the version of the api we are using and a flag to indicate we have the results in sequential format). More useful information about using the civicrm api can be found here: http://wiki.civicrm.org/confluence/display/CRMDOC40/CiviCRM+Public+APIs

After calling the civicrm api, we will have the results in the variable $UFMatch which structure will be:

<?php
array( ["values"] => array( [0] => array_with_row_1_info
                                       [1] => array_with_row_2_info...
                                    )
        )
?>

In our case we will have only one row returned so we can find the civi contact id related to the drupal user in $UFMatch['values'][0]['contact_id']. If the drupal user doesn't exist $UFMatch['values'][0]['contact_id'] will be NULL.

The third step, if the drupal user has a related contact in civicrm, is compare the account info which changes we want to control. We are going to use for example, the user's email.

So first we see what's the email value in the related civicrm contact. For that we use again the civicrm api to make a query against the contacts table:

<?php
$contact_params = array(
'contact_id' => $contact_id,
'version' => 3,
'sequential' => 1,
);
$contact = civicrm_api( 'contact','Get',$contact_params );
$civi_email = $contact['values'][0]['email'];
?>

And after that we compare it with the drupal user's email to check for any possible change:

<?php
$drupal_email = $account -> mail;
if ($civi_email != $drupal_email) {...
?>

If they are different then this information was updated in the user's account info so it will have to be changed in his related contact info, using again the civicrm api:

<?php
$update_params = array(
'contact_id' => $contact_id,
'email' => $drupal_email,
'version' => 3,
);
civicrm_api('contact', 'create', $update_params);
civicrm_api('UFMatch', 'create', $update_params);
?>

In civicrm api v3 some tables don't have defined the "update" operation, in that case we can use the operation "create" so if we provide a row id in our query array and that id exists, then the information will be updated for the row with that id.

If we check our query array ($update_params), we can see we are saying to the drupal api that we want to change the email of the civi contact with id $contact_id to the email of the related drupal user ($drupal_email).

As you can see, civicrm api is a very powerful tool, very easy to use inside any of your drupal modules to cross information between the cms and the crm.

tags:

civicrm

Bulk import tickets into CodeBaseHQ from CSV file

Neil — Sun, 20 Nov 2011 21:20:41 +0000

Here at C&W we've used CodeBaseHQ since day one for task tracking, time tracking, bug tracking and hosted version control. It's a great product, they've recently released their v4 version of the app which is even nicer to use.

During project kickoff we will typically create a specification in a spreadsheet which details each feature as a user story or task description. As a result, there is typically a very boring task of copying and pasting the tasks into CBHQ so we can track them.

My weekend project this weekend was to use the CBHQ API to import the tickets from a CSV file. The results of this are in the zip at the bottom of the post. The whole thing works really well. I have some plans to create something a bit more interactive allowing you to select the project from a drop down (populated form the API). However, if you stick with the CSV format, and don't mind editing the php script, this will do the job nicely.

The script works by converting the CSV into an multi-dimensional array and then this is converted into a series of XML objects which are posted to the API URL.

PHP array

<?php 
    [0]=>
      array(7) {
        ["summary"]=>
        string(14) "my test task 1"
        ["status-id"]=>
        string(1) "1"
        ["reporter-id"]=>
        string(4) "123"
        ["assignee-id"]=>
        string(5) "123"
        ["priority-id"]=>
        string(1) "1"
        ["ticket-type"]=>
        string(4) "task"
        ["milestone-id"]=>
        string(5) "24342"
            ["estimated-time"]=>
        string(2) "33"
      }    
?>

XML

<?php 
<ticket>
  <summary>my test task 2</summary>
  <status-id>1</status-id>
  <reporter-id>123</reporter-id>
  <assignee-id>123</assignee-id>
  <priority-id>1</priority-id>
  <ticket-type>Task</ticket-type>
  <milestone-id>24342</milestone-id>
  <estimated-time>163</estimated-time>
</ticket>
?>

It's worth noting CBHQ have a rate limit of 80 posts per hour and each ticket needs to be done as a seperate post, so if you have hundreds of tickets to import you might want to contact CBHQ to up your limit.

Attachment	Size
cbhq_importer.zip	21.46 KB

tags:

codebase

XML

Getting started with custom profiles, distributions and make files on Drupal 7

Ed — Thu, 13 Oct 2011 15:41:59 +0000

I've spent some time recently building a Cameron & Wilding Drupal 7 installation profile. Whenever we set up a new site from scratch, be it for testing or development on a new project, we always end up installing a host of useful modules: Devel, Views, CTools and Features, to name just a few of them. I was inspired by Antonio and Andrea's great presentation at DrupalCon London last month. They go into detail about maintaining a Drupal distribution with the Features module. This helps you keep all your configuration in code and means you can do really quick, clean deployments.

I'll let you look at the DrupalCon presentation to discover the full potential that this method holds, because in this post I'm going to look at creating a basic install profile with a Drush make file, a git repository and a minimum of coding. I've also borrowed from the documentation for Drupal 7 Install Profiles.

One of the great things about Drush makefiles is that it's possible to put together a complicated module or profile with just a single Drush command. This has been especially useful in the office because I can share the C&W installation profile with the rest of the team just by emailing them a single make file which they run to install Drupal. The installation profile itself is still a work in progress and we're deciding exactly which modules to include in it. It will no doubt evolve over time, so for the purposes of this post we'll use an example profile (that I've put in a Git repository on github) and I'll explain how to customise it for yourself. Wondering why you should use Git?

The new installation profile will provide an additional installation option when you set up your Drupal 7 site. When we've finished, you should be able to choose from a list of options like this:

So, here's the contents of the makefile:

core = 7.x

api = 2

projects[drupal][version] = "7.8"

projects[candw][type] = "profile"

projects[candw][download][type] = "git"

projects[candw][download][url] = "https://github.com/edwardcrompton/candw.git"

projects[candw][download][branch] = "master"

Yep, it's just 7 lines. In fact this is going to be the first of two makefiles that we'll use, but this is the one that we'll run with drush to build the site. Essentially, the make file contains two things: The version of Drupal core that we're going to use and a reference to the github repository where I've put my installation profile files. You can find more details on the structure of a Drush makefile here.

You should be able to run this makefile with Drush. To do so, you'll need to install Drush (but you've already seen the light and started using Drush, right?) and Drush Make. There are various tutorials for installing both these things, so I won't go into that here. It's worth noting though that although the current version of Drush Make at the time of writing is 6.x-2.3, it also works with Drupal 7.

I'm going to assume in this example that you're doing your Drupal installation on some kind of Linux server. I'm using Ubuntu 10.04 here.

To run the makefile, save the example above as candw-stub.make. We'll put our Drupal installation in a folder called candwexample in our web document root (on Ubuntu this is at /var/www). Change director to your web root and then run this command:

drush make candw-stub.make candwexample

You should see something like this:

Project information for drupal retrieved.

drupal downloaded from http://ftp.drupal.org/files/projects/drupal-7.8.tar.gz.

candw cloned from git@github.com:edwardcrompton/candw.git.

Found makefile: candw.make

Project information for views retrieved.

Project information for ctools retrieved.

views downloaded from http://ftp.drupal.org/files/projects/views-7.x-3.0-rc1.tar.gz.

ctools downloaded from http://ftp.drupal.org/files/projects/ctools-7.x-1.0-rc1.tar.gz.

high_contrast cloned from http://git.drupal.org/sandbox/acontia/1281604.git

This will download everything you need for your new Drupal 7 site. You'll notice that we've downloaded Views, CTools and another module called high_contrast. This is a module my colleague Pablo built the other day and I thought I'd use it as a good example of how a custom module can be fetched from a Git repository. The module is in a sandbox on drupal.org, but could just as easily be in a repository on github or elsewhere.

Let's take a look at the structure of the Drupal filesystem that we've just set up. The installation profiles in Drupal 7 are contained in the profiles folder and each one contains any modules that it requires. If you look in the profiles directory inside the candwexample module you'll notice that the Views and CTools modules are downloaded to the path candwexample/profiles/candw/modules/contrib. If you poke about a bit further you will discover that our custom high_contract module has also been downloaded to candwexample/profiles/candw/modules/custom/high_contrast.

But how did Drush know to download these extra modules? Views and CTools are not part of Drupal 7 core and we didn't mention them in our makefile. Neither did we make any reference to the high_contrast module. Well, look back at one of the lines that drush outputted when you ran the makefile:

Found makefile: candw.make

Drush has cloned the profile that we pointed it to on github and it has found another make file in there, which it has also run. Let's have a closer look at the files in the git repository. Drush will keep running makefiles recursively for any of the stuff that it downloads or clones from a repository as a result of the first make file we ran from the command line. In this example we will only come across two makefiles: The first 'stub' (candw-stub.make) that we just ran and the second (candw.make) that is in our custom profile. The stub makefile is kept as simple as possible and it does not change very often. Because it does not change very often we don't need to keep it under version control.

The make file in the profile is part of our git repository and it will change whenever we decide to add or remove a module from our custom Drupal profile or we want to change the version of a module we're using. In this example the profile is in a repository on github. Github is free to sign up for and I'd recommend that you set up an account and put your own custom installation profile in a repository there. As long as you change your makefile to include the path to your repository Drush should always be able to find your installation profile wherever you are installing Drupal.

This is what the makefile in our profile looks like:

core = 7.x

api = 2

; Contrib

projects[views][subdir] = "contrib"

projects[views][version] = "3.0-rc1"

projects[ctools][subdir] = "contrib"

projects[ctools][version] = "1.0-rc1"

; Custom

projects[high_contrast][type] = "module"

projects[high_contrast][download][type] = "git"

projects[high_contrast][download][url] = "http://git.drupal.org/sandbox/acontia/1281604.git"

projects[high_contrast][subdir] = "custom"

In this makefile we're providing details of the three modules we want to download so that they can be used in our profile. Each line in the makefile here (as with the first makefile) is of the format project[name_of_module][some_parameter]. For example, project[views][version] = "3.0-rc1" tells Drush to get the 7.x-3.0-rc1 version of the Views module and projects[views][subdir] = "contrib" tells Drush to download it to a path at candwexample/profiles/candw/modules/contrib. (Without this line it would by default download to candwexample/profiles/candw/modules). Because Views and CTools are contributed modules Drush knows where to download them from automatically.

As you can see, there are extra parameters for our custom high_contrast module. Amongst other things Drush needs to know the location of the repository to get it from and the type of that repository (Git on drupal.org in this case, but it could also be an svn repository).

Drupal.org has full documentation on the makefile format.

The other really important file in the repository (https://github.com/edwardcrompton/candw) is candw.info. This sets the modules on which the profile depends. It is very similar to the .info file you find in every Drupal module. You might think that the modules your profile is dependent on will always be the same as the modules that we've specified in our makefile, right? Not necessarily - the modules listed as dependancies in the candw.info file will be automatically enabled when the installation profile is run. You may not immediately want to enable all the modules that you have downloaded with the Drush makefile, so these would not be included the .info file. Additionally, there may be optional core modules that you want to enable but which you don't need to explicitly mention in your Drush makefile because they are included as part of Drupal core. We will include these in the .info file.

This is what the candw.info file looks like in our example:

name = C&W Blog

description = Install as a Cameron & Wilding blog site.

version = 1.0

core = 7.x

;Standard install core modules

dependencies[] = block

dependencies[] = color

dependencies[] = comment

dependencies[] = contextual

dependencies[] = dashboard

dependencies[] = help

dependencies[] = image

dependencies[] = list

dependencies[] = menu

dependencies[] = number

dependencies[] = options

dependencies[] = path

dependencies[] = taxonomy

dependencies[] = dblog

dependencies[] = search

dependencies[] = shortcut

dependencies[] = toolbar

dependencies[] = overlay

dependencies[] = field_ui

dependencies[] = file

dependencies[] = rdf

;Contrib modules for C&W

dependencies[] = views

;Custom modules for C&W

dependencies[] = high_contrast

The first four lines specify some general details about our profile: The name and description of the installation profile as well as the version of the module and the version of Drupal core we will be using. The name and description will show up when you run your custom Drupal installation profile when installing Drupal.

The 20 or so lines under the 'Standard install core modules' heading is a list of the core modules that we want to enable. These are the modules that Drupal enables by default when you do a standard installation. The last two lines in the file tell Drush to enable Views and our high_contrast module. You'll notice that CTools is not listed as a dependancy. This is because it is a dependancy of Views so it will automatically be enabled when Views is enabled.

Apart from the README.txt file in the github repository, there are two other files that we haven't yet looked at: candw.install and candw.profile. These are very similar to the mymodule.install file and mymodule.module file that you find in a Drupal module. candw.profile can contain PHP code that will dictate the behaviour of your install profile. You can use it to set variables, create user roles and also add extra steps to your Drupal installation process. For the purposes of this example install profile we are going to leave the candw.profile file blank. (I did say this would be a simple example!) There are a few really useful hooks that you can put in this file.

In the candw.install file we'll just place a single hook with a few lines of code:

/**
 * Implements hook_install().
 *
 * Perform actions to set up the site for this profile.
 * We'll just call the install method from the standard install here. 
 * Everything else we do will be on top of that.
 */

function candw_install() {
        include_once DRUPAL_ROOT . '/profiles/standard/standard.install';
        standard_install();
}

Hook_install runs when the installation profile is installed. You'll know all about this hook if you've ever done any module development. In a module it's where you do all your initial configuration and a profile is no different. We're going to cheat a bit here and use some code that's already in Drupal core - we'll simply call the hook_install from the standard Drupal installation. This does some of the stuff that you take for granted when you install Drupal. Amongst many other things it sets up some basic blocks (for example login and search) and content types (Basic Page and Article).

Now we have a working Drupal 7 install profile which can be extended to include any number of custom or contributed modules. Do take a look at the DrupalCon presentation I mentioned at the start to see how powerful these basic concepts become when you start incorporating Features modules as part of your install profile. Using Features you can create install profiles that contain all your configuration so you can set up Views, new content types or taxonomy vocabularies to name only a few of the possibilities.

If you've run your Drush makefile in your document root to download to a folder called candwexample you should now be able to navigate to http://localhost/candwexample and choose your new custom installation profile from the list as shown in the screenshot at the top of this post.

tags:

Doxygen and best practices

Indy — Fri, 07 Oct 2011 13:55:35 +0000

So what is doxygen?

Pulled from Wikipedia, the official definition.

"Doxygen is a documentation generator for the programming languagesC, C++, C#, Fortran, Java, Objective-C, PHP, Python, IDL (CORBA and Microsoft flavors), VHDL, and to some extent D. It runs on most Unix-like systems, Mac OS X and Windows.

The first version of Doxygen borrowed some code from an old version of DOC++; later, the Doxygen code was rewritten by Dimitri van Heesch.

Doxygen is a tool for writing software reference documentation. The documentation is written within code, and is thus relatively easy to keep up to date. Doxygen can cross reference documentation and code, so that the reader of a document can easily refer to the actual code."

What is the syntax/best practices to use?

In this case it all comes down to the quality of code you write. As a guide, follow this documentation here or go to http://www.cisst.org/resources/software/cis/doxygen-documentation.html.

The basics:

Use doxygen commenting in the .h (include) files of your code
Use the /*! sequence to indicate the start documentation blocks and */ to indicate the end of commenting. The same rules apply when ever you start a documentation block you must close it and be careful not to place source code within these documentation blocks. If one text line is sufficent for documentation use //!. Inbetween the start of a documentation block, the use of asterisks is optional at the beginning of each line.

Classes for OO:

Use the /*!before the declaration of a class to provide some descriptive information about the class.
Use the \brieftag to provide a short one line brief description of the class.
Use the \sa (See Also) keyword to link to related variables, methods, or classes. This is the equivalent command to \see tag. The command sets the next section to trigger a link to anything that might possibly be a keyword and also in the html document creates the phrase "See Also:" followed by the linked parameters.
Use the \ingroup to specify which group this class's documentation should be grouped under. Refer to the section on Grouping below.

/*! \brief classB: A normal class
  \ingroup common
  This class likes bob.
  This class is part of the group common.
  This class inherits class A  
  \sa classA 
 */
class classB: public classA{
...

File listing:

At the top of every .h file (except for template and inline files) after the cvs header, one should place a \filetag to label the file. The syntax actually takes in a filename as a parameter and can use the \briefbut for our purposes this is not required
for example one should use:

 
/*! 
  \file 
  \brief
  \ingroup tracking 
 */

For the erc libraries, the requirement is the use of \filewithout any arguments, \briefto give a short blurb on the purpose/contents of the file and a \ingroupwith the group to which the file belongs to in order to link this file's documentation to the group. The \file listing must be used or else documentation for the #define's and the typedefs will not be used.

/**
 * Summary here; one sentence on one line (should not, but can exceed 80 chars).
 *
 * A more detailed description goes here.
 *
 * A blank line forms a paragraph. There should be no trailing white-space
 * anywhere.
 *
 * @param $first
 *   "@param" is a Doxygen directive to describe a function parameter. Like some
 *   other directives, it takes a term/summary on the same line and a
 *   description (this text) indented by 2 spaces on the next line. All
 *   descriptive text should wrap at 80 chars, without going over.
 *   Newlines are NOT supported within directives; if a newline would be before
 *   this text, it would be appended to the general description above.
 * @param $second
 *   There should be no newline between multiple directives (of the same type).
 * @param $third
 *   (optional) TRUE if Third should be done. Defaults to FALSE.
 *   Only optional parameters are explicitly stated as such. The description
 *   should clarify the default value if omitted.
 *
 * @return
 *   "@return" is a different Doxygen directive to describe the return value of
 *   a function, if there is any.
 */

A typical documentation block

/**
* My example connectors function example checking to see if a and b exist the doing some stuff.

*
* @param $name
* The name of the string being passed in and called at module level when returned.

function connectors($name) {

// Check connector_one matches to A or connector_two matches to B

if ((connector_one == 'A') || (connector_two == 'B')) {

// Variable name of string

$name = 'example-name';

}

Doxygen would pick this up as a function call and the developer picking this up would quickly identify why it's there and what it's doing.

So by doing Drupal development work for the past few years I thought a full proof module to help me achieve this clean coding practice is the coder module.

Two benefits:-

If you are pushed for time and need to have well documented code, run this through as a project (will update this in a bit to reflect how to achieve this) and coder will automatically tidy it up for you.
Or if you are like me and want to figure it why the hell your code is poor work through each bug and it will help you improve in the longrun.

So how do I run doxygen on one of my projects?

If you have ubuntu run sudo apt-get install doxygen or download tar gzip/installer from here or go to http://www.stack.nl/~dimitri/doxygen/download.html
Download Matt Farinas excellent file from here or go to https://github.com/mattfarina/drupal-6-doxygen and put this into your drupal root directory
CD to the directory you want to doxygen
Next on command line execute doxygen drupal.doxy
Once its finished compiling navigate to your html docs so it could be something http://sitename/docs/html/index.html

So as an overview use Doxygen as a great developers guide for any documentation you have written for specifically custom written functionality and use the Coder module to help you achieve those best coding standards in Drupal.

tags:

coding best practices

Doxygen

The 10 most critical Drupal security risks

Pablo — Tue, 20 Sep 2011 10:02:08 +0000

One of the most interesting sessions I attended at the DrupalCon London was "Doing Drupal security right", given by Gá bor Hojtsy, a Drupal 6 maintainer involved in the security team.

It was a very useful presentation, with lots of tips and advises that you don't always realise when you write custom code or setup a website environment.

The presentation was based on the "Top 10 Most Critical Web Application Security Risks" guide, written by OWASP, adapting each one of the cases to a Drupal site.

The 10 most critical Drupal security risks

SQL Injection
Cross Site Scripting (XSS)
Authentications and sessions
Insecure direct object references
Cross Site Request Forgery (CSRF)
Security misconfiguration (Main source of attacks!)
Insecure cryptographic storage
Failure to restrict URL access
Insufficient transport protection
Unvalidated redirects

Resources

Points of contact with the Drupal security team

The 10 most critical Drupal security risks

1. SQL Injection

You shouldn't use data that you don't trust (as a feed, a user input, another database, etc) directly in a database query without escape it:

index.php?id=12

mysql_query("UPDATE mytable SET value = '". $value ."' WHERE id = ". $_GET['id']);

Instead you should use Drupal functions passing the user input as parameters:

db_query("UPDATE {mytable} SET value = :valueWHERE id = :id", array(  ':value' => $value,  ':id' => $id);

If you need to include dynamic table or column names in your query, you can use db_escape_table().

2. Cross Site Scripting (XSS)

XSS is injecting data to the HTML output. This is a very important one: there is a 64% likelihood a website has a XSS issue.

You should always escape the variables you send to the output if they come from a non trusted source, like the parameters of a URL. For example, this code is unsecure:

index.php?id=12
print $_GET['id'];

It's also very common see codes like this, specially in custom themes:

$output .= $node->title;

The problem in the code above is that Drupal doesn't usually filter the user input when a node is saved. So the user could save malicious code in the title and then it will be printed without any kind of filtering.

One more obvious mistake is giving full HTML permissions to not trusted users, or allow to use unsafe tags as <script>.

If we grant Full HTML permissions to users, they could add this JS code to a page, which would change the admin password if he views the content:

jQuery.get('/user/1/edit',
  function (data, status) {
  if (status == 'success') {
      var p = /id="edit-user-edit-form-token" value="([a-z0-9]*)"/;
      var matches = data.match(p);
      var token = matches[1];
      var payload = {
        "form_id": 'user_edit',
        "form_token": token,
        "pass[pass1]": 'hacked',
        "pass[pass2]": 'hacked'
      };
      jQuery.post('/user/1/edit', payload);
    }
  }
);

This technique, with code changes, works up to Drupal 6. (Example from Heine Deelstra, Drupal Security team lead http://heine.familiedeelstra.com/change-password-xss)

So, what can you do? Drupal provides a set of functions you should use:

Use placeholders in functions like t() or format_plural(): %name, @url, !insecure:

t(' %name has a blog at <a href="   @url " _fcksavedurl="   @url " _fcksavedurl="   @url " _fcksavedurl="   @url "> @url </a>', array('@url' =>   valid_url($user->profile_blog), '%name' =>   $user->name));

Use Drupal.t() , Drupal.formatPlural() in JavaScript.

3. Authentications and sessions

You need to deal with these issues:

Weak password storage and account management
Session hijacking / fixation
Lack of session timeout / logout

Drupal has good solutions for that, so don't need to worry too much about these:

Passwords are stored hashed
Session IDs changed when permissions change
Drupal works with Apache's SSL transport
Modules to set certain URLs to use SSL

4. Insecure direct object references

We sometimes don't check if the user has permission to access objects:

index.php?id= 12
db_query("SELECT * FROM {node} WHERE nid = :id", array(':id' => $_GET['id'] ));

When using Views, sometimes we don't make sure the user has access to nodes. One common issue is forget to add "published = Yes" to the view filters.

Drupal approach:

Menu system handles permission checking
user_access('administer nodes', $account)
node_access('edit', $node, $account);
$select->addtag('node_access');
Form API checks for data validity

5. Cross Site Request Forgery (CSRF)

If there is an image like this the user who loads it will be logged out.

<img src="http://example.com/user/logout" />

In the same way some content could be deleted:

http://example.com/index.php?delete=12
<img src="http://example.com/index.php?delete=12" />

That's why Drupal always asks "Are you sure you want to delete this?"

Again, we can avoid this issue doing things in "The Drupal Way":

Form API works with POST submissions by default (makes it harder)
Form API includes form tokens, requires form retrieval before submission, checks valid values
drupal_valid_token() provided to generate/validate tokens for GET requests.

6. Security misconfiguration

Secure server

First you should check security holes behind Drupal, like the server configuration
Avoid using FTP at all cost and check your client tool. There are software to decrypt passwords stored in FTP clients.
In shared servers, know who do you share the server with. You might be sharing hosting with a site of a politic party, which could be the objective of an attack.
Which applications are running on the server? Applications like phpBB2 have lots of security holes.
Keep your OS, PHP, SQL server, etc. up to date.

Secure Drupal

Is your admin password "admin"?
Look at all "administer *" permissions
"administer filters" can take over a site
Use Update module, watch the security news (security updates are made on Wednesdays)
Avoid any kind of PHP input, write your own modules instead. Look into using Paranoia module
Watch your input formats, you can be googled!. If we enable Full HTML for anonymous users, somebody can find our site searching the filter description ("Full HTML, Web page addresses and e-mail addresses turn...") and could hack our site.
Check out the security_review module .

7. Insecure cryptographic storage

Drupal approach:

Drupal stores user passwords hashed with a one-way hash
Different randomly generated private key is provided on each site, which can be used to do reversible encryption
Modules exist to help encrypt more data
Up to you to ensure backups are properly protected

8. Failure to restrict URL access

Drupal approach:

Menu system uses access callback and access arguments
Continually review permissions

9. Insufficient transport protection

Tools like Firesheep, get data flowing in the same network you are connected. So somebody connected to the same network as you could see Facebook connections, pictures, etc. You could even login as this user grabbing the login session id.

Drupal approach:

Run Drupal on top of full SSL, which is expensive.
Use securepages and securepages_prevent_hijack to wall your important pages.
http://drupalscout.com/knowledge-base/drupal-and-ssl-multiple-recipespos...
Use a valid certificate.

10. Unvalidated redirects

Having custom redirect systems can be also unsafe:

http://example.com/index.php?target=evil.com

Drupal approach:

Drupal has various internal redirections, which use local paths and generate URLs based on them
Look for use of drupal_goto() and Form API #redirect instances in your modules to validate their compliance

Resources:

Session video: http://london2011.drupal.org/conference/sessions/doing-drupal-security-r...
Session slides: http://london2011.drupal.org/sites/default/files/DrupalSecurity2011Londo...
http://drupal.org/writing-secure-code
Book: Cracking Drupal, A drop in the Bucket (specially chapter 8)
The ten most critical web application security risks: https://www.owasp.org/images/0/0f/OWASP_T10_-_2010_rc1.pdf

Points of contact with the Drupal security team:

Releases at http://drupal.org/security
Reporting issues: http://drupal.org/node/101494
Reporting cracked sites: http://drupal.org/node/213320
Discuss general issues: http://groups.drupal.org/best-practices-drupalsecurity

(image credit)

tags:

security

drupalcon

How to Make Drupal Perform and Scale Like a Rockstar!

José — Fri, 16 Sep 2011 11:16:24 +0000

Some of the developers of our team had the opportunity to attend the drupalcon session:
Damn Quick Drupal: How to Make Drupal Perform and Scale Like a Rockstar! by Michael Cooper (Soyarma), you can find very useful information about the mentioned conference here:

http://london2011.drupal.org/conference/sessions/damn-quick-drupal-how-m...

It was very useful but also very quick, so we've written up the most important points in this quick reference guide.

There are 2 main parts: drupal configuration and apache configuration.

Drupal configuration

Consider always caching your pages (including front and 404s pages):
This can be achieved in a lot of ways: the use of apache cache modules (as you will see in the server configuration section), the use of drupal default cache settings and you can use too extra modules like: http://drupal.org/project/authcache (APC apache module + AuthCache drupal module is a WIN combination for your page caching).
Don't let anonymous hit imagecache generation URLs:
An easy way that helps to avoid this is the use of generic hotlinking protection, you can find a lot of useful information about this here: http://bit.ly/piLnel
Be careful with cookies and session data caching:
An easy win to reduce pages load time in your site in relation with session data is to use the module: http://drupal.org/project/no_anon which allows drupal to avoid session data and cookie caching for anonymous users.
Consider Path alias cache:
You can use a module for this purpose: http://drupal.org/project/pathcache
Examine your views queries and views pages:
The main problem in views queries is the excesive use of JOINs, but the performance of views queries can be significantly improved by using index tables. There is a very interesting module which automates the mentioned process: http://drupal.org/project/shadow
Double check your headers to avoid your browser requests continuously for freshed pages:
This option is included by default in drupal 6 pressflow and drupal 7 core.
Drupal aggresive cache:
This kind of cache management can improve a lot the load times for your site pages but the problem is that a lot of modules are incompatible with this kind of caching becasuse some of their hooks are not called or called in the non appropriate way, so if you are thinking about enabling this setting be sure about your modules compatibility with it and about the existence of patches that can solve this kind of issue.

Server configuration (Apache, MySQL)

Enable MYSQL query cache. If enabled, be sure about you are using sane settings:
http://www.techiecorner.com/45/turn-on-mysql-query-cache-to-speed-up-mysql-query-performance/

Use the following settings:
key_buffer_size=12M (key cache)
query_cache_size=24M
query_cache_limit=2M
table_cache=96
sort_buffer_size=12M
myisam_sort_buffer_size=12M
tmp_table_size=12M
Use Advanced PHP Cache (APC) module for Apache: Every time a file is read by PHP it is compiled (checked for syntax errors, optimized, compiled into byte-code).
APC (advanced PHP cache) will do this once and then cache the results. APC will check the file every time it is accessed to determine if it is still the same. If that happens rarely, set apc.stat to 0 and you will save that check.
The cacherouter module for drupal will integrate APC with the standard cache clearing functions:
http://drupal.org/project/cacherouter
Set the appropiate value for your max-clients apache parameter. Yo can calculate this parameter in this way:
- Use a tracking system (GA, statistics, devel performance logging, webserver logs) to determine how many actual page loads you get at your peak traffic time.
- Depending on how sharp the spike is, pike a time period that sits at around the top 90% of that spike.
- Work out how many pageviews you get a minute during the peak of that spike.
- Since you know how long (on average) it takes to generate a page, you can determine how many of those requests are concurrent. You can calculate this value in this way:
  (P / M) x (E / 60) = C
  P = Number of page views that hit Drupal
  M = Minutes page views collected over
  E = Average Execution time per page in seconds (obtained from your tracking system)
  C = Concurrent requests (the right value for apache max-clients parameter)
  
  Example:
  Number of requests: 5000 (P parameter)
  Time taken: 28.9 seconds = 28.9/60 = 0.482 minutes (M parameter)
  Mean time/request: 578ms = 0.578 sec (E parameter)
  C = (5000 / 0.482) x (0.578 / 60) = 100
  So the max-client parameter should be set to 100

Extra Tips: Improving front-end performance

Mod Pagespeed, help out your users by helping their browers load your site faster:
Pagespeed is an apache module developed by Google team and based on its performance philosophy, you can find a lot of useful information here: http://bit.ly/bBPcYl

Reverse proxies save Apache/PHP from running when they don't have to:
The best drupal accelerator based on reverse proxies idea is Varnish, you can find a lot of information here: http://drupal.org/project/varnish
CDNs offer shorter round trips for your users, but are often notfaster than a good reverse proxy and can cause some confusion when clearing caches.
Domain Sharding, a good way to help your users get content faster:
Browsers open a limited number of connections per domain. Most notably, Internet Explorer 6 and 7 open only two connections per domain. Domain Sharding is a technique based on serving your site contents from different domains (normally used to improve the pages load in relation with images). There is a drupal module which can help you to configure a domain sharding to serve your images faster: http://drupal.org/project/parallel_css

Conclusions

Yo can easily improve a lot your site performance only doing the following:

Use default cache options available in drupal (take care of the aggresive mode).
Use APC + cacherouter module to cache compiled php code, APC + authcache will help you too with the page caching.
Beware of: hotlinking, anonymous session data caching, url alias caching, views queries and headers.
Use basic mysql cache settings (first point of the server configuration tips).
Calculate the right value for your max-clients apache parameter and set it.

tags:

performance optimisation

How to add custom Rules to your module in 5 minutes

José — Fri, 16 Sep 2011 10:56:31 +0000

It's very interesting to know that something so simple as adding custom Rules to your module can make it much more flexible and allow you to integrate it with other modules and processes.

You can do it in two simple steps:

Step 1: Define your custom events using the hook_rules_event_info for your modules:

<?php
return array(
      'my_event_1' => array(
                      'label' => t('Title of my event 1'),
                      'module' => 'yourModule',
                      'arguments' => array (
                                            'arg1' => array ('type' => 'arg1_type', 'label' => t('your_arg1_label')),
                                            'arg2' => array ('type' => 'arg2_type', 'label' => t('your_arg2_label')),
                                            ),
                      ),
      'my_event_2' => array(
                      'label' => t('Title of my event 2'),
                      'module' => 'yourModule',
                      'arguments' => array (
                                            'arg1' => array ('type' => 'arg1_type', 'label' => t('your_arg1_label')),
                                            ),
                      ),
              );
?>

This function returns an array containing the definition of every event you wish to define. The definition of every event is at the same time another array with the keys:

'label' to define your event title (as it will appear later when you create a triggered rule for it).

'module' to specify the name of your module.

'arguments' to define the arguments you will need when you invoke your event in your module.

Every argument is defined as an array in which you need to define the type (for example you could 'node', 'user', 'comment'... depending on what you need), and a label that is the title which will be used if you want to access that argument when you are creating your triggered rule by interface.

Step 2: After defining the hook_rules_event_info you can invoke your events in any place of your module related to the condition you wish to call an specific event (or events). You can do this in this way:

And that's it, once this is done you can choose your custom events when you are creating your triggered rules by interface. If you want to see an example out in the wild, have a look at our recent patch for the Antispam module

(image credit)

tags:

rules

coding

DrupalCon London- Team C&W's takeaways

Neil — Fri, 16 Sep 2011 10:41:05 +0000

The whole team made it down to the recent DrupalCon that took place in London. DrupalCon happens twice a year, once in the US and once in Europe. London's DrupalCon saw 1,750 people attend three days of sessions, trainings and group discussions. Attending DrupalCon is a lot like "drinking from a fire hose", you are bombarded with more information, new concepts and ideas than it is possible to sensibly absorb. It's thrilling, exhausting but utilimately a great rejuvenator.

We recently carried out our DrupalCon debrief session where we exchanged what we took away and how we can integrate it into day to day work in the business.

This is what we learnt:

Neil

Anybody doing anything interesting or significant on Drupal is using Drupal 7. The time is right to make the switch.
Workbench module solves a whole bunch of content and permissions problems that many of our clients have had. I can see this module becoming a one of our default instalation modules for all new sites.

Managing config and DB changes through code is more than just a bragging right. Although it requires a shift in working methods and extra time to test deployment functions, it will yield time savings in managing merges and deployment and improve quality.
Distributions are really powerful and easier than ever to create. The first thing we'll be doing is creating a starter D7 distro with selection of modules pre configured.

Indy

With more OO code in D7 core and contrib, we should start to make more use of OO for custom modules in order to make the code more efficient and maintainable.
We should make more use of coder module to ensure strict adherence to best practice.

Pablo

The Drupal security talk was really interesting and highlighted some points you don't normally think about. We should make a security checklist we can through during and after a site build.
The performance talk was also really useful (so useful, we've written up our notes on it), this will make a great first list of things to check and implement when looking at the performance of a site.

Tom

Content staging between environments is still one of the big challenges for Drupal. Lots of people are talking about it, but there is no one good solution yet.
Responsive design is being talked about more and more, it would be great if our themes were responsive by default

Jos?

Node.js looks really interesting, but we can't quite figure out what we would use it for other than real time chat. Also seem to some serious security loop holes.
Features and code driven development should be used as much as possible.

Ben

The very rapid and recent injection of money and corporate culture into the Drupal ecosystem seems to be causing some tensions. It will be interesting to see how it affects the community.

(image credit)

tags:

drupalcon

A video introduction to Drupal

Ben — Thu, 15 Sep 2011 18:48:38 +0000

Our friends over at the Brighton Drupal Association put together this video, which is a really good three minute overview of what actually constitutes Drupal (more than just code). If you are interested in finding out more about Drupal and how it can benefit your organisation, consider going along to the Discover Drupal Day in Brighton on September 16th.

What is Drupal? from Larchmont Films on Vimeo.

tags:

Drupal