William Durand

Propel, And Symfony2 A Year Ago

2012-04-25T00:00:00-07:00

Did you know that the PropelBundle was part of the Symfony2 core? Yes, at the very beginning, and it has been dropped after. So basically, I didn’t create the PropelBundle but started to learn Symfony2 by using it.

Hi,

You seem to work on the PropelBundle actively, and that’s great news. This bundle is under my account right now because it is orphan. But if you want to take care of the maintenance of it, then I think it’s time for your fork to become the official repository for the bundle. What do you think?

Fabien

To be honest, this bundle didn’t really work, and I spent a lot of time to both learn Symfony2 internals, Propel 1.6 internals, and how to fix this bundle. It was fun. After few weeks, the bundle became interesting to use, and in the same time, I joined e-TF1 where I had the opportunity to work with both Symfony2, and Propel, for real.

Few months later, I decided to take the lead of the Propel project, and I added the PropelBundle as an official Propel project. It was the best choice because people helped me a lot to improve this bundle, and now it is fully compatible with Symfony2 (kudos to Toni, the PropelBundle manager). To spread the word, I also wrote quite complete documentation about Propel, and Symfony2.

To be reliable was the hardest part of the work, even if the bundle worked great, and even if we were there to answer questions, or to fix issues. That’s why I asked Fabien to create a Propel Bridge in Symfony2. It was the first step to avoid comments like “it’s not the default ORM, lalala”. In the same time, Symfony2 took the decision to be a View Controller Framework, without any Model layer bundled by default.

Today, I’m proud to announce that the last step has been unlocked thanks to Joseph. Propel has its own chapter in the official Symfony2 book which means you have to think about which Model layer to use now. No more excuse to not try Propel in Symfony2, or to use Doctrine2 by default, and not by choice.

Try Propel, really! It could fit your needs depending on what you expect. Some great bundles already support Propel (like the FOSUserBundle). And please, avoid comments like “Doctrine2 is much nicer” which make no sense. Why is it much nicer? I never got any good arguments in favor of Doctrine2 I didn’t already know by myself. I can hear everything which is constructive. I used Doctrine2 for more than six months, both ORM, and ODM (MongoDB). I have strong arguments against it, but I also like some parts of it. Who can say the same thing about Propel?

Now, you have the choice, it’s up to you! But, give Propel a chance before to complain about it, or to comment it.

I Will Be Speaking At Symfony Live 2012

2012-04-11T00:00:00-07:00

Heya, I’m glad to announce that I will be speaking at Symfony Live 2012 in Paris, on 7-8th of June.

I will speak about Propel2, the upcoming version of Propel ORM. It will probably present the Propel philosophy, how it works in real life, how it’s built, and some other parts you will discover.

To be honest, it will be my very first conference talk, so I decided to speak in French. Even if some people complained about that, I’m not yet comfortable with my English to hold a talk in English. But, I will write my slides in English. That way, it will be quite easy to understand for people who don’t understand French.

Hope to see you there.

Crazy Idea #1: Converting My Blog Posts In Audio Files

2012-02-29T00:00:00-08:00

Yesterday, I discovered the say command on my Mac. Its aim is to convert text to audible speech, and it works really well. Then, I had the idea to create audio files from my blog posts, because it’s fun, and because it could improve accessibility.

The say command generates aiff audio files, which is not really understandable by the audio html5 tag. So, I had to convert this output in a mp3 file. I used the well-known ffmpeg tool, and that’s all!

Please, welcome Speaker, my fun work of last evening. Speaker aims to convert my blog posts in markdown syntax to a mp3 file. This is a tiny shell script I enjoyed to write.

USAGE:
  ./speaker [-h] [-d <output directory>] <filename>

I used roundup to test it. I love shell scripts, but to write them without any tests is a pain, there is often a condtion which is not good, a typo, or something else. That often makes me crazy! roundup helps you write strong shell scripts. Here is my test suite output:

$ ./speaker-test.sh
speaker
it_shows_help_with_no_argv:                      [PASS]
it_shows_help_with_h_option:                     [PASS]
it_creates_mp3_file:                             [PASS]
it_creates_mp3_file_in_existing_directory:       [PASS]
it_creates_mp3_file_in_new_directory:            [PASS]
=========================================================
Tests:    5 | Passed:   5 | Failed:   0

To sanitize the text to speech, I used some regular expressions. It probably needs some improvements but it works pretty well:

# Sanitize markdown content
content=`echo "$content" | sed -e 's/[\*_]//g'`
content=`echo "$content" | sed -e 's/\[\(.*\)\](\(.*\))/\1/g'`
content=`echo "$content" | sed -e 's/:[p|D]/./g'`
...

For the other parts, check the code :)

As I wanted to provide my blog posts as audio files, I tweaked my templates to integrate an audio html5 tag. And I used html5media to render this tag in all major browsers (don’t know if there is a better solution).

<audio src="/mp3/my-blog-title.mp3" controls preload></audio>

And to automagically generate audio files, I wrote a pre-commit script to build the audio file when I commit blog posts. That’s all folks!

Deploying With Git

2012-02-25T00:00:00-08:00

Yeah, I know yet another blog post on this topic. The main difference with others is that I wrote it myself, and it’s quite up to date :p

I often rely on Capistrano to deploy web applications. I ever talked about this tool in my previous blog, and it’s not the purpose of this article.

Actually, when you need to deploy a simple web application on a server, like this blog for instance, there is no need to use Capistrano or Fabric. You just need to copy files. So you could rely on rsync… But it’s not fun!

Actually, you just need to rely on Git, assuming you are using it.

The first step is to configure your local repository by adding a new remote:

git remote add -t master production ssh://<server>/path/to/project_prod

The -t <branch> option allows you to track a given branch instead of all branches. You can add more remotes if you want like a testing remote.

git remote add testing ssh://<server>/path/to/project_test

To deploy the application in production, run:

git push production

And, to deploy in a testing environment, just run:

git push testing <branch>

You can push your code on your server, but it won’t deploy new changes at the moment. So let’s configuring Git. First, add the following lines to your .git/config file:

[receive]
    denyCurrentBranch = false

It allows to push code on the current branch, it’s important to deploy new changes. Old Git versions don’t need to set this parameter by the way.

Then, enable a post-receive hook with the following content:

#!/bin/sh

SUBJECT="Deploy successful"
BODY="You've successfully deployed the branch:"

while read oldrev newrev ref
do
    branch=`echo $ref | cut -d/ -f3`

    if [ "$branch" ] ; then
        cd ..
        env -i git checkout $branch
        env -i git reset --hard

        EMAIL=`env -i git log -1 --format=format:%ae HEAD`
        BODY="$BODY $branch."

        echo "$BODY" | mail -s "$SUBJECT" "$EMAIL"
    fi
done

This script updates the working tree after changes have been pushed, and send an email to the last committer. Keep in mind that it always deploys the last branch you push.

The important part is:

cd ..
env -i git checkout $branch
env -i git reset --hard

Feel free to decorate these three lines as you want.

In a production environment, or because you are using a Git Branching Model, you should modify the previous code as below. It ensures to always deploy the master branch:

    if [ "$branch"  == "master" ] ; then
        cd ..
        env -i git reset --hard

        // Send an email
    fi

You’re done, each time you’ll run a git push production, you’ll deploy your application in production.

Easy. Fast. Powerful.

Why You Should Love Code Generation

2012-02-06T00:00:00-08:00

What a nice title, right? I often see people complain about code generation, and I would like to explain my point of view in a pragmatic approach. As you may know, I’m the lead developer of Propel, a great PHP5 ORM which uses code generation a lot.

Code generation is about to use code to write code. Pretty easy, isn’t it? In Propel2, we will use Twig to generate PHP code for instance. The golden rule is to never use the same programming language you want to generate, otherwise it will be a pain to maintain (try to read Propel 1.6 builders). But in order to generate code, we need to know information that describe the code we want to generate. The term metadata defines these information. A metadata is a data about data, that’s exactly what we need!

In Propel, the schema.xml (a XML file) contains all metadata we need to generate both PHP and SQL code. As Propel is an ORM, data will be a database name, tables, columns, primary keys, etc. This is your business model! We can call it a Platform Independent Model (PIM) because this schema is database vendor agnostic. It doesn’t know anything about MySQL, Oracle, or whatever you want. It just describes your model with its own notation.

To generate code, Propel relies on builders and platforms. Builders own the logic to write PHP code, and Platforms contain the logic for each database vendor (whether to quote table names or not for instance). This is specific, right? So let’s call this layer a Platform Specific Model (PSM).

Propel uses both a PIM and a PSM to generate code. Actually, it uses the PIM to drive the PSM which depends on configuration parameters also known as build properties in Propel terminology. That’s all, here is how Propel works at buildtime :-)

But wait, what? It’s all about Model Driven Architecture! Did you ever hear about that? It’s the state of the art in research in Computer Science (at least in France). This is a software design approach for the development of applications.

But this is a formal approach, and I would like to keep a pragmatic mind here through Propel. Then, what I love in code generation is that your code is easily debuggable, you can read it, and learn from it. Few years ago, generated code was a mess, especially in symfony 1.x, but today we are able to generate clean code, as you could write yourself. And, as you write code to generate code, you can test both, and avoid more errors. To generate code speeds up your application, and you can pre-calculate some parts of your code. In Propel (again), we are able to pre-generate SQL statements at buildtime. That way, we by-pass the need to generate SQL code at runtime. It’s really really faster than to use the whole Propel stack.

In Propel, we also have behaviors. This is a self-contained logic which extends the current implementation by using hooks at buildtime. A behavior is reusable, testable, and easy to write. Then, you can adopt a behavior driven development: write behaviors you will reuse in order to avoid code duplication.

I know there are drawbacks to use code generation, especially the need to generate code before to deploy your application in production, or the lack of implementation details. But, for the most part of an application, to generate code helps a lot, and it helps you to get what you really want to get too.

Tu sum up, code generation is not a crazy idea. It’s close to MDA, it has pros and cons but in my opinion, code generation is definitely not a bad practice, and you should use it instead of relying on in memory code or complex architectures just to avoid generation.

Component Driven Development: it's like Lego!

2012-02-01T00:00:00-08:00

Did you ever hear about Component-Based Development? Maybe about Component Driven Development? Both concepts are the same, and if you’re involved in Symfony2, Spring, or something similar, then you do Component Driven Development (CDD).

It’s all about separation of concerns. You design components with their own logic, each component does one thing well, and only one thing. To separate business logic in self-contained components requires to focus on interactions between components. In other words, you have to think about right interfaces. To help you, UML provides a component diagram which allows you to easily show components, interfaces, and relations.

In a component diagram, a provided interface is modeled using the lollipop notation, and a required interface is modeled using the socket notation. Small squares represent ports, which are features with distinct interaction points. So, a port is an internal component of a classifier.

Most of the time, you won’t use this kind of diagram because you don’t like to draw UML diagrams :-). But, as I said previously, you probably know Component Driven Development if you know Symfony2 or Spring. These two frameworks use the Inversion of Control architectural pattern through the Dependency Injection design pattern (I consider IoC as an architectural pattern because it’s not a design pattern you can implement).

A well-known issue when you use Component Driven Development is how to manage dependencies between components, and how to instanciate all your components. Dependency Injection Container to the rescue! By describing component’s dependencies, and thanks to a container, you’ll write an highly decoupled application, with components you can reuse.

The main drawbacks are the time you’ll spend to think about your architecture, and to know how your components will interact (don’t forget UML in that case). So, try to find a component before to write your own. In the PHP world, we have magic things:

The Symfony2 components: a reusable set of standalone PHP components;
Composer: the package manager you need;
Packagist: to discover awesome PHP components.

Since few months, I work on a project for a French startup, and it’s my first Component Driven Development experience. I started by listing all my needs (an OAuth client, a iCalendar parser, …). Then, I tried to find one component per point. And, as a glue, I decided to use Composer. After the end of the first sprint, I was a bit confused because I wrote two hundred lines of code (including some test classes), and the project was ever ready to use. I focused on the only important thing: the aim of the application.

Thanks to tools like npm, gem, composer, it’s like Lego!

Designing A Software By Naming Things

2012-01-24T00:00:00-08:00

There are only two hard things in Computer Science: cache invalidation and naming things.

Phil Karlton

Naming things is hard, that’s right, but naming things by their rights names is even harder!

When you start writing a new application, the first step is often to rely on some UML diagrams. It allows you to think before to code, what a nice idea!

Even if use cases or user stories are essential, a class diagram is probably the most useful UML diagram. A first step can be to find packages names (more or less sub-namespaces), and to connect them. It will give you the main components of your application like the controllers, the services, the model, or the configuration part for instance. It’s quite easy to separate concerns at this level. If you use the well-known Model View Controller design pattern, you almost ever know your packages.

Once you’ve defined your packages, you have to find names for your classes. To help you finding the rights names, you can write interfaces first. An interface describes a contract between two entities. This is not related to naming, but if you start by writing interfaces, you start by thinking about interactions between your components. Program to interface is a nice way to design an application, and reduces coupling but it’s not the purpose of this post.

My Golden Rule is: if you’re not able to find a name for a class, then ask yourself if this class makes sense, or if you can decouple things a bit more.

A wrong name or a non-explicit name often lead to errors. If you’re not satisfied with your naming, then think again because you probably missed something. Inspire yourself by reading good code, thanks to GitHub, it’s easy to browse awesome code. As a PHP developer, I often use Symfony2 naming in my own projects.

By following these advices, you’ll build a software with a better separation of concerns, and each component will be decouple, and will own its own logic, nothing more. That way, you’ll be able to unit test your application without any effort. And, as you have a good separation of concerns, each unit test can cover a use case.

This is my last point, always write unit tests that make sense, and think them as use cases. Take a look at the Propel2 test suite, each test method has a readable name which explains a use case: testIsValidReturnsFalseIfNoUserFound() is really explicit for instance. Don’t forget, the best documentation you can write is tests.

My Git Branching Model

2012-01-17T00:00:00-08:00

We all probably know a successful Git branching model which is a very interesting model for teams who want to use Git. However, this model is a bit too complex for common needs. So here is my lightweight model.

I use two main branches:

master : the code in a production-ready state;
develop : the integration branch.

I also use feature branches. A feature branch contains a work in progress. Keep in mind that a feature branch should reflect a feature in your backlog. I use a convention for these branches, I always prefix them with feat-.

> git branch
feat-my-feature
* master

A feature branch has two constraints:

the code must come from the develop branch;
the code must be merged in the develop branch.

To create a feature branch, I use the following command:

> git checkout -b feat-my-feature develop

To merge a feature branch into develop, I use the following set of commands:

# Go back to the develop branch
> git checkout develop

# Get last commits
> git pull --ff-only origin develop

# Switch to the feature branch
> git checkout feat-my-feature

# Time to rebase
> git rebase develop

# Then, switch to the develop branch in order to merge the feature branch
> git checkout develop

> git merge --no-ff feat-my-feature

# Push
> git push origin develop

# Finally, delete your branch
> git branch -d feat-my-feature

I always merge a feature branch into develop using --no-ff to keep a clean log:

The --no--ff option allows to keep track of a feature branch name which is quite useful. The following git log output shows you a feature branch merged with this option:

commit 481771556824c4ae2e6da73ef14d6ce757fb5870
Merge: 6abdd70 8cfe5a7
Author: William DURAND <william.durand1@gmail.com>
Date:   Tue Jan 17 11:31:56 2012 +0100

Merge branch 'feat-my-feature' into develop

commit 8cfe5a7da159663cc09a850bee49a59ce046c67e
Author: William DURAND <william.durand1@gmail.com>
Date:   Tue Jan 17 11:31:19 2012 +0100

Added a new feature

commit 6abdd707aace50ee5aad72a3c6fcff2f36cdea7f
Author: William DURAND <william.durand1@gmail.com>
Date:   Sun May 15 14:07:19 2011 +0200

Initial commit

Without the --no-ff option, you’ll get the following output:

commit 0d5805d52e55e4941ce23585a4cd559e5e643207
Author: William DURAND <william.durand1@gmail.com>
Date:   Tue Jan 17 11:35:43 2012 +0100

Added yet another feature

commit 6abdd707aace50ee5aad72a3c6fcff2f36cdea7f
Author: William DURAND <william.durand1@gmail.com>
Date:   Sun May 15 14:07:19 2011 +0200

Initial commit

In a team, you will probably have more than one feature branch, and you could have a depencency between two branches (this should be avoided). In this case, I use another branch in which I merge two or more feature branches.

> git checkout -b feat-my-feature-with-another-feature develop

Then, I can merge the two feature branches, and solves conflicts in it:

> git merge feat-my-feature

> git merge feat-another-feature

I don’t use any other branches. The last part of the model is to merge develop into master. To avoid conflicts, there should be only one person who owns this responsability: the release manager.

I experimented this model with different teams in terms of number of people and skills, and I never had more needs. I know some people use releases but it can be handled in another way.

All credits go to Vincent Driessen and his Git model.

Services Status Dashboard

2012-01-16T00:00:00-08:00

Today, I was looking for an Open Source alternative to Pingdom as I needed something free, customizable, and lightweight. I love the GitHub’s Status Panel, unfortunately they didn’t open source it.

Basically, this kind of application is simple. You just need a crawler, and few rules to know whether an application is alive or not. Additionally, you may want to store results in order to make graphs or to calculate uptime average, and so on. Before to write this kind of application myself, I tried to search existing projects on GitHub.

Good news! I found a really nice project: Status Dashboard created by Olivier Bazoud. This project is easy to configure, you can use provided plugins (Twitter, IRC bot, History, …) or add your owns, and you can monitor various services (not only HTTP). If you don’t want to use the beautiful interface, then you’ll find a REST API.

Installation

Run npm install to setup the project and its dependencies, then add your own configuration in settings.js. Mine is:

// ...
settings['william'] = {
  services: [{
    name: 'williamdurand.fr',
    label: 'William DURAND (blog)',
    check: 'http',
    host: 'williamdurand.fr',
    port: '80',
    path: '/',
    headers: {
    'Host': 'williamdurand.fr'
    }
  }],
  plugins: {
    history: {
    enable: true,
    host: "127.0.0.1",
    port: 6379,
    namespace: "statusdashboard",
    options: {
    },
    client: true
    }
  }
};

Then, start the server:

node server.js

Check the server is running by browsing http://127.0.0.1:8080/ or by querying the API:

> curl http://127.0.0.1:8080/api/services
{"lastupdate":"Mon, 16 Jan 2012 13:29:11 GMT","services":[{"name":"williamdurand.fr","label":"William DURAND (blog)","status":"up","statusCode":200,"message":""}],"summarize":{"lastupdate":"Mon, 16 Jan 2012 13:29:11 GMT","up":1,"critical":0,"down":0,"unknown":0}}

The project is available at: https://github.com/obazoud/statusdashboard.

Did I Tell You Open Source Was Awesome?

2012-01-16T00:00:00-08:00

Few months ago, my father gave me a Karotz (the new Nabaztag) for my birthday. In the same time, I started to learn how to write a Jenkins plugin, and I knew this kind of rabbit could be programmed. That’s why I decided to write a Karotz-Plugin.

As an Open Source Software enthusiast, I published my code on GitHub right at the beginning. The plugin was in a work in progress state, but quite usable. Unfortunately, I was unable to improve it for few weeks.

When I got some free time to work on it, I saw my repository was forked. Surprise! Someone improved my work, and published it as an official Jenkins Plugin. Wonderful!

So, I pulled the repository, and learnt a lot from the developer who worked on the plugin. Isn’t it awesome? First, my code has been reused. Second, I learnt new things. Oh! And I got an awesome plugin for Jenkins and my Karotz!

To sum up, it’s a often really good idea to open source everything, even if it’s a draft, a work in progress, or just an idea. Someone could find this idea awesome, and could work on it. And thanks to GitHub, to open source something is really really easy :)

Hello, World!

2012-01-15T00:00:00-08:00

Hi folks!

Few months ago, I decided to lock my plain old blog in French. To own a blog is really important but I was bored to write in French, and I had no free time to answer comments, to write new entries, and so on.

That’s why I’m starting this new blog, in English (as well-written as possible) powered by Jekyll, HTML5 Boilerplate, and hosted by GitHub.

You’ll get the code by browsing the following repository: willdurand/willdurand.github.com.

William