nvie.com

vim-flake8: Flake8 for Vim

2012-02-13T23:00:00Z

Just a quick post to let you know that I discarded my vim-pep8 and vim-pyflakes Vim plugins yesterday in favor of vim-flake8.

As you may know, PyFlakes is a static analysis tool that lets you catch static programming errors when you write them, not when you run into them at runtime. And pep8 is a Python style checking tool that enforces PEP8 guidelines on your code.

Flake8, though, seems to be a much better option to use these days. It integrates both of PEP8 and PyFlakes and even combines it with a cyclomatic complexity checker (which is irrelevant for the Vim plugin, by the way). To install Flake8, simply use:


$ pip install flake8

After installing the plugin in Vim, you can add the following command to your .vimrc file to have it executed after every save of a Python source file.


autocmd BufWritePost *.py call Flake8()

To avoid specific error messages from being reported, put a # noqa comment at the end of that line.

Installation

Assuming you already use vim-pathogen (which you really should), you can simply install the plugin by cloning the repository into the ~/.vim/bundle folder.

Introducing Times

2012-02-01T23:00:00Z

Lately I’ve been getting sick of working with datetimes and timezones in Python. The standard library offers many different conversion routines, but does not prescribe a best practice way to deal with them. Luckily, Armin Ronacher did in his article Dealing with Timezones in Python.

The summary is to never ever work with local datetimes. When a local datetime is input, immediately convert it to universal time and only ever store or calculate with those. Only when presenting datetimes to the end user, convert them to local time again.

This seems simple enough, alright. But to actually do it in Python, you still have to think about how to implement it correctly. Every. Single. Time. pytz does help a bit here, but it still isn’t trivial. It should be.

Meet Times, a very small Python library to deal with conversions from universal to local timezones and vice versa. It’s focused on simplicity and opinionated about what is good practice.

Example use

Imagine you’re building a web app that allows your users to set an alarm. Say that someone in the Netherlands sets an alarm to 9:30 am. You can use times to simplify this:


>>> import times
>>> import datetime
>>> 
>>> local_time = datetime.datetime(2012, 2, 3, 9, 30, 0)
>>> universal_time = times.to_universal(local_time, 'Europe/Amsterdam')
>>> universal_time
datetime.datetime(2012, 2, 3, 8, 30)

Now, this universal_time variable is safe to store or calculate with.

Once you want to show this date to the user again, simply format it for the given timezone:


>>> times.format(universal_time, 'Europe/Amsterdam') 
'2012-02-03 09:30:00+0100'

If your app allows users to share alerts, it is just as easy to present the alert date to an end user in New Zealand as well:


>>> times.format(universal_time, 'Pacific/Auckland') 
'2012-02-03 21:30:00+1300'

Current time

If you ever need to record the current time, you can use


>>> times.now()
datetime.datetime(2012, 2, 2, 16, 4, 40, 283090)

Which is actually just an alias to datetime.datetime.utcnow().

Converting from other sources

I’ve added the ability to create universal times from two other sources: UNIX timestamps and date strings. To use any of these, simply pass them to the to_universal function, like so:


>>> time.time()
1328729274.982
>>> times.to_universal(1328729274.982)
datetime.datetime(2012, 2, 8, 19, 27, 54, 982000)

Note that UNIX timestamps must be in UTC (which the output of time.time() is). Local UNIX timestamps are not accepted.

To create universal times from string representations, Times uses the advanced parser from the python-dateutil library. Time zones are automatically recognized if such info is encoded in the string representation. In any other case, you are required to provide it explicitly. Two examples to illustrate both variants:


>>> # Timezone-aware date formats don't require a source timezone
>>> date_str = '2012-02-08 19:27:54+0100'
>>> times.to_universal(date_str)
datetime.datetime(2012, 2, 8, 18, 27, 54)

>>> # Timezone-less date formats require an explicit source timezone
>>> date_str = '2012-02-08 19:27:54'
>>> times.to_universal(date_str, 'Asia/Singapore')
datetime.datetime(2012, 2, 8, 11, 27, 54)

Installing

Times is on PyPI (link), so just pip install times to use it.

Of course, you can fork me on GitHub.

As usual, Times is licensed under the liberal terms of the BSD license.

Thank you, Steve

2011-10-05T22:00:00Z

This night, I woke up to the news that Steve Jobs passed away. I did not expect to be this touched by the news, although it was to be expected.

Steve Jobs has revolutionized the industry several times, each time showing customers what is possible, before customers could even dream about it.

He sold personal computers to households, applied the graphical user interface and the mouse to those, and then, in 2007, he saw a future for touch interfaces that fit in your pockets. Not only did he dream, he was able to actually build them. Each of these have set the standard in the industry. It is incredible how a single man was able to sketch this vision, inspire people to move mountains, and to reach these goals with Apple.

Reshaping the world

It is illustrative that the news of Steve’s death had not even made it to our news paper this morning, but was read by millions of people on their iPhones instead.

Obama used beautiful words to express his impact on mankind:

“Brave enough to think differently, bold enough to believe he could change the world, and talented enough to do it.”

Any person that ever touched an Apple device knows it breathes the dedication and craftsmanship that has been put into them, and it is an awesome feeling. Our kids will take this for granted, but we are still aware of where we came from.

Impact on developers

But Steve has also been able to make developers around the world more aware of their impact to users. That you should push hard to leave out unnecessary details. That it is important to pay attention to detail. That building software might be hard, but using software shouldn’t.

It is his heritage that every creative person on earth will feel Steve looking over his shoulder to push him just a little bit further at each detail. It will result in better products everywhere.

The ripple Steve caused in our industry will at least ripple on for a few decades throughout new products and technologies and will be the basis for how our children communicate, are educated, and perceive life.

What changed today is that we need to figure it out by ourselves, without his vision. That is scary, but we might just be old enough now ;)

Thanks for teaching us.

Chords + Lyrics

2011-10-04T22:00:00Z

It’s been quite a while since I took the time to update this blog. Many things have happened in the meanwhile, though. The most important happening for me is that I launched an iPad app and I founded a company called 3rd Cloud last week.

Hello, Chords + Lyrics!

An annoying problem amateur musicians might be familiar with is that chords or tablature websites all look very differently, format their song data in various formats, and oftentimes are just plain ugly. Oh, and they’re generally paved with ads, too. So to scratch our own itches, I teamed up with @jr00n from StudioWolff to create Chords + Lyrics.

Chords + Lyrics is a simple music manager for your iPad that allows you to easily import songs and lyrics from your favorite chords website (that means: any website—it recognizes the chords semantically):

Once imported, the chords become editable objects in the form of bubbles, which makes it easy for you to edit or finetune the imported songs. A carefully selected choice of fonts is used to create a readable and uniform look and feel for your song’s chords and lyrics:

Furthermore, once the editing is done, you can simply take Chords + Lyrics on stage with your band or solo performance and leave your stack of sheet music at home. When the device is rotated into landscape orientation, the user interface transforms into a big music stand that arranges the songs of choice as a stack of virtual music sheets in the order of your playlist:

You can get a sneak peak of the app by watching this video:

The app is sold at $5.99. Check us out in the App Store.

The Role of Appsterdam

Jeroen and I met during iOS Dev Camp last March and immediately were excited with the idea for Chords + Lyrics. We even received the Most Likely to Succeed award from Dom Sogolla and Mike Lee at the end of those two days, which got us even more excited about the project.

We started hacking away at it in our spare time for the next few months. While at the same time, by some kind of lucky coincidence, the same Mike Lee started building an awesome developer community for app developers: Appsterdam.

Many of our thanks therefore go out to all of the Appsterdammers, in particular Mike and Judy, for inspiring us at the moments we got stuck and for the helpful pieces of advice we got from them. Without the Appsterdam community, the project may have never seen the light of day, or be much less awesome.

Let this be the first of many apps!

A git-flow screencast

2011-01-26T23:00:00Z

Mr. Dave Bock of Code Sherpa’s put together a nice screencast demonstrating a few of the most important git-flow features on their publications page.

Many thanks to Dave for creating this!

Survey: Python vs Ruby Test Ecosystems

2010-11-15T23:00:00Z

tl;dr Please help out and take the survey. It only takes one or two minutes.

In my spare time, I love to read up on many different topics that are happening in the modern world of software development. On many occasions, the initiatives that I find most interesting are happening in the Python and Ruby communities.

I immediately have to admit that I’m not too actively involved in the Ruby community yet. It is a language that I don’t have much “handson experience” with. Although I think the Ruby community is a bit more chaotic than Python’s, I also believe that more exciting things are happening there. Ruby definitely got the more snazzy and sexier appeal of the two, but I also think it’s the more naive brother of the two that gets in trouble in its teen years (think drugs and jail).

Lovable and Loathsome Language Features

I have always found it really hard to express what I dislike about Ruby. In my college years, I really enjoyed functional programming and I absolutely adore the influences Ruby got from that world. Solving programming problems with the typical Ruby language constructs like blocks is very satisfying and feels really elegant. Next to that, Ruby makes a gorgeous language for creating DSLs.

Yet, the “black magic” and the implicit behaviour that also accompany Ruby give me the eerie feeling that I’m not in control.

When I stumbled upon this video by Gary Bernhardt: Python vs Ruby: A Battle to The Death, all the pieces of the puzzle fell together for me. In this talk, Gary puts to words what my feelings about the two languages are to a great extent. Result: I love both and I hate both.

Uncovering Test Ecosystems

Especially the conclusions Gary draws about RSpec being a superiour specification language is something I share. There simply is no good equivalent of RSpec for Python. In fact, it’s impossible to ever create one in Python, due to Python’s inability to inject methods into object and its lack of blocks, and therefore, its lack of DSL capabilities. (Oh yes, there have been attempts. None of them are as good as the real thing.)

This made me think. Does Ruby have a more mature testing culture? If I browse around interesting projects on Github, for example, I always get the feeling that the Ruby projects have better tests and better coverage than the Python ones (sorry, no scientific proof here). <bait>Maybe they have to because of the magic in Ruby? ;)</bait> But most importantly, the Ruby culture seem to care more (or at least have higher awareness) about actually testing their code at all.

At the same time, I get the feeling that the Ruby culture has less testing tools available. Maybe this is because I simply don’t know the Ruby community as well as I know Python’s, but maybe there has been a lot more consensus and standardisation already.

Only one way to find out

This made me curious, so I decided to pull up a quick survey. I hereby invite all Python and Ruby developers out there to participate in it. Please help spread the word on Twitter to get as many participants as possible. The more answers received, the stronger the map of available and popular tools will be. I’ve also included a few questions on the use of continuous build and integration systems. After taking the survey you can see the results so far.

I’ll make sure to blog about any interesting conclusions that can be drawn from the numbers that I will gather with this.

How I boosted my Vim

2010-09-13T22:00:00Z

A few weeks ago, I felt inspired by articles from Jeff Kreeftmeijer and Armin Ronacher. I took some time to configure and fine-tune my Vim environment. A lot of new stuff made it into my .vimrc file and my .vim directory. This blog post is a summary describing what I’ve added and how I use it in my daily work.

Before doing anything else, make sure you have the following line in your .vimrc file:


" This must be first, because it changes other options as side effect
set nocompatible

Step 0: make the customization process easier

Before starting configuring, it’s useful to install pathogen. Plugins in Vim are files that you drop in subdirectories of your .vim/ directory. Many plugins exist of only a single file that should be dropped in .vim/plugin, but some exist of multiple files. For example, they come with documentation, or ship syntax files. In those cases, files need to be dropped into .vim/doc and .vim/syntax. This makes it difficult to remove the plugin afterwards. After installing pathogen, you can simply unzip a plugin distribution into .vim/bundle/myplugin, under which the required subdirectories are created. Removing the plugin, then, is as simple as removing the myplugin directory.

So, download pathogen.vim, move it into the .vim/autoload directory (create it if necessary) and add the following lines to your .vimrc, to activate it:


" Use pathogen to easily modify the runtime path to include all
" plugins under the ~/.vim/bundle directory
call pathogen#helptags()
call pathogen#runtime_append_all_bundles()

Next, I’ve remapped the leader key to , (comma) instead of the default \ (backslash), just because I like it better. Since in Vim’s default configuration, almost every key is already mapped to a command, there needs to be some sort of standard “free” key where you can place custom mappings under. This is called the “mapleader”, and can be defined like this:


" change the mapleader from \ to ,
let mapleader=","

Once that is done, this is a little tweak that is a time-saver while you’re building up your .vimrc. Here, we start using the leader key:


" Quickly edit/reload the vimrc file
nmap <silent> <leader>ev :e $MYVIMRC<CR>
nmap <silent> <leader>sv :so $MYVIMRC<CR>

This effectively maps the ,ev and ,sv keys to edit/reload .vimrc. (I got this from Derek Wyatt’s .vimrc file.)

Change Vim behaviour

One particularly useful setting is hidden. Its name isn’t too descriptive, though. It hides buffers instead of closing them. This means that you can have unwritten changes to a file and open a new file using :e, without being forced to write or undo your changes first. Also, undo buffers and marks are preserved while the buffer is open. This is an absolute must-have.


set hidden

These are some of the most basic settings that you probably want to enable, too:


set nowrap        " don't wrap lines
set tabstop=4     " a tab is four spaces
set backspace=indent,eol,start
                  " allow backspacing over everything in insert mode
set autoindent    " always set autoindenting on
set copyindent    " copy the previous indentation on autoindenting
set number        " always show line numbers
set shiftwidth=4  " number of spaces to use for autoindenting
set shiftround    " use multiple of shiftwidth when indenting with '<' and '>'
set showmatch     " set show matching parenthesis
set ignorecase    " ignore case when searching
set smartcase     " ignore case if search pattern is all lowercase,
                  "    case-sensitive otherwise
set smarttab      " insert tabs on the start of a line according to
                  "    shiftwidth, not tabstop
set hlsearch      " highlight search terms
set incsearch     " show search matches as you type

There is a lot more goodness in my .vimrc file, which is put in there with a lot of love. I’ve commented most of it, too. Feel free to poke around in it.

Also, I like Vim to have a large undo buffer, a large history of commands, ignore some file extensions when completing names by pressing Tab, and be silent about invalid cursor moves and other errors.


set history=1000         " remember more commands and search history
set undolevels=1000      " use many muchos levels of undo
set wildignore=*.swp,*.bak,*.pyc,*.class
set title                " change the terminal's title
set visualbell           " don't beep
set noerrorbells         " don't beep

Oh, and man… never ever let Vim write a backup file! They did that in the 70’s. Use modern ways for tracking your changes, for God’s sake.


set nobackup
set noswapfile

There have been some passionate responses about this in comments, so a warning may be appropriate here. If you care about recovering after a Vim or terminal emulator crash, or you often load huge files into memory, do not disable the swapfile. I personally save/commit so often that the swap file adds nothing. Sometimes I conciously kill a terminal forcefully, and I only find the swap file recovery process annoying.

Use file type plugins

Vim can detect file types (by their extension, or by peeking inside the file). This enabled Vim to load plugins, settings or key mappings that are only useful in the context of specific file types. For example, a Python syntax checker plugin only makes sense in a Python file. Finally, indenting intelligence is enabled based on the syntax rules for the file type.


filetype plugin indent on

To set some file type specific settings, you can now use the following:


autocmd filetype python set expandtab

To remain compatible with older versions of Vim that do not have the autocmd functions, always wrap those functions inside a block like this:


if has('autocmd')
    ...
endif

Enable syntax highlighting

Somewhat related to the file type plugins is the syntax highlighting of different types of source files. Vim uses syntax definitions to highlight source code. Syntax definitions simply declare where a function name starts, which pieces are commented out and what are keywords. To color them, Vim uses colorschemes. You can load custom color schemes by placing them in .vim/colors, then load them using the colorscheme command. You have to try what you like most. I like mustang a lot.


if &t_Co >= 256 || has("gui_running")
   colorscheme mustang
endif

if &t_Co > 2 || has("gui_running")
   " switch syntax highlighting on, when the terminal has colors
   syntax on
endif

In this case, mustang is only loaded if the terminal emulator Vim runs in supports at least 256 colors (or if you use the GUI version of Vim).

Hint: if you’re using a terminal emulator that can show 256 colors, try setting TERM=xterm-256color in your terminal configuration or in your shell’s .rc file.

Change editing behaviour

When you write a lot of code, you probably want to obey certain style rules. In some programming languages (like Python), whitespace is important, so you may not just swap tabs for spaces and even the number of spaces is important.

Vim can highlight whitespaces for you in a convenient way:


set list
set listchars=tab:>.,trail:.,extends:#,nbsp:.

This line will make Vim set out tab characters, trailing whitespace and invisible spaces visually, and additionally use the # sign at the end of lines to mark lines that extend off-screen. For more info, see :h listchars.

In some files, like HTML and XML files, tabs are fine and showing them is really annoying, you can disable them easily using an autocmd declaration:


autocmd filetype html,xml set listchars-=tab:>.

One caveat when setting listchars: if nothing happens, you have probably not enabled list, so try :set list, too.

Pasting large amounts of text into Vim

Every Vim user likes to enable auto-indenting of source code, so Vim can intelligently position you cursor on the next line as you type. This has one big ugly consequence however: when you paste text into your terminal-based Vim with a right mouse click, Vim cannot know it is coming from a paste. To Vim, it looks like text entered by someone who can type incredibly fast :) Since Vim thinks this is regular key strokes, it applies all auto-indenting and auto-expansion of defined abbreviations to the input, resulting in often cascading indents of paragraphs.

There is an easy option to prevent this, however. You can temporarily switch to “paste mode”, simply by setting the following option:


set pastetoggle=<F2>

Then, when in insert mode, ready to paste, if you press <F2>, Vim will switch to paste mode, disabling all kinds of smartness and just pasting a whole buffer of text. Then, you can disable paste mode again with another press of <F2>. Nice and simple. Compare paste mode disabled vs enabled:

Another great trick I read in a reddit comment is to use <C-r>+ to paste right from the OS paste board. Of course, this only works when running Vim locally (i.e. not over an SSH connection).

Enable the mouse

While using the mouse is considered a deadly sin among Vim users, there are a few features about the mouse that can really come to your advantage. Most notably—scrolling. In fact, it’s the only thing I use it for.

Also, if you are a rookie Vim user, setting this value will make your Vim experience definitively feel more natural.

To enable the mouse, use:


set mouse=a

However, this comes at one big disadvantage: when you run Vim inside a terminal, the terminal itself cannot control your mouse anymore. Therefore, you cannot select text anymore with the terminal (to copy it to the system clipboard, for example).

To be able to have the best of both worlds, I wrote this simple Vim plugin: vim-togglemouse. It maps <F12> to toggle your mouse “focus” between Vim and the terminal.

Small plugins like these are really useful, yet have the additional benefit of lowering the barrier of learning the Vim scripting language. At the core, this plugin exists of only one simple function:


fun! s:ToggleMouse()
    if !exists("s:old_mouse")
        let s:old_mouse = "a"
    endif

    if &mouse == ""
        let &mouse = s:old_mouse
        echo "Mouse is for Vim (" . &mouse . ")"
    else
        let s:old_mouse = &mouse
        let &mouse=""
        echo "Mouse is for terminal"
    endif
endfunction

Get efficient: shortcut mappings

The following trick is a really small one, but a super-efficient one, since it strips off two full keystrokes from almost every Vim command:


nnoremap ; :

For example, to save a file, you type :w normally, which means:

Press and hold Shift
Press ;
Release the Shift key
Press w
Press Return

This trick strips off steps 1 and 3 for each Vim command. It takes some times for your muscle memory to get used to this new ;w command, but once you use it, you don’t want to go back!

I also find this key binding very useful, since I like to reformat paragraph text often. Just set your cursor inside a paragraph and press Q (or select a visual block and press Q).


" Use Q for formatting the current paragraph (or selection)
vmap Q gq
nmap Q gqap

If you are still getting used to Vim and want to force yourself to stop using the arrow keys, add this:


map <up> <nop>
map <down> <nop>
map <left> <nop>
map <right> <nop>

If you like long lines with line wrapping enabled, this solves the problem that pressing down jumpes your cursor “over” the current line to the next line. It changes behaviour so that it jumps to the next row in the editor (much more natural):


nnoremap j gj
nnoremap k gk

When you start to use Vim more professionally, you want to work with multiple windows open. Navigating requires you to press C-w first, then a navigation command (h, j, k, l). This makes it easier to navigate focus through windows:


" Easy window navigation
map <C-h> <C-w>h
map <C-j> <C-w>j
map <C-k> <C-w>k
map <C-l> <C-w>l

Tired of clearing highlighted searches by searching for “ldsfhjkhgakjks”? Use this:


nmap <silent> ,/ :nohlsearch<CR>

I used to have it mapped to :let @/=""<CR>, but some users kindly pointed out that it is better to use :nohlsearch, because it keeps the search history intact.

It clears the search buffer when you press ,/

Finally, a trick by Steve Losh for when you forgot to sudo before editing a file that requires root privileges (typically /etc/hosts). This lets you use w!! to do that after you opened the file already:


cmap w!! w !sudo tee % >/dev/null

Use plugins

Ah, finally. Arrived at the magical stuff that is Vim plugins. This is a listing of the Vim plugins I depend on most and that really offer added value when you work with Vim every day.

Almost any person I know who owns a Mac has at least tried or purchased the TextMate editor. It is a great programmer’s editor that has a lot of nice features, but of course, lacks Vim-style navigation :)

Some of its features have inspired Vim plugin developers to clone the awesomeness. TextMate’s best two features (super-quick file opening and snippets) have been “ported” to Vim plugins.

Command-T: TextMate-style file opening NERDTree explorer

The NERDTree plugin is an absolute must for almost any Vim user. For those who don’t know it yet, NERDTree is a visual file browser that allows you to quickly open files by navigating onto it and pressing Return.

You can open the NERDTree using the :NERDTree command, or by using :NERDTreeToggle. To close the tree window, use :NERDTreeClose.

I have I’ve mapped some shortcuts to these commands, for quick access to the list:


nmap ,n :NERDTreeCloseCR:NERDTreeToggleCR
nmap ,m :NERDTreeCloseCR:NERDTreeFindCR
nmap ,N :NERDTreeCloseCR

You can add quick bookmarks to directories that you often visit (like project directories), which works really well. I have configured the following settings, which tells the plugin where the bookmark file is, which extensions to ignore, to show hidden files and to quit on open (which I like to have on, to have screen real estate for my code):


" Store the bookmarks file
let NERDTreeBookmarksFile=expand("$HOME/.vim/NERDTreeBookmarks")

" Don't display these kinds of files
let NERDTreeIgnore=[ '\.pyc$', '\.pyo$', '\.py\$class$', '\.obj$',
            \ '\.o$', '\.so$', '\.egg$', '^\.git$' ]

let NERDTreeShowBookmarks=1       " Show the bookmarks table on startup
let NERDTreeShowFiles=1           " Show hidden files, too
let NERDTreeShowHidden=1
let NERDTreeQuitOnOpen=1          " Quit on opening files from the tree
let NERDTreeHighlightCursorline=1 " Highlight the selected entry in the tree
let NERDTreeMouseMode=2           " Use a single click to fold/unfold directories
                                  " and a double click to open files

You can add quick bookmarks to directories that you often visit (like project directories), which works really well, too.

More documentation available at: the plugin page.

I used to have a whole discussion on the NERDTree explorer plugin here, but I was pointed towards Command-T by a few readers. Once I tried that plugin for a few minutes, it became clear that there would be no need for NERDTree anymore.

The name Command-T is a reference to the original shortcut for “Go to File” in TextMate, which opens the quick file opener. Setting up this plugin is a bit more tricky than usual, because some files need to be compiled, but the instructions on the website are very clear and simple. There’s even a screencast for Windows users.

The plugin registers itself under leadert (in my case that’s ,t). And if it does, it shows a list of all the files from the current directory and a search box in which you can type any character progression. By typing more characters, the list of files is narrowed down to contain only files that have that string as a subset. But here’s the real added value: you can type any sequence of characters that are in the file’s path, they don’t have to be subsequent characters. For example, if you have the following list of files:

foo/bar.py
foo/qux.py
tests/test_foo/test_bar.py
tests/test_foo/test_qux.py

You could type the sequence fb to select the file foo/bar.py, or tfb to select tests/test_foo/test_bar.py. As you can imagine, this is a huge time saver. There is much more goodness in there. For that, please refer to Wincent’s screencasts, or his website.

You can download the plugin at vim.org.

Note: On Vim instances that don’t have Ruby support enabled (type :version to check this), the Commant-T plugin won’t work. If it isn’t an option to recompile Vim to add Ruby support, the NERDTree explorer still is a good alternative.

Snipmate: TextMate-style snippets for Vim

Another killer feature of TextMate are the intelligent snippets: you type some text, press Tab and TextMate creates a snippet for you, with placeholders at key positions within the snippet. The first placeholder is selected, your overwrite it with the text you want, then press Tab to select the second placeholder, etc. This lets you enter code super fast.

Now it’s available to us Vim users, too, in the form of the snipMate plugin (dubbed after TextMate). There’s a great introductory screencast to get you excited about it.

Writing your own snippets

There are already lots of specific language ~~plugins~~ snippets for snipMate available, but writing your own is simple in case you ever miss functionality. Just create a file called ~/.vim/snippets/foo.snippet, where foo is the filetype you want to load the snippets for.

Then, declare your snippets. That’s really as easy as this:

snippet def
    def ${1:fname}(${2:arg}):
        ${3:pass}

If you declare this, you can simply use defTab and it will expand to a Python function definition. The ${1:fname} means put the first placeholder at this position, and fill it with a default value of “fname”.

Other cool plugins

In order to make the article not any more longer than it already is, here’s a list of other plugins that are really worth checking out (I use each of them regularly):

localrc: lets you load specific Vim settings for any file in the same directory (or a subdirectory thereof). Comes in super handy for project-wide settings.
Sparkup: write HTML lightning fast by jotting down a “CSS selector”-like line and pressing a shortcut. Super time saver when you edit HTML manually. Go see the demo video.
YankRing: quickly cycle through your registers when pasting, to select the appropriate paste. How often have you “cut” a line, then “deleted” a line, only to find the “deleted” line in your last paste register?
Pastie: lets you visually select a piece of code and submit it to pastie.org. It even has automatic filetype recognition.
surround: quickly change surroundings of a piece of code. For example, to change dict(mykey) into dict[mykey], put the cursor on mykey, then type cs(] (change surroundings from () to []).
repeat: lets you repeat the changing of surroundings using the default Vim repeat operator . (dot).
abolish: bulk-define autocorrections (abbreviations) for many conjugations of words and smart conversion of words from snake_case (crs) to camelCase (cr_) or MixedCase (crm) to UPPER_CASE (cru) and vice versa.
LustyJuggler: a quick buffer switcher and a great companion to Command-T. See this screencast (Lococast) to get excited.

Other resources

Some of the resources from where I have collected inspiration for my .vimrc file, plugins, and tricks:

Vimcasts
Lococast
Derek Wyatt’s videos (on Vimeo)
Steve Losh blogged about moving back to Vim and has some great tips and tricks.

I hope you like these tips. Please add your comments below if you do, or if you have any Vim improvements or tips you’d like to bring on yourself. You can have a look at my full Vim configuration in my Github repo.

A whole new blog

2010-09-12T22:00:00Z

Finally, I’ve made the move to a static blog engine! I’m using nanoc now (bye bye WordPress). nanoc is a very flexible and customizable static site generator, written by Denis Defreyne.

As with all static site generators, nanoc lets you write your source files in a simple markup language. Out of the box, nanoc offers you the choice of using Markdown, Textile, reStructuredText or plain HTML (with or without embedded Ruby). In fact, nanoc is nothing more than a generator honoring a Rules-file that tells it how to compile, layout and route the site’s items.

Compiling items

An “item” is a file on your website. It can be any kind of file, like a web site page (HTML), an image, a JavaScript or CSS file or an RSS feed. During the compile phase, you specify which sequential actions should be performed on the content of that item. These actions are called filters. Some examples of filters are an embedded ruby filter, a Textile-to-HTML converter, a less compiler, or minify CSS. Filters can be chained, for example:


compile '/static/css/*/' do
    # compress CSS :)
    filter :less
    filter :rainpress
end

Which turns .less-files into compressed CSS:

Any filter you can imagine, nanoc can handle. nanoc comes with a lot of filters out of the box, but even writing your own filters really is a piece of cake.

Routing items

After compiling (i.e. transforming content through filters) comes the routing of the items. This is a means of assigning file names to compiled content. nanoc calculates default files names from the input, but you can use this to influence the default naming. A special case is where you set the route to Nil which doesn’t write the file at all. I use this to test draft posts locally, like this (oh, did I mention the Rules file is 100% Ruby?):


route '/posts/*/' do
    if $include_drafts or @item[:published] then
        '/posts/' + @item.slug + '/index.html'
    end
end

Laying out items

Finally, layouts are applied. Layouts are kind of templates that can be used to “frame” the item’s contents. This is typically used for HTML files only, but isn’t limited to it. For example, the blog posts are compiled into (partial) HTML, and the layout rules put the site’s container HTML around it, adding CSS styling, jQuery scripts, the header, sidebars and footer and Google Analytics tracking (these go for each page). There’s a special extra layout rule for blog post pages, which additionally adds Disqus comments.

Summary

Each build of this blog also automatically:

Converts Textile content to HTML
Highlights syntax using pygments
Converts less to CSS
Minifies CSS
Minifies JavaScript
Downsizes source images
Generates redirect pages for alternative (old-style) URL’s (for user that have existing bookmarks to old WordPress URL’s)
Generates a new blog post RSS feed

In short, now nanoc is fully configured to my wishes, I can simply focus on writing blog content, without preparing image content (it is done automatically), and without having to choose between either a “WYSIWYG” editor or writing HTML manually. And I can do it in an offline fashion, too, which was one my main complaints about WordPress.

So I’m happy.

Oh, and since I have been converting my blog anyway, I also created a new look and feel for it. I hope you like it. Feel free to comment.

An upgrade of gitflow

2010-03-03T23:00:00Z

Last week, I silently tagged gitflow 0.2. The most important changes since 0.1 are:

Order of arguments changed to have a more “gitish” subcommand structure. For example, you now say:
```
git flow feature start myfeature
```
Better initializer. git flow init now prompts interactively to set up a gitflow enabled repo.
Added a command to list all feature/release/hotfix/support branches, e.g.:
```
git flow feature list
```
Made all merge/rebase operations failsafe, providing a non-destructive workflow in case of merge conflicts.
Easy diff’ing of all changes on a specific (or the current) feature branch:
```
git flow feature diff [feature]
```
Add support for feature branch rebasing:
```
git flow feature rebase
```
Some subactions now take name prefixes as their arguments, for convenience. For example, if you have feature branches called “experimental”, “refactoring” and “feature-X”, you could say:
```
git flow feature finish ref
```
And gitflow will know you mean the “refactoring” feature branch.
These actions are: finish, diff and rebase.
Much better overall sanity checking.
Better portability (POSIX compliant code)
Better (more portable) flag parsing using Kate Ward’s shFlags.
Improved installer. To install git flow as a first-class Git subcommand, simply type:
```
sudo make install
```

Major and minor bug fixes.

That’s all for now.

Unexpected side effects in Python classes

2010-03-02T23:00:00Z

Today, I lost several hours while debugging a language implementation detail in Python that I did not know of and that really feels counterintuitive and dangerous to me.

I was writing unit tests for a Python class that I was implementing, when one of the tests that had repeatedly been passing suddenly failed. Moreover, the failing test case was really for testing some completely unrelated piece of functionaly. This simply could not be broken!

After at least an hour of scrutinizing the code, I was able to distill the real problem, which I think is summarized here in the most compact way:


class Foo:
  x = {}
  
  def __init__(self, id):
    self.x[id] = id

f1 = Foo(5)
print f1.x     # {5:5}, as expected
f2 = Foo(6)
print f2.x     # {5:5,6:6} ?!

Creating a simple Foo instance twice exposes the ugly side effect: the second Foo instance has an already initialized x instance variable when the constructor enters! Yuck! Moreover, now, too:


print f1.x     # {5:5,6:6}, too!

Apparently, the x “instance variable” is a shared object, much like a global or class variable.

To be even more confusing, this doesn’t seem to hold for basic data types. For example, change the dictionary to an integer, and the example behaves as expected:


class Foo:
  x = 3

  def __init__(self, id):
    self.x = id

f1 = Foo(5)
print f1.x     # 5, as expected
f2 = Foo(6)
print f2.x     # 6, as expected

The behaviour demystified

The real confusion here is that I was thinking that I was creating “instance variables”, like you would in C++ or Java. As the Python documentation mentions:

“data attributes correspond to […] to data members in C++. Data attributes need not be declared; like local variables, they spring into existence when they are first assigned to.”

Yes, I knew that, but nonetheless my real-world class is much bigger than Foo and I wanted an explicit overview on which instance variables are in this class. Hence the data member.

However, this is not how the Python interpreter processes Python code. In fact, upon class definition, the statement x = {} is executed within the scope of the newly defined class. To prove this:


class Bar:
  x = {}

print Bar.x     # {}

Even without a constructor or instance variable, we can access the data member x. Of course. Now this suddenly seems obvious.

But what about our instance variables? Apparently, when we create a new instance of Bar, the instance data member x is initially pointing to the same object as the class data member x. The following example proves this:


class Foo:
  x = {}
  
  def __init__(self, id):
    self.x = { id: id }

f1 = Foo(5)
print f1.x     # { 5:5 }, as expected
f2 = Foo(6)
print f2.x     # { 6:6 }, as expected
print Foo.x    # {}, didn't intend this!

This example also demonstrates the subtlety of the accidentally discovered side-effect. Remember how we were changing the dictionary in our initial example? self.x[id] = id
The instance data member was pointing to the same object as the class data member. By updating the dictionary, the single dictionary object was changed, causing unwanted side effects in other class instances.

In the listing above, x is forced to point to a new dictionary by the assignment self.x = { id:id }. In other words, x points to a new object! This also perfectly explains why the integer example worked—it’s the same kind of assignment.

Conclusion

To summarize, I learned some important lessons today:

All the time, I have been creating class data members in all my classes, without knowing this.
I initialized those members to default values, effectively creating useless objects that are never accessed and just claiming memory.
Although it can be explained, a seemingly innocent statement like x = {} can have very ugly side effects. Be warned!
Never underestimate the power of unit tests. It is absolutely worth the investment.

gitflow 0.1 released

2010-01-25T23:00:00Z

After the overwhelming attention and feedback on the Git branching model post, a general consensus was that this workflow would benefit from some form of proper scriptability. The workflow works seamlessly if you perform the steps involved manually, but hey… manually is manually, really.

UPDATE 2/4/2010: Anyone reading this: I recommend NOT USING this very early release, but to jump on the current develop tip, which is much more mature. Release 0.2 is coming very soon.

An assisting tool (dubbed gitflow) was therefore created to provide simple, high-level commands to adopt the workflow into your own software development process. It’s free and it’s open source. Feel free to contribute to it if you like.

Fork me on Github: http://github.com/nvie/gitflow

Since this morning, the first working release 0.1 was tagged, albeit very basic.

A quick walkthrough

The gitflow script essentially features six subcommands: paired start/finish commands for managing the different types of branches from the originating article:

Feature branches:
- gitflow start feature myfeature
- gitflow finish feature myfeature

Release branches:
- gitflow start release version-id
- gitflow finish release version-id

Hotfix branches:
- gitflow start hotfix version-id
- gitflow finish hotfix version-id

Each of these scripts exactly reports what actions were taken and what follow-up actions are required by the user. This output will be polished in future versions to improve the UX. An example output:


$ gitflow finish feature foo
Branches 'develop' and 'origin/develop' have diverged.
And local branch 'develop' is ahead of 'origin/develop'.
Switched to branch "develop"
Your branch is ahead of 'origin/develop' by 12 commits.
Merge made by recursive.
 README |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
Deleted branch foo (cd3effb).
 
Summary of actions:
- The feature branch 'foo' was merged into 'develop'
- Feature branch 'foo' has been removed
- You are now on branch 'develop'

Limitations

The script is very limited at the moment yet, but future versions will fix that, too. Some of the main limitations:

Branch names (master, develop) and the remote repo name (origin) are currently fixed.
There is no support for dealing with merge conflicts yet.
There is no support for support-* branches (see the original comment that proposed this extension)
There is no documentation.
There is no installer.

However, as this post is written, some of the limitations are already taken care of by community members. Power to the open source!

Building Git from scratch on Snow Leopard

2010-01-12T23:00:00Z

When you try to build Git from scratch on a Snow Leopard machine, you may have ran into the following problem:


$ git clone git://github.com/git/git.git
Initialized empty Git repository in /Users/nvie/git/.git/
remote: Counting objects: 111619, done.
remote: Compressing objects: 100% (28007/28007), done.
remote: Total 111619 (delta 82192), reused 111264 (delta 81852)
Receiving objects: 100% (111619/111619), 27.60 MiB | 531 KiB/s, done.
Resolving deltas: 100% (82192/82192), done.
$ cd git
$ make prefix=/usr/local/bin/git
GIT_VERSION = 1.6.2.rc0.90.g0753
    * new build flags or prefix
    CC fast-import.o
    CC abspath.o
    :
    :
ld: warning: in /opt/local/lib/libexpat.dylib, file is not of required architecture
    ...
ld: symbol(s) not found
collect2: ld returned 1 exit status

Then, you have a pretty big change you are having an old Darwin ports (macports) collection in use which has not yet been upgraded to Snow Leopard’s new x64 architecture.

There is, however, a simple solution to this, namely to have the git build ignore the Darwin ports, simply by adding the following parameter to the build:


$ make NO_DARWIN_PORTS=1 prefix=/usr/local/bin/git

A successful Git branching model

2010-01-04T23:00:00Z

In this post I present the development model that I’ve introduced for all of my projects (both at work and private) about a year ago, and which has turned out to be very successful. I’ve been meaning to write about it for a while now, but I’ve never really found the time to do so thoroughly, until now. I won’t talk about any of the projects’ details, merely about the branching strategy and release management.

It focuses around Git as the tool for the versioning of all of our source code.

Why git?

For a thorough discussion on the pros and cons of Git compared to centralized source code control systems, see the web. There are plenty of flame wars going on there. As a developer, I prefer Git above all other tools around today. Git really changed the way developers think of merging and branching. From the classic CVS/Subversion world I came from, merging/branching has always been considered a bit scary (“beware of merge conflicts, they bite you!”) and something you only do every once in a while.

But with Git, these actions are extremely cheap and simple, and they are considered one of the core parts of your daily workflow, really. For example, in CVS/Subversion books, branching and merging is first discussed in the later chapters (for advanced users), while in every Git book, it’s already covered in chapter 3 (basics).

As a consequence of its simplicity and repetitive nature, branching and merging are no longer something to be afraid of. Version control tools are supposed to assist in branching/merging more than anything else.

Enough about the tools, let’s head onto the development model. The model that I’m going to present here is essentially no more than a set of procedures that every team member has to follow in order to come to a managed software development process.

Decentralized but centralized

The repository setup that we use and that works well with this branching model, is that with a central “truth” repo. Note that this repo is only considered to be the central one (since Git is a DVCS, there is no such thing as a central repo at a technical level). We will refer to this repo as origin, since this name is familiar to all Git users.

Each developer pulls and pushes to origin. But besides the centralized push-pull relationships, each developer may also pull changes from other peers to form sub teams. For example, this might be useful to work together with two or more developers on a big new feature, before pushing the work in progress to origin prematurely. In the figure above, there are subteams of Alice and Bob, Alice and David, and Clair and David.

Technically, this means nothing more than that Alice has defined a Git remote, named bob, pointing to Bob’s repository, and vice versa.

The main branches

At the core, the development model is greatly inspired by existing models out there. The central repo holds two main branches with an infinite lifetime:

master
develop

The master branch at origin should be familiar to every Git user. Parallel to the master branch, another branch exists called develop.

We consider origin/master to be the main branch where the source code of HEAD always reflects a production-ready state.

We consider origin/develop to be the main branch where the source code of HEAD always reflects a state with the latest delivered development changes for the next release. Some would call this the “integration branch”. This is where any automatic nightly builds are built from.

When the source code in the develop branch reaches a stable point and is ready to be released, all of the changes should be merged back into master somehow and then tagged with a release number. How this is done in detail will be discussed further on.

Therefore, each time when changes are merged back into master, this is a new production release by definition. We tend to be very strict at this, so that theoretically, we could use a Git hook script to automatically build and roll-out our software to our production servers everytime there was a commit on master.

Supporting branches

Next to the main branches master and develop, our development model uses a variety of supporting branches to aid parallel development between team members, ease tracking of features, prepare for production releases and to assist in quickly fixing live production problems. Unlike the main branches, these branches always have a limited life time, since they will be removed eventually.

The different types of branches we may use are:

Feature branches
Release branches
Hotfix branches

Each of these branches have a specific purpose and are bound to strict rules as to which branches may be their originating branch and which branches must be their merge targets. We will walk through them in a minute.

By no means are these branches “special” from a technical perspective. The branch types are categorized by how we use them. They are of course plain old Git branches.

Feature branches

May branch off from: develop
Must merge back into: develop
Branch naming convention: anything except master, develop, release-*, or hotfix-*

Feature branches (or sometimes called topic branches) are used to develop new features for the upcoming or a distant future release. When starting development of a feature, the target release in which this feature will be incorporated may well be unknown at that point. The essence of a feature branch is that it exists as long as the feature is in development, but will eventually be merged back into develop (to definitely add the new feature to the upcoming release) or discarded (in case of a disappointing experiment).

Feature branches typically exist in developer repos only, not in origin.

Creating a feature branch

When starting work on a new feature, branch off from the develop branch.


$ git checkout -b myfeature develop
Switched to a new branch "myfeature"

Incorporating a finished feature on develop

Finished features may be merged into the develop branch definitely add them to the upcoming release:


$ git checkout develop
Switched to branch 'develop'
$ git merge --no-ff myfeature
Updating ea1b82a..05e9557
(Summary of changes)
$ git branch -d myfeature
Deleted branch myfeature (was 05e9557).
$ git push origin develop

The --no-ff flag causes the merge to always create a new commit object, even if the merge could be performed with a fast-forward. This avoids losing information about the historical existence of a feature branch and groups together all commits that together added the feature. Compare:

In the latter case, it is impossible to see from the Git history which of the commit objects together have implemented a feature—you would have to manually read all the log messages. Reverting a whole feature (i.e. a group of commits), is a true headache in the latter situation, whereas it is easily done if the --no-ff flag was used.

Yes, it will create a few more (empty) commit objects, but the gain is much bigger that that cost.

Unfortunately, I have not found a way to make --no-ff the default behaviour of git merge yet, but it really should be.

Release branches

May branch off from: develop
Must merge back into: develop and master
Branch naming convention: release-*

Release branches support preparation of a new production release. They allow for last-minute dotting of i’s and crossing t’s. Furthermore, they allow for minor bug fixes and preparing meta-data for a release (version number, build dates, etc.). By doing all of this work on a release branch, the develop branch is cleared to receive features for the next big release.

The key moment to branch off a new release branch from develop is when develop (almost) reflects the desired state of the new release. At least all features that are targeted for the release-to-be-built must be merged in to develop at this point in time. All features targeted at future releases may not—they must wait until after the release branch is branched off.

It is exactly at the start of a release branch that the upcoming release gets assigned a version number—not any earlier. Up until that moment, the develop branch reflected changes for the “next release”, but it is unclear whether that “next release” will eventually become 0.3 or 1.0, until the release branch is started. That decision is made on the start of the release branch and is carried out by the project’s rules on version number bumping.

Creating a release branch

Release branches are created from the develop branch. For example, say version 1.1.5 is the current production release and we have a big release coming up. The state of develop is ready for the “next release” and we have decided that this will become version 1.2 (rather than 1.1.6 or 2.0). So we branch off and give the release branch a name reflecting the new version number:


$ git checkout -b release-1.2 develop
Switched to a new branch "release-1.2"
$ ./bump-version.sh 1.2
Files modified successfully, version bumped to 1.2.
$ git commit -a -m "Bumped version number to 1.2"
[release-1.2 74d9424] Bumped version number to 1.2
1 files changed, 1 insertions(+), 1 deletions(-)

After creating a new branch and switching to it, we bump the version number. Here, bump-version.sh is a fictional shell script that changes some files in the working copy to reflect the new version. (This can of course be a manual change—the point being that some files change.) Then, the bumped version number is committed.

This new branch may exist there for a while, until the release may be rolled out definitely. During that time, bug fixes may be applied in this branch (rather than on the develop branch). Adding large new features here is strictly prohibited. They must be merged into develop, and therefore, wait for the next big release.

Finishing a release branch

When the state of the release branch is ready to become a real release, some actions need to be carried out. First, the release branch is merged into master (since every commit on master is a new release by definition, remember). Next, that commit on master must be tagged for easy future reference to this historical version. Finally, the changes made on the release branch need to be merged back into develop, so that future releases also contain these bug fixes.

The first two steps in Git:


$ git checkout master
Switched to branch 'master'
$ git merge --no-ff release-1.2
Merge made by recursive.
(Summary of changes)
$ git tag -a 1.2

The release is now done, and tagged for future reference.
Edit: You might as well want to use the -s or -u <key> flags to sign your tag cryptographically.

To keep the changes made in the release branch, we need to merge those back into develop, though. In Git:


$ git checkout develop
Switched to branch 'develop'
$ git merge --no-ff release-1.2
Merge made by recursive.
(Summary of changes)

This step may well lead to a merge conflict (probably even, since we have changed the version number). If so, fix it and commit.

Now we are really done and the release branch may be removed, since we don’t need it anymore:


$ git branch -d release-1.2
Deleted branch release-1.2 (was ff452fe).

Hotfix branches

May branch off from: master
Must merge back into: develop and master
Branch naming convention: hotfix-*

Hotfix branches are very much like release branches in that they are also meant to prepare for a new production release, albeit unplanned. They arise from the necessity to act immediately upon an undesired state of a live production version. When a critical bug in a production version must be resolved immediately, a hotfix branch may be branched off from the corresponding tag on the master branch that marks the production version.

The essence is that work of team members (on the develop branch) can continue, while another person is preparing a quick production fix.

Creating the hotfix branch

Hotfix branches are created from the master branch. For example, say version 1.2 is the current production release running live and causing troubles due to a severe bug. But changes on develop are yet unstable. We may then branch off a hotfix branch and start fixing the problem:


$ git checkout -b hotfix-1.2.1 master
Switched to a new branch "hotfix-1.2.1"
$ ./bump-version.sh 1.2.1
Files modified successfully, version bumped to 1.2.1.
$ git commit -a -m "Bumped version number to 1.2.1"
[hotfix-1.2.1 41e61bb] Bumped version number to 1.2.1
1 files changed, 1 insertions(+), 1 deletions(-)

Don’t forget to bump the version number after branching off!

Then, fix the bug and commit the fix in one or more separate commits.


$ git commit -m "Fixed severe production problem"
[hotfix-1.2.1 abbe5d6] Fixed severe production problem
5 files changed, 32 insertions(+), 17 deletions(-)

Finishing a hotfix branch

When finished, the bugfix needs to be merged back into master, but also needs to be merged back into develop, in order to safeguard that the bugfix is included in the next release as well. This is completely similar to how release branches are finished.

First, update master and tag the release.


$ git checkout master
Switched to branch 'master'
$ git merge --no-ff hotfix-1.2.1
Merge made by recursive.
(Summary of changes)
$ git tag -a 1.2.1

Edit: You might as well want to use the -s or -u <key> flags to sign your tag cryptographically.

Next, include the bugfix in develop, too:


$ git checkout develop
Switched to branch 'develop'
$ git merge --no-ff hotfix-1.2.1
Merge made by recursive.
(Summary of changes)

The one exception to the rule here is that, when a release branch currently exists, the hotfix changes need to be merged into that release branch, instead of develop. Back-merging the bugfix into the release branch will eventually result in the bugfix being merged into develop too, when the release branch is finished. (If work in develop immediately requires this bugfix and cannot wait for the release branch to be finished, you may safely merge the bugfix into develop now already as well.)

Finally, remove the temporary branch:


$ git branch -d hotfix-1.2.1
Deleted branch hotfix-1.2.1 (was abbe5d6).

Summary

While there is nothing really shocking new to this branching model, the “big picture” figure that this post began with has turned out to be tremendously useful in our projects. It forms an elegant mental model that is easy to comprehend and allows team members to develop a shared understanding of the branching and releasing processes.

A high-quality PDF version of the figure is provided here. Go ahead and hang it on the wall for quick reference at any time.

Update: And for anyone who requested it: here’s the gitflow-model.src.key of the main diagram image (Apple Keynote).

Git-branching-model.pdf

Feel free to add your comments!

Auto-generate classes for your Core Data data model, revisited

2009-09-18T22:00:00Z

A few months ago, I wrote about automatically generating classes for your Core Data entities and how to automate Xcode using users scripts, such that, when your model changed, you only needed to run your custom script again and your intermediate model files would reflect the new situation.

Well, the guys from the mogenerator project have come up with a far superior solution in the mean time. The newest version of mogenerator comes with an Xcode plugin named Xmo’d, which monitors your *.xcdatamodel file for changes and, as soon as it changes, regenerates all of the neccessary files.

This means that there is officially no more reason not to use mogenerator.

To set it up, download the installer package from their (improved) project website and install it. (Before installing, please read the important release note about the renamed method +newInManagedObjectContext:.)

When installed, all you need to do is Command-click your *.xcdatamodel file, click Get Info, switch to the Comments tab and add the string “xmod” to the comment field. This is the trigger for Xmo’d to start (re)generating your machine-classes (the underscored class files) when the data model changes. Brilliant!

Oh, the default location at which the generated files will be emitted, is in a folder named after your project, right next to where your *.xcdatamodel already sits:

Enjoy it and spread the word!

Automatically generate classes for your Core Data data model

2009-06-29T22:00:00Z

When designing a Core Data data model for your Xcode projects, you can choose to create Objective-C object wrappers for your entities, so that you can profit from type-safe code. The normal, tedious, workflow for this is that you select each entity from the model designer, select all of its attributes and relationships, Ctrl-click it and from the contextual menu first select “Copy Obj-C 2.0 Method Declarations To Clipboard”, paste it into the appropriate class header file, then do the same thing for the method implementations in the class implementation file. Waaaaaay too much work. Not to mention the manual copy-pastes are really hard to keep in sync once you start adding functionality to these class files, since you don’t want to overwrite those additions, but you want to keep replacing everything else.

Meet mogenerator

Fortunately, there is a great way for automating this process, using mogenerator. The tool can be downloaded as a DMG installer (Aral Balkan’s blog mentions a workaround for older Xcode versions, but for Xcode 3.1.3 it worked out of the box for me), or you can checkout the sources from github and build it yourself.

The mogenerator command line tool eases this generation process by reading the *.xcdatamodel file and generating both class files and intermediate class files for each entity. The intermediate classes (called machine classes) are continuously overwritten by subsequent regenerations, so you should never edit the contents of these files. The actual model object classes (called human classes) inherit from those intermediate classes with a default empty implementation, allowing for all manual extensions.

For example, when you design a model with two entities Foo and Bar, mogenerator can be invokes as follows:

mogenerator -m MyDocument.xcdatamodel -M Entities -H Model

The flag -m sets the input model file, while -M and -H specify the output directories where the machine and human classes should be generated respectively.

This does a few things:

In the Entities subdirectory, there will be generated header and implementation files for NSManagedObject subclasses called _Foo and _Bar;
In the Model subdirectory, there will be generated classes called Foo and Bar—respective subclasses of _Foo and _Bar. These are only created if not available yet. Otherwise, they are left as is.

Wrapping it up

The trick of how mogenerator works is that you can run the script as often as you want. After every change in your model, you’ll want to re-run the generation again to update the machine classes. You could easily leave Xcode, switch over to Terminal and issue the command above. But you’ll get quite tired of that after a few times.

Therefore, I’ve written a custom user script that can be added to Xcode (see figure), which does the following:

You can configure the output directories in the first lines of the script. There is no per-project configuration, so choose them as you would like to use them with all your projects;
Mind that these generated files are not automatically included in your Xcode project. Drag them there once and ideally put the machine generated classes into a group under “Other resource”, so you never have to see them again. Whenever you add a new class to your model, new files will be generated, so again you must drag the new files to reference those, of course!
The script can be run with any file in the project opened. It starts out with that file and walks up the directory tree to search for your Xcode project. If found, it executes all the rest from your project directory. (Suggestions are welcome, I could not find a better implementation since a variable like %%%{PBXProjectPath}%%% does not seem to exist.)
It invokes mogenerator to generate all model classes for the project. It is smart enough to detect whether you are using Brian Webster’s BWOrderedManagedObject in your project. If so, your generated machine classes will inherit from BWOrderedManagedObject instead of NSManagedObject.

To add this script to Xcode, open the menu Scripts (the icon) > Edit User Scripts… Click the “+”-button on the bottom-left and select “New shell script”. Set the values for Input, Directory, Output and Errors as in the screenshot above, then copy-paste the script below into the code window. Add a nice keyboard shortcut to this action to top it off :-) I’ve chosen ^⌥⌘G for this.

Please feel free to leave any comments if this helped you.


#!/bin/sh
#
# Automatic (re)generation of model classes for all *.xcdatamodel files.
# Written by Vincent Driessen
#
# You are free to use this script in any way.
# The original blog post is http://nvie.com/archives/263
#

# Define output directories
MACHINE_DIR="Entities"
MODEL_DIR="Model"

# Look for the Xcode project directory for this file
cd `dirname "%%%{PBXFilePath}%%%"`
while [ `ls -d *.xcodeproj 2&gt;/dev/null | wc -l` -eq 0 ]; do
    cd ..
    if [ "`pwd`" = "/" ]; then
        echo "No Xcode project found."
        exit 1
    fi
done

echo "Project directory is `pwd`"

#
# Check to see whether the base class is just a default (NSManagedObject) or
# maybe Brian Webster's excellent BWOrderedManagedObject.
# http://fatcatsoftware.com/blog/2008/per-object-ordered-relationships-using-core-data
#
# NOTE:
# The check really is quite arbitrary: if there exists a file called
# BWOrderedManagedObject.h somewhere below the project root directory, we
# assume that we want to use this as the base class for all generated classes.
#
EXTRA_FLAGS=
if [ `find . -name BWOrderedManagedObject.h | wc -l` -gt 0 ]; then
	EXTRA_FLAGS+="--base-class BWOrderedManagedObject"
fi

# Generate the model classes using mogenerator
for model in `find . -name '*.xcdatamodel'`; do
   # The output directories have to exist, so create them
   mkdir -p "${MACHINE_DIR}" "${MODEL_DIR}"
   mogenerator ${EXTRA_FLAGS} -m "${model}" -M "${MACHINE_DIR}" -H "${MODEL_DIR}"
done

NSManagedObjectContext extensions

2009-06-21T22:00:00Z

The Core Data framework rules, and its API is really really powerful. But really, why does the Core Data API require us to write so much boilerplate code? Simple things need to be simple.

Why is the deletion of a managed object from the NSManagedObjectContext so easy:


[context deleteObject:someObject];

Compared to its creation:


[NSEntityDescription insertNewObjectForEntityForName:@"someObjectClassName"
                              inManagedObjectContext:context];

Extending NSManagedObjectContext

Add the following category on NSManagedObjectContext to all of your Core Data projects and your pains will be history.


@implementation NSManagedObjectContext(NSManagedObjectContextConvenienceMethods)

- (id)newObject:(Class)entity {
   return [NSEntityDescription insertNewObjectForEntityForName:[entity description]
                                        inManagedObjectContext:self];
}

@end

Now, a call to create a new object is as easy as deleting it.


[context newObject:[someEntity class]];

Further enhancements of NSManagedObject

Matt Gallagher has written an excellent article about how to further enhance NSManagedObject for adding simple, one-line fetch support. Be sure to check it out.

NSPredicateEditor tutorial

2009-06-19T22:00:00Z

Cocoa offers a nice visual editor for editing NSPredicate objects templates, called NSPredicateEditor. The NSPredicateEditor can be set up using code or in Interface Builder, which is preferable for simple use. The setup is fairly easy once you know how to do it. In this tutorial, we’ll be building a simple predicate editor example which shows the basic functionality of the predicate editor.

Setting up the AppDelegate

Begin by creating a new Xcode project (Cmd+Shift+N). Name your project wisely and create a new class in the Classes group, called AppDelegate.

Switch to the header file and declare two IBOutlets for the main window and the sheet on which we’re going to display the editor in a few minutes. Also, add two IBActions called -openEditor: and -closeEditor:. Finally, add an ivar that holds the NSPredicate we’re going to be editing.

Next, we’re going to fire up Interface Builder to build the UI. Double click on the MainMenu.xib file under the Resources group.

Drag an NSObject object from the Library into the XIB and call it App Delegate. Hit Cmd+6 and make it a subclass of the AppDelegate class we just created. Then, hook it up to the delegate property of the File’s Owner.

Drag a new NSWindow to the XIB-file and call it Sheet. Make sure the checkbox “Visible At Launch” is deselected or the sheet will not display properly at runtime. Open the main window and add a NSButton and a NSTextView to it. To the sheet window, drag a NSPredicateEditor and a NSButton. They should look somewhat like this now:

Now, we can hook up the outlets and actions as usual. Hook up the Edit Predicate button on the main window to -openEditor: and the OK button on the sheet window to closeEditor:. Then hook up the mainWindow and sheet outlets of the AppDelegate class to the respective NSWindow objects.

Configure the NSPredicateEditor

Once we have all of the connections between Xcode and Interface Builder set up, we can continue to configure the predicate editor itself, which is actually what this tutorial is all about. An NSPredicateEditor control uses a list of NSPredicateEditorRowTemplate objects that can handle individual (simple) NSPredicate objects. Combining these row templates enables the NSPredicateEditor to edit compound predicates. There is no limitation to the depth of nested compound predicates, although nesting too deep would not be advisable from a usability perspective.

In the edit window, click a few times until the “name contains” row template is selected. In this row template, you define which key paths are supported. Supported here means two things:

matching—given an existing predicate with this key path in it on the left-hand side, this row template can be used to alter the predicate;
generation—when using the editor to create new predicates, adding a new rule for this key path will generate a predicate for this key path.

Gotcha

A small gotcha, at least one that initially put me on the wrong foot, is that there is quite a difference between the rows that you see design-time in Interface Builder and the rows that are available run-time. At design-time, you define the NSPredicateEditorRowTemplate objects while at run-time you see instances of them. Hence, the number of rows at design-time is the number of different row templates available. At run-time, however, the number of rows is the number of (simple) predicates within the compound predicate (which each has an associated row template instance that handles it). Subtle difference.

In short, in Interface Builder, create a row template for each type of match that you want to allow. Typically, this means for each data type that you want to support. In our example, we have the following setup:

Row template #1 is for all string matches. Here, we have defined it for the key paths “firstname”, “lastname”, “address.street” and “address.city”. They, per definition, have the same allowed operators. If we want to have an other set of operators for a specific key path, we need to define a separate row template for it.
Row template #2 is for date matches, i.e. our “birthdate” key path.
Row template #3 is for all integer matches, i.e. our “address.number” key path.

The result looks like this:

Using bindings to connect the predicate to the UI

Next up, we simply connect both the text view from the main window and the predicate editor from the sheet window to the predicate key path using Cocoa bindings. In order to do so, select the NSPredicateEditor (first click the control to select the scroll view, then click again to select the inner NSPredicateEditor), hit Cmd+4. Then, unfold the “Value” binding and hook it up to the App Delegate’s “predicate” key path.

Do the same for the text view in the main window, but this time hook it up to the “predicate.description” key path (since only strings can be displayed in a text view). When you do this, make sure that the text view is read-only, since the description property of objects should never be set.

Writing the code to wrap it all up

Finally, we have only a bit of code to write in our AppDelegate implementation, so let’s go:

//
//  AppDelegate.m
//  PredicateEditorTest
//
//  Created by Vincent on 20-07-09.
//

#import "AppDelegate.h"

#define DEFAULT_PREDICATE @"(firstname = 'John' AND lastname = 'Doe') " \
                    @"OR birthdate &gt; CAST('01/01/1985', 'NSDate') " \
                    @"OR address.city = 'Chicago' " \
                    @"AND address.street != 'Main Street' " \
                    @"OR address.number &gt; 1000"

@implementation AppDelegate

- (id)init
{
   self = [super init];
   if (self != nil) {
      predicate = [[NSPredicate predicateWithFormat:DEFAULT_PREDICATE] retain];
   }
   return self;
}

- (void)dealloc
{
   [predicate release];
   [super dealloc];
}

- (IBAction)openEditor:(id)sender
{
   [NSApp beginSheet:sheet
      modalForWindow:mainWindow
      modalDelegate:nil
      didEndSelector:NULL
        contextInfo:nil];
}

- (IBAction)closeEditor:(id)sender
{
   [NSApp endSheet:sheet];
   [sheet orderOut:sender];
}

@end

In the -init: method, we initialize the AppDelegate by setting and retaining a reference to a rather complex default predicate. When the XIB is loaded at run-time, the textbox shows exactly this predicate and it can be edited by invoking the edit sheet.

The actual implementation of the -openEditor: and -closeEditor: methods aren’t too exciting.

Downloading the source

You can download the source code for this tutorial as an Xcode project here.

PredicateEditorTest.zip

Have a blast!