Pool of oblivion

Fosdem 2012 impressions

2012-02-09T21:05:00.000+01:00

Back from FOSDEM 2012 and it was great again. More than 400 talks, about 5000 people attending, spread over multiple devrooms on ULB campus Solbosch in Brussels.

Listing everything I've done and seen is almost impossible, so I'll mention some highlights:

Coreboot is now available for a limited set of laptops
CERN is using Open Source Hardware
Tips and tricks for caching in web development
History of Open Source licenses, and other license related stuff
Team meeting with phpMyAdmin, outlining the future of the project
The talk of Freedombox mentions dreamplug : looks like a nice toy :)
Meeting a lot of people, both from my own team, people I've met at GSoC mentor summit, people from UGent and people I haven't met before
Community building : what works
Why Open Source should get non-geeks involved in their projects

And much, much more.

Looking forward to the next edition already : about 360 days to go. :)

Fosdem 2012

2012-02-03T17:43:00.001+01:00

The yearly meeting of Free and Open Source Software Developers and enthousiasts is about to start. This weekend about 5000 people will come to Brussels for FOSDEM, to attend presentations, meet people and share knowledge.
I intend to go as well, if my health allows : I was hit by a virus in the beginning of this week and I am still recovering.
Anyway, apart from a lot of talks and presentations I want to go to, there is also a team meeting planned of my project, phpMyAdmin. And, at some point, I hope to run into some people I met at GSoC mentor summit last year.

Concept of a home made icy transistor

2012-01-03T22:42:00.001+01:00

Some time ago I found out that ice (solid water) is a semi-conductor, like f.e. silicon and germanium. So this means that theoretically a diode or transistor can be made out of it, even at home with common products.

What follows is a back of the envelope reasoning, not a proof of concept, as I haven't tried it, but it might work.

First some rudimentary background about semi-conductors and how to make a simple component, like a diode or a transistor. A semi-conductor is basically an insulator that conducts electricity only in some circumstances, for instance when a semi-conductor is doped with another material to add electrons (negative, n-type) or to create 'holes' (positive, p-type), which can conduct electricity.
If a n-type semiconductor is joined by a p-type, you get a diode. If another p-type or n-type is added to create a npn or pnp-sandwich, you get a transistor.

So if frozen water (pure water (H2O), distilled) forms a semi-conductor lattice, it needs to be doped with something else to create the holes and electrons. More research is necessary to find suitable candidates, but for instance kitchen salt consists of sodium (Na) and chlorine (Cl) atoms which, when diluted in (pure) water, become positively and negatively charged ions.
By adding an electrical charge on two sides of the solution (pure water with f.e. a little bit of kitchen salt), most of the positively charged ions would be transfered to one side and the negatively charged ions to the other side. Freezing the solution at that moment would create an ice cube with a p-type and an n-type side, thus a diode.
Cutting a second diode ice cube in half and joining it with the other one, creating a npn or pnp sandwich results in a transistor.

This is just an idea, a lot of things need to be solved to make it work. For instance, what materials should be used to make n-type and p-type semi-conductors out of frozen water? How much of the material is needed? Will this work on a macroscopic scale (regular sized ice-cubes), or should the ice-cube be very tiny?
Where should the electrodes be positioned to separate the ions and what charge is necesarry?
Should the ions be evenly spread, and how can this be achieved?
And if it works, is it a practical diode? It would need subzero temperatures to remain solid, but it could melt by the heat dissipation of the electrical current flowing in the icy diode.

What a year

2012-01-01T00:01:00.002+01:00

2011 has been amazing. There were a lot of firsts and a few big changes in my life :

I bought a house. One of my resolutions for 2011 was to find a new place to live and I managed to do this. This time last year I had no idea I would now be sitting in the living room of my own house, writing this post.
It all went rather quick, first looking for a new apartment/house to rent, then deciding to buy one, followed by the search, finding one, and then all the administration and negotiating with the bank and finally, moving in and doing small construction works.

In the mean time I was a first time mentor for phpMyAdmin in the Google Summer of Code (GSoC) program. My student made phpMyAdmin compliant with Drizzle database, a MySQL fork.

Because of this mentoring, I was invited by Google to attend the GSoC Mentor Summit, which was a great experience, meeting a lot of interesting people and discussing FOSS and GSoC related issues at Google Headquarters.

This also meant I got to travel to the US, another first, and visit San Francisco and a part of California. This was an amazing experience.

For 2012 I have some goals and resolutions :

continue contributing to Open Source
be a GSoC mentor again
write an extensive article/blogpost about good passwords practices, or set up a website/wiki dedicated to it.
build a homemade transistor using ice (solid water), or write about the theoretical possibility
start reading again and keep a steady reading pace. I didn't manage to keep up reading in 2011.
further improve my house

California

2011-11-07T01:40:00.000+01:00

Waiting in the lobby of the hotel for the shuttle to take me back to the airport. And that will be the conclusion of a 2 week adventure starting at the Google campus for the GSoC Mentor Summit, a road trip visiting several places in North California and an extensive exploration of San Francisco city.

To summarize : it has been amazing. The things I've seen, the nature, the wildlife, the atmosphere, the people I've met and talked to.

So what did I learn :

One can pay almost anything with a credit card, even drinks in a bar or a parking meter.
Having an insurance is a good thing. If you have one, you will be helped immediately and with a smile. If you don't have one, you will still get a smile, but you are presented a bill before you are helped.
San Francisco is a nice city, full of friendly and patient people (exception for cab drivers, but that counts for probably all cab drivers in the world) but also a city of controversy. Rich and poor live together, or more precisely, next to each other, and you can take that quite literally. One moment you're walking down a fancy shopping street, and one block further you find yourself surrounded by homeless people.
Automatic gearboxes are nice for driving in the city and on freeways. Cruise control is also a very nice feature. But when doing some hills you just crave for a manual gearbox.
Americans are very good at putting signs for anything, usually stating the obvious. It can be a helpful reminder, or in my case, being a tourist visiting the US for the first time, very convenient. But when you need a sign you can't find one.
I still haven't come to grasp with traffic rules for crossroads. On crossroads without a traffic light and stop signs on all corners, I haven't figured out yet who gets to drive first. The one who came to the crossroad first, or the one coming from the right.
On cross roads with traffic lights, it is usually allowed to drive when it's red if you are going right, but not always. So I usually stopped for red anyway, just to be safe. But this sometimes resulted in being notified by the cars behind me that I should be going.
Driving by bike in San Francisco is a very nice experience. The challenge is to find streets that are not too steep.

It was a nice experience and I hope to return some day.

GSoC Mentor Summit 2011

2011-10-24T03:21:00.002+02:00

I'm on the bus back to the hotel, after a weekend of meeting and collaboration with a lot of inspired Open Source developers. We met at Google Headquarters for the annual GSoC Mentor Summit, discussing last year's Google Summer of Code in particular and Open Source collaboration in general. More than 300 people of 175 participating organisations, from all over the world, were present.

It was my first time, both as a GSoC mentor for the phpMyAdmin project as on attending the summit, but it was great (or awesome, like some would state it). I've met and talked with many people. I learned a lot from the different sessions I attended and was pleasantly surprised about how much drive, knowledge and experience was present.

Awesome!

5 years

2011-08-15T20:20:00.001+02:00

Exactly 5 years ago, I posted a 'First Post' on this blog. Back then I was wondering how long this blog would survive. Apparently it still exists. Going over a few posts of the past 5 years I noticed there were some active periods and some slower ones. Times were I posted things that I saw on the streets or read in the newspaper, but I think my blog has evolved into a space were I share IT and technology related stuff that I discover, am working on or I shouldn't forget.
By the way, the top 3 of most visited posts over the last 5 years are IT related :

BTW : A top 5 of the most popular post of the last 30 days is in the column on the right.

Under construction

2011-07-08T22:26:00.001+02:00

Today my holidays start and I will spend them ... working : Tomorrow I will start redecorating my new house. Basically it will be just some cleaning, painting, and small repairs, no major de- and reconstruction.
In two weeks I plan to move to the new place and I hope to be ready by then, and I still have to do a lot of packing.
The next three weeks will be quite busy, just like the last 3 months actually. ;)

House

2011-05-03T22:42:00.001+02:00

The front

For the last few weeks and months, I've been looking for a new place to live, but today I found a house that looked nice and affordable. So I bought it.
It sounds very simply put that way, but basically that's what I did. I visited it, inspected, decided to buy and came to an agreement with the current owner.

It's big enough for me, situated in a quiet neighborhood, close to my work, in a good state, and for a fair price.

But this is of course only the beginning of it. Now starts the hassle of securing a loan, planning to move, looking for cardboard boxes (to move things in, or if something goes wrong, to live in ;) ), start packing, notify my current landlord, and a lot of administration, ...

A lot of things to do in the next few weeks, that and going to work, mentoring for GSoC, and my normal every day and social live. :)

But it's all very exciting!

The view, old picture, the weather is much nicer at the moment ;)

The living room (not my furniture)

The stairs, quite fond of them actually.

git subtree module

2011-04-20T15:21:00.002+02:00

Did you ever want to merge an external git tree with yours, while keeping the commit history? Or do you want to create a new git repository from a folder in your project, keeping the history?

Then the git subtree module is what you need. It let's you import a complete git repo (with commit history) into your project, and add the files to a folder you specify.

For example :

$git subtree add --prefix=other_project \
     git://github.com/your_tree/your_project.git master

imports the master branch of your git repository located in git://github.com/your_tree/your_project.git into the folder other_project.

BTW: the subtree module is not part of the core git package. So if you want to use it, you will have to install the module first.

Download the git subtree module and extract it, or clone it :

$git clone https://github.com/apenwarr/git-subtree

In the git-subtree directory, run

$chmod u+x install.sh

and as root :

#./install.sh

This will copy the git subtree module to the git script folder. You can now use the git subtree module.

Reduce firewall configuration complexity using iptables with chains

2011-04-04T17:57:00.004+02:00

Introduction

Setting up a firewall on your *nix box, being it a workstation, laptop, or server, is always a good idea. In most cases, you can do with some simple firewall rules, f.e. on your laptop, block all incoming requests (except the established connections, i.e. the replies on the outgoing requests you made), or on a simple webserver (allow port 80 only).

But if you need more complex rules, f.e. a server that hosts a website available for the entire internet, but with an ssh and samba service that should only be available for the local subnet, or even some specific IP addresses, it becomes a bit more complex.
And if you want to filter the outgoing traffic as well, your iptables rules get a mess after a while, and when you want to change anything, chances of a mistake or forgetting something are high, which may result in locking yourself out of your box (at least for remote access), or leaving something open that shouldn't.

To make your rules more manageable, you can make use of chains in your iptables rules. I got some inspiration in an article that uses chains to make iptables more efficient (faster). My goal was to get easier to read and configure iptables rules, but it will result in faster handling of packets as well.

Setup

A web service should be available from all networks (i.e. internet) on port 80 (http) and 443 (https)
The server can be managed remotely using ssh (port 20) and webmin (port 10000), but only from a limited set of IP addresses (admin PC's).
The server hosts a samba service (several TCP and UDP ports), that should only be available from a limited set of IP addresses (admin + webmaster PC's).
Outgoing connections will be filtered, but some services should be allowed (dns, dhcp, smtp, ntp) and some external websites should be available to get updates.

Concepts

ESTABLISHED state

When using this option, you can filter for established connections. If you define it in both the INPUT and OUTPUT rules, you only have to define in the INPUT rules which NEW incoming requests should be allowed, and in the OUTPUT rules which NEW outgoing request are allowed. The established connections will be allowed and should not be redefined (making the configuration a lot more readable and maintainable). An example allowing only an ssh service without using the ESTABLISHED state would be :

# iptables -A INPUT -p tcp --dport ssh -j ACCEPT
# iptables -A INPUT -j REJECT
# iptables -A OUTPUT -p tcp --sport ssh -j ACCEPT
# iptables -A OUTPUT -j REJECT

Basically, every incoming/outgoing connection is dropped, except if the incoming packet has port 22 (ssh) as destination, or if the outgoing packet was sent from port 22 (which is the reply of the ssh server).

When using ESTABLISHED state, this will be :

# iptables -A INPUT -p tcp --dport ssh -j ACCEPT
# iptables -A INPUT -j REJECT
# iptables -A OUTPUT -m state --state ESTABLISHED -j ACCEPT
# iptables -A OUTPUT -j REJECT

Now, every incoming/outgoing connection is dropped, except if the incoming packet has port 22 (ssh) as destination, or if the packet belongs to an established connection. Because incoming connections to port 22 are allowed, the firewall will remember a packet coming in, creating a 'connection' for the host/port the packet originates from when the ssh server replies to it. So when the reply of the ssh server is sent out, it matches an 'established' connection and will be allowed out.

In this example, the benefit of using the connection state is not clear, but when more allowed incoming services are added, they only have to be added on the INPUT chain, but not on the OUTPUT chain, because they are covered by the ESTABLISHED rule.
In the first example (without the ESTABLISHED rule), every allowed incoming connection should be repeated in the OUTPUT chain, matching the packets sent for the outgoing connection, which results in an equal amount of rules on both chains.
If you want to do filtering in both directions (allowing incoming request for listening services and outgoing request for remote services), this can become very messy, and almost unmaintainable without making mistakes.

Introducing chains

When two services (on different ports) should be available to a limited but identical list of IP addresses.
Without using chains, for every combination of port and IP a rule should be created :

# iptables -A INPUT -p tcp -m tcp -s 10.100.2.3 --dport 22 -j ACCEPT
# iptables -A INPUT -p tcp -m tcp -s 10.100.2.4 --dport 22 -j ACCEPT
# iptables -A INPUT -p tcp -m tcp -s 10.100.2.7 --dport 22 -j ACCEPT

# iptables -A INPUT -p tcp -m tcp -s 10.100.2.3 --dport 10000 -j ACCEPT
# iptables -A INPUT -p tcp -m tcp -s 10.100.2.4 --dport 10000 -j ACCEPT
# iptables -A INPUT -p tcp -m tcp -s 10.100.2.7 --dport 10000 -j ACCEPT

Resulting in a lot of rules, and when an IP address has to be changed/added/removed, this has to be done for every corresponding rule.

When using chains, this can be much easier. Imagine, that you first check if the packet matches the destination port, and if it does, jump to a new chain, where a list of IP addresses is checked. :

// create new chain admin_IP
# iptables -N admin_IP

// add rules to chain admin_IP
# iptables -A admin_IP -s 10.100.2.3 -j ACCEPT
# iptables -A admin_IP -s 10.100.2.4 -j ACCEPT
# iptables -A admin_IP -s 10.100.2.7 -j ACCEPT
// drop all packets that are not matched by previous rules
# iptables -A admin_IP -j DROP

// filter ports in INPUT chain
# iptables -A INPUT -p tcp -m tcp --dport 22 -j admin_IP
# iptables -A INPUT -p tcp -m tcp --dport 10000 -j admin_IP

As you can see, there is are several benefits of putting the IP addresses in a separate chain :

the list of IP addresses in the separate chain can be reused for both ports, so they have to be defined only once.
adding/changing/removing an IP address is much easier
there is a better overview of the firewall rules.

Actual configuration

INPUT chain

# iptables -A INPUT -i lo -j ACCEPT
# iptables -A INPUT -p tcp -m tcp --tcp-flags ACK ACK -j ACCEPT
# iptables -A INPUT -m state --state ESTABLISHED -j ACCEPT
# iptables -A INPUT -m state --state RELATED -j ACCEPT
# iptables -A INPUT -p icmp -j icmp_in
# iptables -A INPUT -p tcp -m tcp --dport 22 -j admin_IP
# iptables -A INPUT -p tcp -m tcp --dport 10000 -j admin_IP
# iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
# iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
# iptables -A INPUT -p tcp -m tcp --dport 139 -j webmaster_IP
# iptables -A INPUT -p tcp -m tcp --dport 445 -j webmaster_IP
# iptables -A INPUT -p udp -m udp --dport 137:138 -j webmaster_IP
# iptables -A INPUT -j DROP

Basically, this is the input filter, allowing :

all local traffic (not leaving the physical PC)
a check for tcp-connections
established and related connections
ICMP packets (ping, etc.) are handled in a seperate chain icmp_in
some services
- ssh (tcp 22) and webmin (tcp 10000) allowed for admins (admin_IP chain)
- website (tcp 80 and 443) for everybody
- samba (tcp 139 and 445, udp 137-138) for webmasters (and admins, see definition of webmaster_IP chain)
everything else is not allowed (dropped)

Very structured and readable, I must say. :)

OUTPUT chain

# iptables -A OUTPUT -o lo -j ACCEPT
# iptables -A OUTPUT -m state --state ESTABLISHED -j ACCEPT
# iptables -A OUTPUT -m state --state RELATED -j ACCEPT
# iptables -A OUTPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
# iptables -A OUTPUT -d 10.1.1.2 -p udp -m udp --dport 53 -j ACCEPT
# iptables -A OUTPUT -d 10.1.1.3 -p udp -m udp --dport 53 -j ACCEPT
# iptables -A OUTPUT -d 10.1.1.4 -p udp -m udp --dport 67 -j ACCEPT
# iptables -A OUTPUT -d 10.1.1.5 -p tcp -m tcp --dport 25 -j ACCEPT
# iptables -A OUTPUT -d 10.1.1.6 -p udp -m udp --dport 123 -j ACCEPT
# iptables -A OUTPUT -p tcp -m tcp --dport 80 -j ext_websites
# iptables -A OUTPUT -j DROP

The output filter, allowing :

all local traffic (not leaving the physical PC)
established and related connections
ICMP replies
some remote services (hosted by different servers, IP addresses do not represent actual situation)
- 2 dns (udp 53) servers (a separate chain could have been created)
- dhcp (udp 67)
- smtp (tcp 25)
- ntp (udp 123)
external websites (tcp 80), listed in chain ext_websites
everything else is not allowed (dropped)

icmp_in chain

# iptables -N icmp_in
# iptables -A icmp_in -p icmp -m icmp --icmp-type 8 -j ACCEPT
# iptables -A icmp_in -p icmp -m icmp --icmp-type 0 -j ACCEPT
# iptables -A icmp_in -p icmp -m icmp --icmp-type 3 -j ACCEPT
# iptables -A icmp_in -p icmp -m icmp --icmp-type 4 -j ACCEPT
# iptables -A icmp_in -p icmp -m icmp --icmp-type 11 -j ACCEPT
# iptables -A icmp_in -p icmp -m icmp --icmp-type 12 -j ACCEPT
# iptables -A icmp_in -j DROP

Basically, all allowed incoming ICMP message types.

admin_IP chain

# iptables -N admin_IP
# iptables -A admin_IP -s 10.100.2.3 -j ACCEPT
# iptables -A admin_IP -s 10.100.2.4 -j ACCEPT
# iptables -A admin_IP -s 10.100.2.7 -j ACCEPT
# iptables -A admin_IP -j DROP

A list of allowed IP addresses of admin PC's.
Everything else is not allowed.

webmaster_IP chain

# iptables -N webmaster_IP
# iptables -A webmaster_IP -s 10.100.2.11 -j ACCEPT
# iptables -A webmaster_IP -s 10.100.2.17 -j ACCEPT
# iptables -A webmaster_IP -s 10.100.2.34 -j ACCEPT
# iptables -A webmaster_IP -s 10.100.2.50 -j ACCEPT
# iptables -A webmaster_IP -j admin_IP

A list of allowed IP addresses of webmaster PC's.
At the end of the list, it jumps to the admin_IP chain, actually combining both chains.

ext_websites chain

# iptables -N ext_websites
# iptables -A ext_websites -d 212.211.132.250 -j ACCEPT
# iptables -A ext_websites -d 212.211.132.32 -j ACCEPT
# iptables -A ext_websites -d 195.20.242.89 -j ACCEPT
# iptables -A ext_websites -d 130.89.149.225 -j ACCEPT
# iptables -A ext_websites -d 86.59.118.153 -j ACCEPT
# iptables -A ext_websites -d 130.89.149.227 -j ACCEPT
# iptables -A ext_websites -d 128.31.0.51 -j ACCEPT
# iptables -A ext_websites -d 86.59.118.153 -j ACCEPT
# iptables -A ext_websites -d 67.228.198.100 -j ACCEPT
# iptables -A ext_websites -d 140.211.166.6 -j ACCEPT
# iptables -A ext_websites -d 140.211.166.21 -j ACCEPT
# iptables -A ext_websites -j LOG

A list of allowed external websites for updates (mirrors of Debian, webmin and Drupal, in this example).
All other requests for external websites are logged. This can be useful for monitoring : notification of abuse, or if you forgot to add an allowed website.

This is of course, just an example. There are many variations possible. Maybe you want to add an ftp server, that is only accessible from a specific subnet and some other IP addresses. Or you can allow outgoing ssh-connections to a pool of servers. If you want to protect your server from scanning techniques (XMAS, NULL, ...) you can create a seperate chain for that as well. Options are limitless and depending on your needs.

But the idea is that the firewall rules are now much easier to understand and change. For example, if you want to make ssh (port 22) available also for webmasters, or for the entire internet, just change the -j option to webmaster_IP or ACCEPT. Adding an IP address for an admin PC, is just adding one line to the admin_IP chain.

BTW: I didn't go into every detail of iptables options that are used in my examples, so if you like more information on all available options of iptables and on how iptables works, you can take a look at this tutorial.

My first Python script

2011-03-02T23:19:00.001+01:00

I've started reading a Python tutorial and this is the first little script I wrote :

#!/usr/bin/env python

a="internationalization"
print a[0] + str(len(a[1:-1])) + a[-1]

a="localization"
print a[0] + str(len(a[1:-1])) + a[-1]

If it is executed, you get :

i18n
l10n

It's not much, but slightly less boring than the usual "Hello World!".
Basically, it just takes the first and last character of each word, and replaces the characters in between with the total number of characters in between. The exciting thing about it, is the sheer compactness of the syntax. In any other language a lot more code would be needed. :)

BTW : i18n and l10n are widely used numeronyms, that refer to internationalization and localization.

bash treats numbers with leading 0 as octal

2011-02-28T12:00:00.006+01:00

In a bash script, the number of the current week is used to select a different tape every week (rotation of 4 tapes) :

#!/bin/bash 

WEEK=`date +%W` # Week number (01..52)
TAPE=$((WEEK % 4))

This worked fine so far, but this weekend (it's the 8th weekend of the year) there was an error :

value too great for base (error token is "08")

After some digging, I found that bash treats numbers with a leading 0 as octal. But value 8 (and 9) don't exist in octal, so that's why it throws an error. With smaller numbers (01-07), there is no problem, because they are valid octal numbers, and for bigger numbers (10-...), the number is treated in decimal format, because there is no leading 0 anymore.

This problem can be fixed in two ways :

1) strip leading 0 from value

In bash you can do this with this command :

WEEK=${WEEK#0} # strip leading 0

This only removes a leading zero, a zero that is not leading will not be removed.

2) convert value to decimal

TAPE=$((10#$WEEK % 4)) # convert $WEEK to base-10 with '10#'

Note that you have to add the $ in front of the variable WEEK, to treat it is a variable, in order for the conversion (with 10#) to work.

FOSDEM 2011

2011-02-13T19:14:00.002+01:00

It's been great again. Over 5000 Open Source developers and enthousiasts meeting to share more than 200 hours of knowledge over the course of a weekend at FOSDEM 2011, held at campus Solbosch of ULB in Brussels, Belgium.
Last year I went there for the first time, for just one day, and I liked it so much I immediately decided to go back this year, but for the whole two days.

So there I was, on Saturday afternoon, after taking two trains, a metro and a bus, entering the ULB campus where FOSDEM is held, seeing the signs pointing to all different buildings and rooms where talks were held, the catering trucks, and a busy bunch of enthousiastic people.

I headed for the MySQL & Friends devroom, where I met some of the people of the phpMyAdmin team, which I joined last year. I hadn't met them before, so this was a nice opportunity.
These are some of the talks I attended on Saturday :

After this, we had a phpMyAdmin team dinner and afterwards I introduced them to some fine Belgian beers.

Sunday was a bit chaotic, I hadn't planned to see that much talks, one of the talks I planned to attend was canceled, and I missed another one, because I left early to catch the canceled one (but I didn't know it at that time), but I still went to some :

After the last talk, it was time to go home. A special FOSDEM bus brought us back to the railway station, were I had some dinner before I headed home.
It was a nice weekend, with a nice Open atmosphere, again perfectly organized by and for the community. In case you missed it, this was FOSDEM 2011 (a video artists impression).

I'm looking forward to next edition already!

Debian 6 (Squeeze) is released.

2011-02-07T17:20:00.002+01:00

A few days ago, Debian 6.0, codename Squeeze, was released after a 2 year development period.

Of course this new release contains a lot of new stuff : lots of new and upgraded packages, faster booting because processes are started in parallel (if dependencies allow it), integration of grub2, volatile repository is replaced by squeeze-updates, ...

Because I want to use Tomcat 6 on a production server, I wanted to try the upgrade on a testbox. Everything went quite smoothly. All steps for upgrading from Debian 5.0 (Lenny) are clearly explained in the release notes (Chapter 4). The only thing that went wrong, in my case was the update of mysql-server. For some reason it was uninstalled, but the replacement package for the new version 5.1 was not installed.

After the upgrade process, I rebooted and I noticed grub 0.97 (now called grub-legacy) was still used, but it transfered the bootprocess to grub2. This worked like a charm, so then I ran (as root) :

# upgrade-from-grub-legacy

to install grub2 in the MBR and boot from it. Rebooting went perfectly afterwards.

So far I'm quite pleased with squeeze and the upgrade process. I'll do the upgrade with my other boxes in the next few days.

XFS check

2011-01-21T11:33:00.001+01:00

I wanted to check the consistency of the data partition on one of my servers. It is 6.5TB and formatted with XFS, so I ran :

#xfs_check /dev/sdb1

And I got :

xfs_check: out of memory

After some searching, it turns out that a lot of memory is needed to perform the xfs_check on a large file system : >6GB and you need to run it on 64bit, to able to address that memory.

My system is a 32-bit with only 4GB, so I would probably not be able to run xfs_check on my system, but there is another way :

#xfs_repair -n /dev/sdb1

This tools tries to repair a XFS filesystem, but with the -n switch no changes are written to the file system, so the effect is quite the same. It still uses a lot of memory if you have a lot of files/inodes on the file system, but 3GB on a 32-bit system should be sufficient.

Of course, if xfs_repair finds a problem, you can still run it without the -n switch, to repair the filesystem.

All time favorites : games

2011-01-04T00:10:00.001+01:00

I've recently rediscovered the wonderful pass time of gaming. And I was thinking of a few great games I played in the past, in alphabetical order :

Assassin's Creed II
Exile (Myst III)
Grim Fandango
Riven, the sequel to Myst

They earned a position in this list because of the mere of beauty of the game, which was usually a combination of graphics, music, scenery, atmosphere and gameplay.

Happy 2011!

2011-01-01T00:01:00.055+01:00

Happy New Year!

May 2011 be a year of friendship, love, joy and happiness, good health and prosperity!

2010

A lot of things happened in 2010. I'm not going to write an exhaustive list, but will just mention things that spring to mind.
On a technological level : I got my new server and installed linux on it, I learned to use vim and git, and started contributing to the Open Source community by joining the phpMyAdmin development team. I visited FOSDEM for the first time and will certainly do so this year.
On a professional level : I created a new website for my department using Drupal, and installed a new backup server.

One of my resolutions for 2010 was to read more, and I achieved this. If possible I read a chapter every evening, and I hope to continue doing this in the new year.
I also spent quite some time playing games, watching films and series and (re)discovered some music. Some of the highlights of the year : Assissin's Creed II, the best game I played in years; Isbells and Mumford & Sons, the latter doing some great gigs; and 1984, classic by George Orwell. :)

And I met a lot of interesting people, made new friends and got to know some people better.

Resolutions 2011

Bit a more of the same of last year. I made some progress, but there is always room for improvement :

Smile (even) more!
Enjoy more and worry less
Continue reading
Be more confident
Look for a new place to live

Cleanup apt cache

2010-12-04T19:41:00.006+01:00

Over time, the cache of apt, the Debian package installation application, keeps growing. Every time a package is installed or updated, the original packages is stored in the cache. After a few months of updates, the cache grows quite big, because nothing is deleted from it automatically.
Keeping the original packages from all software you have installed is certainly a good idea, but the packages that are replaced by newer ones are kept as well.

The cache can be cleaned. using this command :

apt-get autoclean

This removes all unneeded (old) package files from the apt cache.

If you want to clear the cache completely, and thus removing all packages, use this command :

apt-get clean

Both commands are quite safe to use, because they only remove the downloaded packages from the apt cache. The installed files naturally remain untouched.
This way you can sometimes free a few hundred megabytes after a few months.

Ubuntu is based on the Debian packaging system, so this applies to every flavor of Ubuntu as well.

Firefox plugin : HTTPS Everywhere

2010-11-08T00:12:00.003+01:00

Yet another Firefox plugin to be added to the list of useful plugins : HTTPS Everywhere.

Essentialy, this plugin forces your browser to use the secure (encrypted) https protocol when visiting certain websites (like Facebook, Google, ...). This way, no data is sent in cleartext over the internet. When visiting these websites, everything is encrypted by your browser before it leaves your computer, and is then decrypted by the server (and vice versa, when the server sends the webpage back to your PC).

Why would you want this?

If you login over a normal http connection, your username and password are sent over the internet, for everybody who can intercept that package to read. The same with sensitive data that might be visible on websites your are visiting.

You are probably wondering who can catch your packets? To do that you must most certainly be an internet wizard? Not true. Recently a Firefox plugin was published, called Firesheep, that allows a computer to look around on your local (wireless) network, and hijack Facebook and other sessions on social networking websites and actually see what they are seeing.
For instance, when you are using a free wireless accesspoint, someone with the firesheep plugin, can easily watch what you are doing. (With more sophisticated software this is also - and was already - possible, but this requires more skill)

So, to protect yourself from these kind of attacks, it's better to use the https protocol as much as possible. And a nice tool to help you do this is, is a plugin like HTTPS Everywhere, or Force TLS. The Force TLS plugin does about the same as HTTPS Everywhere, but is more difficult to setup, because you have to tell the plugin for which website you want to use https. Downside of the HTTPS Everywhere plugin, is that it only works for the preprogrammed websites.

Microcode

2010-10-19T13:52:00.004+02:00

This morning, one of my servers (Dell Poweredge R510, with a quad-core Intel Xeon E5520 CPU running Debian 5.0.4) had crashed. It didn't respond to anything : ping, ssh, smb. Not even when physically attaching a screen and a USB keyboard to it, did it respond.
After a reboot, the system started without a problem and everything seemed to work again.

Skimming through the logs I found some things occuring :

kernel: BUG: soft lockup - CPU#7 stuck for 61s!

and

kernel: __ratelimit: 35 messages suppressed
kernel: nf_conntrack: table full, dropping packet.

After Googling, I didn't really find a sound explanation, but some of the things that was mentioned was a bug in the Intel CPU, which could be solved by updating the CPU Microcode.

I'm not sure this will solve my problem, which only happened once since I started using the server about half a year ago, but as was mentioned, it does little risk, doesn't slow down your machine and might solve a few problems.

So I installed two packages (you need contrib and non-free repositories):

apt-get install intel-microcode microcode.ctl

Package intel-microcode contains the updated microcode for Intel CPU's, while microcode.ctl does the update. Because the update is done in memory, the update is lost after a reboot, so you will have to do it again, but this package takes care of that.

Update 06Dec2010 : The microcode is automatically updated after a reboot. :)

Open Source Social Networking

2010-10-13T23:19:00.000+02:00

For quite some time now, I've been thinking about a social networking website alternative, where you, as a user, can keep control of what you submit : who can see it, what can be seen, and most important, if you delete it, that it is really gone (at least from the website you originally put it on, as one can not be sure that something that is published on the internet can be completely removed from it). But I got an insight on how it could be done. It's only a rough idea, with a lot of conceptual and practical things still to be sorted out :

A way to reach this goal is to use encryption. When encrypting a message for instance with OpenPGP, you use the public keys of the ones that should be able to decrypt the message. To do so, they use their own - private - key, which is of course private, and can only be used by the one who owns it. So this way, only the ones you intend to be able to read your message will be able to read it.
So, in case of this conceptual open source social networking website, every user has a PGP-key. If you want to submit something to this website, for instance your place of birth, you encrypt it, using the public keys of a selection of your friends and submit that encrypted chunk of data to the website, where it is stored in the database.
If one of your friends accesses your page on this social networking website, all the data, in encrypted form, is requested from the website and decrypted using the private key of that friend. The data that was encrypted with that persons public key, will be decrypted, while the rest will remain unreadable, thus showing only the data that you intended for that person.
If on a later moment, you decide you want to change the list of people that will be able to consult your place of birth, you simple encrypt the same data with different keys and replace it in the database.

The kind of data you can encrypt is of course not limited to short texts, but can also be a picture, a piece of video, a link to a website, a piece of you DNA, ...
The upside of storing an encrypted version of the data you share, makes it unreadable to anyone who doesn't have the right decryption key. So even if your data remains in the database of the website, it will only be readable for the persons it was originally intended for. It will even not be readable by the maintainers of the website, unless you include their public key when encrypting.

So, the bottom line is that YOU should be able to keep control over your data. Encrypting the data is one thing, somewhat trusting the software that makes it happen and of course also the ones hosting it all, is as important.
And here the Open Source model comes in. The software is freely available, so the way it works can be checked and improved by the Open Source community.
And because the software is freely available, anyone can set up a social networking website. So as a user, you can choose who to trust when you join a group.

Of course, this is all just an idea and a very general concept.

One final remark : in order for this to work, the whole encryption mechanism should be invisible for the end user. And the encryption should be handled client side, for obvious reasons, otherwise the data you intend to be limited available, might end up unencrypted on some kind of server.

change permissions of files and folders

2010-10-07T11:45:00.004+02:00

On *nix, when you want to change the access permission of a directory and all files and folders in it, you could use :

cd your_dir
chmod -R 750 *

But this doesn't make a distinction between files and folders. So if you want different permissions for files and folders, you can do this :

cd your_dir
find -type f -exec chmod 640 {} \;
find -type d -exec chmod 750 {} \;

New department website launched

2010-05-16T16:35:00.004+02:00

Last week, the new website of my department, that I've been working on for the last few weeks, was launched. In its first week, it got about 1200 visitors.
Although not completely finished, I thought it had reached a stage were it could be made public. I consider it a starting point anyway, a basis to start improving on by adding new features.

The previous website was created about 4-5 years ago, and was in need of a face lift. I could just have replaced the css file, to refresh the layout, but the code running in the background was replaced as well.

After some comparing, I decided to use Drupal 6 as the CMF of the new website. I hadn't worked with Drupal before, but heard and read good things about it, being developed in PHP and strong focus on modules as strong points.
First, I started developing a custom theme that reflects the corporate style of my university. I didn't do this before either, so it took me some time to understand how a page is built in Drupal and what parts it consists of, but in the end I got it working.

The new website should be bilingual (English-Dutch), so I had to look for a way to do this. Luckily, Drupal provides some modules (Locale, Internationalization) to make a website multilingual, and tools to do the translation (core module Content Translation).

I wanted to integrate the login system into the single sign-on system (CAS) of my university, so that the users of the website wouldn't have to create yet another account with matching password for this website.
There is a CAS module available for Drupal, but I got a custom one from a colleague, which I modified further, to take advantage (through CAS) of groups that are defined in the LDAP system of my university and of groups I defined in a seperate database, and thus assigning the right roles to each user on login.

With the different roles defined and assigned, I used the taxonomy module (in Drupal core) to define different parts of the website. My department consist of three research groups, so each lab has it's own section within the website. All sections have a public and private (intranet) part, so by using Taxonomy Access Control access to these pages could be linked to the defined roles.

Some pages are just static HTML, being maintained using a WYSIWYG editor, provided by Drupal module CKeditor.
Others are dynamic, getting their data from a database table, using a PHP script to generate the page. Module Cache Exclude was needed to keep these PHP generated pages from being cached by Drupal. Otherwise changes made in the database, would not show.
And core module PHP filter, to be able to execute PHP code in a page.

As I mentioned before, I'm new to Drupal so I'm still discovering possibilities, new modules and ways to do things, every day. Things that are still on my list is learning how to write modules, find a way to integrate my current database management PHP code with Drupal, basically using the Form API, not only to insert data into a database, but also update and delete data, and expand the features of the new website (both on the intranet as on public pages). Luckily I've got a good guide for the upcoming developing journey.

LHC detects something unexpected.

2010-04-01T09:41:00.008+02:00

A few days after the first collisions in LHC, something strange has happened at the new particle accelerator at CERN.
Physicists noticed, this morning, some unexplainable activity in the detector of CMS, one of the experiments.
Remarkable, because there were no active beams at the time.
Investigations have started, to try and find an explanation for this fishy behaviour.
LHC managed to collide two 3,5 TeV beams of protons, for the first time on the 30th of March 2010, only a few days ago.
Something might have been created during those first collisions, that only now was detected.

Fascinating for some, troubling for others, as some critics feared that LHC might swallow the Earth.
Other options are faulty hardware in the detector, causing these unexpected events.
Only after some more tests, it will be certain if there is a flaw in the detection, or the discovery of something amazing.
Later today, more news will be available.

Update (2Apr2010) : As far as I know, everything is working fine at LHC. All experiments detect collisions as they should, because there were beams colliding now and at that moment. If you are wondering what this post is ranting about, check the date it was published and the first letters of each sentence. It might give you a clue. :p

Update 2 (2Apr2010) : Apparently, something was found.