You should know that tilllate is using a self-made “ORM” tool: Our ORM framework maps table columns to object properties. But it is agnostic of relationships between tables. And when you want to write a complex query, you end up programming long lines of native SQL. Unreadable.

ORM tools increase productivity

I brought up the discussion as we have to make technology decisions for a new project. I know Doctrine 1.2 very well from a spare time project. I am very satisfied with it:

• No need to think about how the objects are mapped to the database. You just save() them (or persist() them as Doctrine2 calls the function more accurately). You can navigate along the object graph without caring on what exact SQL statements and DB calls are needed.
• The code is more readable as there is no mix between PHP and SQL
• Because of the query language is integrated in PHP the IDE can help you with autocompletion.
• Useful features like data validation or event listeners (“onSave“, “onCreate“) become possible.

Just like back in 1975

The advocates of the “ORM framework” we’re currently using argue that with an ORM tool hiding all SQL from you you lose control of what happens. Also they are worried about performance. Funny, exactly that’s what developers objected 35 years ago against high-level languages like PL/I or APL when they had to switch from assembler. “It does not let you do what you want and it is slow.” ¹

As to function, after a learning phase I was able to do whatever I wanted to with it.

As to speed, I believe that the additional abstraction layers certainly cause some performance loss. But for one part the loss is absorbed by built-in caching. And for the rest: With the time you save on development and maintenance you can buy additional hardware.

What do you think?

What is your opinion on ORM tools? Is there a way around them when you do OOP? Are there occasions when writing plain SQL makes sense? Or should we forget about relational databases altogether and switch to NoSQL as jpkoenig suggests?

1. Brooks, Frederick P., Jr (1975). “The Mythical Man-Month”. Addison Wesley, pp. 135

In my position as team leader I attach great importance to the first day of work for new members in my team. A good first impression is a foundation for a good relationship with the direct report. Here’s what I do for new employees on their first day:

1. I make sure that his desk, computer, email-account etc. is ready.
2. I organize a welcome card and a small gift if put on his or her desk.
3. I organize an introduction program where the new team member meets someone from every department.
4. I introduce him to every staff member. I am making sure that everyone knows his role. And the he knows what people are relevant to him. Him being able to build an internal network quickly is essential to become productive quickly.
5. I make sure that I am available for him.
6. I organize a lunch with the team.

By observing these points I make sure that the new staff member feels comfortable and welcome. And that he knows what to do. That extra effort is worth it.

In diesem Fall ging es darum ein System komplett auf ein anderes zu klonen. Sind die beiden Systeme baugleich, ist das mit dd und ssh relativ einfach einfach möglich.

Dazu müssen zuerst beide Systeme mit einer Linux-BootCD (z.B. sysrescuecd) gebootet- und das Netzwerk konfiguriert werden.

Dann gibt man auf dem Quell-System einfach:

quelle~ # dd if=/dev/sda | ssh root@zielsystem "dd of=/dev/sda"

ein und wartet bis der Vorgang abgeschlossen ist. Wer sich noch einen Status des Kopiervorganges anzeigen lassen möchte:

ziel~ # kill -USR1 $(pgrep '^dd$')

-Die Ausgabe erfolgt dabei auf der Konsole vom Quellsystem.

Nach dem Kopiervorgang sollte man das System mounten und die Netzwerk-, sowie Hostname noch anpassen; sonst gibt es danach einen IP-Adresskonflikt im Netzwerk.

Achtung: Dass das ganze klappt sollten beide Systeme (zumindest die Disks) relativ baugleich sein – Und natürlich gehen die Daten auf dem Zielsystem unwiderruflich verloren.

Erweiterungen

Komprimieren: Es wäre nun noch Möglich die Daten durch gzip zu pipen um Bandbreite zu sparen.

Erweiterter Status: Mit pv kann man sich erweiterte Statusinformationen über den Kopiervorgang zusammenbasteln.


We may start with this non-fictional example:

/**
* Returns array of uids of selected items
*/
getSelected:function() {
	var returnArray = new Array();
	var modelFriendListItems = this.modelFriendListItems.getSelected();

	for (var index = 0 ; index < modelFriendListItems.length ; ++index) {
		if(!modelFriendListItems[index].isFacebookUser)
			returnArray.push(modelFriendListItems[index].id);
	}

	return returnArray;
},

It’s just one function out of a class that does quite something typical. This function goes over a list of items (iterate) and builds an array out of some of this items (filter). A friend of mine does some ruby coding and in ruby this would look something like this:

this.modelFriendListItems.getSelected().select{|item| !item.isFacebookUser}.collect {|item| item[:id]}

This is quite short and still yet readable. In JavaScript you can go as well in the direction of more functional programming. So let us refactor the JavaScript example above over some iterations. The following examples use prototype.js because this framework extends the JavaScript array with some fancy functions.

First we replace the “new Array()” term with the shorter term ‘[]‘. Then we try to get rid of the for loop which is quite a easy place to create a bug. Prototype does extend each iterable object (array is one of them) with a set of functions. One of them is the “each” function which does apply the anonymous function given as a parameter to each element of the iterable.

/**
* Returns array of uids of selected items
*/
getSelectedFB:function() {
	var result = [];
	this.modelFriendListItems.getSelected().each(function(item) {
		if(item.isFacebookUser){ result.push(item.id);  }
	});
	return result;
},

This is in my point of view already easier to read and it might be harder to create bugs as in a for loop. I do not like that you have to define a result array ‘outside’ of the main function. So the following code will do exactly the same in a more recursive style. You give as a starting value a empty array to inject and the anonymous function does now have to parameter where the first one is the array with the values and the second one is again the current item. You have to return the array at the end of the function. This makes the code again a bit shorter.

getSelected:function() {
	return this.modelFriendListItems.getSelected().inject([], function(array,item) {
		if(!item.isFacebookUser){ array.push(item.id);  }
		return array;
	});
},

You get the code might get a bit more readable it you use two other functions of prototype. ‘findAll‘ does return a array of all items that do match the given filter function. ‘pluck‘ does return an array with the values of a given field from all items.

getSelected:function() {
	return this.modelFriendListItems.getSelected().findAll(funtion(item){
	return !item.isFacebookUser;}).pluck('id');
},

The only way, I can think of, to make this short and more readable would be to have the function ‘filterAllBy’ and ‘filterAllByNot’ in prototype. This is currently not the case, so the example bellow will not work.

getSelected:function() {
	return this.modelFriendListItems.getSelected().filterAllByNot('isFacebookUser').pluck('id');
},

Some people did ask me if this code is not slower. Well it is! During the iteration of refactoring you lose some speed, but that should usually not matter in a real world application. But I spend also some time on how you could make your code faster with smart functional programing. I found out that generators (the one from python) are quite a good way to save cpu time under some circumstances.

For this I made up this ridiculous example. It’s the same thats in the article about phyton in the german computer magazin c’t (“Neues aus der Schlangengrube”, c’t 5.2009, p. 198).

var squares = $R(1, 1000000).map(function(n) {
	return n * n;
});

var result = [];
squares.each(function(n) {
	if (n > 100) throw $break;
	result.push(n);
});

$("output").innerHTML += result;

Off course this example is horible slow and will nearly crash your browser. I measured something like 24 seconds on a IE 8, something around 6.5 seconds on a Firefox 3 and around 400 milliseconds on chrome. A lot of the numbers you generate into the array squares are never used. With a generator you only generate the numbers you actually need.

I actually implemented a generator for prototype.js. It’s quite easy to create a own enumerable in prototype you just have to extend enumberable and implement the ‘_each‘ method. I did this in the following example with the class Generator. I also did implement the method mapGen directly into the Enumerable, it does have the same function as prototype.

var Generator = Class.create();

[...]
 mapGen: function(iterator, context) {
		var result = new Generator(iterator, this);
		return result;
	},
[...]

Object.extend(Generator.prototype, Enumerable);
Object.extend(Generator.prototype, {

	initialize: function(generator, enumerable) {
		this.generator = generator;
		this.enumerable = enumerable;
	},

	_each: function(iterator) {
		this.enumerable.each(function(item){
			iterator(this.generator(item));
		}, this);

	},
});

The code looks afterwards still nearly the same:

var squares = $R(1, 1000000).mapGen(function(n) {
  return n * n;
});

var result = [];
squares.each(function(n) {
  if (n > 100) throw $break;
  result.push(n);
});

$("output").innerHTML += result;

The real difference is the execution time. As the first example took some seconds on the most browser, this code here runs in around 1 millisecond on all browsers. Nice looking code does not have to be slow, with some smart tricks you can write short, readable and fast code.

Die letzte Woche habe ich im Silicon Valley verbracht. Mit einer Gruppe von 12 Leuten haben wir verschiedene Firmen und Organisationen besucht. Hier zähle ich ein paar wild zusammengewürfelte persönliche Eindrücke auf:

1. Die zwei heissesten Themen in US-Firmen sind Cloud Computing und Green IT.
2. Trotz Green IT ist die durchschnittliche Temperatur im Sitzungszimmer nie höher als 18°. Warme Kleidung ist empfohlen.
3. Je niedriger die Temperatur im Sitzungszimmer, desto grösser das Unternehmen.
4. Die NASA arbeitet mit Original-Anlagen aus den 50er-Jahren. Gut amortisiert!
   

   Von Silicon Valley Tour 2009

5. Google ist jene Firma, welche am häufigsten unsere Fragen mit “I cannot comment on that” beantwortet hat.
6. Executive Briefing Center sind Gebäude, wo grosse Corporation ihre Kunden mit blinkender Hardware, PowerPoint-Präsentationen und Donuts von ihren Produkten überzeugen wollen. Diese Gebäude sind in sicherem Abstand zu den Anlagen, wo ernsthaft gearbeitet wird.
7. PowerPoint-Folien von US-Firmen enthalten mehr Text als eine Telefonbuchseite.
8. Mozilla Corporation’s Stolz ist (neben alter Netscape-Memorabilia) deren Sammlung an Alkoholika.
   

   Von Silicon Valley Tour 2009

9. Will man Security-Guards kennenlernen, so packe man auf den Geländen von Apple, Google oder Sun den Foto-Apparat aus (Quelle: @marcammann).
10. Die meisten Firmengebäude sind ein- bis zweistöckig und sehen aus wie Bungalows. Das verleiht dem Valley den Groove eines Club Med Feriendorfs.
11. Ein echtes IT-Startup hat ein wandfüllendes Whiteboard und ein Kühlschrank voller Gratis-Getränke.
12. Ein Semester an der Stanford University kostet $20’000 an Studiengebühren. Man kauft sich damit in ein Kontaktnetz.
    

    Von Silicon Valley Tour 2009

13. Die Stanford University besitzt 2 Olympische Schwimmbecken, ein Stadion, und um 6 Uhr morgens sind die umliegenden Hügel übersät mit Joggern.
14. Man joggt im Baumwoll-Shirt. Damit wird anhand der Schweissflecken die Leistung für das Umfeld sichtbar.
15. Party auf amerikanisch: Ein 30-Pack Bud Lite kaufen und im Zug, auf der Fähre oder im Park konsumieren (Gilt für Männer und Frauen).
16. Parkieren in San Francisco kostet $7.50 pro Stunde.
    

    Von Silicon Valley Tour 2009

17. Es gibt den Öffentlichen Verkehr. Er ist langsam (Zug fährt rund 30km/h) und unpünktlich.
    

    Von Silicon Valley Tour 2009

18. Die Varianz der Kundenfreundlichkeit ist weitaus grösser als in der Schweiz: In einigen Geschäften wird man keines Blickes gewürdigt. In anderen wird man umarmt, wenn man erwähnt, dass man aus der Schweiz kommt.
19. Jedes zweite Auto im Silicon Valley ist ein Toyota Prius.
20. Nur die Obdachlosen rauchen.

Weitere Fotos sind hier zu finden.

yahoo performance-team proved this very well.

Also, pages (like tilllate.com) more often make heavy use of JavaScript with the goal of providing a better user experience. The problem is, that most developers work on modern hardware and develop on their favourite web browser – which is usually a recent one. They forget that a big amount of visitors still

surf with their first computer bought 7 years ago at Interdiscount and is mostly surfing on the developers least favourite browser (like IE 6). As a result, those visitors often do not get the optimal user-experience and the website loses traffic (=money).

The conclusion out of this is:

• We have to optimize our page following yahoo’s findings.


• We have to optimize our JavaScript-execution.

Where to start?

Most of our traffic is generated through the photo album. We analyzed and improved following use-case in terms of loading-speed and responsiveness:

User comes to homepage…

…goes to the a photoalbum…

…clicks through the photos

Tweaks

We then did the following things:

• Reduced HTTP requests needed to load the JavaScript files: from 45 to 1-2 requests by merging the multiple files into a single one.


• reduced file sizes of CSS- and JavaScript files by using yui-compressor


• reduced amount of external scripts (like Google Analytics) or load them after everything else to reduce the dependency on third party hosts

Also with the photo album being a JavaScript app we went through the code and implemented the following improvements:

• reduced DOM-operations


• reduced amount of written code by refactoring


• found and removed some common memory leak-patterns


• removed fancy fading-effects as they were useless and took a lot of CPU time

The results

For example on IE7

(50% of our users are using it) we had following improvements:

• time loading and set up the photoalbum decreased from 4.8 to 2.9 seconds


• time switching between thumbnailpages decreased from 1.3 to 0.3 seconds


• the general loading/parsing-time of JavaScript decreased from 1,2 to 0.6 respectively 0.5 to 0.3 seconds (Firefox).

Once we released the tweaks, the pageviews per visit increased by 20%.

Then you should apply for a job at tilllate.com: Our team is looking for dedicated web developers. You’ll find more information in our job ad:

• Web developer (English)
• Web-Entwickler (Deutsch)

The motivation for the sprint retrospective is:

• Visualize the accomplishment – important for the team morale
• Review any impediments and discuss measures on how to avoid them in the next sprint

Here’s how we are structuring the sprint retrospecives:

The set up

The team and the product owner are allocating one hour in the meeting room. We’re looking at the wall with the task board showing the user stories, the burn down chart and the impediment backlog. This is a big paper with a post it for every impediment encountered during the sprint. We collected the impediments during the bi-weekly scrum meetings (aka daily scrum).

Visualize the achievements

First, I go through all done user stories and say a few words about every story. Time for praising the team. Developers often think “we haven’t achieved anything”. So it’s important to visualize the finished user stories.

Getting the metrics

Second, we update the burn down chart and calculate the achieved velocity ([accomplished story points] / [available man days] = [velocity]). In our case we had 74 man days available. We accomplished 38 story points. That’s a velocity of 51%. Better than last time (39%). But still quite low as we committed for 50 story points. Fail. No praise here.

Learning the lessons

Third, we do the the retrospection round: This round is split in two parts.

First, we look at the conclusions from the last sprint retrospection a month ago. Were we able to implement the lessons learned? For example: We found that the product owner Martina did not have enough time to quickly reply to the questions of the team. As a measure we gave her an assistant. Did it help? - “Yes”, the team confirms, “Martina was did have a lower response time than during the former sprint”.. Good.

Then, we go through all the impediments we collected during the daily scrums. We discuss what we can do to avoid them during the next sprint. Example: “The changes on the CSS were much more complicated than expected because our CSS is the mess”. The conclusion is: “We plan a CSS clean-up”.

Results

This meeting took about one hour. The result is:

• key metrics about the past sprint (expected velocity vs. actual velocity etc)
• A list of impediments along with the measures to avoid them during the next sprint

The team now has one day of slack time before we do our third sprint planning on Thursday.

“It’s 99% complete”, I was told. Yeah, right. I counted 2 story points (without looking at the code). Soon I knew I was too optimistic: Classes calling other classes without logic. Randomly named variables. Data, creation of DOM elements, AJAX calls spread all over the place:

Architecture before refactoring (dramatized)

That’s not going to be easy. To make things worse: I am a Javascript beginner. What I did up to now was procedural Javascript code.

I felt the urge to begin everything from scratch. But I remember Joel Spolsky saying that we should resist so I decided to go for a refactoring.

As I know the MVC pattern very well from Zend Framework I tried to apply this pattern. Separation of concern! Each class has a well defined purpose. Here’s the architecture I refactored the code to:

1. Model: All operation on data (i.e. the friends to be selected) is in the classes “ModelFriendListItems” (which aggregates “ModelFriendListItem”). The Ajax request fetching the friend objects from the server is located ther as AJAX is about data.
2. View: The three views for the panes “Parent container”, “All friends” and “selected friends”. DOM manipulation only in these classes.
3. A controller in charge of instancing and controlling the model and the views. The controller handles all events. No AJAX.

Here’s the sum up of the concerns of the parts:

As every class has his well defined concern you quickly know where to look when you need to maintain or extend the control. The name of the class and files reflects the role (ViewAllFriends.js, ModelFriendsListItems.js). Logical event and data flows.

You can try it here. Here’s how the final version looks:

Well… It took much more than the 2 story points I counted in the beginning. But it feels much better having clean code! And my colleagues having to extend this control (Leo?) are surely happy for this clean up

I am sure there are other ways to model a Javascript control. What is your best practice?

Sprint planning meeting

Here’s how we do it: Martina, our product manager for the website is the product owner. She collects all feature requests from the clients. In the case of tilllate this might be photographers, sales guys, visitors from the website. These features make up the user stories. She prioritizes these user stories lets the CEO sign them off. This creates the product backlog.

Sprint planning

Enters the IT team: During the sprint planning we go through every item on the product backlog. Martina explains each user story. The team does an estimate in story points. Once the estimation is done, every team member can choose which user story he would like to implement. This takes roughly 2 hours for a 3 week sprint.

This information is tracked on our scrum board which has a swim lane for every developer. Every user story is a post-it.

Scrum board

At the end of the sprint planning every developer knows what he has to do in the upcoming sprint.

Daily scrums?

No, we don’t do daily scrums. Yet. As my developers are not used to daily meetings I did not want to shock them (“Change management”). We started with bi-weekly meetings where we update the scrum board and the burn down chart.

Sprint review

At the end of the sprint we are doing a demo and a review of the sprint. In the demo every developer shows what he has accomplished during the sprint. This anticipates the feeling of “I did not get anything done” that so many of us have. The sprint review is a performance and problem analysis of the sprint. Did we meet our goals in term of story points? Why not? What can we do to meet them in the next sprint?

First experience

Today we had the sprint review for the sprint “1″ and had the sprint planning for sprint “2″. Here’s our first experience:

• Hot: The IT team is much more motivated than before we used scrum: They can choose themselves, what they would like to work on. And they have a goal
• Hot: As the IT team is doing the estimates they are committed to meet them. They are more focused.
• Hot: The product owner and the customer is happy as they have a high transparency. They always know what the team is up to. This increases trust in the IT team
• Not: We committed to 58 story points. At the end we only accomplished 23. That’s an awful velocity 39%. Bad. We had to take a number of measures for sprint two. For example: Decrease velocity down to 70% and better estimations.

Burndown chart of the first sprint

*Thanks, Peter Stevens, Leo Büttiker and Maarten Manders for motivating me to try it.