Debuggable.com - Blog

JavaScript Meetup Hamburg + Slides

felix@debuggable.com — Thu, 04 Mar 2010 11:43:53 +0000

Update: Andy Wenk posted a very nice summary of the event.

Last night Tim and I took a little road trip to Hamburg. I had no idea the Reeperbahn looked like Las Vegas : ).

Anyway, our actual destination was the first Hamburg JS meetup where Malte Ubl invited me to speak about node.js. The turnout was fantastic, and thanks to SinnerSchrader's hosting of the event, there was plenty of pizza, beer and an absolutely fantastic location.

I've also updated my previous node.js talk, all the examples should now be 0.1.30+ compatible, and the section about "The Future" of node has a more recent and interesting list of things that are on the radar:

Node.js - A Quick Tour II

You can also download the slides as PDF (164 kb).

--fg

Apology to the CSF and CakeDC

felix@debuggable.com — Thu, 11 Feb 2010 17:37:44 +0000

After our previous post, we are very happy to announce our formal apology to the CakeDC and the Cake Software Foundation.

After further discussions with the CSF we have come to understand why the CSF needs to have an agreement with the venue, this is required to free the funds for making this transaction since the CSF would be legally responsible for the payment. Larry E. Masters has helped us to work out all misunderstandings in that regard, and we would also like to formally apologize to him.

Beyond that, we are also apologizing to the CakePHP community, for jumping the gun early, and having caused damage to the image of the project. This was not our intention, and we only did that, as a last resort to see the venue paid and defending our business from harm.

As previously announced, we are still planning on being valuable members to the community, releasing plugins & components as open source.

We are grateful to all those who donated, and will reimburse everybody completely once the venue has been paid in full, or in no longer than 60 days from the time of your donation, whichever comes first. We will take down the pledgie site also. Until then we will continue to pay the venue 500 EUR / month, so that any eventual delays will not be to their disadvantage. We also had a misunderstanding with the agreement Larry E. Masters had made to personally cover the 500 EUR / month while the CSF was trying to clear up the situation with the 501(c)(3) status. He also clarified, that the CSF is an independent entity from the CakeDC with its own board of directors which currently hold these positions temporarily and that a new board will be in place when all of this legal stuff is completed.

Please, do not let any of this reflect on your opinion about the framework. Every community has their difficult moments, and the CakePHP community has just shown how strong it is, and that all parties are willing to cooperate for its prosperous future without leaving any hard feelings behind.

Sincerely Yours,

Felix Geisendörfer, Director
Tim Koschützki, Director

Thank You!

felix@debuggable.com — Thu, 11 Feb 2010 10:07:29 +0000

Latest Update: The whole situation has been resolved, please read our apology to the CSF and CakeDC.

We are truly thankful. Since putting up our post last night, people have already donated 2,075.00 EUR. There were almost 30 donators, some of them donating as much 600 EUR, with others who have pledged continuing support over the next months.

We cannot thank everybody enough for this, and all the kind comments. But as a start, here is the list of people who have donated so far, we'll keep it updated. If you want your name to be linked, just email us.

Garrett Woodworth
Shawn Stratton
Phillip Hile
Aaron Forgue
Rob Wilkerson
Clay Loveless
Gilles Grinfeder
Jeff Loiselle
Lars Schenk
Nathan Abele
@elventails
@indiefan
Yusuke Ando
@jperras
Jon Adams
Jose Diaz-Gonzalez (@savant)
Dirk Olbertz
Mark Boas
Charlie van de Kerkhof
Lucian Lature
@leomelzer
Robert Scherer
Guillaume Sautereau
Daniel Hofstetter
Arne Psczolla
@pzwiers

Regarding the question: Did we publish this to harm the CakePHP community?

The answer is no, we are sorry if this shines a bad light on the project. We had two motivations to publish this post. For one, we wanted to reach out for help with this bill. However, we also wanted to share our story with the community, giving them the chance to push for improvements of the situation CakePHP is in.

But you don't have to take our word for it. We already have a few new plugins & components lined up for release. This project has a community of great people, and just as they have supported us in this tough moment, we plan to continue supporting them.

Again, thanks to everybody who helped, also those who are not involved with CakePHP at all!

--fg

The High Cost of Open Source

felix@debuggable.com — Wed, 10 Feb 2010 19:58:14 +0000

Latest Update: The whole situation has been resolved, please read our apology to the CSF and CakeDC.

6 months ago, I posted about a great event which we helped to organize: CakeFest #3, here in our home city of Berlin. This event helped to connect developers from all over Europe, and even as far away as Australia, Tokyo and Los Angeles. There were great people, talks, and food, and everything ran extremely smoothly. By any measurement, it was a huge success.

This did not happen by accident. A number of people made a big effort to put the event on, starting with lots of planning and communication months beforehand. Debuggable helped by finding and booking the venue, researching, and coordinating logistics here on location. We didn't make money from this, and that's okay. We wanted to support the CakePHP community by hosting another awesome event.

Over 6 months later, the GLS, who provided us with first-class service and what was probably the best venue ever, has still not received payment.

Don't be mistaken, this was not due to any errors in the organization of the event. The turnout was great, and the event was well-sponsored. The Cake Software Foundation, Inc., which collected all revenues, has already reimbursed its own organizing participants, but completely neglected to pay the venue (the bulk of the conference's expense). To make matters worse, the CSF has had their accounts frozen by the IRS.

How do we know this? Unfortunately, we are in the middle of this mess. In our efforts to help organize the event, we signed the proposal for the location and catering. We did so in good faith, because we were working in cooperation with the CSF, and of course had all such details approved.

When, in early October, we discovered that the venue never received payment, we initially believed it to be a mistake, and started delivering messages between the venue and Larry E. Masters, who in the meantime had taken over the position of CSF president after Garrett Woodworth had resigned. Larry told us he would take care of it.

Weeks passed, and we talked to the GLS staff every other day, asking them for their patience, as there appeared to be "problems" with getting the payment sent. A month later, on November 10th, Larry agreed to send a token payment of USD $4500 through the CakeDC via PayPal, which we forwarded to the venue (they do not accept PayPal).

Unfortunately, this is still the latest update on the payment. Of the total invoice of EUR 7665, the first payment (after currency conversion and PayPal fees) only amounts to EUR 2807, leaving an outstanding balance of EUR 5058 (USD $6929).

The venue has been incredibly patient and understanding through this whole process, but their patience has run out, and they're considering legal action against Debuggable. We signed the proposal, so technically they're not wrong: we are responsible.

At this point it's pretty clear that the CSF has no intention of taking responsibility for this, which leaves us with some hard choices. We could argue that, by invoicing the CSF and not us, that the venue implicitly signed over the contract to the CSF. We have to admit, we briefly considered doing that. 5K is not pocket money for us, and this definitely isn't our bill.

However, that would be wrong. Not to say that us paying this bill isn't wrong, but this is a wrong we can actually right. The venue deserves to get paid. So, we decided to go ahead and set up a plan to pay back the venue over the next few months.

Since a few people who have been aware of the situation have asked us to provide a way for them to help, too, we have put up a page on pledgie.com:

Donate to the CakeFest Venue Payment

We greatly appreciate any support, as it helps ease our burden, but the venue will be paid regardless of the outcome of the pledge. Right now, the most likely outcome is that we'll have to take care of this on our own, but we're still holding onto the hope that this last-ditch effort of going public might help change Larry's mind.

What's worse is his company, the CakeDC, now has exclusive controlling interest over a project which was once driven by its community. Despite his slowness to work this out with us, he has shown no hesitation in taking full advantage of the benefits of this control. If the CSF's position changes in any way, we will certainly send out an update.

--fg

PS: There is much more to this story, the CSF shake-ups, and why the their accounts are frozen, but it goes well beyond what we can verify. In the end, it doesn't matter; this isn't about finding fault, but about taking responsibility.

Unit testing with node.js

felix@debuggable.com — Sat, 30 Jan 2010 18:41:42 +0000

Even so most stuff is fairly easy with node.js, unit testing takes a little bit of getting used to. The main difference to regular unit testing is that just because your test finishes running without an error, it doesn't mean there was no problem. But lets start with the basics.

Node.js has adopted the CommonJS assert module specification. You may also find references to mjsunit, but node no longer uses this library.

Lets have a look at how to write a basic test with node, lets call it my-test.js:

var assert = require('assert');

// Will pass
assert.ok(true);

// Will throw an exception
assert.ok(false);

Running this test will produce the following output:

$ node my-test.js
AssertionError: true == false
    at Object. (my-test.js:7:8)
    at [object Object]. (node.js:928:23)
    at [object Object].emitSuccess (node.js:240:15)
    at [object Object]. (node.js:653:21)
    at [object Object].emitSuccess (node.js:240:15)
    at node.js:510:29
    at node.js:985:1
    at node.js:989:1

One thing to note here is that, when node dies from an exception (which a failing assertion is), the exit code will be 1:

$ echo $?
1

But lets have a look at a more interesting unit test. Let's say you are writing a "hello world" web service called hello.js:

var http = require('http');

exports.server = http.createServer(function(req, res) {
res.sendHeader(200, {'Content-Type': 'text/plain'});
res.sendBody('hello world');
res.finish();
});

To test this service, we can write a file called test-hello.js like this:

var
assert = require('assert'),
http = require('http'),
hello = require('./hello'),
callbackFired = false;

hello.server.listen(3000);

http
.cat('http://localhost:3000/')
.addCallback(function(data) {
callbackFired = true;
assert.equal('hello world', data);
hello.server.close();
});

process.addListener('exit', function() {
assert.ok(callbackFired);
});

What's important in this test is that we are checking for two things here.

The first is the obvious assertion that the web service outputs the content we expect, nothing too exciting here.

The second is declaring the variable callbackFired and checking its value in the process 'exit' event. You could get away with not doing it for this particular test, because the test would never finish without the hello.server.close() line getting executed. For more complex tests however, it becomes incredibly important to keep in mind that a callback may never fire, something you can only catch with little helper variables like this.

If you need some more inspiration for testing different stuff with node, you should start by going over the test suite that comes with it.

Let me know if you have any questions!

-fg

Git Fake Submodules

felix@debuggable.com — Wed, 20 Jan 2010 14:07:44 +0000

Git submodules suck. They are a pain to use, difficult to explain and you cannot check out partial trees. The later is an inherent limitation of git, but I have a fix for the rest.

Meet fake submodules. The idea is simple, instead of using actual submodules, you just trick git into thinking the files belong to the main repository while having the respective sub-dirs remain independent clones. Doing that is simple:

$ cd my-project
$ git clone <subproject-url> my-subproject
$ git add my-subproject/

The important part is the "/" (slash) at the end of the last command. If you omit that, git will automatically assume you want to add 'my-subproject' as a submodule. But if you don't, git just sees the files in the sub-directory and ignores that fact that its a git-repo of its own.

The cool thing is that you can now update a fake sub-module to the latest version as simple as:

$ cd my-subproject
$ git pull

This works because when you are inside my-subproject, git uses the '.git' folder closest to it which is my-subproject/.git. If the sub project is your own, you could even push your changes to it upstream without changing projects.

Now as far as collaboration is concerned, none of your collaborators will have a clue about your fake submodules. All they will see is some code in some folder. They will not be able to go into the my-subproject folder and git pull on it. However, if they need to do that, they can just rm -rf my-subproject and replace it with a clone themselves.

To me this is a pretty perfect & easy solution. I mostly use it for CakePHP plugins & node.js modules I am working on, but it works just as great with other peoples code you want to depend on. You can even apply hacks against their code, have them in your main repository, and make sure they stay applied on top by using 'git pull --rebase' when updating the sub project.

Let me know what you think!

--fg

JavaScript Meetup Berlin + Slides

felix@debuggable.com — Wed, 13 Jan 2010 10:08:46 +0000

Last night I had the opportunity to speak about node.js at the first Berlin JavaScript user group meeting. I'm really thankful to Robin Mehner, Jan Lehnardt and the awesome people at upstream and co-up for organizing the event and hopefully many to follow.

Anyway, here are the slides:

Node.js - A Quick Tour

View more documents from Felix Geisendörfer.

You can also download them as PDF (344 kb).

The meetup itself was rather awesome. ~35 people showed up with many coming from different backgrounds (Rails, Symfony, Zend, Python, ...) united by their interest in JavaScript.

-- fg

CakePHP Authsome - Debuggable's Xmas Gift

felix@debuggable.com — Fri, 25 Dec 2009 12:49:42 +0000

Merry Xmas everybody. We hope you are having a lovely time and get to renew the batteries for an awesome 2010 over the holidays.

To ease the withdrawal symptoms, Tim and I decided to do some light development over the holidays in order to create a little Xmas gift for the CakePHP community:

The Authsome plugin: Authentication for people who hate the AuthComponent.

We have been using a system very similar to Authsome for all our CakePHP projects over the years, but it wasn't until now that we had a chance to properly decouple it from the individual projects we used it in. A big thanks for that goes to ThreeLeaf Creative, the makers of a fantastic CakePHP CMS system who paid for the refactoring.

Next on the radar is Righteful: Acl for people who hate the Acl component. We hope to have it ready at some point early next year.

Alright, so go ahead and read the authsome docs over at Github and let us know what you think.

Merry Xmas,

-- Tim & Felix

RightJS 1.5: 6-8 times faster than jQuery

felix@debuggable.com — Wed, 09 Dec 2009 17:16:48 +0000

My journey of mastering procrastination has led me to an interesting article on Hacker News today:

RightJS 1.5: 6-8 times faster than jQuery

(The title has been updated to "RightJS Version 1.5.0 Is Out" since I started writing this.)

Wow, I thought! This sounds like an excellent example of cargo cult science, one of my favourite subjects.

I mean I love innovation in this field just like everybody else. But seriously, jQuery is not exactly know for being slow & heavy. So anybody claiming a 6-8x speed improvement must have achieved an unbelievable breakthrough. Either that, or he must be using using the cargo cult method.

Applying the cargo cult method to performance testing is rather simple, which probably explains its popularity. You pick a random series of tests that can be run against the various implementations you want to compete with. Then you spend a few hours hacking away at your implementation until it is the clear winner. Don't give up if it becomes too hard, just tweak the test cases to slightly favor your implementation. It's really as easy as that.

I can totally understand why people are doing that. The opposite would mean that you have to apply the scientific method, which is really cumbersome. First you have to collect data, lots of it. In our case that means performing very detailed analysis and profiling of a large enough set of real world JavaScript applications. Using this data set, you should be able to answer questions like: What are the most common selectors people use? What DOM operations are popular? Which of those are actually relevant to the performance of the analyzed applications? With those answers you can attempt to come up with a series of tests that will rank the various implementations according to their performance. But actually writing those tests will be very hard. Should you use the most distinct and sexy way in each implementation? Or should you use the most effective techniques people have come up with?

Luckily there is a third option. It is called specialized benchmarking. You start by admitting that the things you are going to test are purely based on your curiosity about them, possibly because they are related to the particular problem YOU care about. Make it very clear that the outcome of those tests should in no way be seen as an indicator for overall performance and try to hide them from people who don't know what that means.

Specialized benchmarking will possibly not answer anybodies questions other than your own, but it beats the hell out of the cargo cult method.

Let's examine why the RightJS performance tests get it wrong and what they could do about it. From this point on I will only refer to material on their page, the post on Hacker News was just how I heard about them.

First of all, they claim that their performance benchmarks are there "To give you some ideas about the project quality and abilities". I think that should be changed to: "We are especially fast for the following operations (...), those however are not proofen to be good indicators for general performance in JS projects.". It's kind of like weight loss advertisement. You can show pictures of people who lost 50 pound, but you gotta put that "* Results are not typical" note there. This way people can pause for a second, and remember that there are no magic bullets to weight loss and consider their purchase with that in mind.

After that, they could start to decide whether some of their tests are worth keeping, and if so, make sure that they are as scientific as possible. I'll just use their test #1 as an example, but check the test suite for yourself, to see that this pattern is repeated throughout the entire thing.

Testing jQuery DOM building (343ms*):

"make": function(){
for(var i = 0; i<250; i++){
$("<ul id='setid" + i + "' class='fromcode'></ul>")
.append("<li>one</li>")
.append("<li>two</li>")
.append("<li>three</li>")
.appendTo("body");
}
return $("ul.fromcode").length;
}

Testing RightJS DOM building (80ms*):

"make" : function(){
for (var i = 0; i < 250; i++) {
document.body.appendChild(
new Element('ul', {
'class': 'fromcode', id: 'setid'+i
}).insert([
new Element('li', {html: 'one'}),
new Element('li', {html: 'two'}),
new Element('li', {html: 'three'})
])
);
}

return $$('ul.fromcode').length;
}

I smell cargo! First of all, why is RightJS using a native DOM method, document.body.appendChild, and jQuery has to use .appendTo('body')? Those are two radically different operations, and just to see how radical lets make the following change:

Optimized jQuery DOM building I (194ms*):

"make": function(){
for(var i = 0; i<250; i++){
document.body.appendChild(
$("<ul id='setid" + i + "' class='fromcode'></ul>")
.append("<li>one</li>")
.append("<li>two</li>")
.append("<li>three</li>")[0]
);
}
return $("ul.fromcode").length;
}

Ouch, an error rate of 43% against jQuery. Let's try harder:

Optimized jQuery DOM building II (72ms*):

"make": function(){
for(var i = 0; i<250; i++){
document.body.appendChild(
$(
"<ul id='setid" + i + "' class='fromcode'>"+
"<li>one</li>"+
"<li>two</li>"+
"<li>three</li>"+
"</ul>"
)[0]
);
}
return $("ul.fromcode").length;
}

If this was a presentation I would have an LOLCat saying "jQuery rulez" right now. But luckily this isn't and I'll try to reason scientifically about this.

jQuery is NOT faster in this example. Don't believe the numbers you see. They have been meaningless all along. The reason for that is simple: While initially it looked like we were performing the same test with jQuery as we were with RightJS, we never actually did! The jQuery example, from the beginning, was creating DOM elements from HTML strings, while RightJS was wrapping the document.createElement API. This is not the same thing and you cannot learn anything from comparing apples to oranges.

The truth as far as this test case is concerned? Well, jQuery simply does not have a document.createElement wrapper. Thus you cannot compare it to implementations that do. And why should you? DOM building like this is largely useless, given excellent alternatives such as John' Micro -Templating engine.

Just to show how useless this test was from the beginning, here is my not so paradox implementation that outperforms the pure DOM test:

Testing Pure DOM building (37ms*):

"make": function(){
for(var
body = document.body,
ul = document.createElement("ul"),
li = document.createElement("li"),
i = 0,
fromcode;
i < 250; ++i
){
fromcode = ul.cloneNode(true);
fromcode.id = "setid" + i;
fromcode.className = "fromcode";
fromcode.appendChild(li.cloneNode(true)).appendChild(document.createTextNode("one"));
fromcode.appendChild(li.cloneNode(true)).appendChild(document.createTextNode("two"));
fromcode.appendChild(li.cloneNode(true)).appendChild(document.createTextNode("three"));
body.appendChild(fromcode);
};
return utility.getSimple.call(body, "ul.fromcode").length;
}

Optimized jQuery DOM building III (36ms*):

"make": function(){
var elements = '<div>';
for(var i = 0; i<250; i++){
elements = elements+
"<ul id='setid" + i + "' class='fromcode'>"+
"<li>one</li>"+
"<li>two</li>"+
"<li>three</li>"+
"</ul>";
}
$(elements+'</div>')
.children()
.each(function() {
document.body.appendChild(this);
});

return $("ul.fromcode").length;
}

As you can see, the cargo cult method is quite powerful : ).

Anyway, I don't want to discourage the development of RightJS in any way. I think it's awesome that there are libraries that are trying to compete with jQuery.

It is really hard to do meaningful performance testing and infinitely easy for some random punk like me to come along and point out all the flaws. To me, even trying to do a general purpose performance test against 6 (!) implementations, that is pure bravery. So in case you decide to do something similar, just admit the odds you are up against and people will be very forgiving and engaged.

Comments, hate mail & suggestions are welcome!

-- Felix Geisendörfer aka the_undefined

* Results not typical - Some recent version of Firefox on my Laptop, picking random samples from runs that looked good!

Parsing form data with node.js

felix@debuggable.com — Tue, 24 Nov 2009 15:43:15 +0000

Many people asked about form parsing in #node.js after the initial buzz-wave yesterday.

Right now node does not include a parser for regular form data (application/x-www-form-urlencoded). However, you can use the http multipart parser to achieve the same thing.

Here is a bare-bone example for that:

Run this code and point your browser to http://localhost:8000/. You will be presented with a form, and when you submit it, you will see the contents of the POST as JSON. For more information check:

The important part is specifying the enctype of your form as "multipart/form-data".

If you need to parse regular form data, have a look at sixtus www-forms module. Chances are good a module like this, with a similar API to the multipart parser, will make it into the core at some point (patches are welcome).

HTH,
-- Felix Geisendörfer aka the_undefined