Michiel Overeem's blog

JavaScript architecture - some lessons learned

2013-02-22T00:00:00-08:00

The secret to building large apps is NEVER build large apps. Break up your applications into small pieces. Then, assemble those testable, bite-sized pieces into your big application. Brian Moschel

In the past year, I’ve been busy with researching architecture in JavaScript. And there are a few things that I’ve found and learned during this year. Now, I am nowhere near being an expert, I’m just someone that loves the language and the platform it offers. I happen to like the fact that it opens up so much possibilities, because JavaScript is everywhere!

I would like to list some principles and observations that follow from my learning.

Create little parts and not one big application

Now, JavaScript is a language that misses some features when you compare it with more mature and better designed language. There are no modules or namespaces, and there is only function-scope. When you do not look for a solution for this missing, you end up with spaghetti code. One of the things I’ve learned is that a module framework is one of the most important parts of your application.

I myself are only in depth familiar with RequireJS, but there are more. A module framework forces you to break up your app in little parts that have their own scope and clearly export what others can use.

#js
define("moduleNam", [/* list of dependencies*/], 
  function ( /* dependencies */ ) {
  
    /* place for private code */
  
    return {
      /* your contract with the outside */
    }
  }
);

You see, a module has clear definition of what it needs and what it delivers. This way, you are forced to let go of easy global variables.

Create abstractions for changing things that you cannot control

One of the really hard points that I discovered is working with a quickly changing environment. The third party libraries are developing fast. And it makes it hard to plan for the future. Of course no one wants to right code that we do not need. But last year, the jQuery UI library was widely used. Nowadays, it is Twitter Bootstrap everywhere. How do you deal with this? Do you choose and never look back? Or do you try to abstract it? Or do you write your own code?

Unfortunately there are no easy answers here. I found it to be possible to abstract some of these choices. But your situation might be different. But it is important to choose widely what you depend on. The JavaScript world is moving fast. Depending on something that is hard to change might be a mistake.

Be careful with accessing the DOM

The DOM is what your environment looks like in JavaScript running in the browser. You will need it. But you do not want it to be everywhere. JavaScript code is not compiled. Unit tests are more important then ever. Nobody likes script errors that make a web application become useless. So testing is required. But using the DOM everywhere makes your code untestable. Writing code that is modular and testable is a must!

There are more lessons and more paths to follow. But these three items are key to success in my opinion.

Side project - drawing signatures in HTML5

2013-02-03T00:00:00-08:00

This weekend I worked on another little side project. This time it consisted of a little HTML5 and JavaScript to use the power of a small tablet (iPad mini in this case). And it amazes me again how easy it is to put some open source projects together and get something working. Of course the proof-of-concept is no big deal, and it is not production ready. But all these little machines that we have give us so many possibilities.

The requirements are easy: enter information about work done at a customer and get a signature of someone. This will then by e-mailed to both the company and the customer, forming the basis of the bill that needs to be payed at the end. The tablets where chosen easily: iPads where already available. The code will hopefully be build in such way that we can easily expand support to other platforms, but for now we will stick with iPads. They come with a modern browser and support most of the HTML5 features. Important is the touch of course, because we will use that to get a signature.

The input of information is easy: put some input elements in the body and write some PHP script to turn it into an e-mail. We need a signature. It will be the official approval of the customer, and we do not want to put it on paper. HTML5 comes with the canvas element. That allows us to get some drawing input. But this is a side project and I do not want to write my own drawing code if it is not necessary. So I turned to Google. There I found Signature Pad, and let me tell you, using it is easy. With some configuration I managed to get a good size canvas displayed on the iPad. The PHP script transforms it to an image and with swiftmailer we build a beautiful HTML mail and send it off.

Follow progress on github!

JavaScript and the flexibility that is sometimes needed

2013-01-16T00:00:00-08:00

Integration of third part libraries. The promises that a library makes often force you to delete your own code right away and start integrating. Finally, ripping out code and replacing it with a tool, framework or library that is developed by a team instead of you. Developed over a couple of months, maybe even years. And all is good.

But is it? Because using a third party library comes at a price. What if integration is not that easy. Or what if some parts are not that stable. What if the demands from your customers are a bit different. Of course, as good developers we start with analyzing, prototyping, proofing that the library does what it needs. And of course, every manager, or stakeholder, approves and gives you the green light. Until a new requirement is found, that was not known at that time. Or maybe just not told to you. And now what to do?

There are a lot of discussions about JavaScript. About it being a language that is bad designed. About the risks in large enterprises. Microsoft started to design a better version. But there is great power in this language. At least, that is my experience. You can shoot yourself in the foot. Sure, you can also hit yourself with a hammer. In a perfect world, we might want to have static analysis that finds all the errors before we release stuff in the world. But that is not going to happen.

If you want to program for the web, there is only one option: JavaScript. And sure, you can use CoffeeScript, TypeScript or ClojureScript. But still, at one point you will hit JavaScript. I do not believe that we are already arrived at the point that JavaScript is only the assembly language underneath it all. I think that you still need to have some knowledge of the underlying core. Of course you can use tools that compile into JavaScript. But at least learn some, I think you will need it when debugging.

Back to integrating third party libraries. One thing that JavaScript stands out in, is the flexibility. The fact that you can change any object after creation makes it super easy to alter behavior of libraries. And you can mimic any object, without a type-system that will not allow it.

Sometimes it feels wrong to do this kind of stuff. And my computer science background screams for type-systems, proves of correctness and a high level of maintainability. But the maintainability of an application does not need to suffer from JavaScripts flexibility. You will need discipline. Maybe some unit tests. Even a little documentation.

But most important, we as developers need to get the job done and deliver what our customers want. Because in the end, all they care about is working applications that do the job for them in the best possible way.

Best tech books in 2012

2012-12-27T00:00:00-08:00

The year 2012 is almost done, and while enjoying my holiday I thought it would be nice to take a look back and see what books I have read and which of them I still would recommend. On looking at the list of books that I have read in 2012 (yes, I do keep a list) I saw that it weren’t that much tech books. I read more then 45 books in 2012, but only seven of them I can call tech or tech-related books.

On the tech side, the book reading was a bit quiet this year. It was my first full year at AFAS Software as a Software Architect, but that did not really drive my into more book reading. I also became a dad for the second time, so reading time is heavily cut down :)

So on to the books that I did manage to read, and after that, maybe a little bit of a wish list.

Thomas Myer - Beginning PhoneGap

A book that I reviewed for the O’Reilly blogger program. I was mildly positive about the book. Pure tech books are becoming less relevant, because stuff is moving hard. Jeff Atwood said it better:

Do highly technical books tied to a specific technology have any reason to exist in an era of ubiquitous, high speed Internet access? I wonder. I think they’re increasingly irrelevant, and almost by definition out of date by the time they manage to hit bookshelves.

But I did like to book as an entry into PhoneGap. And PhoneGap still looks very cool. I should make it a goal for 2013, to finally get something into the App Store…. I should, really…

Clay Johnson - The information diet

This book was a gift from the O’Reilly blogger program and not really technical. But it did touch on something that I recognized in my life. There is so much information coming from Twitter, Blogs and other sources that it is becoming more important to actively decide on what to read and what to ignore. Important stuff!

Jordan Mechner - The making of Prince of Persia

This book read more like a biography then a tech book, but the topic of course being very much tech. I remember playing Prince Of Persia when I was a kid, so it was really fun to read about the creative process behind it. And being a programmer in current modern times, it is really useful to read about problems like memory management and CPU limitations.

Nicholas Zakas - Maintainable Javascript

My first six months at AFAS mostly consisted of working on JavaScript and the architecture of a JavaScript application. Nicholas Zakas was a big source of inspiration, so this book was a must have. It is a nice overview and summary of knowledge.

Marty Cagan - Inspired

This book was a little disappointing because it did not talk about inspiration, but more about the job of a Product Manager. It was nice read, but it was not what I was looking for…

Donald Norman - Living with complexity

Donald Norman is well known for his books on design. And this book is no exception. This one really goes into the problems around complexity and complicated software. There is in fact a difference between complex and complicated software. Something that I was not aware of.

Sander Hoogendoorn - Dit is agile

Last but not least is a dutch book about agile software development. At my previous job I was a scrum master and implemented agile with a colleague of mine. Agile stuff is still very interesting, but often it is used more like a religion then a useful way of developing software. Sander Hoogendoorn is very pragmatic in his thinking about agile, and therefore this book is highly recommended! A German translation is already finished, and an English version is planned. Visit the website for more information!

So far for the books that I have read. What is next? What will follow next year? … I hope to continue diving into the world of JavaScript, HTML5 and front-end architecture and design. Maybe some UX. And for books? The list keeps growing, but on it are

Which books should I add? Let me know on twitter!

Suggested books

Clojure Programming by Chas Emerick, Brian Carper, Christophe Grand via @borkdude

The rise of CSS Frameworks and some thoughts on how to use them

2012-10-06T00:00:00-07:00

The advance of HTML5, CSS3 and responsive design has given rise to a whole new type of (open source) projects: CSS frameworks. The first time I saw these kind of projects I found it rather strange. I am used to frameworks in terms of reusable libraries giving some functionality. Like ASP.NET MVC, helping you serve content over HTTP. CSS however consists of rules and style definitions, there is no code, there is no functionality. These definitions can be reused of course, but it often becomes a pain to maintain large CSS files. The language itself does not have constructs to deal with reuse in a way like programming languages.

My first initial reaction was to view these frameworks not as ordinary frameworks, but rather as bootstraps: a basic starting point for your CSS design. Much like CSS resets. Such a bootstrap file gives you a certain amount of basic rules to build your layout. This approach does comes with some warnings.

Bootstrapping your project

Let us look at the relation between CSS and HTML and then how a bootstrap file can go wrong. CSS stylesheets consists of style definitions. These style definitions are annotated with selectors. These selectors link the style to your HTML file. In this way you get a clean separation: the HTML tells you what you are presenting and the CSS tells you how to present it. But they do have a stricter relationship then simple a separation of concerns. This relationship can go two ways. The selectors used in CSS can use some basic constructs: HTML tag names, CSS classes, ID attributes and some special cases (like hover, when a cursor is over an element). These constructs can be combined to match the structure of your HTML. The CSS file can use your nesting of HTML tags to link style definitions, but then the relationship becomes very fragile: a change in nesting can mess up your style. The other way around is also possible: you can annotate the HTML with class names and id attributes, but then you can wound up with to much knowledge about your CSS file in your HTML.

The two ways to use a CSS bootstrap are to use only the style definitions and not the selectors. This gives you freedom in both your HTML and CSS selector structure. However, it is lots of work and to be honest: you would only use the knowledge of the CSS in the bootstrap file and not the file itself. Another approach is to use the bootstrap file as is and change your HTML structure to match the file.

In the last approach, you get the extra benefit that it is possible to update the bootstrap file to incorporate bug fixes. But the usage of class names from the CSS file does violate the principle of separation: HTML tells what, and CSS tells how. These class names do not carry semantics for your project. They carry knowledge about what the rule definition does (for instance how much columns wide an element is presented).

Extending CSS to gain reuse

This is where two popular languages that extend CSS come into play: LESS and SASS. These languages add extra features on top of CSS to make the language more flexible; they add constructs for re-usability. They both add variables and mixins to the language.

Variables allow you to reuse certain key values, like colors or sizes. This makes it easy to change the color palette in one place, instead of throughout your CSS rules. Mixins on the other hand allow you to reuse a set of style definitions inside other definitions. This allows you to reuse for instance a set of prefix specific definitions without writing them out every time.

These language allow you to distribute a CSS framework in terms of mixins and variables. That way you can set up your CSS file and HTML file in the cleanest way, but use the already defined style definitions and gain development speed.

Reflection

So far I’ve used two CSS frameworks: Skeleton and Zurb Foundation. I like the Zurb framework because of the easy responsiveness that it adds. My approach is to add the Zurb class names throughout my HTML. Yes, this is not really clean, but the projects sizes are small and the likeliness of switching a framework or updating to a newer version are small. I am not sure of a larger project would succeed with such an approach. In such a project I would switch to just using the knowledge and style definitions, but not the actual code.

I am not sure if the LESS/SASS approach is any better. In my opinion, CSS and HTML are in a very strict relation, enforced by the language capabilities. Positioning and sizing within style definitions are depended on context of parent elements, and often you cannot view the CSS separate from the HTML. No language extension or framework will fix that problem.

But without the CSS frameworks, LESS/SASS do add better maintainability to your stylesheets. There are multiple tools to integrate it with your IDE. LESS even comes with a client-side library that transforms it on the fly to CSS. Performance would keep me from going that way, but it adds speed to the development process.

Depending on the size of a project, I would choose to learn from the CSS frameworks and write my own code or integrate them into the projects like any third party library.

Fall 2012 update on projects

2012-10-04T00:00:00-07:00

I thought it would be nice to give an update on some of the side projects that I’m working on. Reflecting on those I see that I currently am focussed on design, CSS3, responsiveness and less on core programming.

The first side project is very simple and plain informative website. I’ve started using Zurb Foundation as a CSS bootstrap and worked with Jekyll as a templating engine. There is no server-side stuff in this website but a simple mail form. I could really focus on (responsive) design on this one and I think it worked out nice.

The second project is a larger website build using ASP.NET MVC. Originally it was a website build on Joomla but I got fed up with it, because it feld limitting. Any CMS will force you in setting up pages a certain way. And styling or plugins are also limited in a way. I wanted more flexibility, so choose to build something from scratch.

I started with the design of the frontpage, again using Zurb Foundation. After the frontpage I started to add dynamic pages to move the current content. As back-end storage it used a mix of plain text files: csv, xml and markdown files. This allows me to manage the content through FTP and also to easily backup the content. There will be an administrative part, but it can wait because I am the sole maintainer.

Another reason to rebuilt this one is that I want to build a Phonegap/Cordova app that builds on stuff in the website. By designing the website in a mobile-ready way, I hope that the app will be easier.

This project is on the edge of ready for release, but I need to get approval from certain people first.

The last project is not really started yet, but I am researching the design currently. Again a simple website, but with a larger backend. Multiple roles will need to login and do their stuff. This one is constrained by the hosting platform. Therefor it will be written in PHP, and of course HTML5 and CSS3. I haven’t written PHP in years, so I am curious on the current state. Hopefully some blog posts will come from that project.

The last project is of course this blog. I hope to get to writting some posts soon. There is enough to write about, but time is limited.

Sharing that what we have received

2012-06-22T00:00:00-07:00

Last week the new MacBooks arrived. And what a gorgeous new model was introduced, with a Retina Display. I myself was waiting for the new models because the old laptop needed to be replaced. The Retina version was a little to high priced for me, but I got a beautiful 13” instead.

While reviewing and researching a new laptop, I also read some books about poverty and how we have a responsibility in al of that. We (at least my readers from The Netherlands) are living in one of the richest countries in the world. We are in a financial crisis, but still we have plenty of everything. And so I made the decision (again) to share from that what I have received.

There are a lot of possibilities if you want to share. You could give money, you could give time, etc. But in the book The hole in our Gospel, Rich Stearns makes the case for using peoples abilities and giving them a platform to work on their own poverty. Yes, we are talking about microfinance here. So that is what I have done. I’ve gone over to kiva and lend some money to people that have far less than me.

Now, I normally blog about tech stuff and software development. The whole microfinance thing has got me thinking: why are we spending money on kickstarter stuff that are run by people that have access to far more resources than those in third world countries? Would it not be cool if we as tech people could help fellow tech people all over the world to use their talents and fight poverty?

Developing, previewing and debugging on mobile devices - how to make your blog beautiful across devices

2012-03-11T00:00:00-08:00

This week I spotted Adobe Shadow: a tool for developing, previewing and debugging websites on mobile devices. The tool consists of a server component, installed on your development machine, a Google Chrome plugin and client apps (both iOs and Android). But before I tell you my experience, watch this demo video:

Pretty amazing, don’t you think? So I installed it to preview my blog. The connection was not without problems, but I managed to get it working on my iPhone and iPad. And of course, there were some problems to fix. So let me give you some things that I did to get a nice blog for mobile devices.

Start with a responsive desing

According to Wikipedia, responsive desing is:

Responsive Web Design (RWD) essentially indicates that a web site is crafted to use W3C CSS3 media queries with fluid proportion-based grids, to adapt the layout to the viewing environment, and probably also flexible images. As a result, users across a broad range of devices and browsers will have access to a single source of content, laid out so as to be easy to read and navigate with a minimum of resizing, panning and scrolling.

So the key is to use CSS3 features to get one design that scales nicely on different screen sizes. Personally, I started with a free CSS framework: skeleton. But there are many more. Pick the one that matches your style best. And of course, adapt to get your own style. I’ve choosen to let my blog be basic black and white, because I did not yet find a nice design that fits my personal style. To be honest, I like the black and white simplicity, so I might keep it longer.

Use your media queries

These things are key, because it let you define custom CSS rules, matching different screen sizes. It is far better then browser sniffing, or other conditional things, because it focusses on features and device-types and not on brands or other vendor specific things. A concrete example is the scaling of images to keep them in the main content.

!css
@media only screen and (min-width: 480px) and (max-width: 767px) {
    .container img {
        max-width: 400px;
    }
}

My blog on iPhone and iPad:

Any more tips on CSS3, mobile web and responsive design? Let me know on twitter!

Building and distributing an iOS app with PhoneGap - part 1

2012-02-19T00:00:00-08:00

I’ve started working on an iOS app. Finally. The goals for this app are multiple. First of all, I wanted to work with PhoneGap, to see what the platform looked like. Second, I found a nice case for an app that I could build and put into production. I try to write up the experience and other stuff that I encounter. This first blog post marks some sources that I found to be useful.

The application will be build with PhoneGap and jQuery mobile. Not because I think that PhoneGap is the way to go, but because I want some experience about the edge cases.

After a day, I had something working and wanted it to deploy. Being new to iOS development, I found this blogpost useful in all the steps. A tweet let me to TestFlight, a useful service for distributing test versions to a group of people.

So the first one is small, but I hope to write some more about the experience.

Javascript sources, a summary

2012-02-18T20:00:00-08:00

I decided to summarize some JavaScript sources that I use, have used and want to share with others. This is by no means complete, but it might serve as a good starting point. Please, send my anything that I have missed :-)

Books

This is a short list of all the books that are available on JavaScript. I only wanted to list the books that I have read. I cannot recommend others.

The Good Parts , by Douglas Crockford
JavaScript: the definitive guide, by David Flanagan
Pro JavaScript techniques, by John Resig

People with weblogs, twitter accounts

Douglas Crockford

http://www.crockford.com/
Inventor of JSON and JSLint

Brendan Eich

http://brendaneich.com
http://twitter.com/BrendanEich
Inventor of JavaScript, currently CTO of Mozilla

Nicholas Zakas

http://www.nczonline.net/
http://www.twitter.com/slicknet/
Front-end tech lead for Yahoo!, author of Professional JavaScript for Web Developers (Wrox, 2012), Professional Ajax (Wrox, 2007), and High Performance JavaScript (O’Reilly, 2010)

John Resig

http://ejohn.org/
http://twitter.com/jeresig
Inventor of jQuery, worked at Mozilla, currently head of JavaScript development at Khan Academy

David Flanagan

Projects and libraries

There are a number of cool projects, build in JavaScript that I try to follow.

Cloud9 IDE: a new IDE for JavaScript build in JavaScript. A very cool project, showing what can be done with JavaScript.
Google docs: what Google has build into docs is awesome.
jQuery: one of the many cool DOM manipulation libraries.

bookreview - Beginning PhoneGap

2012-02-18T16:00:00-08:00

Beginning PhoneGap by Thomas Myer

Beginning PhoneGap aims to give the reader a push in the right direction when using PhoneGap. PhoneGap is a platform for writing mobile applications that can run on iPhones, Blackberry’s, Androids and more. By using HTML5 and JavaScript inside a native app users get the benefits of native apps, while developers get the benefits of code sharing and reusing. Of course by abstracting away the native platform you also loose something.

The author starts by giving a very practical tutorial explaning how to start with PhoneGap. Screenshots and hyperlinks explain every step from downloading to installing. After that the key API’s are explained: media, camera, storage and more. The author goes into great dept to show all details. Every chapter describes the key elements for iPhone, Android and BlackBerry development. The other platforms are also mentioned, but in less detail.

The book itself was good to read, and it showed that the author did everything he could to be as precise as possible. However, I am not sure if the content justifies a book. Some chapters (like chapter 7 “Compass”) feel more like a large blog post. The web is stuffed with blog posts about these techniques. Although it is nice to have everything combined, a book suffers from the speed of development. During writing PhoneGap released a new version, the author updated everything. But what if PhoneGap releases a new version next week? Or maybe tomorrow? To justify a book, I would expect eloboration about the technique and platform, not screenshots of applications using a compass. But that would also need another title, not Beginning PhoneGap.

Find the book at http://shop.oreilly.com/product/9781118156650.do

Disclaimer: I have joined the O’Reilly Blogger Review Program (http://oreilly.com/bloggers/). It means that I receive free books that I review. I am not forced to write nice things, so this review is an honest one.

Moved to Github Pages

2012-02-08T00:00:00-08:00

Yep, that’s right, I have moved my blog. I thought that posterous was the place to be. And the integration with google docs worked nice. But I wasn’t satisfied. I missed some control, and posterous was growing into a social media platform instead of a blogging platform. I missed some features, like good syntax highlighting and markdown support.

So, why did I go with Github Pages? Well, first I thought let’s write this myself. But there are already so many blogging platforms, and to be honest: I wouldn’t know when to build this. Now I have something that I can easily port or host myself. The posts are easily maintained in markdown.

What is Github Pages and how do my posts look like?

Github Pages is a simple way to host static html files. There is no dynamic pages support. But the pages are run through Jekyll before being hosted. Jekyll converts markdown files and … to html files and copies everything else over. The files are pulled from a github repository. This means that everything is under version control, you can create branches and you can fork someone else’s website. Go ahead, fork my blog :-)

The moving itself was just a lot of work, but that is because I did this manually. I wanted all my posts to be converted to markdown, and not simply copy the html files. There are converters, but I didn’t try that out.

I really like the concept of generated files. I allows me to maintain the templating seperate from the actual content, but doesn’t lay any requirements on the hosting so moving again is easy. Downside is that hosting comments on your own blog is impossible. There are comment-services, but I decided to redirect feedback to google+ or twitter. That way reacting for me is easier.

The autoposting to my twitter account is done through if this then that. I hope that they will add google+ support in the near future.

Let the posts start flowing!

JavaScript and the runaway timer

2012-01-26T00:00:00-08:00

One of the things that you probably will run into when creating a JavaScript application is the runaway timer. The runaway timer? Yes, the runaway timer. Ever seen this message?

or this one?

That is the runaway timer telling you that something is running out of control. It is the thing that keeps your browser from crashing or not responding because of a long running script. JavaScript in browsers is executed in a single thread. But, any DOM manipulation or event handling is also done on this single thread (yes I’m ignoring web workers in this post). When your script runs for seconds, no other thing can be done thus making your browser unresponsive. Clicks on buttons or hyperlinks will not be executed, hovers or other effects will do nothing. You should be timing your JavaScript code and split everything that takes more than 100 ms. Why 100 ms? The different runaway timers have enforce limits of multiple seconds. Usability research however shows that humans find a User Interface responsive when it reacts within 100 ms.

I myself ran into problems with the runaway timer when implementing the Mandelbrot figure using JavaScript and the Canvas element. The basis of the mandelbrot figure is an algorithm that calculates a color for every pixel on the canvas. A nested for-loop that is, visiting every pixel and doing some calculation. With a 400 by 400 pixel canvas, I was already running into problems.

#js
var paint = function () {
    for (var screenX = startpoint_x; screenX &lt; endpoint_x ; screenX++) {
        for (var screenY = startpoint_y; screenY &lt; endpoint_y ; screenY++) {
            var xynumber = getNumber(screenX, screenY, center_x, center_y);
            var color = getColor(xynumber);
            setColor(color);
            canvascontext.fillRect(screenX + canvas_half_width, screenY + canvas_half_height, 1, 1);
        }
    }
};

The solution was to split this up in parts that run within the 100 ms limit (actually I choose 50 ms blocks to stay on the safe side).

#js
var paint = function () {
    paintPart(startpoint_x);
};

var paintPart = function(start_x) {
    var start = new Date();
    for (var screenX = start_x; screenX &lt; endpoint_x &amp;&amp; (new Date() - start) &lt; 50 ; screenX++) {
        for (var screenY = startpoint_y; screenY &lt; endpoint_y ; screenY++) {
            var xynumber = number.getNumber(screenX, screenY, center_x, center_y);
            var color = number.getColor(xynumber);
            setColor(color);
            canvascontext.fillRect(screenX + canvas_half_width, screenY + canvas_half_height, 1, 1);
        }
    }    
    if(screenX &lt; endpoint_x) {
        setTimeout(function() { paintPart(screenX); }, 25);
    }
};

The effect of applying this pattern is that the browser stays responsive. Of course, the splitting also gives some overhead. But the site and the browser stay usable, and that is key in every good UI.

Want to more about creating responsive JavaScript applications? This presentation by Nicholas Zakas is a must see.

Inheritance and reuse in a prototype world

2012-01-12T00:00:00-08:00

One of the things that I struggled with writing JavaScript, is the prototype-based programming. Coming from a Java and C# background, it is hard to let go of classes, interfaces and normal inheritance. In a Class-based object-oriented language the classes define how objects behave. An object is an instance of a class. Classes can inherit from other classes and interfaces. In JavaScript (and other prototype-based languages like IO) there are no classes. There are objects, constructors and prototypes.

The prototype way in JavaScript is simply shown in the following example. Let us first define a constructor:

#js
function Vehicle() {
  this.speed = 10;
}

Now you shouldn’t think to much of this constructor. It is a normal function in JavaScript, but by convention we let it start with a capital letter. The important thing is that you use it in combination with the new keyword.

The next step is to define functions on the prototype object of this constructor:

#js
Vehicle.prototype.setMaxSpeed = function (speed) {
  this.speed = speed;
};

Vehicle.prototype.maxSpeed = function () {
  return this.speed;
};

When we construct a new object, we can use the constructor to create an instance:

#js
var myVehicle = new Vehicle();
var isVehicle = myVehicle instanceof Vehicle; 
// isVehicle is true

The constructor makes sure that the new object is based on the prototype. This makes all defined functions available in the prototype chain. There are two advantages of this approach: we can inherit from this object, and we can use the instanceof operator.

#js
function Car() {
}
Car.prototype = new Vehicle();

var myCar = new Car();
var isVehicle = myCar instanceof Vehicle;

Now we have an instance that is more specific then our Vehicle object, but can be used as a Vehicle none the less. As one might notice, the inheritance is achieved by setting the prototype of a constructor. But it is not set to another constructor, but to the instance of an object. The inheritance chain is thus build on instances and not on classes.

There are other approaches possible to share code between objects. One of them is object augmentation. Because we can alter the JavaScript objects at any moment, we can augment objects with functionality.

#js
function makeVehicle(o) {
    var speed = 10;
    o.maxSpeed = function() {
       return speed;
    };
    o.setMaxSpeed = function(s) {
       speed = s;
    };
    return o;
}

var myVehicle = makeVehicle({});

function makeCar(o){
    o = makeVehicle(o);
    // add extra functionality
    return o;
}

We cannot use instanceof anymore, but we can easily create objects with the same functionality. Of course there are some (subtle) difference with the prototype approach. If we decide that vehicles need more functions later on, we cannot alter all instances by changing the prototype. As a positive gain, we can use a Mixin pattern to get more flexibility.

The prototype approach can also be used without constructors and the new keyword. ECMAScript 5th Edition added a new function, Object.create. This function creates a new object using the given object as prototype.

Douglas Crockford has some nice articles (this and this) about inheritance and reuse in JavaScript. Because objects in JavaScript are soft and can be changed at any moment, he hardly sees use for the Classical Inheritance approach. Peter Michaux also writes about the augmentation approach, seeing the Decorator Pattern in it.

Another discussion raised by looking at the SOLID principles is the usage of inheritance. Most of the time, inheritance is used for method overloading and polymorphic methods. The Liskov Substitution Principle states that

Subtypes must be substitutable for their base types.

But JavaScript is not statically typed. This gives us the freedom to do more dynamic typing. Instead of looking at inheritance, we can look at behavior. By checking if certain functions are present we can be much more flexible in the range of objects that we can act on.

#js
function move(vehicle){
    if(!vehicle instanceof Vehicle)
        throw Error('not a vehicle');
    // ...
}

function moveDynamic(vehicle){
    if(!vehicle.move)
        throw Error('move function not present');
    // ...
}

The moveDynamic function shows that we can move any object that has a move function. The move function only works on instances of our constructor function. The last fits the dynamic model and gives more flexibility.

Things learned from the Game of Life

2011-12-24T00:00:00-08:00

I mentioned in the previous blog post that I am digging deeper into JavaScript lately. This post is the second in a series about JavaScript.

I love mathematical stuff. Things like fractals, but also Conway’s Game of Life are amazing. So I started implementing the Game of Life using JavaScript (github and a running example). While implementing, I used a number of patterns and techniques that are worth explaining. These patterns are useful because they let you work more effective with the language.

But let me first explain two differences between JavaScript and traditional object-oriented languages.

Scope

JavaScript only knows two scopes: the global scope and the function scope. This is different then for example C#. In C# block structures also start a new scope. So when you use a conditional or a loop structure, you know that you enter a new scope. This gives advantages in reuse variable names, but it also means that the garbage collector can free up memory when your done. The next example shows how this works in C#.

#java
if(condition) {
    int i = 9 + 8;
    // do something with i
}
return i; // error, because i is out of scope

If we would try the same in JavaScript, it would lead to different results.

#js
if(condition) {
    var i = 9 + 8; 
    // do something with i
}
return i; // fine, i is in scope

This can really fool you. Because these are the only two scopes we have, libraries that define global variables (like jQuery does) can conflict with each other easily. So you want and should be careful to not pollute the global scope. That is also why you should always use the var keyword to declare your variables. Declarations without this keyword cause global variables, even if you only use them inside of a function.

Access modifiers

One of the first things that we learn in object-oriented languages is the usage of private and public modifiers. With those modifiers it possible to hide information inside of object instances. It is one of the core principles of good code design: information hiding.

In JavaScript there is no such thing as access modifiers. So we are left with two choices: give away our implementation details, or come up with some clever way to hide them anyway.

These two key differences between JavaScript and other object-oriented languages make that we should come up with some clever way’s to structure and design our code. There are three things that I would like to point out.

Namespaces

The first thing that I choose to do is adding namespaces to my code. Namespaces are used to create unique names within a certain naming scope. C# has them, and so does XML. Unfortunately, JavaScript doesn’t know namespaces. But you can still simulate them by using object properties.

#js
// create the first level
var overeemm = window.overeemm || {}; 
// create a subnamespace
overeemm.conway = overeemm.conway || {}; 
// continue creating as many levels as you would like

What happens is actually very simple: we just create empty objects to simulate a namespace level. By using the OR notation I am sure not to redefine another namespace and thus clearing functions and objects created on that level. By putting this into every file, I know that the namespace that I will use is available.

With this simple technique we prevent naming conflicts with other libraries.

Hiding your private stuff

We want to prevent all of our internal implementation details to leak out of our objects. At least I would. So we need some clever way to do this. There are a couple of things that you could do, I have been using closures to achieve this.

A closure is a function stored together with his environment (the variable bindings). When you execute this function, it has access to the variable bindings from the stored environment. It is said that the function is closed over the free variables. Because the function keeps access to all those variables, it is possible to create an hidden environment. This hidden environment is only accessible by the functions defined in this environment.

#js
var f, g;
function doSomething(x, y) {
    var z = x + y;
    f = function () { return z++; };
    g = function () { return z--; };
}
doSomething(1, 4);
// f and g are free to use

After calling doSomething we can use the functions f and g. But they are defined within the scope of doSomething! Because they are defined in the same scope, they both have access to the same variable z. This variable is not available outside of doSomething, making it a private variable.

If we change the code snippet a little bit, we can create an object with his own private scope.

#js
function createObject(x, y) {
    var z = x + y;
    return {
        f : function () { return z++; },
        g : function () { return z--; }
    }
}
var o = createObject(1, 4);
// o.f and o.g are free to use

With this technique we actually achieved two things:

we created an object o with a private state.
we did not pollute the global namespace with f and g, but only with o.

Singletons through self executing functions

But what if we have a library with utilities. Or we want to create a singleton object? We can use the same technique, but place it in a different context.

#js
var singleton = (function(x, y) {
    var z = x + y;
    return {
        f : function () { return z++; },
        g : function () { return z--; }
    }
})();
// singleton.f and singleton.g are free to use

We change the create-function to an anonymous function that is directly executed. Because of the object that we return, we have created a private scope. And by executing the anonymous function, we make sure that nobody can create a second instance.

For me, these examples show a nice and elegant side of JavaScript. One could argue that they are solution around problems in a bad designed language. However, we have to deal with our inheritance even if we do not like it. And to me, these techniques show creativity and possibilities.

Singleton revisited

After a twitter discussion I wanted to revisite the singleton pattern a bit. One of the aspects of the singleton pattern is the fact that you can ask for the thing you want and get the same instance back, no matter how often you ask for it. That is something that was not captured in my code example.

On stackoverflow a nice example is given. The principle is that you redefine the creation function and let it return the same instance. I thought it was a creative solution, and again I am amazed by the things you can do :-)

#js
function singleton() {
  var instance = (function() {
    var privateVar;

    function privateMethod () {
      // ...
    }

    return { // public interface
      publicMethod1: function () {
          // private members can be accessed here
       },
      publicMethod2: function () {
        // ...
      }
    };
  })();

  singleton = function () { // re-define the function for subsequent calls
    return instance;
  };

  return singleton(); // call the new function
}

The rise of JavaScript

2011-12-22T00:00:00-08:00

I’m digging deeper into JavaScript lately, and I like what I find, see and try.

I remember that when I first saw JavaScript I was building a website for our graduating class in high school, it was around 2001. It was the period that JavaScript was used in the marketing term DHTML. It was primarily used for dynamic menu’s with overlays and for the snow effects during winter. And yes, I am guilty for doing that too.

During college, fast forward three years, we used JavaScript for client-side validation. It was still advertised as a scripting language, useful for dynamic HTML.

Then during my first full-time job, the rise of AJAX happened. Microsoft added dynamic capabilities through various WebControls (like the UpdatePanel). Prototype.js was the first major library. Everything and everybody that used AJAX was hot. People started pointing out the downsides: JavaScript could be disabled, so websites should work without it. The battle of browser compatibility began, Internet Explorer 6 was still mainstream.

And then, slowly it all evolved from sprinkling extra’s into an HTML environment to full web applications build in JavaScript. Great examples are of course gmail, google docs and Cloud 9 IDE. JavaScript has grown into a serious programming language, that people should know and care about. With Node.js it even became possible to write your web application’s server-side in JavaScript. Microsoft is enabling you to use it for desktop applications on Windows 8.

Some saw this evolution, thought about it and stated bold things ( Yes, it was written in 2007, I know that I’m late to the game. ):

Any application that can be written in JavaScript, will eventually be written in JavaScript. (Atwood’s Law).

and

JavaScript is an assembly language. The JavaScript + HTML generate is like a .NET assembly. The browser can execute it, but no human should really care what’s there.(Erik Meijer on hanselman.com)

Atwood’s Law is getting confirmed every day: real life web applications and games are written in JavaScript, not only for fun! Hanselman’s statement is seen in things like the Google Web Toolkit and the Cappuccino framework. But also in ClojureScript and CoffeeScript. They all try to lift some of the problems in JavaScript by using other languages or creating new languages that compile into JavaScript.

I am not sure if that is the way to go. I like JavaScript. I’ve read some great books by Douglas Crockford (JavaScript The Good Parts), John Resig (Pro JavaScript Techniques) and David Flanagam (JavaScript The Definitive Guid). Still I think that I scratched nothing but the surface. There are so many tools, libraries and frameworks from which I can learn. However, I can also see where those abstraction are coming from. I never learned C or C++. It is great that we have C#, Java, Ruby and so on on top of those languages to make development easier, faster and less error prone. But is JavaScript already in that stage? I am not sure… only time will tell.

I hope to digg into JavaScript and some libraries in the next couple blog posts.

First steps in a new language

2011-11-03T00:00:00-07:00

One of the tips given in the book is

Learn at least one new language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut. Additionally, learning many languages is far easier now, thanks to the wealth of freely available software on the Internet.

Although I read through the seven languages in seven days book and the book of Ruby this year, I was still not comfortable enough with Ruby to use it when necessary. So one of my goals was to really learn Ruby. And the best way to learn is by using it. Like Knuth already said in his classic The Art Of Computer Programming:

It is difficult, if not impossible, for anyone to learn a subject purely by reading about it, without applying the information to specific problems and thereby begin encouraged to think about what has been read. Furthermore, we all learn best the things that we have discovered for ourselves.

This write-up is a summary of my first (baby) steps in Ruby.

I started out with doing some katas from the pragmatic programmers. I quickly found out that although they are good exercises, they did not interest me. And because of that I was not really motivated in doing them.

The second exercise was a rewrite of some tool that I build in C# to encode, upload and publish mp3 files to a Joomla-website. The .NET tool is a WinForms executable, using lame to encode, using ftp to upload and using XML-RPC to publish the post. The Ruby version is one script, doing all those things in sequential steps. This exercise was more fun, because I needed it (the .NET tool didn’t work on my MacBook).

My scripts can be found on github.

So far so good. I like the language, but I do realize that these are just scripts. And by no means they are building a real life system. So the next exercise should be bigger, forcing me to use more advanced features. Maybe I should rewrite the Joomla-website in a simple RoR version?

How do you learn a new language?

Using Google APIs through OAuth 2.0

2011-09-09T00:00:00-07:00

Lately I’ve been working on a web application that connects to Google docs: gdocerous. During the development I’ve stumbled up on OAuth and other authorization protocols. This blog post is a small write-up of the experience.

As the major cloud provider, Google has lots of content available through their applications. Fortunately, they also have a nice API with lots of support and samples. But also lots of ways to connect and authenticate. They support AuthSub (their own proprietary protocol), OAuth 1.0 and OAuth 2.0 (still in beta), but also a plain user name and password connection. They do recommend that you use OAuth, and because the specs for OAuth 2.0 seem stable enough, I would recommend to use the 2.0 version. This post shares some example code on how to use it in a .NET web application.

OAuth 2.0 in short

The OAuth 2.0 protocol is an evolution from OAuth 1.0 (really? yes it is!). If you are interested in knowing why a new version was necessary, I suggest that you read the following article from Eran Hammer-Lahav: Introducing OAuth 2.0.

The global idea is that you redirect a user to the OAuth-server. There he can authenticate and grant access to the application that redirected him in the first place. The user gives access for a certain scope. In the case of Google, this scope can be Google docs, gmail or some other application. It is of course important that the scope for which access is requested is as narrow as possible. After the user has authorized the application, the OAuth-server redirects to the application, giving an authorization code along. This code can be used to request an access token. And finally, the token can be used to access APIs and resources.

The main advantage is that the user does not have to give his user name and password to the application. Instead the application requests access and receives a short-lived access token. This makes it easy to revoke the access in a later stage, without having to change any credentials.

Is all good in the Google OAuth 2.0 world? No, not really. Some things remain hard. For instance using Google’s IMAP or SMTP. Although the protocol documentation is clear, there is no easy way to use this from .NET.

Show me the code!

So how do you get started with these APIs? Let me walk you through the basic steps.

First, register your app in the Google APIs console. This gives you a clientid and clientsecret. It also allows you to give your application more trust by supplying a description and logo. This way users can recognize the application that asks for trust.

On requesting authorization, your application starts with redirecting the user to https://accounts.google.com/o/oauth2/auth, following the description of Using OAuth 2.0 to Access Google APIs. This part is not that hard. Users are shown a recognizable Google page on which they grant access for your application.

After these two steps we need to write some real code. First retrieve the code from the request en get an access token from Google.

#csharp
string queryStringFormat = @"code={0}&client_id={1}&client_secret={2}&redirect_uri={3}&grant_type=authorization_code";
string postcontents = string.Format(queryStringFormat
                                   , HttpUtility.UrlEncode(code)
                                   , HttpUtility.UrlEncode(clientid)
                                   , HttpUtility.UrlEncode(clientsecret)
                                   , HttpUtility.UrlEncode(redirect_url));
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("https://accounts.google.com/o/oauth2/token");
request.Method = "POST";
byte[] postcontentsArray = Encoding.UTF8.GetBytes(postcontents);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postcontentsArray.Length;
using (Stream requestStream = request.GetRequestStream())
{
    requestStream.Write(postcontentsArray, 0, postcontentsArray.Length);
    requestStream.Close();
    WebResponse response = request.GetResponse();
    using (Stream responseStream = response.GetResponseStream())
    using (StreamReader reader = new StreamReader(responseStream))
    {
        string responseFromServer = reader.ReadToEnd();
        reader.Close();
        responseStream.Close();
        response.Close();
        return SerializeToken(responseFromServer);
    }
}

The result is a JSON string that, with the help of DataContractJsonSerializer, can be serialized into

#csharp
public class GoogleOAuthToken
{
    public string access_token;
    public string expires_in;
    public string token_type;
    public string refresh_token;
}

The given access token comes with an expiration time. To extend your timeperiod, you can use the refresh token to request a new access token. The code to do this is similar to the example above.

After this we can finally get to work. Let’s get a list of all folders from google docs.

#csharp
var settings = new RequestSettings("your appname", "oauth access token");
var request = new DocumentsRequest(settings);
foreach (Document doc in request.GetFolders().Entries)
{
    /* do something */
}

This snippet uses the Google Data Protocol, using the C# library available from google code. The Data Protocol it self is REST-inspired, but the library gives a nice object-oriented wrapper around that.

The library wraps all major APIs from Google, so anything is possible.

Have fun coding!

Blogging in the cloud through gdocerous

2011-08-21T00:00:00-07:00

I’m trying to find more time and topics to blog about. While doing that, I reviewed the work flow that I have for blogging. My blog is hosted on posterous.com, a great service (although they do have some downtime-issues). It makes posting a blog easy: just send a mail to posterous@posterous.com and your done. The sender e-mail is recognized and the e-mail is converted to a blog post. They automatically convert images to galleries, embed youtube links and so on. If you are working on multiple blog posts at a time, you could store your mail as a draft or store it somewhere else. Somewhere else is no fun, because it means that you will have to have some other intermediate form. Storing it as a draft means that you do not have history on your blog post.

Lately I’ve started to use google docs more and more. And it looked like a great place to store my draft-blog posts. But about posting the final version? In that case you had to copy-paste it over to your e-mail and reformat it. Not that easy and fun anymore.

And then I got this great idea: wouldn’t it be great if you could choose your google document and post it to posterous? Of course it would be :) So that became my weekend-project: gdocerous. ( the domain is not yet active, it should be this week ).

I have a working version, but unfortunately it is not open for public yet. I still have some things to finish before opening it up:

I want to use the OAuth API instead of the AuthSub version
I still need to figure out how to use gmail’s smtp through OAuth

The whole thing is pretty simple, but I think useful. One of the things is that I do not want to store sensitive data or user accounts. That’s why the posterous API is not usable. Another thing is that I only want to select and post, no mutation of articles and such. Last is that the complete source code is open for everyone to look at.

Reading != learning

2011-08-07T00:00:00-07:00

I think that I have always been a reader. There is a box with books from my childhood, and now I have shelves full of books. A great number of them are technical. But there are also novels and history related books. I even signed up to review books for O’Reilly. I already did a number of them (including two reviews that were not for O’Reilly).

But you know what, I did not really learn anything from them yet. The problem with technical books, programming books is that you have to do something with the knowledge. Otherwise it will not stick. So I’m taking a break from the reviewing stuff. I’m going to set two goals for myself, after which I can pick up the reviewing again. What are those goals?

1. Build an iPad/iPhone app and release it to the Appstore

After the iOS book I was eager to write something, but then again another book came by and the app is left unfinished. So I’m going to finish it and release it. I have an idea and already started on it. It is time to focus and finish it. Probably will take a couple of months though.

Progress will be visible in my github repo.

2. Learn Ruby and change my default language for a certain period

I have just finished the book of Ruby and before that touched on Ruby while reading the seven languages book. But again I’m lacking some real experience. So I will change my default script/tool language for a certain period and see where that will take me. I hope to get some fluency in Ruby and try on IronRuby and MacRuby. That will give me some flexibility in developing on my MacBook or on my Windows laptop. I’m planning of working through some of the coding Katas as described here. That will give me some guidance as I’m trying stuff.

The big eye-opener here is that I have to stop reading technical books, but I have to start doing them. Let’s see how I will do.

bookreview - the book of Ruby

2011-08-07T00:00:00-07:00

The book of Ruby by Huw Collingbourne

This book covers every part of Ruby that a newbie Ruby developer should know. It starts out with the basics around classes, objects, collections and moves over to more advanced stuff like threads and marshaling. Not only the language is covered: developing, debugging and the framework Rails are also explained to the reader. The book is full of example code snippets and good explanation. For the more experienced developers the author has added ‘digging deeper’ sections, that take you to more advanced topics. The book finishes of with more reading material and guidance around development tools.

The book has a subtitle telling us that this is ‘a hands-on guide for the adventurous’. That sure does sound promising. Unfortunately, the book does not read as a ‘hands-on guide’. It reads more like a reference manual. There are no assignments to practice the language and the examples are short. It misses a real life example that is build from start to end. You would expect some real application coming from a hands-on guide. The book self is well written and reads very easily, but the ‘reference manual’ style makes it somewhat boring.

I definitely recommend this book to people starting in Ruby, but not as a first book. Start with a real hands-on book that walks you to the creation of a basic application. Then pick up this book and walk to all the different topics. That will give you a good understanding of Ruby.

Find the book at http://oreilly.com/catalog/9781593272944/

bookreview - Head First iPhone & iPad Development, 2nd Edition

2011-07-16T00:00:00-07:00

Head First iPhone & iPad Development, 2nd Edition by Dan Pilone & Tracey Pilone

The Head First series are always a great way to get started in some new technology. This book is no exception to that. This second edition is completely rewritten to match the new iOS and Xcode versions. The authors take the reader with them on a path to learn these new technologies. Every aspect of developing for the iDevices is shown and discussed. Real life examples are used to build complete applications that can be submitted to the App Store. The book starts out with a simple iPhone application which is then expanded to incorporate more advanced stuff such as iPad support, device orientation, Core Data and usage of the Camera. Every part is explained in detail with code samples and screenshots.

Overall the book is a great way to get introduced to development for iOS. The authors try to cut lose from boring introductions. Code samples are alternated with screenshots, in dept explanations and silly interviews. This way the book stays fun and interesting. However, I found the style a bit to childish from time to time. Interviews with GUI controls are not the content that I would expect. It does make this book useful for teenagers that want to get started. I think the content could be made a little more dense and less childish.

Find the book at http://oreilly.com/catalog/9781449387822/

bookreview - JavaScript, The Definitive Guide, 6th edition by David Flanagan

2011-05-15T00:00:00-07:00

Javascript, The Definitive Guide, 6th Edition by David Flanagan

This book is called ‘the definitive guide’ for a reason: it is by far the most complete book on Javascript that you can buy. Every subject, from syntax to popular libraries is covered. The 6th edition is completely updated to reflect the latest standard, ECMAscript 5. The newest features such as localstorage and geolocation are explained in this new edition. The book follows a clear outline. It starts by explaining the language and it’s syntax, followed by it’s features. Next is the usage of the language, both as a client-side language and as a server-side language. The book also covers the most popular Javascript library: jQuery.

I found the book, although it is written as a reference, easy and entertaining to read. The book is a great resource to use while developing Javascript, because every little detail is covered. A good indication of its quality is that the book survived five editions and is still going strong! What I did miss is some guidance around real world Javascript development and tooling. The book does not explain anything around unit testing, development environments and other useful tooling. That would be a welcome addition for the 7th edition!

Find the book at http://oreilly.com/catalog/9780596805531/

Two years of reading summarized

2011-05-05T00:00:00-07:00

So I like to read… a lot. And not just books about programming or computer science, but like everything. Exactly two years ago, I started to use @deboekenlijst. It is a twitter service that builds a list of the books that you have read. So every time you have read a book, you tweet the information to that account. Once a week (or less), a website is build with statistics and an overview. This week I thought, let’s see how much I read. I’m currently in the top 10 of the most active readers:

I started on May 3 2009, the last book was entered on April 28 2011. In that period I’ve read 108 books in 104 weeks.

The analysis of my reading started with getting the information from the website, parsed it to JSON and expanded the list with books that weren’t yet analysed. After that, I added information about language and type (ebook vs dead-tree). With google charts, I build some nice charts.

44 books are ebooks, 64 are paper books. The ebooks are in ePub, Kindle and PDF format. They were read on sony’s PRS-300 and on Apple’s iPad 1. And although eInkt is very nice to read, I like the iPad more. The screen is larger and it supports more formats. Downloading and reading without the need to link it to my laptop is also very nice.

I think that ebooks are the future, and I try to buy and read them as much as possible. I do not like DRM however.

I’ve read 31 English books, the rest is in Dutch. The English books are mostly technical, although I buy some non-technical books in English too. Number of books per month

On average, I read 4 books per month. Some months were more productive, reading wise. How come?

July 2009: two week holiday
May 2010: two week holiday
October 2010: the service went down (due to twitter errors), so I re-added a number of books at once
March/April 2011: became father and had two weeks holiday

On to some more reading!

bookreview - Seven Languages in Seven Weeks by Bruce Tate

2011-04-29T00:00:00-07:00

Seven Languages in Seven Weeks by Bruce Tate

I love to try out new programming languages and different to ways to develop software. But my time is limited. So it is hard to really learn new languages and keep up with what happens outside of my comfort zone (I develop mostly in C# and JavaScript). So a book that promisses to learn you seven languages in a short period makes me curious.

The concept of this book is great: show you enough of a language to give you an idea, but keeps it short enough to not make you bored. Programming language books often become lists of syntax rules and library calls. This book gives you enough to become interested and curious, but moves on before you become bored.

My reaction on every one of the seven languages:

Ruby: The language for cool web developers. What I like is the dynamic aspect: quick prototyping and developing of software. Together with the Rails framework it really blends well with the client-side aspect.

Io: Prototype based, so you recognize JavaScript aspects. Not really useful in a commercial environment, but great to learn the prototype based stuff. Hard to find stuff about this language through google :)

Prolog: I already learned this during college, but it is nice to revisit. Prolog really makes you think about programming. Solutions become more declarative, instead of imperative.

Scala: The new language for Java developers. I like how it blends functional and object-oriented. But with C# becoming more functional, it didn’t capture my interest.

Erlang: A functional programming language. It strength is in the concurrency library. Didn’t trigger my curiosity.

Clojure: Awkward syntax because of the prefix notation. Lots of parentheses. Looks a bit to academic.

Haskell: Did not really put a lot of time into this chapter. During college we did a lot of Haskell, so nothing new for me in this chapter. The language still looks to academic, but seeing a number of dutch startups using it (http://tupil.com/, http://www.silkapp.com/), it still triggers interest.

The downside of this book is that it doesn’t show you how to use these languages in real world problems. You should develop larger applications with these languages to really get the feeling. What’s next? I hope to dive into Ruby and Clojure, because these did trigger something. Not so much with Io, because I do enough JavaScript programming. Scala and Erlang do not really have my interest either. Haskell and Prolog are out, because I did that in college already.

Notes from the class Advanced Agile Development

2011-04-01T00:00:00-07:00

Last week I attended the Advanced Agile Development class, organised by Xebia and given by Alistair Cockburn. With this post, I try to make my notes persistent for future reference.

these are my notes. they are far from complete. as seen from this tweet, the class was really advanced :)

But I did survive!

I survived @TotherAlistair :)moby.to/z4s4u2
— Michiel Overeem (@michielovereem) March 30, 2011

Content

craft; deepening (life long learning) skills in a medium (for instance a progr language); shu-ha-ri
cooperative game; all about strategies (invent, decide, communicate)
flow management; manufacturing lines; palette size
self awareness/ personalities; everybody is different, look at the personalities in your team
design as a knowledge acquisition; try to get as much knowledge as you can with every step

Craft

Learning takes place in three stages:

SHU - learn a skill; copying a technique step by step
HA - collect techniques; search for other techniques to try.
RI - blend and invent techniques; knowledge and experience are combined into something new

Do not try to become RI without really archiving that level. Otherwise you are just picking techniques and doing something that you do not understand.

good trick > technique > process > dogma It starts with a ‘good trick’, that becomes a technique, which is formulated into a process and that becomes a dogma. You have to bring everything back to a ‘good trick’. There are no rules. Master agile and reach RI level.

Cooperative game

The speed of the project is the speed at which ideas move between minds

what are conditions to facilitate speed: colocation, …
attitude: ego, fear, self preservation, …

Programming is Community Theory Building Communication is touching into shared experiences

Development is a cooperative game: Invent, Decide, Communicate primary goal: deliver working software secondary goal: set up for the next game (maintenaince, new release)

Flow management

Design looks like manufacturing if we use ‘decisions’ as inventory. Let the decisions flow through the team. Find the bottleneck and fix the process.

Remove waste. Example: tracking bugs is waste, we need to fix them.

Design is knowledge acquisition

Software development is like knowledge acquisition. Eliminate risks early on.

business risk: are we building the right thing
social risk: can they/we build it
technical risk: performance, compatibility
cost schedule risk: see if features change the costs of other parts of the project.

trim the tail: first do risk, then do value, then do the fine-tuning/glossy/polishing slicing helps with trimming the tail; build complete vertical-slices of user stories.

Properties of success projects

frequent delivery; quick visible progress; stakeholders can give feedback; little steps so little errors
reflective improvement; evaluate to improve
close/osmotic communication; communication should be easy, everybody should be involved
personal safety; willing to listen to others & ability to speak without fear to be damaged; trust for everybody to be part of the team
focus; knowing what to work on, having time to work on it; distraction
easy access to expert users; quick and better feedback on decisions
technical environment; trust to try stuff out; unit tests/ CI/ version control
sunshine/visibility; transparency; be honest when failing; motivation

Some aha’s

More reading: Jeff Patton website, Book Agile Software Development, the cooperative game

bookreview - Gamestorming by Gray, Brown, Macanufo

2011-03-22T00:00:00-07:00

Gamestorming, A Playbook for Innovators, Rulebreakers, and Changemakers by Dave Gray, Sunni Brown, James Macanufo

Gamestorming is all about using games to bring up ideas and solutions in your team. The games are used to give a creativity boost. The book starts out with an explanation of the different concepts that are used in gamestorming. The essentials are explained to give a better understanding of using games in business. A game is split up in three parts: an opening, an exploration and a closing phase. During the exploration phase the team should produce nodes. This can be done by sketching and modeling. Important things during this phase are allowing randomness and improvisation. In the closing phase it is important that an artifact is produced. Through selection the best results can be taken out of the game. The book also points out some important skills; for example asking questions, visual language and improvisation. After the more theoretical part the authors give a list of games that can be used in gamestorming. These games can be used as a reference list that can be used to guide a team in producing new ideas and solutions.

The book is a fun read. The authors are clearly speaking from a lot of experience. The theoretical part gives a solid understanding of the different parts and gives body to the book. The larger part of the book is a list of games that can be used. This is somewhat disappointing, because it makes it more of a reference book. After the first part I was tempted to put the book away and leave it there until I could use the games. That is a shame, because there is a lot to learn from those games. The authors could have taken more information from the game-descriptions to make the theoretical part larger. I cannot wait to try these new ideas in real life.

Find the book at http://oreilly.com/catalog/9780596804183/

My photo library over the years

2011-01-06T00:00:00-08:00

After reading Hanselmans blog I became curious. How does my photo library look?

The journey, or better adventure, started with seeing if I could get those statistics from my library. I use iPhoto to manage all my photo’s, but it doesn’t come with these kind of statistics. I does have some .xml file, so I thought lets use XSLT. But no, the necessary information (like resolution) is not in this .xml file. Next step was to do it myself. So after copying everything to a Windows environment (yes, I do use Windows and I like Visual Studio and C#) I started to do some gathering. The opensource library jpegdata came to the rescue.

But then you also want to view charts, right? Nobody wants to look at some numbers. So I dove into google charts to generate some nice pictures. And the result is:

So what does this tell me?

I don’t invest a lot in camera’s (I often make photo’s with my iPhone), so the resolution does not increase a lot.
I probably need to clean up my library, because there was a lot of garbage in it :-)

Again, not very interesting, but a fun exercise.

bookreview - Continuous Integration in .NET by Marcin Kawalerowicz and Craig Berntson

2010-09-27T00:00:00-07:00

Continuous Integration in .NET by Marcin Kawalerowicz and Craig Berntson

A bit about my background: I am a .NET developer for almost 5 years (3 of them full time). I am familiar with Visual Studio 2003 up to 2010, TFS 2005 and 2008, SVN and SourceSafe. I am very interested in using the right tools to speed up development and increase software quality.

Continuous Integration is getting more and more attention from the .NET community. With Microsoft focusing on ALM tooling and releasing new and better versions of Team Foundation Server, CI is getting a place in the spotlights. However, the barrier for starting with CI is still high. There are lots of tools, and starting with CI can be overwhelming. Many aspects and things need to be thought out. This book brings a lot of information about starting with CI to the .NET community. The book doesn’t presume any knowledge except for some basic .NET and Visual Studio things.

The book is very pragmatic and practical. The authors take you through the complete process of installing and configuring a CI server, visualizing this process even more with an example project. The strong part of this book is that the authors aren’t focusing on one technology by forcing Microsoft tools on the reader. They show different approaches, and give you the pros and cons of Microsoft tools, open source tools and commercial third party tooling. So the reader can make a solid choice based on functionality, complexity and pricing.

Although the book gives a lot of information, there are also some downsides to the book. One of those downsides is the current collection of blogs, twitter accounts and other resources that are available through the Internet. Many pieces of the information found in the book can be found online. The danger is that the book becomes outdated very fast.

By making the book very practical, the writers have made the book also less readable. The content forces you to read the tutorial steps one by one, because sometimes good information is hidden between those steps. Not all readers are interested in every technology, so it could be wise to split up the content into different parts. In that way readers can decide to skip some parts of the book. By splitting up the book, it can also become more of a timeless resource. Readers interested into the concepts and the big picture can focus on the main content part, while readers wanting to get started on implementing CI into their organization can go to the tutorial section.

A piece of information that I missed is the fact that most of the readers probably aren’t starting with a fresh project. Many developers are already maintaining pieces of software that could really benefit from CI. However, how does someone start with CI on an existing piece of software? That could really be a valuable addition to the book. A smaller piece that is missing if a little bit of information about Mono and how to make sure that you software is compatible with Mono. I would be nice to spend a chapter on that part as well.

To summarize this review: I would recommend this book to any one who is starting fresh on the CI process. It is a very good starting point with practical guides on the implementation of CI. For people already familiar with CI and using it to some extend, the book can be a next step. Experienced developers familiar with CI and the different steps will probably find little new information in this book.

ebooks uitlenen (dutch)

2009-10-08T00:00:00-07:00

Zoals de meeste die mij volgen op twitter wel weten lees ik nogal veel boeken (zie bijvoorbeeld mijn pagina op @deboekenlijst).

Wat mij betreft zijn ebooks dan ook helemaal geweldig. Ik koop mijn boeken toch al, dus waarom niet op een formaat dat ik ook makkelijk mee kan nemen. In plaats van 10 keer een boek van 3 a 4 centimeter dik heb ik nu alleen mijn reader nodig op vakantie. Maar, het moet natuurlijk wel net zo gemakkelijk zijn als een boek. Als het even kan wil ik mijn boeken kunnen uitlenen en wil ik boeken van vrienden en familie lenen. En dat laatste bleek toch iets lastiger…

Ik heb een reader van bol.com en heb daar inmiddels ook een aantal boeken gekocht. Bol.com gebruikt het epub formaat, wat betekent dat je je boeken moet registreren met Adobe Digital Editions. Je autoriseert je computer, je reader en daarna het ebook. Zolang het 1 reader betreft is er niks aan de hand. Maar nu wil ik mijn ebook uitlenen. Dus ik stuur het .acsm bestand op dat je krijgt van bol.com. Ze adverteren immers met het feit dat je boeken kan uitlenen (Het lijkt erop dat dit terug gedraaid is? Ik kan dit niet meer op de website vinden). Helaas kan het ebook bestand niet met een ander Adobe ID geregistreerd worden. Dus dan maar proberen het boek met mijn ID registreren voor een andere reader. En weer helaas: een reader kan maar aan 1 ID gelinkt zijn! Oftewel: wil je boeken (uit)lenen? Dan moet je 1 Adobe ID gebruiken. Dat is natuurlijk niet handig.

Daarom ben ik maar op zoek gegaan om de DRM van de epub files te slopen. Let op, niet met het doel om al mijn boeken te verspreiden. Ik verspreid ze alleen maar naar vrienden en familie, met daarbij natuurlijk de voorwaarde dat zij het niet weer verder verspreiden. Lijkt mij geen echt probleem, ik koop immers al mijn boeken nog steeds… (overigens zijn er via Google en Projet Gutenberg ook gratis boeken te downloaden).

Het verwijderen van DRM is eigenlijk vrij simpel (Dit is alleen voor Windows, ik heb niet verder gezocht naar een MacOSX handleiding). Even googlen levert al snel 2 links op: http://rapidfind.org/upload/showthread.php?p=1035319 http://klungvik.com/index.php/2008/how-to-remove-drm-from-ebooks/

Wat heb je nodig?

Python voor Windows (versie 2.6)
De library Pycrypto
Adobe Digital Editions
Een tweetal scripts: ineptkey.pyw en ineptepub.pyw - deze zet ik niet ter download hier, maar even googlen levert ze al snel op.

Nu begint het echte werk:

Eerst moet je een ebook bij bol.com kopen. Dit levert een .acsm file op.
Deze open je met Adobe Digital Editions, unlock hem met je Adobe ID.
Zoek nu de map ‘My Digital Editions’ op. Hierin staat het echte .epub bestand.
Kopieer het .epub bestand naar de map met python scripts.
Run eerst het script ineptkey.pyw. Dit levert een adeptkey.der bestand op.
Run het script ineptepub.pyw. Je krijgt een schermpje dat vraagt naar het key bestand (zie vorige stap) en het .epub bestand. Specifieer het output bestand en runnen maar.

Het bestand zelf kan nu met bijvoorbeeld Calibre omgezet worden naar andere formaten, of op de reader geplaatst worden.