In the next couple of days (understand weeks) I will force myself to go through the notes I took during the last 3 days and come up with a couple of posts. I swear.

My schedule

Day 1

• Spring into the Cloud by Josh Long & Chris Richardson
• A Practical Introduction To Kanban by Marcus Hammarberg & Joakim Sundén

Day 2

• Enterprise Java in 2012 and Beyond by Juergen Hoeller
• Player Framework 2.0 by Peter Hilton
• Maven vs Gradle, On your marks, get set, go! by Hardy Ferentschik
• Unleash Your Domain by Greg Young
• An Intro to Hadoop by Eva Andreasson
• Application Security for Rich Internet Applications by John Wilander
• The road to REST by Rickard Öberg

Day 3

• 7 Things: How to make good teams great by Sven Peters
• HTML5 with Play Scala, CoffeeScript and Jade by Matt Raible
• Zero Downtime Continuous Deployment of Java Web Applications by Fabiane Bizinella Nardon
• It Is Possible to Do Object-Oriented Programming in Java by Kevlin Henney
• Scala in Action by Heiko Seeberger
• We visualized, we saw, we changed by Leonard Axelsson
• Cool Code by Kevlin Henney

Install Apache

sudo apt-get install apache2

Install MySQL

sudo apt-get install mysql-server

Install PHP (with MySQL support)

sudo apt-get install php5 php5-mysql

Enable the rewrite mod in Apache

This is only needed if you plan to get rid off “index.php” in your CodeIgniter application’s URLs.

sudo a2enmod rewrite
sudo gedit /etc/apache2/sites-available/default

<!-- Edit the configuration and change AllowOverride None to AllowOverride All -->
<Directory /var/www/>
   Options Indexes FollowSymLinks MultiViews
   AllowOverride All
   Order allow,deny
   allow from all
</Directory>

Add your site to your local web server

You could probably just extract the CodeIgniter archive into /var/www/ but you also could have your application in your home folder (/home/scandinabox/www.mysite.com/ for instance). In that case, all you need to do is create a symbolic link in the Apache web directory pointing to your application folder.

cd /var/www
sudo ln -s /home/scandinabox/www.mysite.com/ www.mysite.com

Restart Apache

sudo /etc/init.d/apache2 restart

Here you go, just point your web browser to http://localhost/www.mysite.com/ and you should see the CodeIgniter example pages.

Let’s say we have a class that represents a square. The only operations we can perform on a square are: getting its surface and doubling the side length.

public class Square {

  private int sideLength;

  public Square(int sideLength) {
    this.sideLength = sideLength;
  }

  public int getSurface() {

    return this.sideLength * this.sideLength;

  }

  public void doubleSideLength() {

    this.sideLength = this.sideLength * 2;

  }

}

To test that piece of code, we could probably write a unit test that will look like:

public class SquareTest {

  public void testSurfaceWhenSideLengthIsDoubled() {

    Square a = new Square(5);
    Square b = a; b.doubleSideLength();

    assertEquals(a.getSurface(), b.getSurface() / 4);

  }

}

From a programmer’s point of view the code above perfectly makes sense but – even if the example is rather simple – it does not really tells us what scenario we want to test here. If this piece code would be read by a product manager, she would probably not understand it (oh well, in that case she probably will but unit tests can be a lot more complex than that).

What about making the code a little bit clearer and telling us a story? There are frameworks for that you could argue (like JBehave) but do we need a framework – and all the configuration, XML files that come with – for everything really? What if we just rewrite the test class so it is more narrative?

public class SquareTestEnhanced {

  private Square square;

  public void test_side_length_x_2_gives_surface_x_4() {

    when_doubling_is_done_on(new Square(5));
    then_surface_must_be_multiplied_by_4();

  }

  private void when_doubling_is_done_on(Square s) {

    this.square = s;

  }

  private void then_surface_must_be_multiplied_by_4() {

    Square s = this.square; s.doubleSideLength();
    assertEquals(this.square.getSurface() * 4, s.getSurface());

  }

}

Isn’t it enough? Can my product manager understand what we are testing between lines 5 and 10?

In the last 5 years the amount of data produced worldwide (texts, images, audio, …) has drastically increased from 161 exabytes to 988 exabytes (one EB being one million TB or one billion GB) and with that come some new challenges (storage capacity, availability, …) that cannot be entirely solved by SQL solutions and relational databases.

Not entirely because “NoSQL” does not mean no SQL at all but not only SQL really and a mixed architecture (depending on the needs) is probably what the final solution to a problem will look like.

What are the problems?

The amount of data is growing at an exponential rate and a relational database (like MySQL) is not really a distributed solution. Even though reads can be performed on slaves, writes most likely have to be done on the master (where would the consistency be otherwise?) which becomes a bottleneck in a transaction intensive system.

Relational models are rather statics and once the model has been defined you better not have to change it. From my personal experience I can say that adding a column to a table that gets 10 millions (roughly) extra rows a day without downtime is a rather complex and costly (just in the human resources involved) operation. And when you need to keep 5 years of data in that same table (which would be 18.3 billions rows) and still have a good performance on reads and writes, it gets even more complex.

When you need to store – as fast as possible – large quantities of data which structure has to be somehow flexible you’ll definitely have to have a look at the following solutions.

What are the solutions?

There are 4 kinds of NoSQL models at the moment:

1. key/value
2. column
3. document
4. graph

Key/value stores design is domain driven. Entities that are tightly coupled go to the same bucket (a customer and her shopping cart for instance) but different instances (customer A and customer B) don’t have to be in the same bucket.
With a key/value store like Redis a throughput of 110,000 database operations per second can be achieved when MySQL shows a good 15,000 (good because it’s still pretty good).

Most key-value stores provide an indexing mechanism and/or a search engine usually based on Lucene, Solr, Elastic Search.

As for the other solutions we can mention Cassandra (from Facebook), Google BigTable and Hadoop HBase (column models), CouchDB and MongoDB (document model) and Neo4J (graph database).

Graph models seems like a pretty interesting topic (even though performance is not what you can expect from a key/value store for instance) and I will definitely have a closer look at Neo4J.

Let’s get started then with notes I took during a session ran by Tobias Järlund, CTO at AftonBladet, the biggest Swedish newspaper (so he claims at least) : high performance web sites, with ads (don’t let third parties make you slow) which was based on AftonBladet’s experience and experiments on web site, embedding third party ads, optimization.

Facts and figures

www.aftonbladet.se lately (due to the elections in Sweden) had all time high traffic with 2.4 millions visitors a day on its front page which, as it has always been, is really, really, … really long and contains no less than 20 advertising spots.

Web performance optimization, iframes vs. JavaScript

“The server side is not the bottleneck” and, according to Tobias, optimization should rather be performed on the client side.

There are plenty of solutions available (iframes and/or JavaScript based) and AftonBladet decided, after evaluating the pros and cons of some of them, to go for friendly iframes that have the following “properties” (but not only):

• may load in parallel
• possible to late-load
• full access to the DOM
• support all types of ads

The setup

1. create an iframe to a small static cacheable HTML page from the same domain
2. inside the iframe, use the JavaScript variable inDapIF = true to tell the ad it is loaded inside a friendly iframe
3. inside the iframe, create a script element with the ad url as the source (load the ad just like a normal JavaScript ad)
4. when the ad has finished loading, resize the iframe to fit the dimensions of the ad

Even if “using iframes is the best option today” to solve that kind of problems, be careful when implementing such solutions since rendering iframes is quite an expensive operation.

Tweaks

• don’t load all ads at once, use queues or delays
• set default sizes (1 by 1 pixel for instance) and then resize
• try not to block the “onload” event (the page loading speed is now taken into consideration by Google for ranking)

The future

• <frag> tag (Google Chrome) to mark a piece of code that should be loaded in parallel, not blocking (not implemented)
• postMessage(…) implemented in all modern browsers for messaging between iframes and the main page

Results

It’s certainly pretty cool when you get the chance to try out solutions and experiment different techniques but it’s even better when you measure the impact of the changes you make on your users’ experience!

By implementing friendly iframes Aftonbladet drastically improved the loading speed of the front page (editorial content loaded at the same time or even before the ads) and readers liked it…

• visits per week up 7%
• page views per visit up 16%
• “latest news” section (which used to show up really late in the rendering process) CTR (click through rate) up 35%
• average page load time down 30%

… and the most remarkable thing being that all those changes have not had any impact on the ads CTR…

Today a consultant from Mejsla AB, new member in my team and committer to the Terrastore project, Sven Johansson, ran a session on Terrastore, a documents store (a documents oriented database in other words) based on the Terracotta clustering technology. As the team is currently investigating NoSQL storage solutions for one of the applications its responsible for, it’s always nice to get to know about existing (or at least under development) products out there.

Terrastore is a modern document store which provides advanced scalability and elasticity features without sacrificing consistency.

The open-source project started in September 2009 and is basically an advanced key/value store (like CouchDB, MongoDB, RavenDB, …) written in Java. Where a relational database deals with a bunch of tables containing rows, a store has buckets in which Java objects can be stored and retrieved from.

/** A class representing a person */
public class Person {
  public String identifier;;
  public String lastname, firstname;
}

/** An instance of the class */
Person aPerson = new Person();
aPerson.identifier = "1234567890";
a.lastName = "Doe";
a.firstName = "John";

Now we want to store aPerson and retrieve it.

TerrastoreClient client = new TerrastoreClient("http://some_host:some_port", new HTTPConnectionFactory());

/** Store the document in the "address_book" bucket */
client.bucket("address_book").key(aPerson.identifier).put(aPerson);

/** Retrieve the document from the "address_book" bucket */
Person aPerson = client.bucket("address_book").key("1234567890").get(Person.class);

Sven also described the different deployment architectures currently available (cluster or multi-cluster) among other things… On September 12, 2010 version 0.7.0 of Terrastore was released (including changes like client instantiation giving a list of hosts instead of a single server address like in the above example) and more functionalities are already in the pipe: MapReduce operations, better integration with Elastic Search, …

If you want to read more on Terrastore, feel free to pay a visit to the project’s Google Code page (http://code.google.com/p/terrastore/).

Those couple of lines were my minutes of Sven’s presentation and, I have to admit, a rather brief introduction to the project. Good luck to the committers and looking forward to the baby’s next steps!

One of the software I had to install was the Tibco Enterprise Messaging Service (an implementation of the Java Message Service, JMS). The installation went fine (I had to deactivate all the visual effects in order to see the content of the dialog boxes though – under System > Preferences > Appearance) but I had some troubles starting EMS.

/opt/tibco/ems/5.0/bin/tibemsd

/opt/tibco/ems/5.0/bin/tibemsd: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory

EMS requires libstdc++.so.5 but only libstdc++.so.6 is nowadays bundled with Ubuntu. After some Googling I finally came across a solution that worked.

wget http://security.ubuntu.com/ubuntu/pool/universe/i/ia32-libs/ia32-libs_2.7ubuntu6.1_amd64.deb
dpkg-deb -x ia32-libs_2.7ubuntu6.1_amd64.deb ia32-libs
sudo cp ia32-libs/usr/lib32/libstdc++.so.5.0.7 /usr/lib32/
cd /usr/lib32
sudo ln -s libstdc++.so.5.0.7 libstdc++.so.5

If you have another solution that you’d like to share, please leave a comment!

sudo add-apt-repository "deb http://ir.archive.ubuntu.com/ubuntu jaunty-updates main multiverse"

Resynchronize the package index files

sudo apt-get update

Install Sun JDK 5

sudo apt-get install sun-java5-jdk

Now let’s look for a call to the “getSomeInformation” method made by the merchant 123 that should hopefully have reached one of the nodes. I will use zcat and cross my fingers…

zcat 10.1.*.*/application-a/http-requests-log.gz | grep 'getSomeInformation&merchant=123'