Browsers that support HTML5 and session storage also have an ‘offline disk cache’, which is comparable to a SQLite database and can be operated via JavaScript. In general, this cache is about 500MB in size and GoogleMail uses this feature in order to make the application available when offline.

In terms of caching, XING focuses on optimizing ‘disk cache’ utilization as the ‘memory cache’ cannot be used in a targeted manner and does not represent a sound method for cache optimizing in the long term. We therefore do not consider the specific use of offline disk caching to be effective due to a lack of general support.

The ideal client caching state can be seen in the following figure:

Ideal state of a page request

As all assets are already cached by the client, no requests are sent to the server with only HTML content and tracking pixels being reloaded. At most, individual assets should be checked to ensure they are up to date by means of a 304 (not modified) request. This in turn equates to the minimum possible number of page views, i.e. minimal traffic, only the bare minimum of requests, and minimal processing time in terms of downloading time.

But can this state actually be achieved in a real-life scenario?

Currently, at least 50% of our PCs run Internet Explorer 8, Opera and FF2/3.x browsers, and this number is expected to rise (see following figure).

Percentage distribution of the top 5 browsers in use at XING

In their standard delivered state, these browsers have a disk cache of 50MB (Opera 20MB). At first glance this seems to be enough as an entire page request hardly ever exceeds 500KB.

However, a number of key aspects need to be taken into consideration:

1. Most users never clear their cache.
2. A rising number of (2.0) websites are constantly increasing in size coupled with well-documented instructions availabe for everyone in the web (e.g. Steve Souders) mean that operators are constantly having to work on squeezing as much efficiency out of user caches as possible.
3. Youtube, last.fm and other streaming stations are blowing up the cache with sometimes just one video/stream.
4. When looked at simplistically, the cache is a kind of “least recently used” (LRU) list, i.e. the files that come in and are not used for week may be dropped out, if meanwhile other files come in.
5. With some browsers, only the newer versions cache SSL content like that of xing.com (Firefox version 3.0 and above), and only if the cache header is ‘public’.

The continual development of the web means that the ratio of kilobytes to page view is likely to rise, as confirmed by our in-house statistics based on the xing.com website.

A combination of the ‘total time of the file in the cache’ and ‘last use’ then determines which files are deleted first.

So what is the upshot of this? Most users don’t have the pleasure of fully retrieving assets from the browser cache. Instead, the assets have to be requested again from the server, which increases loading time by at least 40%. When measured precisely, the number of visitors to the site with an empty or partially empty cache is surprisingly high.

Our own JavaScript-based measurements showed that 45% to 62% of users had an ‘empty cache experience’ while visiting xing.com, which is a surprisingly poor value.

See the green line in following figure for more information.

Image caching rate at xing.com

Our measurements correlate with those of a study conducted by Yahoo which used a backend-based method and returned an ‘empty cache experience’ of around 40% to 60%.

Conclusion: Well-designed caching has been proven to have a positive effect on user experience and is a genuine must both now and in the future. However, this isn’t the only area we need to focus on as there are too many other factors out of our control which may rapidly lead to the entire client caching being cleared out so that the user has to download all of the assets again. To this end, strategies such as minimizing the code and modularization need to be implemented together with lazy loading. Efficient caching and minimal requests are the only combination that can guarantee positive user experience in the long term.

By predicting the next page we can now load certain assets into the browser cache before they are actually needed. If the assets are then referenced via the source code of the returned page, they are then loaded from the browser cache and not requested from the server. This technology is best described as “anticipated preloading”. The goal is to anticipate what the user will do next, ideally based on precise and detailed tracking information on user behavior, and then to load static content in advance.

The success of theese measures were clear (see the orange line which shows the loading time of the users logged-in startpage):

Average response time on xing.com (green is the average time over all measurements)

Since October 2009, we’ve used “anticipated preloading” for all login forms. As of December 2009 we are explicitly pre-loading graphics of the most frequently viewed pages on xing.com. By preloading graphics that are not immediately used, we give visitors a “primed cache experience”, and we help to avoid the problem of poor browser caching.

We’ve seen results improve by over 10% since we began pre-loading.

Improved preloading support is expected from Mozilla in the future. Since Firefox Version 3.0 for http and since 3.5 for https as well, the browser offers a “prefetch tag”:

<link rel="prefetch" href="/images/big.jpg" />

The tag directs the browser to begin loading the image specified by ’href ’ once the current page hascompletely loaded. When the next page is called up, there is one request necessary. Prefetching also works with complete web pages:

<link rel="next" href="2.html" />

If you’re interested in learning more, please see
https://developer.mozilla.org/en/Link_prefetching_FAQ.

A migration changing an existing database table ‘people’ might look like this:

class AlterPeopleTable < ActiveRecord::Migration
  def self.up
    change_column :people, :first_name, :string, :default => "John", :null => false
    add_column :people, :last_name, :string, :default => "Doe", :null => false
    remove_column :people, :street
    rename_column :people, :phone, :telephone
    add_index :people, :last_name
    remove_index :people, :zip
  end
end

Now, we see two problems with this code. First, the table name ‘people’ has to be repeated in every line. Second, and more important, the code will lead to six separate ALTER TABLE statements being sent to the database. This will slow down the execution of the migration significantly, especially for large tables.
The good news is that databases like MySQL and PostgreSQL allow an ALTER TABLE statement to contain multiple clauses, so no matter how many alterations you want to apply to a table, they can be combined into a single ALTER TABLE statement.

Truthfully, we were surprised Rails did not already provide this.  So, we went ahead and added an alter_table method that follows the syntax already used in Rails migrations.

To take advantage of that, here is how you would write the same migration using the alter_table plugin:

class AlterPeopleTable < ActiveRecord::Migration
  def self.up
     alter_table :people do |t|
       t.change_column :first_name, :string, :default => "John", :null => false
       t.add_column :last_name, :string, :default => "Doe", :null => false
       t.remove_column :street
       t.rename_column :phone, :telephone
       t.add_index :last_name
       t.remove_index :zip
    end
  end
end

The produced SQL would look like this:

ALTER TABLE `people`
CHANGE `first_name` `first_name` varchar(255) DEFAULT 'John' NOT NULL,
ADD `last_name` varchar(255) DEFAULT 'Doe' NOT NULL,
DROP `street`,
CHANGE `phone` `telephone` varchar(64),
ADD INDEX `index_people_on_last_name` (`last_name`),
DROP INDEX `index_people_on_zip`

Currently the plugin only supports MySQL databases, but other database types will be added soon.

If you find yourself making changes to large tables in your Rails project, you should consider using the plugin.

The full README, including how to install and use it, and the code are here:
http://github.com/xing/alter_table

We at XING are constantly making use of new technology and techniques. So we decided to use the CSS3 declaration “border-radius” for our text buttons. “Boring, old, I know that” you might say and you’re right – it’s not really new – although it’s not been rendered properly by all browsers up until now. What’s new, and here’s the point, that we at XING have embraced the idea, that our site doesn’t have to look the same way in every browser (also see Ingo Chao’s blog post Degradation without grace). The text buttons are rectangular – obviously – in Microsoft Internet Explorer, because it doesn’t understand rounded corners via CSS3. Users of the newest version of Safari, Firefox and Opera (10.5) will continue to see rounded text-buttons. Not the same as last week, but they look nearly the same.

What did the text buttons used to look like?

A short time ago, we styled the links with additional HTML and background images (CSS sprites), so they look round on all corners.

Screenshot of old XING links

Not only did we have to use additional (and meaningless) markup, the CSS declaration was massive as well. And here’s what the new text buttons look like today:

Screenshot of new XING links

.text-button,
.text-button-right {
   background-color: #d6ecea;
   border: 1px solid #bcd9d7;
   -moz-border-radius: 7px;
   -webkit-border-radius: 7px;
   border-radius: 7px;
   color: #333;
   display: -moz-inline-box;
   display: inline-block;
   font-size: 11px;
   line-height: 13px;
   margin: 0 5px;
   padding: 0 7px;
   text-decoration: none;
}
 
a.text-button:hover,
a.text-button-right:hover {
   background-color: #bcd9d7;
}

What’s the benefit in using rounded corners via CSS3?

Using CSS3 to round the corners saves a lot of HTML and CSS. To demonstrate, we put the new text buttons to a test. For this we built a little page with ten old text buttons as well as a page with ten new ones. We tested the speed of the page-loading with a tiny JavaScript and were pleased with the result we achieved.

• The HTML code is only 60% the size of the old one
• Because we don’t need the sprite image, we can, in the future, only have one Http-request, instead of two
• No more unnecessary HTML equals a reduced DOM depth, now at 2 instead of 4
• The CSS had 62 lines, now it only has 23 lines of code

Comparing the old with the new CSS, it should be obvious why we decided to use the rounded corners via CSS3. The functionality of the webpage is still intact. Everyone with a “modern” browser, i.e. one that understands CSS3, will see the rounded corners, everyone else has to live with rectangular corners.

Is XING trend-setting?

Well, ‘trend-setting’ might be a little bit too much, but we can say, that XING is looking forward – to the future. We use CSS for Webkit and for Mozilla browsers as well, to display rounded corners. But we also put the required border-radius into the CSS – without any browser-specific prefixes. By the way: Opera 10.5 will be the first browser supporting the border-radius.

And for all of you wondering why we used the display: -moz-inline-box; – well, more on that issue later. Come back and find out.

Microsoft introduced Compatibility View for IE8 in accordance with their “Don’t break the web”-maxim. Previous versions of Internet Explorer did not follow web standards, IE’s development seemingly came to a standstill, consequently frontend engineers used an awful lot of workarounds to fix the non-standard rendering. However, IE8 follows the standards, and the old workarounds caused many sites to break – hence Compatibility View. In Compatibility View, IE8 renders a page like IE7 did. This was meant as a helper for pages that were not updated yet.

Meanwhile, Compatibility View became a cause of stagnation: Sites will likely be slow to update their strategy, keeping IE in Compatibility View. Most big sites run their pages in Compatibility View nowadays, because “it just works” in IE7, and because the costs to re-engineer and to re-test the pages are fairly high. In contrast, the benefits of using Standards View in IE8 are not obvious; performance, of course – but you cannot measure that until you try it.

Browser Mode and Document Mode

There are two modes that rule Compatibility View: browser mode and document mode.

Browser mode is to be set in the Compatibility View settings by the user and via a list hosted by Microsoft. This Compatibility View List is distributed via Windows Update to the IE8-client. XING used to be on the list, but was removed by Microsoft, so some users will land on XINGs platform in compatibility browser mode, some in standards browser mode. Although Microsoft doesn’t make this clear, site owners have to opt-in to standards mode because of the ever-changing Compatibility View List.

Document mode can be set set by the site owner: as a meta-flag in the HTML document or as a response header by the server. XING used to set the document mode via <meta http-equiv="X-UA-Compatible" content="IE=7" /> in the HEAD section of your HTML documents.

For the beta-test migration to IE8 standards mode, we removed that meta tag. To set the document mode, our servers answered with a response header instead:

• “X-UA-Compatible” “IE=8” for XING developers and
• “X-UA-Compatible” “IE=EmulateIE7” for all other users.

This server-side switch allows for forking depending on the IP-address.

Again, you cannot rely on leaving the response header or meta tag unset because you cannot exactly know what Compatibility View List settings the browser is using. Users may or may not have used Windows Update recently; and a third party controls the list.

Fig.1: Analysis of IE8 browser mode set for Alexa’s Top 100 Sites (Nov. 2009). More than 50% are set via the Microsoft-hosted Compatibility View List.

Fig. 2: Analysis of IE8 document mode set by Alexa’s Top 100 Sites (Nov. 2009). Quirks mode: Without a standards-mode-doctype. IE7-Standards: Render mode set via Response-Header, Meta-Tag, or Compatibility View List. IE8-Standards: One of the mentioned trigger or no trigger at all. Only 30% use the most standards adherence mode.

Beta Test Findings

Our internal beta tests revealed a couple of problems:

• Bug with vertical-align: text-top for inline elements in our transitional doctype (solution: ie8-specific workaround, since MS stated that they will not fix it for IE8)
• Bad hyphenation in some texts: ‘Nicht-\nMitglieder’ becomes ‘Nicht\n-Mitglieder’ (solution: won’t fix)
• Prototype JavaScript library had to be updated when available (solution: update)
• User-Agent sniffing breaks for those users that had the old Compatibility View settings. They land with an IE7-like User Agent, receive the response from the server, and proceed with a IE8 User Agent string. (Solution: User Agent sniffing on a pre-sanitized User Agent string.)

“Haha. So, you are coming from IE6/7-specific workarounds and got IE8-specific workarounds in addition?” This is partly true. On the other hand, we gained a lot more standards compliance and run a comparatively small set of workarounds, with the benefit of better performance for a growing set of users who switched to IE8. More users are getting less workarounds.

Performance Results

Because there is no need to serve IE7-related workarounds to IE8 any more, the amount of loaded CSS rules per page decreased significantly, and we saved one HTTP-request on every page load. The page load times (Page loaded, Dom loaded, Interaction ready) are better – measured and perceived. IE8 was known to be faster than its predecessors, and we could measure it on-site, not just on laboratory conditions.

Was it worth it?

Yes, definitely. We stopped using a comfortable mode that meant a dead-end-street for development, and we got measurable benefits for our users.

We will also be providing you with little peeks into our daily work life, which, as it turns out, is quite exciting here at Europe’s leading business networking site. We have development teams based in Hamburg and Barcelona working on new and improving existing features, and also making sure that things run as smoothly as possible, which every now and then can be an interesting challenge. Watch out for a bunch of future posts about our technical setup, how stuff is glued together, and how we keep it all running without major hiccups. You will see how developing a site like xing.com can indeed be quite an adventure — of awesome!

Which, of course, you’re very much invited to be part of. If you’re into stuff like Ruby on Rails, Perl, Java or JavaScript, fond of agile development, and would like to work on an established product that’s always changing and moving forward, check out our current job openings — we’d love to hear from you.

Have a good 2010 everybody — rumors are this one’s going to be good.