WEBO Site SpeedUp Blog

WEBO Site SpeedUp for Joomla! screencasts

noreply@blogger.com (Nikolay Matsievsky) — Tue, 09 Mar 2010 14:58:00 +0000

We are starting with a set of product screencast to help users with product tuning. The most important features will be described in step-by-step manuals.

Here is the first one - JavaScript / CSS files merging tuning.

All screencasts will be available on our website - www.webogroup.com/home/site-speedup-joomla/video/

v1.0.3 released

noreply@blogger.com (Nikolay Matsievsky) — Sun, 07 Mar 2010 17:29:00 +0000

A lot of minor usability and accessibility (both in administrative interface and application core logic) is this release. More info:

Added option 'Safe JS combine'. It guarantees that in case of broken JavaScript it will be loaded separately and won't harm other scripts. Here is used try-catch envelope (which reduces a bit performance, so it can be disabled).
Added Basic configuration set. Basic configuration includes all first-priority performance actions, and it is a bit better than Safe one. But it can fail on a few environments.
Added welcome screen after installation. Major UI improvement - now there is welcome screen with quick actions and new features right after WEBO Site SpeedUp installation.
Added Error screen on failed automatic installation. With welcome screen also added Error screen to help understand what is worng (very rarely) with WEBO Site SpeedUp installation (and which actions can be applied).
Other UI fixes and improvements.
CSS Sprites logic improvements.
Improved server environment detection.

Download the latest WEBO Site SpeedUp or update from the product.

HTML Sprites - silver bullet against small images

noreply@blogger.com (Nikolay Matsievsky) — Fri, 26 Feb 2010 19:47:00 +0000

Last builds of WEBO Site SpeedUp include amazing client side optimization approach - CSS Sprites for HTML images. Let's review underneath logic and all related techniques.

CSS Sprites - why HTML?

Actually we are speaking about combining images to a larger one which can be positioned with CSS property background-position. It's impossible to convert raw HTML images to CSS Sprite (to be correct - it's possible but leads a lot of restrictions and drawbacks, i.e. we will have to drop a huge part of images' functionality and properties).

So before any actual operations with images in HTML code (like <img src="..." />) we should prepare them. How? We should convert all images to 'fake' ones - transparent stubs which have initial image as a background, and the same width and height.

Converting HTML images to CSS images

Well. To reduce amount of data (we are going to speed up website, not to increase its load speed) we should choose very efficient approaches to insert transparent stubs. And we have 2 choices:

Insert call to 1x1 transparent GIF image (yeah, bad known spacer.gif which was used a lot several years ago to make table layout). Its size is only 43 bytes (but 1 more HTTP request).
For modern browsers we can use 82-bytes data:URI chunk (which represents this image - data:image/gif;base64,R...Ow==. A bt more data (with gzip usage - about 100 bytes for 1 HTML document, or nothing). But minus 1 HTTP request (cool!).

To provide the best practices here we can combine both approaches - for IE7 and lower use the first one, for all the other browsers - the second one.

Playing with HTML Sprites

It seems now we have all parts for HTML Sprites logic and can reveal it.

First of all - parse initial HTML code to find all images, get their sizes (and skip some large ones).
Then create an array of CSS rules based on given HTML images - just background image (plus no-repeat and zero background position), width and height.
Then just combine all images in 1 CSS Sprite (using existent CSS Sprites logic over current set of CSS rules).
Finally (with a number of CSS classes) replace initial HTML images with stubs with calculated styles.

All this looks very simple. But what are drawbacks?

Troubles and solutions

As far as we need to check size for any image (just to get its dimensions an skip it if it's very large) - we will have a lot of I/O requests with any HTML document. So this can significantly increase server side load time. That's bad. But we can cache calculated dimensions for known images (such dimensions are being changed very rarely) - and skip all future recalculations.

But with every HTML document (with cached calculated dimensions) we have one more expensive CPU operation - CSS background positions calculation (even with completed CSS Sprite we need to get actual positions of given images inside it). But good news here is that we can also cache such calculations (to a set of CSS rules for given HTML image).

Also we must have per-page or per-website modes. Just because you can have thousands of small images through the whole website - but you don't want your visitors to download them all at once.

With all such approaches HTML Sprites technique works both on a high-end servers and weak VDS/shared hosting. And reduces by 20-30% number of HTTP requests for every your web page. That's really amazing. And it's ready to use - just download WEBO Site SpeedUp or update to the latest version.

WEBO Site SpeedUp v1.0.2

noreply@blogger.com (Nikolay Matsievsky) — Thu, 25 Feb 2010 10:57:00 +0000

New version with HTML Sprites support and more performance.

Added HTML Sprites. There are now 3 new options: apply HTML Sprites, dimensions limit for them, and 'per-page' mode (you can combine small images for the whole website, or for the currect page only). HTML Sprites combine small HTML images and replace initial images with transparent one. This can greatly reduce total number of objects on your website.
CSS Sprites refactored. To meet strict resources limit on shared hsoting / VDS core logic for images' placement in CSS Sprites was refactored, and now it consumes much less memory, but a bit more CPU.
Added option 'Days to store cache files'. Now you can restrict number of files in cache with this option (outdated files will be deleted automatically when new ones are being created).
Improved unobtrusive JavaScript. Fixed layout bugs in a few cases and reduced DOM tree size.
Added support for ICO files into CSS Sprites. Now ICO files (as well as JPEG/PNG/BMP/GIF ones) can be combined both to CSS or HTML Sprites.
Improved JavaScript merging for DLE.
Social Engine (3.19) added to supported systems.

WEBO Site SpeedUp v1.0.1

noreply@blogger.com (Nikolay Matsievsky) — Tue, 16 Feb 2010 21:59:00 +0000

We didn't stop with product improvements, and after major 1.0 release continued to make it better. So here is a scheduled 1.0.1 update.

Greatly improved CSS Sprites. There were added a few complicated cases to CSS Sprites logic, fixes memory consumption (approximately by 20-50%). Also improved CSS properties in case of complicated selectors restoration.
Added automated request for 7-day trial key. Now you can test Premium Edition for 7 days (for 1 website) after installation, no performance or feature limits.
Fixed endless redirects on 404 errors. In some cases side caching with wo.static.php lead to recursive redirects. Such situations were fixed.
Added 'Share results on Twitter' buttons. Now you can post your optimization results directly to Twitter.
Minor UI fixes.
Improved static resources caching in case of symlinks usage.
Improved external resources fetching (i.e. in case of dynamic background images).

Download the latest WEBO Site SpeedUp or update from the product.

WEBO Site SpeedUp 1.0 released!

noreply@blogger.com (Nikolay Matsievsky) — Wed, 10 Feb 2010 18:53:00 +0000

After 2 months of creating, tuning, and polishing of the new interface, old and new bugs resolving and general compatibility improving we have released version 1.0 of this outstanding application to speed your website up.

Also we have released our new corporate website (formerly web-optimizer.us) to allow wide range of website acceleration products and services - www.webogroup.com.

Here are few words about new features in 1.0 (in comparison to 0.9.7).

Added option 'Performance -> Restore CSS properties'. It is responsible for correct CSS Sprites creation (parsing CSS rules' tree), but it can be CPU expensive, so it's disabled by default.
Improved Unobtrusive JavaScript. Added BlogBang? and eBuzzing added.
Improved external files fetching. There were fixed some bugs with external redirected files (or 404 errors on them).
Improved paths automatic calculation. Added better support for symlinks in directories.
Improved JavaScript merging behavior on HTML/JS/CDATA mess (added more rules for pseudo HTML comments).
Improved HTML minify and fixed its compatibility with System - Cache in Joomla!. Now HTML documents are even less in size!
Improved .htaccess rules (fixed unstyled websites).
Minor improvements in UI. Reduced number of files (with CSS Sprites), fixed minor layout bugs.

Get the latest WEBO Site SpeedUp or update it from internal interface.

WEBO Software increases efficiency for the website of charity fund "Happy World"

noreply@blogger.com (Nikolay Matsievsky) — Fri, 05 Feb 2010 20:05:00 +0000

WEBO Site SpeedUp increased efficiency of Russian charity fund - "Happy World".

WEBO Site SpeedUp is a wholesome product to speed up a website. There can be not enough time to tune hosting environment, sometimes you need any solution out of the box - that's about us. And it's much easier to have only one module to optimize website performance than apply handy changes to CMS or web server.

Children's charity fund "Happy World" (www.hworld.ru) was created in 2005. The foundation raises funds and provides assistance to children with serious illnesses. Only the last year 199 children received the necessary treatment at $2.6 million. Wards fund are children from poor families. They want to get well and together with their peers go to kindergarten, to school. Strong, reliable adult hand help to cope with the seemingly insurmountable problems, - says about fund activity Olga, its coordinator.

Geography of the project is quite broad - it covers the whole Russia territory. For this reason site of a charity fund for the last few years changed 6 hosting providers, which were periodically unable to bear the load. And now the website uses a distributed hosting from Media Temple (virtual hosting on the "clouds") with Joomla! + Apache + MySQL.

Average load on www.hworld.ru is about 20k unique visitors a month. But during the fund-raising campaigns it can increase several times. Once a campaign to raise funds for the treatment of one of our wards began, - says Michael, a fund technical specialist - and there were more than 8 000 unique visitors per day. It was necessary to accelerate the website load time for visitors and fit a dedicated hoster power. Due to the rapid increase in attendance it urgently needed to optimize Joomla! CMS. There was chosen the product for the universal website acceleration to achieve this goal - WEBO Site SpeedUp.

As Michael says, after the installation WEBO Site SpeedUp reduced consumption of GPU (unit of measurement of power consumption which is used in Media Temple). This allowed to fit the provided power and not to overpay for a non-optimized site. There have been indications that the site was being loaded faster. Both the first and returned visits were speeded up with competent caching. Latter is extremely important for the fund - there are a lot of regular visitors, and it is a huge benefit for them - fast website load speed and website stability.

What is the most important thing - in the most tense moments for the host when a child's life hangs in the balance, and is going active raising for medical treatment, we are confident that the site is working as quickly as possible, saving the lives of children - concludes Michael.

Website load speed is connected not only with the robust website performance, but also with the confidence of visitors to the website information. For charity fund the last fact leads the priorities list. And if there is any possibility to increase confidence of visitors to the charity - you need to do it now, do not fish or cut bait. Because the child's life depends on the decision about donation.

High Performance for Contra Costa County

noreply@blogger.com (Nikolay Matsievsky) — Mon, 01 Feb 2010 22:05:00 +0000

WEBO Site SpeedUp was installed in Halfway to Concord website and significatnly increased its profitability.

"Traffic is seasonal based on local election cycles and holidays, but we can document a 20 per cent increase in visits since implementing WEBO Site SpeedUp."

Launched in early 2004, Halfway to Concord (www.halfwaytoconcord.com) is a blog covering news, politics, culture, business and various events for Contra Costa County. The site's hyper local focus has helped it become a leading independent online news source for East Bay of Northern California, USA.

Halfway to Concord has more than 12k unique visitors monthly, most of them return to the website at least once a week. The website core is a WordPress installation on a Bluehost dedicated server. But even this didn't provide sufficient page load time which was more than 5 seconds for every page. And Bill Gram-Reefer, Halfway to Concord founder and editor, decided to improve user experience with speedier loading site to increase readership.

Because of its hyper local focus and audience, Halfway to Concord cannot currently accelerate with CDN usage or any number of particular WordPress hacks. There must be a solution to provide maximum website performance. "We had slow loading pages", says Gram-Reefer. "Many visitors chose to move on before page load."

CDN or other network distribution solutions didn't fit because the website is located very closely to the vast majority of its visitors and server response time is not crucial. But a complete set of performance optimization practices - including file compression to multiple host usage and unobtrusive logic implementation - provided by WEBO Site SpeedUp resolved all issues.

"While experiencing some expected bumps in the road during implementation, WEBO Software tech support was helpful and gracious, eager to help customers instead of berate them for asking questions", Gram-Reefer adds. Through close collaboration, Halfway to Concord was accelerated approximately 2.8 times, especially for return visitors. And they were grateful!

Finally YSlow was improved from 56 to 90. This brought about 20 per cent increase in visits. With a few simple actions automatically handled with WEBO Site SpeedUp, the website became faster and could handle more advertisement and visitors.

Only a full range of website load speed optimization methods can help such local websites or small blogs to provide new levels of user experience for their readers. And this can be easily achieved with very little effort.

WEBO Site SpeedUp v0.9.7

noreply@blogger.com (Nikolay Matsievsky) — Tue, 26 Jan 2010 06:38:00 +0000

We are moving to completely stable and featured v1.0 and here is one more update on this way. In the version 0.9.7:

Improved UI. All icons were recreated and combined to a few of CSS Sprites (via Auto Sprites tool). Now the whole interface loads faster and there are less files in package (update is going also faster).
Added option to exclude WEBO Site SpeedUp logic. You can define in System Status -> Settings a number of parts of URL to exclude from processing - and they won't be accelerated with WEBO Site SpeedUp. This field has the same syntax as for Server Side Cache exclude option.
Improved smush.it callbacks. Reviwed some more issues with bunch image optimization of a huge amounts of data. Callbacks from smush.it tuned a bit.
Improved static gzip tool. Now it can use gzip console tool if it's allowed to be executed from PHP.
Improved CSS Sprites logic. Fixed a minor issue with ancesting seletroes lookup and improved CSS Tidy merging qualities.
Improved gzip on CGI enviroments for CSS/JavaScript files.
Fixed minor compatibility issues.

Download new version from here or just update it from your product interface.

WEBO Site SpeedUp v0.9.6

noreply@blogger.com (Nikolay Matsievsky) — Tue, 19 Jan 2010 21:46:00 +0000

One more intermediate version before stable 1.0 release. We fixed a lor of UI bugs (makeup / client side logic) and improved general stability.

Changes log:

Improved general UI stability / fixed minor issues.
Improved Apache modules detection / .htaccess changes.
Added rollback to stable feature.
Fixed issue with rewriting current JavaScript / CSS files with wo.static.php equivalents.
PHP Fusion (7.0.5) added to supported systems.

Now there is the complete set of native modules (WordPress / Joomla / Drupal / Bitrix).

Download the latest WEBO Site SpeedUp

WEBO Site SpeedUp v0.9.5

noreply@blogger.com (Nikolay Matsievsky) — Tue, 12 Jan 2010 02:03:00 +0000

Today we have released the first (official) version of new WEBO Site SpeedUp. Its general state is beta (release candidate), but all unstable issues are related to UI only. Main core stability is improved (since version 0.6.7) and some new features were added.

Get the latest version.

Before public 1.0 release we are going to review and improve current user interface (in terms of usability and response times). Also it seems support for punypng optimization will be added to 1.0 (since they have API get up).

Feel free to submit any bugs here, we appreciate your help in improving this awesome software package.

You can also check public demo of WEBO Site SpeedUp.

WEBO Site SpeedUp public beta

noreply@blogger.com (Nikolay Matsievsky) — Thu, 07 Jan 2010 21:15:00 +0000

Here we are with the hottest performance optimization software. We are still working to pplish it but you can enjoy this stuff under you favorite CMS. Here are WEBO Site SpeedUp 0.9.5b download links:

Installation is the same as usual - just copy plugin in WordPress or Drupal to general plugin directory and activate it via administrative interface. Then go to Settings page and play with WEBO Site SpeedUp interface.

For Joomla! 1.5 you just need to install appropriate package (via Extensions installer) and go to the component's page.

Upgrade procedure

If you are using old version and want to upgrade to the newest one -- please store file web-optimizer/config.webo.php somewhere (just as a backup), than upgrade your package to this beta version. Usually you need to update it once more - from WEBO Site SpeedUp interface (Update is located in System Status -> Updates -> Show me beta versions). Then configuration file will be altered to fit new structure. If you have any troubles with upgrading - please contact us. It seems you should have at least backed up configuration file to perform restoration.

Joomla! users please note that there is no standard way to upgrade current Web Optimizer plugin to WEBO Site SpeedUp component + plugin. So please somehow save settings from plugin interface / database. You will be able to set them back (WEBO Site SpeedUp by default is in a debug mode, website won't be hurted, cache files will be saved).

Known issues

There a few minor bugs we are still working on:

Not all blocks are loaded at once on Control Panel (home) page in IE.
Interface doesn't look perfect under IE6/7.
Optimization results may be counted incorrectly sometimes (just not countered properly).
Some pages / UI elements will be changed a bit to make overall design better.

Complete ist of WEBO Site SpeedUp / Web Optimizer packages also is located here: http://code.google.com/p/web-optimizator/downloads/list

WordPress Performance: Step-by-Step Every Day

noreply@blogger.com (Nikolay Matsievsky) — Wed, 06 Jan 2010 13:37:00 +0000

We are going to completely review all sides of WordPress-based blog performance with a set of articles. For this purpose we created a simple blog on a shared hosting — www.speedingupwebsite.com — and every day there will be new articles about WordPress performance (both server and client sides).

We will review all possible plugins variations (on this website) and give you complete information how you can speed up your WordPress blog. No more '10/20/30/40+ tips about WordPress speeding up'. All information in one place. All failures and successes and all detailed manuals right here. And all of them will tested on a real website.

Keep tuned: www.speedingupwebsite.com or via RSS. If you have any ideas to cover in tests and comparisons - please share them in comments.

WEBO Site SpeedUp - configuration sets & debugging

noreply@blogger.com (Nikolay Matsievsky) — Thu, 31 Dec 2009 03:49:00 +0000

In the coming new version of WEBO Site SpeedUp we prepared a lot of technical innovations and UI enhancements. One of them is sets of configuration - now you can simply choose one of the predefined settings' sets, or create your own based on one of them.

Configuration sets

As you can see from the upper screenshot there can be unlimited sets of configuration with 3 predefined - safe (which is completely safe for all environments, but doesn't provide a lot of acceleration), optimal (balanced one), and extreme (which should be re-configurated before it can be used for a live website).

Application debugging

We new concept of debug / live mode for WEBO Site SpeedUp application you can simply tune / debug current configuration set and save it. There a lot of options to configure, so basic process can be a bit hard for newbies.

But we prepared a lot of hints through the user interface - they should help you to speedup your website with WEBO Site SpeedUp.

Configuration export / import

After basic configuration is ready you can prepare one-two more configuration sets (if you have time, or just on a test website). And copy them to the live website. After this you can apply any of available sets 'on fly' painless (also in debug mode to be re-sure that all is OK with your website).

With new dashboard cache refresh and debugging application is much easier. Your just need to press 'Enable' or 'Refresh cache' - all the other actions wll be performed automatically.

Get WEBO Site SpeedUp

All these options are avilable in the latest alpha version - http://code.google.com/p/web-optimizator/downloads/detail?name=webo.site.speedup.v0.9.1b.zip. It already has update-to-beta procedure (you just need to go to "System Status -> Updates" and check 'Show information about beta versions'. After this just press 'Install' (below current beta version change log) - all files will be downloaded from beta repository and applied automatically.

WEBO Site SpeedUp alpha version

noreply@blogger.com (Nikolay Matsievsky) — Wed, 23 Dec 2009 12:11:00 +0000

Actually in Xmas eve we prepared alpha preview of WEBO Site SpeedUp (formerly Web Optimizer). You can get it from official repository here (link updated to alpha2): http://code.google.com/p/web-optimizator/downloads/detail?name=webo.site.speedup.v0.9.1b.zip. Main feature is new interface and a few more tools + improved overall stability. Some screenshots:

Control Panel

Here are all important blocks which should help you to increase load speed of your website - Settings overview, System status, Cache, Load speed, optimization tools, Updates, News, etc.

Cache

On cache screen you can refresh overall cache and get all information about actual files.

On cache reload you see all performed actions.

System status

Here are listed all warnings / troubles with server environment, also some useful information about application.

Optimization tools

Optimization tools (static gzip and image optimization) allow you to compress your files in interactive mode. Image optimization will be performed via smush.it / punypng.com services (in alpha only former is available).

Get alpha2 version http://code.google.com/p/web-optimizator/downloads/detail?name=webo.site.speedup.v0.9.1b.zip and leave your feedback.

Any issues can be also submitted to issue tracker.

Web Optimizer for CDN

noreply@blogger.com (Nikolay Matsievsky) — Wed, 16 Dec 2009 01:51:00 +0000

After a couple of questions (how Web Optimizer can be used to move all assets to CDN) we prepared a quick quide to let you use Web Optimizer with your CDN easily.

So step-by-step

First you need to have CDN host setuped as a mirror for your website. I.e. if you have an image www.site.com/img/image.png you must have cdn.site.com/img/image.png (right now Web Optimizer correctly supports only 1 subdomain for CDN - cdn.yourwebsite.com).
After this initial CDN setup you new to set only 2 options in Web Optimizer configuration. First if 'host' (the very first screen of settings, near license key field). Set it to cdn.site.com. This turns on CDN usage for CSS / JS merged assets.
Then you should enable multiple hosts. For this purpose in the group "Multiple hosts" just enable their usage + disable option "Check hosts' availability automatically" (this seems don't work properly in case of CDN). And enter 'cdn' (without quotes) to the field "Allowed hosts, i.e. img i1 i2".

That's all! So you will have the following options:

Website host (to include before static resources), i.e. site.com ->
    cdn.yoursite.com
Enable multiple hosts -> Yes
Check hosts' availability automatically -> No
Allowed hosts, i.e. img i1 i2 -> cdn

So this will turn on CDN usage for your website fast and easily.

A good example of CDN mplementation is www.maxcdn.com.

Web Optimizer in Bitrix

noreply@blogger.com (Nikolay Matsievsky) — Fri, 11 Dec 2009 09:12:00 +0000

This night Web Optimizer has been added to official Bitrix repository (Bitrix provides large varierty of solutions to create corporate websites / intranet systems / e-stores). It seems now you can install for any Bitrix product (with the same license policy as for the other plugins / standalone Web Optimizer version - Community, Lite, and Premium Editions).

Also Web Optimizer is listed on Wordpress and Joomla! Extensions Directory. Feel free to leave your comment / vote for us there.

Rocket Boost your site

noreply@blogger.com (Nikolay Matsievsky) — Wed, 09 Dec 2009 20:30:00 +0000

To make your website ready for Xmas shopping season we offer 20% discount for all editions of Web Optimizer.

Simply use promotional code webo_nwy20092010 and get 20% off. Site optimization was never that easy!

Also you can order basic or advanced installation — our engineers will help to make your website the fastest one.

Promotion will end at December 31, 2009.

Conditional caching: several approaches

noreply@blogger.com (Nikolay Matsievsky) — Thu, 03 Dec 2009 15:26:00 +0000

Last month we talked about various caching layers last month. The latest 'line of defense' is conditional caching.

What is Conditional Caching?

Browser usually has a lot of resources in its local cache. Some of them can be expired but browser can check if they can be used once more. So there is a way (frankly speaking two different ways) to check if a resource can be used once more.

First we can use Last-Modified (specified in HTTP/1.0) and its pair — If-Modified-Since. Server sends header Last-Modified with the resource modification date. Browser can request if resource was changed since the last request and send this date back with If-Modified-Since header. If there were no modifications server responses with 304-answer.

There is almost the same situation with ETag and If-None-Match (or If-Match) headers. Expect the only thing that Entity Tag can be any string (date, set of numbers, file name, etc). So this allows you to define it any way. But ETag belongs to HTTP/1.1 specification.

Benefits

With 304-answers server actually doesn't send any content to browser so you save transfer time for these resources.

If can be also useful for a kind of dynamic content which can't be cached for a long time but can't be changed often. So browser re-requests such content (if cache is expired) but receives answer 'Not modified'.

Also if helps with pages' reload - when visitor presses Ctrl+R in his/her browser the latter must request from the server all resources. So such kind of refresh can be made faster with conditional caching.

Practical examples

Apache server usually sends ETag among Last-Modified. There is mod_headers that is responsible for such behavior (and it seems for 304-answers). You can add ETag header (which indicates file modification time only) this way:

FileETag MTime

You can also unset each of these headers:

Header unset Last-Modified
Header unset ETag

With PHP you can send the equivalent of these headers by:

@date_default_timezone_set(@date_default_timezone_get());
header("Last-Modified: " . gmdate("D, d M Y H:i:s", $time));
header("ETag: \"" . md5(gmdate("D, d M Y H:i:s", $time)) . "\"");

If you want to emulate 'classic' ETag which Apache server sends by default you need to use:

header("ETag: \"" . dec2hex(@fileinode($filename)) . '-' .
    dec2hex(@filesize($filename)) . '-' . dec2hex($mtime) . "\"");

where dec2hec is a helper to convert numbers for correct conversion numbers from decimal to heximal.

Issues

First of all you need to send different conditional headers with compressed and not compressed content. Generic approach here is to add '-gzip' to the end of the ETag (there is nothing to do with Last-Modified so it's a bit less usable).

Then you need to make such headers equal through all servers you use to server content. Because common ETag header in Apache includes information about inode (server-related, but not actually file-related), so it must be eliminated or replaced.

Then there is information (not approved yet) about excessive requests from browsers with conditional headers. Please be careful with this.

For static resources Web Optimizer unsets Last-Modified header for all static resources and sets ETag based on modification time. For dynamic ones it (if HTML documents are cached) sets ETag based on content hash and Last-Modified with static PHP proxy.

Web Optimizer 0.6.7 released

noreply@blogger.com (Nikolay Matsievsky) — Mon, 30 Nov 2009 23:06:00 +0000

Last 0.xx version before major 1.0 release has been published today - 0.6.7 aka 'frost'. We tuned a lot of minor stuff and improved a number of parts of application.

Added delayed load for iframes. Now you can exclude iframes (i.e. ads) from general waterfall of website load (to prevent their blocking nature) — the same way as for Unobtrusive logic.
Improved behavior for 'Uniform cache files' option. Now conditional comments are striped for all browsrs except IE and HTML cache files relate to current set of options.
Enabled optimization for cached content in Drupal. There was a bug in Drupal native module with not optimized content on caching. Now Web Optimizer works in any case, it doesn't matter have you cached your website on server side or not.
Separated Upgrade / Install to stable / beta branches. One more big change before 1.0 release — now you can upgrade either to the latest stable branch, or to the latest beta (the most featured) version on the product. And switch between them.
Added Web Optimizer module for Bitrix. Bitrix is the most known CMS for corporate websites, and now Web Optimizer supports it in native mode.
Improved unobtrusive logic (added some ads).
Improved files combine (added exclusions, general logic).

Download the latest version of Web Optimizer

Static gzip is your best friend

noreply@blogger.com (Nikolay Matsievsky) — Thu, 26 Nov 2009 18:01:00 +0000

After we touched several aspects of caching let's return to gzip and review very simple and powerful technique 'Static gzip'.

What is Static Gzip?

Static Gzip is a way to serve compressed content w/o its actual compression 'on fly' (here is a blog post about gzip). Very roughly we have gzipped files and serve them instead of common ones. How can we do this?

General algorithm

First of all we need to have gzipped versions of initial files. Usually they are named with a postfix .gz, i.e. main.css.gz. As far as these files are static we can have them compressed at maximum. With Linux you can do the following
```
gzip -c -n -9 main.css > main.css.gz
```
to get the smallest compressed file from the initial one.

Secondly we must have any way to route HTTP requests to take compressed version of the file. Via Apache and mod_rewrite we can add the following rules to .htaccess file or Apache configuration.

<IfModule mod_rewrite.c>
 RewriteEngine On
 RewriteCond %{HTTP:Accept-encoding} gzip
 RewriteCond %{HTTP_USER_AGENT} !Konqueror
 RewriteCond %{REQUEST_FILENAME}.gz -f
 RewriteRule ^(.*)\.css$ $1.css.gz [QSA,L]
 <FilesMatch \.css\.gz$>
  ForceType text/css
 </FilesMatch>
</IfModule>
<IfModule mod_mime.c>
 AddEncoding gzip .gz
</IfModule>

Rip off the veil of the mystery

What does this set of rules actually do?

First of all we enable RewriteEngine (it may have been already enabled in your configuration).
Then
```
RewriteCond %{HTTP:Accept-encoding} gzip
```
rule selects all HTTP requests with
```
Accept-Encoding: gzip
```
header and allows Apache to perform other rules.
Then we skip Konqueror browser
```
RewriteCond %{HTTP_USER_AGENT} !Konqueror
```
because it seems it doesn't understand compressed content for CSS / JavaScript files.
Then we check if there is physical file with postfix .gz
```
RewriteCond %{REQUEST_FILENAME}.gz -f
```
And only after all these checks we perform actual internal rewrite rule.
```
RewriteRule ^(.*)\.css$ $1.css.gz [QSA,L]
```
We redirect all requests to .css files to .css.gz files.
After this redirect (which is the last one — modificator [L]) we force Content-Type of such files to be text/css. Most servers send .gz with default archive encoding, and browsers can't properly detect this content.
Finally with AddEncoding gzip .gz rule sets proper Content-Encoding header for compressed content.

Of course this logic can be applied not only to CSS files but to all files which can be efficiently compressed — JavaScript, fonts, favicon.ico, etc.

That's all?

Graceful Degradation

What if Apache doesn't have mod_rewrite module? Or we need to serve CSS content via PHP? Or there is environment which doesn't support such logic (CGI / IIS)?

Well we can perform the whole algorithm actually via PHP too. All what we need here is the following.

Form compressed file name (usually just with a postfix .gz). And check for this file's existence.
If no such file exists, create compressed version of the file (i.e. via gzcompress function).
Then write this compressed content to a new file (with already defined 'gzipped' file name).
And set for this file time of change (mtime) to the value equals to initial (non-compressed) one. Why it's required? Because we can with check for existence also check if gzipped version of the current file has the same mtime and skip its re-creation in case of equivalence. If the current file and its compressed version have different time of change — it seems we need to re-create latter.

So with this logic we just check for file's existence and perform 2 checks for mtime (all such checks can be cached on file system level, or cache folder can be mapped to shared memory) and serve gzipped version. CPU is saved (with 80-85% of transferred content size)!

So Web Optimizer has all such approaches integrated and with version 0.6.7+ allows you to create .gz versions of CSS / JS files within any folder on your website.

Web Optimizer on Facebook

noreply@blogger.com (Nikolay Matsievsky) — Fri, 20 Nov 2009 08:29:00 +0000

Yeah, now you can join us on Facebook:

http://www.facebook.com/pages/Web-Optimizer/183974322020

Also there is a Twitter account

Several layers of caching

noreply@blogger.com (Nikolay Matsievsky) — Thu, 19 Nov 2009 13:53:00 +0000

The last two topics were about various aspects of client side caching. Also there was a topic about cache integrity check. For now let's review possible overall caching schema:

Web Optimizer can get optimized HTML code from its own cache. And output it to the browser. It's the first layer of caching — on the server side.
Otherwise Web Optimizer gets raw results from CMS engine (HTML code). But they can be already cached (in CMS). Web Optimizer doesn't touch internal CMS caching logic, only uses it if it's available. So here can be the second layer of caching logic — also on the server side.
During client side optimization performance Web Optimizer usually checks (view a complete description of this logic) if there are any files ready to be served (merged and combined ones). If yes -- all is OK here, Web Optimizer uses them. It's a one more server side caching layer here.
Also on serving files Web Optimizer (but usually Apache web server) checks if there are gzipped versions of files — .gz ones — and uses them (via mod_rewrite or static gzip) rather than 'gzip on fly'. The fourth level of server side caching. And overall website configuration can have 1-2 more levels (i.e. on frontend proxy, nginx or squid, or on shared memory virtual disk).
But before files are served browser receives ready HTML code with assets' URLs. And tries to fetch them in its own cache — browser's one. If yes (Web Optimizer sets strong caching headers) - such files aren't requested from server. It's a client side caching layer.
If there are no such files in local cache browser requests files but local (or not very local but intermediate) proxy server can have them cached (read more about caching on proxies). So proxy server gives such files faster than initial website (and request doesn't reach the website at all). Actually client side caching layer.
If request reaches initial server but has conditional caching headers (ETag or Last-Modified) server can response with 304-answer (and don't send content). So it's client side caching layer too — all content is taken from client side.

Wow! It seems that there is all. As you can see there is about 7-9 different caching layers somehow smudged between end client and end server (various chain links of requests' way from your browser to the website and back to you).

If the next posts we will try to light some of these layers in details.

Web Optimizer 0.6.6 released

noreply@blogger.com (Nikolay Matsievsky) — Mon, 16 Nov 2009 17:41:00 +0000

A lot of new features and bug fixes in new version.

Added separation for Community / Lite / Premium Editions. Now free version of Web Optimizer becomes Community Edition and it's prohibited to be used on commercial websites. For this purpose you can also buy Web Optimizer Lite Edition (data:URI + performance included, $19.99). Version comparison.
Added option to move all scripts (w/o merging) to </body>. If you choose to move all scripts to </body> but Minify JavaScript option is disabled — all scripts will be just moved one-by-one to the end of the document. Very useful.
Added option 'Uniform cache files for all browsers'. In some cases (i.e. if you use different from Web Optimizer caching engine) it's incorrect to differ HTML code through browsers (yes, this disables data:URI group of options but allows you to cache any page for any browser only once).
Added option 'Cache external files'. Now light PHP proxy can be applied to external files too. They can be downloaded, gzipped, and cached — to improve your own website load speed.
Added option 'Enable chained optimization'. There are a few issues with chained optimization algorithm (due to buggy server side environments or mix of rights). So it can be disabled to prevent any Web Optimizer tries to pre-optimize your pages. With this option disabled all website pages will be optimized by first visit only.
Added events onBeforeOptimization, onAfterOptimization, onCache to plugins API. These events can be used for standalone version to add any dynamic code to PHP, apply any internal logic, or add any dynamic pieces to cached HTML documents.
Improved behavior (content skipping). For content different from (X)HTML and for Ajax requests it's very tricky to append and optimization techniques (in most of cases such content has been already optimized, so it's just skipped).
Improved general logic in case of </body> absence. Sometimes HTML document has no </body> tag (yes, that's true). So we need to apply all logic in such cases too.
Improved multiple hosts behavior (especially for dynamic images).
Improved unobtrusive logic (minor fixes, added counters).
Improved Apache modules detection on CGI environments (especially mod_rewrite).
Fixed several tiny bugs in files fetching (after dozens of unit tests integrated).
VaM Shop and Gekklog added to supported systems.

Download the latest Web Optimizer.

Client Side Caching: proxy servers and forced reload

noreply@blogger.com (Nikolay Matsievsky) — Sat, 14 Nov 2009 11:38:00 +0000

In the previous blog post we have talked about caching basics. Let's review now how proxies actually work and how can we force cache reload.

Cache reload

The main issue with far future expires headers is that browser doesn't re-request resource but takes it from local cache. So if you have made any changes on your website they won't be visible for all 'old' users (with cached styles and scripts, HTML documents usually aren't cached so aggressively).

So what can we do with this trouble? How can we tell browsers to re-request such resources?

Main cache reload patterns

There are two main patterns to force browsers (user agents) to request the current asset once more.

Add to the file name any GET parameter (which should indicate new state of this asset). For example
```
styles.css -> styles.css?20091114
```
Change "physical" file name. For example
```
styles.css -> styles.v20091114.css
```

Both approaches change URL of the asset and force browser to re-request it.

Cache reload and proxy servers

As you can see the first approach is simpler than the second. But there a few possible issues with it. First of all some proxy servers doesn't cache URL with GET parameter (i.e. our styles.css?20091114). So if you have a lot of visitors from a network behind one firewall we will serve this asset to each visitor separately, without its caching of a proxy server. This will slow down overall website speed and sometimes this can be critical.

But how can we apply new file name without actual changes on file system? Is there any way to perform this with only change in HTML code? Yes!

Apache rewrite rules

Apache web server has a powerful tool to perform 'hidden' redirects for local file (this is called 'internal redirects'). We can manage the first way with just one predefined rule for all files (in our case it's a set of numbers after .v):

RewriteEngine On
RewriteRule ^(.*)\.v[0-9]+\.css$ $1.css

So all such files will be redirected to their physical equivalents but you can change a part of URL with .v at any time — and browsers will request this asset once more.

Automated cache reload

There are several ways to automate cache reload process for all changed files. As far as Web Optimizer combines all resources into 1 file, it's required to re-check file mtime (time of change) for all files and re-combine all resources.

Issues with re-checking all combined files have been already described last month, so it's not generally good to check them all with every web page visit. We can cache all previous checks into 1 file and check only its mtime. So it's done by default. By default we can check time of change of the only file (CSS or JS one) and add as a GET parameter or as a part of file name.

So this is applied for all such files (that should be cached on a client side) and results in the following:

/cache/website.css?1257927769

/cache/website.wo1257927769.css

As you can see there are two timestamps in these CSS files, one goes as a GET parameter, the other — as a part of URL (and with Apache mod_rewrite rule is transformed to /cache/website.css).

Overall schema

So what is overall caching algorithm for the website?

Check if we have combined file. If no — create it.
Check mtime of the combined file. If it's required add mtime to URL (using one of the described ways).
Browser receives HTML code with the URL of combined file.
Browser checks if it has this URL cached. If yes, all finished here.
If not browser requests cached file (which is already prepared on the server or is cached on the proxy).