blog

Login to HDRLab Using your Facebook Account

Thu, 18 Aug 2011 00:21:41 -0500

It is now possible to log in to this website and post on the forums using your Facebook account. This was added so that people don't have to create and remember yet another user and password combination. Like many people, I have an increasing number of accounts on multiple websites, and remembering all of those usernames and passwords gets harder and harder (particularly with websites that have "special" rules regarding what a password should have). Special thanks to Will Rossiter for developing the Silverstripe Facebook Connect module.

To log in using Facebook, simply click on the "Login with Facebook" button at the top of the forums, and enter your facebook login in the window that opens. If you already have an account on this website using the same email address as your facebook account, then your existing account will automatically be linked to your Facebook account. Otherwise, a new account will be created for you.

NOTE: New accounts created when logging in using Facebook will use your first name as forum "nickname" by default. You can edit your profile at any time using the "Profile" link shown next to the "Log Out" button at the top of the forums pages.

GfxBench2D's Traffic Spike

Thu, 21 Jul 2011 02:41:09 -0500

I haven't posted to this blog in a long while, so I thought that I would share some interesting traffic data related to the newest section of the website. Just over a month ago I released GfxBench2D, a 2D benchmarking tool and web-application. This tool measures 2D graphics performance and can (optionally) upload results to this website for display. Well, this triggered the biggest traffic spike that this website has ever seen. Take a look a graph of the number of daily visits that have occurred since this website was first created (up to 10 July 2011):

As you can see, it dwarfs the next highest traffic spike, which coincided with the announcement that compositing support for Radeon HD 2000-4000 cards had been added to the RadeonHD driver. Not only did it dwarf all previous traffic spikes, but it was also the first spike not generated by the RadeonHD driver project (the first spike was the very first entry to the RadeonHD Development Log). The crazy thing is that this spike in traffic could have been even higher if I had submitted the news simultaneously to the various AmigaOS related forums. I deliberately staggered the news posts to spread the traffic out over several days. The graph of unique visitors follows the same pattern. Clearly, GfxBench2D appeals to a lot more people than than the Radeon HD driver project does despite only an AmigaOS 4.x version of the tool being available.

The effect of GfxBench2D's release becomes even more pronounced when looking at the number of page-views per day:

Not only did more visitors check out GfxBench2D than any of the RadeonHD development log posts, but they also collectively viewed more pages while they were here. This difference comes as no surprise. While all of the RadeonHD development log posts are well contained within a single page, the GfxBench2D section provides pages and pages of results (182 results at present) to examine. As a result, visitors tended to browse through the available benchmark data instead of reading a single page and then leaving.

The final thing that is noticeable in the graphs above is the slow but relatively consistent rise in traffic to this website. This is a trend that I hope will continue or, preferably, will be able to accelerate.

Website Search Feature Restored

Sat, 23 Oct 2010 00:00:00 -0500

I got an email this morning from someone who tried to use the search functionality of this website, and got a "Generated with the default ContentController.ss template" error. Apparently, the search function has been broken since I upgraded to Silverstripe 2.4.0, and no-one noticed (obviously, I didn't), or bothered to tell me up to now. Anyway, it is now fixed, and I have updated the Silverstripe JRank Integration page with a new archive.

There were two issues. The first is that the API of Silverstripe's XML class was changed, breaking my code in the process. However, even with this fixed, the Generated with the default ContentController.ss template" error persisted. This was despite the template being present, and even being partially executed (according to stack dumps that I deliberately triggered). This confused me for a while. Even more confusing was that it suddenly worked, albeit with a broken layout, when I changed the SearchResultsPage class to be a child-class of Page, and its controller to a child-class of Page_Controller.

Eventually, I realised that the template file was in the wrong place. Since the SearchResultsPage is a direct child-class of SiteTree, its template is supposed to be in the templates directory (where the base Page.ss is), not templates/Layout, where it had been put. For some reason having the template in the wrong place used to work pre-Silverstripe-2.4.0, but not anymore.

NOTE: I made the SearchResultsPage class a child-class of SiteTree so that I could eliminate the left-column navigation bar for the search results, which would have made the layout too cramped. If you want to use the same base layout as the rest of your pages, change the SearchResultsPage to be a child-class of Page, the SearchResultsPage_Controller to be a child-class of Page_Controller, and move SearchResultsPage.ss to templates/Layout.

Heater Recommentation for a Cleaner, Healthier Room

Fri, 03 Sep 2010 18:26:10 -0500

This post may seem a bit out of place in this blog, but seeing as it relates to health, I think that it's important enough to post anyway. It is also an example of how good design and attention to small details makes a difference. The short story is that I recently bought a new heater, and it has done wonders to the air quality in my room. One small change has made the room so much more pleasant to work in.

The Problem - Poor Air Quality when Heating the Room

I do most of my work in a small room without central heating, and had been using a small fan heater to stay warm during winter. While the warmth was welcome, the air in the room quickly became very stuffy. It seemed like the heater was sucking up dust off the floor, heating it, and then blowing it into the room, sweeping even more dust into the air. Yuck! As it turns out, this was more or less exactly what was happening.

Needless to say, it wasn't particularly pleasant breathing in that heavy dust/particle laden air, and I frequently opened the window just to flush it out again. Opening the heater to clean it out, and vacuuming the room made a little difference, but it still wasn't good enough. Since I spend a lot of time in that room, it was time to find a new heater. One that would heat the room without the stuffiness.

The Solution - DéLonghi Ceramic Heater

The DéLonghi TCH7592PB
Ceramic Heater

Having decided that I needed a new heater, I set about looking for one that would keep the air clean and breathable. The room was too small to bother with a heat-pump, so the search was on for a conventional heater. Whatever it was, it had to have the following two features:

An air/dust filter,
Relatively quiet, and
An electronic thermostat.

To my surprise, almost no heater had a dust filter on its intake, something that would help reduce dust particles in the air. After much searching, I found that the best option was the DéLonghi Ceramic Heater (NOTE: Model numbers may vary from country to country, in New Zealand, it's the DéLonghi TCH7592PB, whereas in the USA, the closest model is the DéLonghi TCH7590EB). I had never thought that I would spend $170 on a heater for a small room, but it has been worth every penny. The difference to the room's air quality was so big, that I wondered how I had put up with the old heater for so long. Keep reading to see why.

Good Design - Attention to Detail

So why do I think that this heater makes such a big difference. It is a combination of attention to detail and the quality. Firstly, the heater stands tall vertically, moving both the air intake and outtake off the floor. As a result, it isn't sucking dust off the floor, or whipping dust of the floor/carpet into the air. This one design decision alone reduces the amount of dust in the air. Next, it has an air filter which, despite being a small and cheap addition, nevertheless helps keep the air in the room clean. Having the air filter is one of the things that separates it from other tall heaters. Those of you who are concerned that a tall heater might fall over and burn the house down, don't be; there is a switch in the base that disconnects power to the heater if it tilts over (or is lifted off the floor). There are other vertical orientated heaters out there, but this is the only one that I found that had an air/dust filter.

It is also quiet. No, not Mitsubishi heat-pump quiet, but much quieter than the rumbling that most fan heaters make. In low heat mode the fan is slowed down, and only a soft air-through-pipe noise is heard. In automatic mode, it will heat the room rapidly, and then switch to the quieter low mode one the room is close to the desired temperature.

The other important point is that it is good qulity both in design, and construction. Despite having a lot of plastic parts, it is solidly built, and doesn't look like it's going to break easily. The plastic is also of a type that can handle the heat. Due to its vertical design it also doesn't take up much space.

The heater has other features, but they aren't important to me, and didn't factor in my buying decision. By far the most important thing is that I now have a warm room without the stuffiness. Breathing clean air makes a world of difference.

I recommend anyone who is using cheap fan heaters to consider replacing it with the DéLonghi Ceramic Heater, or any other heater that has an air/dust filter. If you don't mind waiting longer for the room to heat up, a convection heater that's easy to clean might be another idea. Just ditch that cheap fan heater, your lungs will thank you for it.

Computer Dies, but it's Only the Power Supply

Thu, 10 Jun 2010 02:03:36 -0500

I got a bit of a scare today when my AmigaOne XE G4 computer stopped working. Unlike standard PCs, this computer cannot be replaced by simply walking down to a computer store and buying another machine. If this machine did have to be replaced, I would have to order one from overseas. Since I am hard at work with various Amiga projects (particularly this one), this would have been quite a setback.

I have had trouble switching it on for a while now, but thought that this was due to a flaky power switch. As it turned out, it wasn't. The power supply was nearing its end. Today, I was alerted to a problem a beeping noise that sounded like a wrist-watch alarm. Looking at my Amiga, I saw that it was coming from the fan controller, which showed that the temperature was 60 degrees, and rising. So, I quickly shut it down. After opening the case and waiting a minute, I tried to switch it back on, but it failed. A few more attempts later, the fans switched on, but rotated very slowly and made a horrible noise. The power LED remained off. Clearly, something was broken.

Since I was already suspecting the power switch, I pulled the power switch out, and performed some quick tests with a multi-meter. Sure enough, the switch was perfectly fine. Besides, the switch could not have caused the strange fan behaviour that was observed.

By this stage I was thinking in dismay about how long it would take for a replacement machine to arrive. Added to that, I'd rather not spend money on a new computer until I can get my hands on an AmigaOne X1000. Anyway, given the way that the fans were sort of turning, but making a dirty rasping noise, I started suspecting that the power supply might have died. Fortunately I happened to have a spare 230W ATX power supply lying around (was going to be a power supply for an electronic project; not any more), so I swapped the power supply around, plugged the machine back in, and pressed the power button. The machine immediately started; no trouble with the power button, no noisy fans, just a computer working as it should. Phew, that was close.

This incident has reminded me just how disruptive hardware failure can be. In this case, shipping a replacement Amiga would take weeks. However, the last time that I had to reinstall Windows on my laptop (after the audio stopped working), it took me two days before I had everything reinstalled, and back to approximately how I wanted it. First, a restore from backups on a network drive failed, then the Windows install CD failed due to some mysterious problem with the DVD drive. Finally, a copy of Windows XP x64 installed, but it still took a very long time to reinstall all the software. I cannot remember if it was Norton Ghost or Acronis True Image that failed with restoring from backups. All I know is that neither program has worked properly for me, and so I'm still facing the possibility of a lengthy reinstall the next time that disaster strikes. This is a problem despite my rigorous backups of all critical data.

Maybe a better idea would be to have a system that backs up only the user data, and keeps a record of what programs were installed with what settings. Then, instead of attempting a whole system restore (which seems to fail a lot), the restore utility would run the appropriate installers to get a fresh copy of all applications, and then restore the user's data. In theory, this should be more reliable, result in a more stable reinstalled system, and save a lot of space with the backups. To date I have never found a product that works like this, so this is a free idea for anyone working in the data backup industry.

In the meantime, all my machines are once again working as they should.

Upgraded to Silverstripe 2.4.0

Sun, 30 May 2010 15:49:08 -0500

This website has just been upgraded to Silverstripe 2.4.0, which adds (amongst other things) hierarchical URLs. While the effort has been worth it, this upgrade was more problematic than previous ones due to version 2.4 including a cleanup of the API, and some rather major changes. Fortunately I now have a test server that I can do trial runs on before making changes to the live website. Here's a quick rundown of the problems that I experienced, and the solutions.

With the first test upgrade, a series of warnings about extraStatics() being deprectated, followed by an error saying: [User Warning] Cookie 'bypassStaticCache' can't be set. After a quick look for this error on the Silverstripe forums, I discovered that a set of modules had to be upgraded. Thus, the following modules were upgraded to either the latest public release (if that release supported Silverstripe 2.4), or the latest daily snapshot:

auth_openid,
blog,
forum,
gallery,
mollom,
newsletter,
securefiles,
spamprotection, and
userforms.

With these modules upgraded, the rebuild (/dev/build) succeeded. However, new problems arose due to some module upgrades being incompatible with older versions. In particular, the forum module had lost all posts, and some of its links (e.g., the login link) were broken. The broken links were due to the templates being out of date, so I took the new templates, and fixed up my theme's CSS files so that the forum looked good again. Next, the forum documentation mentioned a migration task (/dev/tasks/ForumMigrationTask). After executing the migration task, all of the forum posts reappeared.

The next issue was with the userforms; the form itself had disappeared, leaving only the submit button. Here too, a migration script was available. Unfortunately it didn't work. It complained about cartain database tables starting with _obsolete were missing. Looking in the database (by executing 'SHOW TABLES;' in MySQL), I noticed that the tables in question had been renamed with all lowercase, and the script was expecting certain letters to be capitalized. So, the script was updated to match the database, and the script executed a bit further. Nevertheless, there was still an error: "Unknown column 'CustomParameter' in 'field list'." Another search on the forums revealed that this was due to a residual obsolete column still being present. Executing the following fixed it:

ALTER TABLE `EditableFormField` DROP `CustomParameter`;

With this done, the migration script completed successfully, and after republishing the forms, the forms worked correctly again.

The last remaining issue was very minor; the blog widgets had disappeared. These were simply reinserted using the Silverstripe CMS pages. Unfortunately the Tag Cloud and Archive widgets are still not working. These are issues that I still need to look in to.

Once the website was working properly on my test server, the upgrade procedure was repeated on this live site. Interestingly, data (e.g., form submissions) that had disappeared on the test site remained intact in the actual upgrade. This is probably due to various steps that I took back then being skipped in the actual upgrade, because I knew by then that they would not work.

So far Silverstripe 2.4.0 is performing well. Small issues that I had previously are now gone, it feels a little faster (although this may be pure imagination), and the hierarchical URLs are a welcome improvement. Don't worry, the old URLs are automatically redirected to the new ones so existing bookmarks should still work.

Adbrite Dumped for Poor Performance

Tue, 27 Apr 2010 00:00:00 -0500

It's been a while since I have posted anything to this blog, mainly because I have been busy with other things. I have, however, still kept an eye on it, deleting spam when necessary, and monitoring how the advertising programs are performing. Today I made the decision to dump Adbrite.

A few months ago, Abrite revenue dropped to almost nothing (over a 10 times drop), and stayed there. I decided to give it a few months before deciding whether to continue using their service or not. Well, revenue has remained very low, and a pitiful few impressions are actually shown. Adbrite advertisements also had a poor click-through rate due to poor relevance. Hence, today I ditched that advertising network.

Ironically, round the time that the sudden unexplained revenue drop occured, I was about to ditch ADSDAQ for providing mostly ugly slimming and teeth whitening advertisements, and stick with Adbrite. Since then, has ADSDAQ improved dramatically, and filtering has removed the offending advertisements.

Acording to this article, someone else had a similar dramatic drop in revenue from Adbrite after they limited his accound for invalid clicks without any notification. This may have happened in my case, because the month prior to the revenue collapse my account recorded a few clicks (yes, performance was that poor). I am not going to bother checking if this is the case because I would still dump Adbrite for the following reasons:

Failing to notify customers/partners of important events (such as restricting their account for alleged violations of terms and services) is completely unacceptable business practise to me, and
If my account has been restricted for alleged "invalid clicks" which I did not make or cause, that would demonstrate a very poor invalid click detection algorithm on which I cannot rely, whether I am advertising, or providing advertising space.

So, either Adbrite has suddenly started performing very poorly, or their "invalid clicks" technology and business practises are unacceptable. Either way, they're gone.

I still haven't found an advertising network that I am satisfied with. ADSDAQ is okay, but has a low fill-rate (impressions per page view). Chitika works quite well, but caters only to visitors from the USA and Canada. Google Adsense was okay, until they disabled my account due to invalid click activity from an unknown source (I still have no idea who was responsible). I have just started testing Brinkin Banner Exchange, which combines concepts from both banner exchanges, and contextual advertising. It is an interesting concept; we shall see how it performs.

Mollom Learns from its Mistakes

Mon, 07 Dec 2009 00:00:00 -0600

Several weeks ago I mentioned that spammers had found a weakness in Mollom. Well, Mollom's learning algorithm has successfully learnt the new tricks that spammers were using, and the spam comments have stopped. The number of spam attempts has not decreased, but I have now had over a week without a single spam comment coming through.

Mollom's anti-spam algorithm learnt the new behaviour fairly quickly. Within a few days, the number of spam comments that made it through dropped from over 100 down to only a few. Getting rid of the last few remaining spam comments took another few weeks. I still hope that Mollom will learn faster in future, but it's current performance is quite good.

Fighting with Linux

Fri, 27 Nov 2009 00:00:00 -0600

Last night I had bit of a battle with my Linux server. The evening had already started out badly with a backup of this website failing due to some mysterious Sqlite error (note, that is was on a Windows machine). Next, backups from my laptop to the server failed, and I could suddenly no longer access network shares on the server by the server's name. So, I logged on to the server from my laptop, opened the Samba config file, and added an entry, specifying its netbios name, and then restarted Samba. At this point, the server was visible in the "network neighbourhood" again, and the backups could proceed. Little did I know that this was only the first problem that I was going to be dealing with that evening.

With the backups working, I turned my attention to the little indicator that said that there were system updates available. "Might as well update those files while I'm here," I thought. Bad idea; had I not done this, my evening would have been much more enjoyable. One by one all of the updates were failing, due to security.debian.org not being accessible. I had noticed this before, and decided to fix this issue. Security.debian.org was actually accessible; it responded to pings, and the web browser had no problems accessing it. After searching the internet, I discovered that you disabling ipv6 solved the problem (I've written a how-to, here, for future reference). With that done, the updates started coming. At the same time I decided to upgrade to the latest version of Debian.

Upgrading a Debian distribution takes a long time, so I left my machine running, and went off to more fun stuff. Returning late in the evening, the remote Gnome session window on my laptop, had gone black, and wouldn't wake up. Normally, I would just close the session and start a new one but, with an update in progress, I did not want to do this. After another search on the internet for ideas, I eventually discovered that one can regain access by killing the screen saver.

My joy at having learnt a new trick was short-lived. The system update had stalled a while ago because one of the updates had a question for me. Grumbling, I pushed enter, and then waited; I knew that there would be many more questions during the update process. Eventually, updates started failing. Security.debian.org was un accessible, again. A system reboot failed to make a difference. This was rather frustrating since it was late, and I thought that I had fixed this problem. Pinging the server gave the answer; this time server.debian.org really was down. Argh!

Security.debian.org remained down till mid-morning today. Now the system is completely up to date, and I'm relying on Debian's reputation for reliability to save me from having another frustrating battle like this for a while. Ah, computers; so much more complicated then they need to be.

WidgetBucks Removed from This Site

Wed, 25 Nov 2009 00:00:00 -0600

I've been trying out different advertisers on this website, WidgetBucks being one of them. The hope is that, one day, this website will cover its own hosting costs. As of yesterday, WidgetBucks is no longer active on this website. This was done because WidgetBucks popped up a message saying that they were going to disable the account in three days because it was failing to meet "minimum requirements," as outlined in the terms and conditions. I checked the terms and conditions, and there was nothing there that I had failed to meet; they didn't even have minimum revenue requirements. Rather than try to fight the decision, I decided to pull all of their advertisements off this site for the following reasons:

The revenue was extremely poor, averaging just one click per month,
WidgetBucks is very North America centric, whereas this website's visitors are spread out globally,
Visitors from most countries get very low quality CPM (Cost per thousand impression) advertisements (e.g., those annoying flashing "you are the 1,000,000th person to see this" advertisements), and
Most advertisements were completely irrelevant to my website's content, and thus served only as a waste of space.

Given all of the above, I decided that WidgetBucks was not worth continuing with. It may work well for sites with primarily North American visitors, who are interested in buying MP3 players and/or printer cartriges, but for other sites, it is likely to perform poorly. Irrelevance was the same reason that I ditched Bidvertiser months ago.

At present, this website still uses two advertising services, Adbrite, and Chitika. Adbrite's advertisements are still marginally relevant but, I can (and do) reject advertisements, and people can purchase advertisements to run on this website simply by clicking the "Your Ad Here" link below the banners.

Most visitors will probably have never seen Chitika advertisements, because they only show up when someone from a supported country (currently only the USA and Canada, but they plan to expand) arrives at a page via a serach engine. The advertisements are tailored to what they are seraching for (so it's likely to be relevant to them). Everyone else (including regular visitors) don't see the advertisement. Despite showing only to American and Canadian visitors who arrived via search engines, Chitika is currently outperforming Adbrite. This is most likely due to the advertisements being shown by Chitika being relevant to what visitors are searching for.

This website is still far from covering its own server costs from advertising. With an average of about 100 page views per day, this is not surprising. Daily page views should be in the thousands or higher in order for advertising revenues to reach a meaningful level.

AVLabs Digital Photo Frame Stops at "Initializing"

Tue, 24 Nov 2009 00:00:00 -0600

If you have a digital photo frame from AVLabs (or some other company) and it simply shows "initializing" when you insert an SD card, and switch it on, try a smaller card. I came across this problem recently when my sister bought a digital photo frame, and a 4 GB SD card. Unbeknownst to both of us, a 4 GB SD card, is in fact a 4 GB SDHC (High Capacity) card, and the SDHC standard is incompatible with the SD card that it is based upon.Yes, that's right, despite having SD written on the front (with HC in a different font), and having the same physical size as the original SD card standard, only SDHC card readers can access the data on 4 GB to 32 GB SDHC cards.

After taking the photo frame and SD card home, photos were copied onto the SD card, the card was inserted into the photo frame's card slot, and the device was switched on. The result was a black screen with "Initializing" written in the top left hand corner. The photoframe could still be switched to AV input, but otherwise it remained "Initializing" perpetually. Unfortunately, none of those who had this problem seem to have fixed it (and posted it on the internet for the benefit of others). This blog post aims to rectify this.

At this stage, we had a quick look through the manual, which was about 16 useless pages of stating the obvious, and providing no useful details (stating that the switch marked power switches the device on and off is not useful). Next, a quick look on the internet, brought up conflicting reviews; some said that it was a great device, while others complained about the poor manual (agreed), and how it simply says "initializing." Okay, others have the same problem.

Next, I tried a USB drive, which it also failed to read. After a few wasted hours, it was discovered that it would read a USB stick, but not a USB hard-drive. Finally, I looked at the Secure Digital entry in Wikipedia, which states:

"Changes to the interface of the established format have made some older devices designed for standard SD cards (≤4GB) unable to handle newer formats such as SDHC (≥4GB). All SD-cards have the same physical shape and form factor however, which causes confusion for many consumers."

Ah, so that is the problem. After copying the photos to a 2 GB SD card (minus the HC bit), everything worked perfectly.

To me, this is an example of poor user interface design. Sticking at "Initializing" when a device isn't readable is very poor feedback to the user; it tells the user nothing about why it doesn't work, and makes it very likely that he/she will take the device back to the shop. Having a user manual that doesn't provide even basic setup/installation instructions only adds to the problem. Granted, when a photo frame works, there isn't much to the installation but, when things go wrong, the manual should provide some assistance.

In summary:

Some digital photo frames (e.g., the one that we tried from AVLabs) cannot handle SDHC cards or higher (i.e., cards with 4 GB or more storage space),
Some photo frames can also only handle USB sticks, not other kinds of USB storage devices, and
If it doesn't work with your existing card (e.g., stops at "Initializing"), try a 2 GB SD card or smaller.

Software-as-a-Service (SaaS) Challenges - Reliability and Trust

Wed, 18 Nov 2009 00:00:00 -0600

I have always had my reservations about Software-as-a-Service (SaaS) due to issues such as reliability and control over my own data. To date, no SaaS vendor has addressed my concerns, indeed some people have been openly agressive, to the point of suggesting that I'm so paranoid that I need my head checked. Well, last night I had a problem that highlighted my reliability concerns: the power supply to my home router broke, leaving me without internet until I could buy a replacement today. Due to Dick Smith electronics becoming increasingly poor in its electronics selection (it is now hopeless for the electronics enthusiast), I had to drive all the way into the other side of downtown wellington to Jaycar, in order to buy a 5VDC 3A power supply. One small failure knocked out my internet connectivity for half a day.

I'll get back to my power supply failure, and SaaS reliability soon, but, for those who aren't familiar with the term, SaaS desrcibes software that is accessed as a service over the internet, rather than buying and installing it on ones own machine(s). From the SaaS vendor's perspective, it's great because:

You have a continuous stream of revenue from subscribers rather than a one time revenue when people buy, and
Pirating the software is almost impossible (bar security breaches) because customers never have the full application; the application's core functionality runs on the server, and the customer only gets the display/user-interface portion.

Reliability

Back to my power supply failure. The power supply is but one small part of the chain between me, and the SaaS system that have to work properly. Depending on where the servers are lkocated relative to me, there are

my computer,
the router (and its power supply),
the DSL modem (if it's separate),
the DSL hardware at the local exchange,
the network connection from the local exchange to the ISP's nearest hub,
the connection from the ISP's hub to the national backbone,
the connections from the national backbone to the server's country (may go through multiple countries), and all routing equipment involved (possibly including satellites),
the server's machines, and
implicitly the power generators, power lines, sub-stations and power supplies that supplies all of the above with power.

If any of these systems fail, then so to does my connection to the server. Sure, each of these systems is very reliable, but the more serially connected systems that you have, the greater the likelyhood of overall system failure. Putting multiple systems in parallel, increases redundancy, and thus, reliability (which is why we backup our data).

Now, let us assume that I was using a service such as Xero, and/or PlanHQ as a critical part of my (fictitious) business. If any critical device fails between their severs, and my business fails, then I can no longer access and update my accounting data (Xero) and/or my business planning and collaboration data (PlanHQ). Both services allow you to save the data in some external format, but is this enough? It still won't be possible to update or modify the information. One could say, "just wait until it's back up," but that does not address the issue. If the failure lay outside of the business, then I would be able to do nothing to make the service come back faster, other than maybe yelling at the vendor, and hoping that this helps. Moreover, there would be that annoying realization that I would have had none of these issues if the server was running within your own organization (and if it did fail, I could fix it).

Things would be even worse if my company also used salesforce.com; then the entire sales team would be unable to do their work effectively. If the eCommerce website also used PayPal, or some other payment gateway then sales would also grind to a halt. The more external systems that the business relies upon, the more likely it will be that something fails, and takes part of the business down. Don't think that it can happen? My power supply failure is a small example of the many things that can go wrong. Other examples include:

Earlier in this decade a major systems failure (I cannot find the details) at the moment) of one of the major data hubs on the USA's west-coast crippled New Zealand's main data link to the rest of the world for several days,
In 2003 the entire north-east of Canada and the USA had a power blackout caused by one fault triggering a cascade of shutdowns (not directly data related, but servers need power); other blackouts have occured around the world and in New Zealand too, over the years,
In 2008, a Pakistani telecommunications company inadvertently knocked youtube offline globally,
In June 2009, Xero was taken offline by a massive data center failure in the USA, and
In August 2009, Paypal went offline for several hours (for the second time this year),

Failures can, and do happen, despite the internet increasingly haing multiple connections and redundant systems.I do have to point out that, in the case of a failure, SaaS vendors may be in a better position to deal with it then a small company would, if that small company were maintaining its own systems. Nevertheless, there are still many more things that could go wrong, many of which are outside the control of both the customer, and the vendor (i.e., anything that happens on systems between the two).

Data Security and Trust

The next major issue is whether or not one wishes to entrust private data to a third party's care. When I mention this, invariably someone pipes in with "if you're doing nothing illegal, then what is the issue?" Cue, accusations of paranoia. This is very easy to say if we're talking about a few personal photos and you live in a country with freedom of speech, and in which persecution of political or religious views is illegal, but it misses the wider picture; maintaining access to ones own data is just as critical as preventing others from reading and using it. Here the reliability issue raises its head again; all that personal/business data is of no use to me if I cannot access it myself. Moreover, Microsoft's computing cloud demonstrated vividly that a computing cloud can lose data when its Sidekick service lost T-Mobile's customers emails, and personal data. A computing cloud is supposed to be highly redundant, and robust to failure, yet here was a massive data failure that no doubt resulted in the loss of highly critical information. This is not the only data failure that has occurred.

Next is the issue of protecting sensitive information. For many businesses their information is a critical part of their competitive edge, so maintaining privacy is critical. Customer's personal details are particularly sensitive, and having that data stolen has serious consequences. Fortunately, this also applies to SaaS companies, so it is in their best interests to make sure that no-one - not even they - can access your data. SaaS companies typically have stringent privacy policies, but this does not necessarily translate to technological barriers such as encrypting customer data. For example, Xero's privacy policy is good, but it does not explicitly say how securely the data is stored; it states that Xero employees cannot access usernames and passwords, and so cannot access customer data without the customer's administrator explicitly giving them permission, but would someone with physical access to the server be able to read the data. Why would someone want to do this? Well, if your company developed some technology that a competitor felt would bankrupt them, then there would be ample motive to have a go at the data center that it was stored on. Most of us will never be in a position where this matters, but it is still something that SaaS companies have to address. One only has to look at the myriad of credit card number theft reports to realize that privacy policies does not equal data security.

I have to point out that data on an in-house server is just as susceptible to theft as data on an SaaS company's servers; in fact, it may be even more so if your company lacks data security experties. However, this is another issue that makes people like me more reluctant to use SaaS software. Plus, keeping the applications and data in-house just feels better, because then I know where it is stored, and I'm in charge of looking after it.

Possible Solution - A Shadow Server

Here is an idea that would address the issue of data reliability and access during a server/internet outage: a shadow server. This would be a local server within a business that maintained its own copy of all data on the SaaS server, and provide access during any external outages. It would also act as a redundant copy of the data should a Sidekick type data loss occur.

Google Gears would be another alternative, but while it works well for things such as Google Docs that operates on single files, it probably wouldn't work well for applications with large databases; you are unlikely to want to cache a company's entire database on every machine, especially using an SQLite database (which is what Gears uses).

The few times that I suggested something like a shadow server to an SaaS company (I can't remember to whom), they never replied. Allowing their server software to run on a customer's machine isn't something that most of them wish to contemplate, given piracy issues, and the many different server configurations that customers could have. However, it could be run on a virtual machine, allowing them to use a standardized installation, to administer it remotely, and lock down the server as much as possible as an anti-piracy measure. Virtual private servers have been around for a while now, so the technology is there. Not everything could be run on the shadow server, e.g., PayPal would be unwise to allow a shadow server to perform transactions, but having full access to their data locally would generally put customer's minds at ease, and make people less reluctant to use SaaS.

There is one major technical issue for a shadow server, data synchronization. Let us say that a company's data link goes down, but the main server is still running. The CEO makes changes which are stored in the shadow server, and so does the sales manager, who is currently remotely accessing the main server whilst on a business trip. When the data link is restored, the servers would have to be resynchronized, and any conflicts (e.g., two people edit the same item) resolved.

I know that at least one person is going to say "ah, but your power supply failure would take down your local network, and cut you off from the shadow server." True, until I plug my machine into the server directly, or into another hub (assuming that I have multiple units). Moreover, I can get up and do something about it instead of waiting and hoping for someone somewhere to do it and, that, is more satisfying.

Final Comments

Everything in life has an element of risk, so the issues above are no reason to reject SaaS. I personally use a few of them myself. However, I do hope that SaaS vendors will consider allowing customers to have a shadow server, or provide some other form of offline capability that is functional; allowing customers to save their data to a text file, PDF, or a report, is not good enough. Cloud computing may sound cool, but there is something reassuring knowing that your data is on a disk next to you, rather than on a disk (set of disks) somewhere on this planet.

I'd be interested to know what others think of these issues, including the opinions of SaaS companies. Feel free to comment below.

Spammers Find a Weakness in Mollom

Sat, 24 Oct 2009 00:00:00 -0500

Logging in to work on this website today, I was greeted by an ugly sight: comment spam! The number of comments on this website had increased by over 150. It appears that spammers have found a weakness in Mollom, and exploited it.

Mollom blocks spam by analysing comments, and then grading it as "ham" (i.e., a genuine comment), "spam," and "unsure." If it is unsure of whether the comment is spam or not, then the submitter is presented with a captcha image, and must enter the characters shown. These images are only readable by humans, at least no-one has shown a spambot that can interpret them so far. If the user enters the captcha characters correctly, then the comment is accepted as ham, otherwise it is rejected as spam.

The spam that made it through has such innocuous content such as "great site," or "Thanks for this." This is made to look like a genuine comment. What makes it spam, is that the comment's "author" is something like "enlarge_your_..." (I leave ... to your imagination, it's not hard to guess), or the name of a certain product. Plus, the "author's" website is the website belonging to the spammer, who almost always trying to sell shady/dirty products and services.

The number of spam attempts that this website receives rose sharply at the end of September from 20-40 per day up to hundreds per day (peaking at 484 spam attempts on 1 October). Up till now almost all of them were blocked. It has just been over the last day or so that they have broken through. Needless to say, I will be monitoring the situation, and contacting Mollom if it continues.

Blocking Hotlinking of Images by Other Websites

Tue, 28 Jul 2009 00:00:00 -0500

There have been several cases of external websites embedding images from this website directly into their pages. Known as hotlinking, this results in bandwidth being used up on this website, for the traffic belonging to another website. Hotlinking is essentially stealing of bandwidth from another website, which someone else (or me, in the case of this website) has to pay for. Up until now I have been fairly relaxed about this, after all, some of these external pages were discussing projects on this website and providing inbound links. However, I have also had totally non-related websites hotlinking websites. Most recently, a classified listing on another website was using a photo of my old Compaq Presario in order to sell their item. Not only is this non-related traffic, but the photo shown in the advertisement was most certainly not of the item being sold; something that could be misleading to prospective buyers (it's clearly not a company promotional shot, so the assumption would be that it is the second hand item being sold). With this latest incident I decided that I was fed up with others stealing my bandwidth for non-related websites, and any page that hotlinks to this website now shows the following image:

How to Block Hotlinking

There are various methods with which hotlinking can be blocked, but the method that I used was to add this to the end of the .htaccess file for this website (below the Silverstripe URL rewriting):

# Block image hotlinking
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(.+\.)?hdrlab.org.nz [NC]
RewriteRule .*\.(jpe?g|gif|bmp|png)$ /images/nohotlinking.png [L]

This works by examining which webpage the file request was linked from (i.e., the referer), and redirecting any image request for which the referer is not this website (or unknown) to the image shown above. Note that "RewriteEngine On" must appear in the .htaccess before the code above, or it will do nothing. Anyone using this code should replace "hdrlab.org.nz" with their own website's URL.

What's Wrong with Hotlinking?

Some people would argue that there is nothing wrong with hotlinking. Their claim is that, since the website owner has made it publicly available, people should be able to do what they wish with it. This is not the case. What appears on one website can be copyrighted content that may not be used on another website without permission. However, this is not the real issue, the real issue is who pays for the bandwidth resulting from website traffic.

Let us say that I owned a website with millions of visitors per month, and I hotlinked an image from your small website with a 1 GiB/month allowance and placed it on the home-page. All of a sudden your website would receive millions of downloads of that image, which, not being a small file, this promptly exceeds your bandwidth limit, which either takes your website offline, or gives you a hefty bill for extra bandwidth charges. Clearly this is unfair; you should not have to pay for the bandwidth of visitors to my website, or any other website for that matter.

This is why websites should either obtain permission to host the image themselves (which they should be doing anyway, regardless), or posting a link to the website containing the image so that it can be viewed within the original website. In a previous argument about this topic, someone pointed out that an external website posting a link to my website would result in even more of my site's bandwidth being used up than if they just hotlinked the image. True, however this bandwidth would be due to people visiting my website, as opposed to visitors of the other website. I would be quite happy to pay for extra bandwidth if this website had a huge influx of visitors; hopefully the advertisements shown to these visitors would cover the extra costs, or at least offset them. What I am not okay with is paying for another website's traffic.

Watermarking Images - An Alternative to Blocking

If bandwidth is not an issue for you, but you object to the lack of acknowledgement when someone hotlinks an image, another option could be to insert a watermark into hotlinked images. That way, your logo and URL could appear in the image itself whenever it is hotlinked. I personally opted for blocking and placing this website's logo and URL into the redirected image; I do want to discourage hotlinking.

Tips for Aligning Satellite Dishes

Tue, 07 Jul 2009 00:00:00 -0500

I have just spent a ridiculous amount of time setting up a satellite TV system for receiving Freeview, free satellite TV for New Zealand. A lot of this time was due to my own lack of experience, and the difficulty in knowing whether something is really working, or not aligned. Here are a few tips for anyone else wishing to set this up themselves. This is not a complete guide, just some ideas that should save time. It assumes that you are installing a Ku band dish for digital TV, but a lot of the advice applies to any satellite dish.

Before Installing the Satellite Dish

Buy/borrow a satellite finder. Do NOT rely on just the satellite finder that is built in to the receiver.
Some receivers are advertised as having a built in satellite finder. What this means is that it can emit the same annoying sound that a satellite finder does. However, it will only indicate the signal strength of the current transponder that it is tuned to. If you have incorrect settings (e.g., the wrong LNB oscillator frequency) then you could have the dish pointed at the satellite and not know it. More importantly, the satellite finder is much more sensitive and, therefore, does not have to be aligned as precisely. This will save you much time and frustration.
Buy/borrow a spirit level.
Taking the time to get the dish mounting mast perfectly vertical will save time later. If you are using a motor, this is essential.
Buy/borrow a compass and be sure to adjust it for the local offset for magnetic north.
It will be much easier to find the satellite if the dish is already pointing in roughly the right direction. Magnetic north and true north are at different locations, and the offset varies from place to place.
Write down the Local Oscillator (LO) frequency of the LNB. NOTE: A universal LNB has two LOs.
I made the mistake of mounting the dish complete with LNB before noting the LO frequency, which just happened to be different from most "standard" LNBs for the KU band. As a result, the satellite receiver was looking for the signals at the wrong frequencies. The box that the LNB comes in usually does not list this information.
Unless you have access to the right crimping tool (i.e., a coax cable crimp tool, not a normal crimp tool), it is much easier to use the screw on connectors.
Use dishpointer.com in order to find out where to point the dish.
Using the dishpointer.com data, check that there are no obstacles blocking the view of the desired satellite(s).
The high frequencies used by satellites are line-of-sight only. What this means is that you will not be able to receive from a satellite that you do not have a clear view of (by which I mean clear sky; you won't be able to see the satellite with the naked eye). Note that most Ku band satellite dishes have the LNB mounted offset from the centre, so they appear to be looking lower than they really are (typically by 20°-30°).

Installing the Satellite Dish

Make sure that the mounting pole is vertical using the spirit level before attaching the satellite dish.
The more accurate you are earlier on, the easier it will be to find that satellite.
Make sure that the you have compensated for the offset between magnetic north and true north.
Dishpointer.com provides a magnetic azimuth compass reading, so it may be easier to use that. However, a compass that has be pre-adjusted should use the true offset. If the arrow is not pointed at 0°, then it may have been adjusted.
Do NOT trust the elevation/lattitude markings on the satellite dish.
Obviously the markings are a good reference, but they can be off by over 5°. In my case, the elevation adjustment mechanism was very loose, and I finally found the satellite 5-10° lower than the markings on the dish. I have read that some people manually check the elevation, but remember to look-up and compensate for the LNB mounting offset (e.g., a dish with a 28° LNB offset should be pointed 28° lower than the actual elevation).
Check if the LNB as a mark on it indicating the horizontal polarization angle.
Satellite signals can be vertically and horizontally polarized (and circularly polarized too, but this is less common for TV). There is also a "skew" angle for the satellite which should be used to adjust the angle of the LNB. If the LNB is rotated to the wrong angle, then a weak signal, or no signal could be received. I made the mistake of assuming which way the horizontal antenna was oriented, and payed for it later (see the section below called "Some Transponders are not Working").
Be as accurate as possible with alignment.
The closer the dish is to the correct alignment, the easier it will be to lock on to the satellite. This includes the azimuth (compass angle), elevation (how high), and LNB skew (rotation of the LNB in its mount).
Even with the satellite finder connected, it is still useful to have the satellite receiver and TV visible/audible from the satellite dish during alignment.
The satellite finder is great for telling you whether the dish is pointed at a satellite, but it won't tell you which satellite; that is where the satellite receiver's signal level and quality indicators come in.

Aligning the Satellite Dish

Start with the satellite finder sensitivity turned right up; it will make a noise when the dish is pointing close to a satellite, and the needle will swing to the right.
Check if the LNB is working (see "Is my LNB Broken?" below) before starting.
Once it starts beeping (squealing really), turn down the sensitivity of the satellite finder, and rotate the dish (very slowly) a bit more until it starts beeping again.
You should not have to move more than a degree from the initial point at which it started beeping. If you do, try turning in the other direction. This process is repeated until the dish is aligned.
Try to align the azimuth first, and then adjust the elevation, and then the LNB skew.
If you cannot find the satellite at one elevation, perform a grid like search swinging back and forth with the elevation adjusted by 1° increments per sweep.
New Zealanders should start their sweep a little to the east, so that Optus D1 is the first satellite that they find (assuming that you want to watch freeview).
Optus D1, C1, and D2 are quite close together, and it is easy lock on to the wrong one.
Make sure that the satellite receiver has the correct LNB LO frequency(ies) set.
As was mentioned above, the receiver must have the right LO frequency, or it will be searching for signals in the wrong place.
Once you have a reasonable lock, check the signal level and quality on the satellite receiver, with it tuned to the desired satellite.
If you are not receiving a signal despite the satellite finder indicating the presence of a satellite, then the dish is aligned with the wrong satellite.
The LNB skew can make a big difference.
I ended up with being able to receive signals from some transponders, but not others. This turned out to be due to the LNB not being rotated to the correct angle. See the section below called "I Can Only Receive Some Transponders but not Others" for more details.
Check that all transponders have a signal.
This assumes that the satellite receiver has been preprogrammed with the channels on the target satellite. If not, the satellite receiver should have the ability to scan for transponders.

Is My LNB Broken?

After scanning the sky for a while (without the satellite finder), and finding nothing, I started to wonder if the LNB might be broken. Searching through the internet, the best advice seemed to be: "point the dish at a satellite, and check if you get a signal." How on earth can you do this if you don't even know if the dish is aligned! Well, I have found out a relatively easy way to check if a Ku band LNB is broken: a satellite finder that is turned up to maximum should start squealing if you put your hand in front of the LNB (in front of the end that should be pointed at the dish). Alternatively, pointing the LNB at the ground should have the same effect. Your hand and the ground are sources of noise which the LNB should pick up, resulting in the signal detected by the satellite finder. If the satellite finder does not make a noise, then the LNB is likely broken.

It is unusual to receive an LNB that is dead, but it does happen. This test is also useful if satellite reception suddenly stops, since this could be caused by a number of different things; it could be caused by a dead LNB, a broken cable, the wind pushing the dish out of alignment, etc.

I Can Only Receive Some Transponders but not Others

This is a problem that I had, which initially had me confused. I was receiving strong signals from some transponders, whilst some others (which had channels that I wanted to see), were completely dead. To make things even more confusing, those very same transponders had worked not too long agon. It turned out that I had rotated the LNB slightly (adjusted the skew), resulting in a lost signal. In fact, my initial skew setting was significantly out from where it should be, and I had switched from a standard LNB, to a universal one. Universal LNBs seem to have stronger polarization characteristics, which can result in a better signal, but also requires better alignment.

Final Comments

This blog entry has become much longer than I had expected, but it contains a good summary of the kind of issues that can come up during satellite dish alignment, and how to avoid them. I hope that it will save other people some time and frustration. There are plenty of satellite installation "how-tos" on the internet, but none of them prepared me for the issues that I encountered.

Online for a Year

Sat, 04 Jul 2009 00:00:00 -0500

Roughly a year ago this website was opened. By opened, I mean first indexed by Google and Yahoo; the web-space and domain name were registered a year and a month ago (the start of June 2008). It took about a month to get this website to the point that it was ready to receive visitors. At that point, I registered a Google Webmasters account, added this website to the list, and started creating external links.

One of the reasons for creating this website was simply to find out how it all works. In particular, I was curious as to what level of traffic a newcomer's website would actually obtain, and how much effort was required. There are plenty of claims on various websites that one could get rich quick by doing the following:

Pick a "niche" topic,
Slap together a website,
Put Google AdSense and/or other advertisements, and
Watch the traffic and watch the dollars roll in.

Pure fiction of course; building anything takes time and effort. No-where could I find solid details on website visitor numbers and advertising revenues, so the only way to find out was to create a real website, and record its performance This is why this blog has monthly reviews/statistics posted; they provide a profile of the first year of a personal website's existence. Unfortunately my Google AdSense was disabled for invalid click activity (which I still do not know who was responsible for), so those results are unavailable. However, the total would not have been more than tens of dollars anyway, as the number of page-views has been far below the hundreds of thousands of views required.

So, how did this website perform? The monthly visits, unique visitors and page views are shown below. The number of visitors quickly rose up to about 1500 in October 2008, when I first announced the RadeonHD for Amiga OS 4.x project. From October onwards the number of visitors dropped off a little, and then rose back to the 1500 level. Most likely there are around 1000 - 1500 people who are interested in my Amiga projects, ignoring the additional visitors who come looking for various things via Google, Yahoo and other search engines. Thus, this website's visitor levels have probably already hit the boundary of the Amiga "niche." The increase in page-views in the last month was caused by the introduction of the forums, and the opening of the RadeonHD driver beta testing program.

Another thing that I was curious about was visitor locations over time. Each monthly review contained a snap-shot of the ClustrMap for that month. These have been collected into an animation (below), showing a record of visitor locations over a year. This animation gives a vague idea of how the animated visitor website location map concept that I posted on the "Free Ideas" page might look.

So there you have it: a profile of visits to a personal website in it's first year for the curious, or the bored. With this complete, I am going to stop posting the monthly statistics. This "experiment" was all about the first year of operation (and creating the animation above), which is now complete.

Site Statistics for June 2009

Tue, 30 Jun 2009 00:00:00 -0500

The statistics for June 2009 are:

Attribute	Value
Number of Pages	194
Size of site	131.98 MiB (HELM provides the total)
Number of Visits	2041 (Google Analytics)
Number of unique visitors	1468 (Google Analytics)
Total page views	5809 (Google Analytics)
Bandwidth	1.66 GiB (server control panel)

A snap-shot of visitor locations on 1 July 2009.

Site Statistics for May 2009

Tue, 02 Jun 2009 00:00:00 -0500

The statistics for May 2009 are:

Attribute	Value
Number of Pages	188
Size of site	127.38 MiB (HELM provides the total)
Number of Visits	1866 (Google Analytics)
Number of unique visitors	1412 (Google Analytics)
Total page views	3768 (Google Analytics)
Bandwidth	1.20 GiB (server control panel)

A snap-shot of visitor locations on 1 June 2009.

Every Man and his Dog has a Web Forum...

Sat, 30 May 2009 00:00:00 -0500

... and now, so do I.

Previously I saw no point in adding a forum, because there was little to discuss, and I didn't want to join the hordes of webmasters with forums on their website simply because "I can." There are enough online discussion boards without a topic to discuss as-is.

This website has slowly grown to the point at which a forum could be useful. In particular, the RadeonHD driver project is now reaching a stage at which I am ready to release a beta version to a select group. Now that there is something to discuss, the HDRLab forums (or fora) are now open.

Marketing Matters, Even for Engineers

Thu, 21 May 2009 00:00:00 -0500

Today a section on marketing was added to this website. It may seem strange for marketing to appear on the personal website of an engineer. However, marketing is an essential component of engineering companies. If you build a better mouse trap, the world will not beat a path to your door unless they know about your mouse trap, understand why it is better, and why it would be useful to them. Without a marketing strategy (which is more than just advertising), that better mouse trap will languish in obscurity.

Motivation for this new section came after reading through Mark Joyner's book on Integration Marketing. This book has details an interesting strategy for low cost (and low risk), high reward marketing. It struck me that engineers could benefit from thinking about marketing, especially entrepreneurial types who will invariably end up trying to sell their own products/services. A little thought about who the target market is, and how they are likely to use the product/service at initial design time would go a long way toward making something that people actually want. Too many good ideas are marred by poor user interface design and/or designing for a non-existent market. Thus, this new section provides reviews and links for material by marketing experts that I think are of value.

Incidentally, Mark Joyner's book can currently be obtained for free.