1. Promise first, don’t deliver.
   VirtuallyDedicated promised me a few times a control panel for my VPS. I didn’t really need a control panel. I just wanted it to be able to tell if my VPS was down by my own fault, or if the node was down, and to reboot my VPS if needed. Yet, I was promised again and again, and no control panel was delivered.
2. Don’t communicate with your customers. Don’t inform them of downtimes or system failures.
   Throughout the time I was with VirtuallyDedicated, my VPS experienced a few occasions of extended downtime. Extended being in the range of hours. The webhost didn’t bother to inform me nor offer any explanation. Unless I e-mailed them to ask about it. Over time, I just gave up and let the downtimes be, since they were rare.
3. Let your servers go down for really long, and not only not inform customers, but don’t reply to their e-mails.
   This last one took the cake. My VPS went down for over 24 hours. No word from the webhost. I submitted a support ticket, but did not receive a reply. I directly e-mailed the webhost, and over a day later, received a reply that the downtime was due to a hardware issue and that I didn’t receive a reply because “for some reason tickets were not reaching the support system”. No further explanation, not even why a hardware issue resulted in downtime of over a day.
   

After that, I simply had enough of VirtuallyDedicated. I decided that for my next webhost, I would go for an established company, one that actually had a real team of technical and support staff. Although there were some negative reviews, I am now on BurstNET. So far, so good. There was an incident where my VPS was performing very slow due to (I suspect) high disk I/O by some other user on the node. But since then, everything is working well and I’m happy with BurstNET.

One of the neat features of HTC Sense is the integration with Facebook, YouTube and Flickr. From the HTC Albums, you can easily upload photos to Facebook. If it works. Unfortunately for me, I couldn’t get that feature to work. Instead, I was prompted to login. After typing in my e-mail and password, I got the following “Login error”: “There has been a connection error while attempting to login”. I never figured this problem out until recently. Apparently this is an obscure problem. I came across a forum posting that mentioned trying to change the default data network, so I tried that. To my surprise, I hit the jackpot with “SingTel Streaming”. Once I selected that and tried to login, it worked! I changed the setting back to “SingTel Internet” (the default network), and was able to upload photos from HTC Albums!

I don’t really know the difference between the different Singtel data network settings, and it is quite strange that even though “SingTel Internet” (aka “IDEAS Internet”) is the default connection, HTC Sense is not able to login. This issue was also encountered on a HTC HD mini.

By the way, to manage your accounts in HTC Sense, go to “Settings” > “All Settings” > “Connections” > “Account Manager”, and you will see the login status of Facebook, YouTube and Twitter. Not particularly intuitive.

A side effect of changing the data network was that all of a sudden, my Opera browser could not surf the Internet. Internet Explorer and other applications were fine. Just Opera. Loading any page would immediately display the network error page.

Thanks to a thread from Mobinauts, I found that the solution is to change the following registry key from ’0′ to ’1′:

HKEY_LOCAL_MACHINE\Software\Opera\Prefs\Network
Conn Mgr Auto Detect Network = 1

Apparently Opera is configured to connect only through “SingTel WAP”. Unfortunately, something I did caused the connection to fail. By changing the registry, Opera will use an available data network setting, in this case, “SingTel Internet”.

Perhaps these are more accurately telco (Singtel) quirks, rather than HTC Sense?

Unfortunately I was facing a very strange situation. When I tethered my phone, the laptop did have Internet. But I couldn’t load anything other than the Google home page. I could ping, and I could do nslookup. I could wget the Google homepage consistently. But I couldn’t search or load any other pages. apt-get refused to work as well. I tried to FTP, and discovered that the connection would choke if I tried to do a directory listing of a large directory.

After much trying and Googling, I wondered if it had anything to do with the MTU. I Googled, and came across a forum post that suggested changing the MTU to 1000. I tried it, and… it worked!

I’m now happily tethering my Windows Mobile 6.1 phone with Ubuntu 9.10, using just SynCE + MTU 1000.

I wanted to upgrade to a plan with more RAM, but the plans on RapidXen were beyond my budget. I looked around and found VirtuallyDedicated. Their 640MB (with 1.25GB swap) was still reasonably affordable for me, and I decided to take that plan. I did not regret my decision. The extra RAM really does help, and my sites load faster, and handle more connections. Even compiling PHP for LiteSpeed took a fraction of the time. Network connectivity is much better, especially in terms of bandwidth. A tip if you’re buying hosting in another country: make sure you do latency and bandwidth tests. Not all datacentres are made equal. 10-20KB/s is really bad.

So far, so good with this VPS. My only issue right now is the lack of a control panel (for Xen VPSes), which means things like reboots have to be requested and done by staff. They keep telling me a control panel is launching soon, but I have not received any word.

To prevent this, I decided to automatically, and regularly backup my VPS and store them on my computer. I searched Google a bit, and found two typical approaches: a simple shell script, or a full-fledged backup solution. I decided that I didn’t need something like Amanda or Bacula. I didn’t feel too inclined to messing with scripts though. However, I found this nifty tutorial to backup web server files and MySQL databases. The tutorial came complete with a script generator, which was really handy.

I made some changes to the generated script though:

1. I prefer the yyyy-mm-dd date format for easier (and natural) file name sorting. I also changed the timestamp of the MySQL dumps to match the file dump timestamp format. 
2. Because I’m not running large websites, I decided to change the script so that the full backup  is only run once a month instead of once a week, to save on resources.

With this script running, I should not have to encounter another situation where I lose months of data. Well, unless the backups stop running without me realising.

As I was adding sites and working to bring them online, I realised that my sites were loading very slowly. I installed WP Tuner, a WordPress plugin, to try and see if the bottleneck was on LiteSpeed or MySQL. From the statistics, the delay occurred at the very first “Start” marker, taking 30 seconds, or even as long as 60 seconds. This wasn’t a problem with MySQL then. Next, I used top to monitor CPU and memory usage. It turns out that each PHP process took up 20MB of memory. With a VPS of only 128MB memory, and each site load using multiple processes, this was a huge problem. I was quickly running out of memory, and there was a lot of swapping. A quick search on Google, and I found this page of documentation, which states:

There are two ways to let PHP handle multiple requests concurrently, Server Managed Mode and Self Managed Mode. In Server Managed Mode, LiteSpeed web server dynamically spawn/stop PHP processes, in this mode “Instances” should match “Max Connections” configuration for PHP external application. To start PHP in Self Managed Mode, “Instances” should be set to “1″, while “LSAPI_CHILDREN” environment variable should be set to match the value of “Max Connections” and >1. Web Server will start one PHP process, this process will start/stop children PHP processes dynamically based on on demand. If “LSAPI_CHILDREN” <=1, PHP will be started in server managed mode.
Self Managed Mode is preferred because all PHP processes can share one shared memory block for the opcode cache.

I configured LiteSpeed to use Self Managed Mode as described, and restarted the server. I then loaded my sites in my browser, and just like that, my websites loaded in under a second. Just to make sure it wasn’t due to caching, I browsed several pages of the sites, and everything worked perfectly. It certainly pays to spend a bit of time analysing performance, reading documentation and tuning the server.

On the positive side, I am now a proud owner of a VPS. I decided to go with a VPS because I loved the flexibility and customisability of a VPS, and not be tied down to the quirks and limitations of shared hosting. A dedicated server is way over budget, so I decided to go with a small, cheap VPS instead. After much reading up, I decided to sign up with RapidXen, and purchased a 128MB VPS. Since I had a coupon for 20% off, I decided to get a slightly higher spec’ed VPS, 128MB RAM with 384MB swap and 10GB disk space.

Over the past few days, I have had fun setting up my VPS, and have just put this website back online (though restored from the May backup). Setting up a VPS requires Linux knowledge though, so this is definitely not for a Linux-newbie. There are a quite a few guides, and I found these particularly useful:

• Yes, You Can Run 18 Static Sites on a 64MB Link-1 VPS, an article on LowEndBox, which shows some steps for optimising memory usage on a VPS.
• Reducing MySQL Memory Usage for Low End Boxes, also from LowEndBox, on how to reduce MySQL memory usage.
• Debian VPS configuration and setup, from usefuljaja.com, with a quick-start on setting up and securing a clean VPS.
• Litespeed tutorials, also from usefuljaja.com, which has many detailed, step-by-step instructions on setting up Litespeed web server.

The VPS seems to be running well at the moment, and I hope this will be a long-term host

A simple script would be able to format the recognition results. Because I am not familiar with shell scripting, I wrote one in PHP instead.

The gist of the script is that it scans the working directory for all JPEGs, runs zebraimg and writes the result to a file. The result is saved in a CSV format with the filename and zero or more recognised barcodes.

First, we define the location of the zebraimg executable, and the allowed file extensions. Note that zebraimg is not limited to only JPEGs. However, since my barcodes are all JPEGs, I’m restricting my scripts to JPEGs.

Next, we get the file listing in the current directory, and open result.txt for writing.

We then do some basic validity checks:

We are now ready to execute zebraimg on the current file:

I used the -q option so that zebraimg only returns the barcodes found, if any (and no other messages).

Next, we do some simple text processing to get the CSV format that we want, and write out the result.

And of course, close the foreach loop and the file pointer:

Sample output from running the script:

And the corresponding entries in the result file:

There is, of course, a slight performance overhead incurred, but this is nothing compared to the effort of cross-checking the recognised barcodes manually.

What performance overhead are we talking about here? Running the zebraimg *.JPG took around 10 seconds for 74 images. Running the script took around 13 seconds. As you can see, in absolute terms, this overhead is minimal, thanks to the speed at which zebraimg processes the images.

This was from my visit to the Singapore Zoo in December. Do check out the rest of the album here, or my dA for selected photos.

Being a Windows person, I searched for and tried Windows programs first. The only candidate I found was bcTester. It recognised the first photo, then choked on the second. I tried cropping that, and it recognised it, but then choked on the third. To make it easier, I adjusted my photos to increase the contrast (easily done in batch), but cropping each photo individually would defeat the whole purpose of my “exercise”, and bcTester did not support batch processing either, so I decided to pass.

The next program I tried was Zebra, a *nix open-source project. I spent some time getting the pre-requisites installed, including messing around with some symbolic links, before I managed to successfully compile and run zebraimg. Once I did, zebraimg was just amazing. It zipped through my barcode images and got most correct.

Out of 75 images, zebraimg failed to process four (three of which were too blur, and one was because the barcode was scraped out partially), got two wrong and was unsure of two (generated more than one barcode).

Recognising the barcodes was only the first step. Books without 13-digit ISBNs have barcodes that are “useless”, because they do not identify the book uniquely. In fact, I had three books in this batch with the same barcode. Hence, I had to manually key in the 10-digit ISBN for those books that did not have barcodes beginning with “978“. Once that was done, I finally had a list of (10- and 13-digit) ISBNs.

There are two routes for managing a home library: online or offline. For online, I settled on two: LibraryThing and Gurulib. GuruLib is free, and allows mass importing of books by uploading a file. Unfortunately, the interface could use some tweaks. While importing my list, I noticed there were duplicates, and some books were not imported. However, the website does not show which books failed to be imported. Since I only recorded the ISBNs of my books, this irked me as I was not able to cross-check the titles. I decided to pass on GuruLib.

LibraryThing is free for the first 200 books, after that, it requires payment of USD10/year or a one-time payment of USD25. I tried to avoid LibraryThing at first, becuase I know I have more than 200 books. Since I didn’t have much of a choice left, I gave LibraryThing a spin, and was impressed. The interface is clean and plain, but it is functional and fast. Importing my books was so easily done, and the import screen showed that two books failed to be imported because they were not found in the default databases (Amazon US, UK and the Library of Congress). I tried Amazon CA and found my books. Amazing! I just might subscribe to LibraryThing.

For the offline option, I only tried Collectorz.com Book Collector. It did the job, but at USD29.95, I wasn’t too keen. While importing my books, it failed on a few. I checked the ISBNs against my images, and found that those barcodes had been processed wrongly by zebraimg. I manually typed in the barcode, searched again, and found the correct books.

This has been an interesting experiment and exercise. And now that I’ve finally figured and tested things out, I’d definitely save time on the subsequent batches of books.

In the next post, I will show a simple PHP script I wrote to easily batch process and manage the decoded barcodes.