I am honestly sorry it had to come to this.  I am not a big fan of Captcha either, but with almost 50,000 links shortened by this site, and less than 2500 of them not spam, well, I really had no choice.  Related to this effort was a human powered purge of spam links.  It wasn’t perfect I’m sure, but it did kill off a lot of really obvious garbage.  They are still in the database for good measure, but they are now flagged as inactive links.  Inactive links can no longer be followed, peeked at, or listed on anything more elaborate than a count.

Also, while I was in there I wrote up a little Credits Page to give credit where it is due.

So, if you use the site, please use it properly.  Seeing 2000 URLs make 7 page views, and $0.00 in a day will not keep this system running.  Thank you.

The side effect of course is that even though the site has been up and working for months, for the last week or two it was useless at showing you the details about a link, or in fact at directing you to where that link goes.  You know, the whole purpose of the site really.  The reason it turns out was that an extremely crafty SQL query (if I say so myself) ended up being a bit too darn crafty to finish in a reasonable amount of time once there was a lot of data in the database.  In this case, “reasonable” means something a bit like the query would “ever finish.”  While I suspect the query would in fact finish someday, it wasn’t showing any obvious signs of doing anything besides making the CPU in my server nice and warm.

That being the case, I was left to either shut down the site for killing my server, or come up with a better way of doing things.  In this case, multiple very high speed queries.  These queries were designed to be parts of a whole picture that very rarely must all be executed.  So some crafty logic to decide which parts need to happen and executing as few as one of the queries (which is actually the most common case) and the peek and link following code are now back to basically instant.  I’m a big fan of when an attempt to speed things up actually speeds things up.

At the same time as I request feedback, I also welcome everyone to tell a friend about this site.  If you like it and want others to use it too, the only way for that to work is to tell everyone you know about it.  Sure, it’s a shameless request for word-of-mouth advertising, but I would love to see this site become popular, can you blame me?

Thanks to everyone in advance for both the excellent feedback I’m sure I’ll receive and for telling others about this site.

The links I like to refer to as “legacy” links, or more accurately, the ones that were produced on a previous edition of the database that sits behind the website are not super likely to be brought forward at this time.  Also, if they do get migrated forward, they absolutely will not have any statistic data attached to them.  That part just isn’t even possible since there is no information to base it on in the legacy database.  Enjoy the new version of the site, hopefully this time for more than two days.

There I was in my home office working on something on my “main machine,”  that’s the one that still runs Windows, and I heard a beep from the web server that I wasn’t expecting.  The beep, it turned out, was the server booting up.  This is an obviously weird situation to be in for a server that generally only goes down for very long power failures (three or so hours is about all I can usually expect out of my batteries.)  In fact it had been happily serving web sites for over three months without a hitch when this happened.  My first thought was that I had somehow been overloaded by Google AdWords, but a quick check on that showed they had sent me a whopping 30 or so hits in two days.  That amount of traffic wouldn’t take down a 20 year old computer acting as a web server much less the hardware I use.  My next thought was a hack attempt, which of course there are all the time on pretty much all publicly exposed servers.  To limit the effect they had on my web server in the future I decided to install software that would auto-detect a certain kind of hack attempt and block people accordingly.

During my efforts to figure out what was happening, the machine rebooted again.  There was no one logged in but me, there was no obvious reason for this, this was when I started to piece together that I was enjoying a genuine hardware failure of some sort.  I downloaded the latest version of MemTest86+ and rebooted the server myself to run it.  It found nothing wrong with the memory and in fact the server shut itself off without warning within about 45 minutes of running the test, again.  The amount of time between reboots was actually shrinking and I was no closer to figuring out why.  With the RAM seeming to be Ok and the power coming and going my next theory was the power supply.  I acquired a replacement (1000W), installed it (never a fun process), and fired up the server once more.

It stayed on for around five minutes, then seven, then two, then it actually failed to complete the POST (Power On Self-Test) several times in a row.  I figured that was about as bad as it was going to get until yet another reboot showed the BOIS Logo screen (re-enabled by resetting to factory defaults during one of my attempts to get things working) came up and was completely scrambled.  In fact it looked a bit like when an 8-bit Nintendo game would freak out and you knew you needed to try to save your progress in Zelda quickly so you could remove the game and blow on it and put it back in to continue.  Several more attempts all left the BIOS logo in some evil state of disrepair and very soon after it wouldn’t get past the logo, enter setup, show the POST screen or anything else someone trying to fix things would try.

Now I was down to the last few possible theories.   Having replaced the power supply and continued to have the exact same problem, and then of course to watch that problem get progressively worse, all that was left was the motherboard and the CPU.  For reasons I can’t explain beyond a simple hunch, I was pretty sure it wasn’t the CPU.  I guess because all the beating around I’ve ever done to upwards of 20 CPUs in all the computers I’ve ever built, I’ve never had an Intel chip crap out.  That left the motherboard.  Some searching on NewEgg for a motherboard that had the same CPU socket, RAM type, RAM slots, and enough SATA ports to run the guts of the server and it was on the way.

Once it arrived I began disassembling the whole server to replace the motherboard.  Never a small task since basically everything in the box is connected to that board in some way or other.  Once everything was back together I fired it up and sure enough the problem was solved.  Almost.  Most operating systems have a heck of a time recovering in a friendly fashion when as many things as are on a motherboard are suddenly changed.  FreeBSD is no exception to that pattern so I found myself reinstalling the OS and restoring the backup.  Once everything was back to a working state again, I realized what had happened.

In a weak moment of extremely poor software engineering practices, I had modified the production code of the site directly to fix bugs as they came up.  This may not seem like a very bad thing, but it actually is way worse than you can imagine when your server chooses to die.  In the two days since my prior backup I had made enough changes to the code that it would no longer work properly without it.  Since the backup was two days ago none of those changes were in it, they were really just lost.  This left the site in a state that requires those two days of work all over again.  In what has now turned into several months, I still haven’t managed to get it running again.  Largely out of a total lack of time to put into it.  For that I apologize and I swear it will be back on line eventually.

The most important piece of lost information is how many times a link has been clicked on.  In the old version this was tracked as a simple count.  What this means is that when someone clicked a link, I simply changed the number in the database that kept track of how many times it has been clicked.  The new version of site now keeps track of much more information about a click to follow a link.  Specifically it tracks when it was clicked, where in the world the click came from (roughly) and a few other things.  Things that the old system simply doesn’t have to move forward.

Rest assured that your links will not go away, generally speaking.  I apologize but there are a couple links that are actually duplicates in the new model, and invalid links will not be recreated in the new database.  Links created in the beta site will be thrown away entirely since they have no real relevance to the production data.   The existing production links will have their hit counts reset to zero.

These minor losses are going to pave the way to great features.  The beta, with any luck, will make everyone that liked the site before, like it all that much more in the future.

This brought me to the idea of crafting unit tests to assist my efforts.  If done properly they could be executed in a reasonable amount of time, could test hundreds or thousands of things in less time than a person could expect to test one or two, would be easily repeatable as many times as needed, and could test both the site and the underlying code.  So for the effort of choosing a unit test framework that fits my needs and spending a handful of hours coding the test cases I could nearly guarantee a solidly functional web site.  I grant you that there are still likely to be things that are missed while creating unit tests, but at any time I can add more to cover the holes, and then they are there in the list of tests forever to be reverified at will in the future.

This is no small effort to be sure, and it will certainly add further delays to the v0.02 release, but it’s already uncovered some warnings in the code and will likely find more.  I feel it is worth it to catch the unit tests up to the current code base, so that is where the focus is right now.  I’ve mostly completed the tests that follow all the links on the pages, and the test results for just that happily spit out:

1/1 test cases complete: 155 passes, 0 fails and 0 exceptions.

That’s excellent indeed.  155 tests, all passes, and the ball is rolling.  Let the games begin.

The current plan has 16 categories and 280 check points to verify.  Think of it like that list at the oil change place, except this one is actually going to get done instead of just getting a bunch of pen marks next to each task that somebody pretended was done.  As I plow through the list I’m also jotting down loads of notes on my work sheets.  These notes indicate things that I want to check or adjust in conjunction with a check point being validated, reasons why I didn’t pass a check point, general ideas about how to improve things later, and even additional test points that became obvious only once I started testing.

The execution of this test plan does not of course mean that I feel testing to be entirely complete, so by all means, keep on suggesting things.  Keep on replying to posts about this site on Reddit, keep on emailing ideas to me, keep on giving me the ideas that you want to see in a URL shortening web site.