What went wrong? Honestly, I am not sure entirely. Based on the symptoms, there was a problem with some administrative server software that caused the entire JDK to become unstable. Based on my hosting service, this was a known problem but it did not have a solution as the problematic service has become unsupported. Obviously, this is not the best answer that I wanted to hear, but I did have options. I could move YackTrack to another hosting service, move YackTrack to a new server or just rebuild the server. At this point, I have taken the simplest option of rebuilding the server. Eventually, I will be looking into my other options in the long term as this type of instability could become a problem in the next year.

Thank you for your patience and I apologize for any inconvenience the system outage has caused.

So, I am happy to announce that a premium API for YackTrack is now available! Thanks to the good people at 3Scale.net, I did have some help getting the premium part created, specifically handling of the payment processing. Otherwise, you can review the API at the YackTrack Developer site. The API has only two methods at this point, searching for mentions of a keyword and searching for mentions or comments on a URL or blog post. I am planning on extending the API and social service coverage as soon as possible, without changing the current fee structure. There will continue to be a free version of the API which allows a limited number of API calls, but functions in the same manner as the premium plans.

This does mean that the current version of the API has been deprecated, and will be removed around October 1, 2011. That gives you about 4 weeks to transition from the first version of the API to the new version.

You may be wondering about the user facing part of YackTrack, as even I have complained that it needs to be updated. While the API will be the short term focus, I do plan on making some improvements to the user interface as there are some things that should not require huge amounts of work. I still plan on adding premium user accounts, but that is currently lower in priority.

So, if you are a social service that requires this type of data, like a social media monitoring service, or you used BackType previously to get some social data, take a look at the YackTrack API!.

First, there is a minor update to the main navigation, which you can see in the image to the left. I removed the expandable menus for registered users as it was obviously confusing people. Now, you just get a link to the “Saved Queries” and a list of blogs that you are watching. Very few people were looking at older posts of the blogs, and the navigation was awkward at best, so I removed it.

I have also made the menu a little cleaner. You can see the simple rounded edges in most browsers, and simple separators help to keep things obvious.

Another new addition, that is more interesting than a cleaner navigation menu, is the inclusion of blog overview pages. If you look at the image below, you will see what the new page looks like.

Click to see a full size image

As you can see, this is much different than previous blog pages on YackTrack. There is now a Yackability score for the entire blog, though these numbers will be changing over time as I see how various blogs compare. There is also a new social activity chart in the top right section of the page. This is social activity for the entire blog broken down by service. As you can see in the example above, Twitter is a large portion of the activity, followed by WordPress comments, and some other services. Beneath this information is a table that shows the 25 most recent posts for the blog. This is a reasonable number for most blogs to see how each post is faring in social media, but for bigger blogs like TechCrunch or ReadWriteWeb, this will only be posts from the past day or two. Also, some blogs are only recently added, so there will be a lag in getting blog social activity and yackability scores.

Keyword queries and blog post details now have the same basic overview information and social activity chart. The difference for queries is that you see the listing of comments or shares from the various social services. The display of those comments has not changed yet, but it will be changing in the near future. I find looking through the services and comments more difficult than it should be, so I would assume most of the users do as well.

What’s Next?

In addition to the query results page redesign, there are a few things that I want to improve. First, there are still some inconsistencies and instabilities in how data is gathered from various services. Twitter support needs to be constantly improved as more traffic comes through that service. I am also looking to improve the blog features, as that was the original purpose of YackTrack, finding the comments on your blog post on various social services. So, if you have any requests for features you would like to see, please let me know by sending an email to support@yacktrack.com or posting on GetSatisfaction.

As always, I am looking to include additional services in YackTrack. As an example, Facebook, LinkedIn and GoogleBuzz have improved their APIs to the point that we may be able to include them without significant amounts of pain. I do not want to start including every service known to man, as that is a maintenance nightmare for very little benefit. That means that niche services like Sphinn or dying services like MySpace will not get included. Basically, I want to provide as much bang for the buck as possible and include more interesting features. If there is a service that you think I should be including, send an email to support@yacktrack.com or post on GetSatisfaction and I will consider it.

The hosting company has been having issues with a whole segment of customers, since about 2:30 AM Eastern time. They are supposedly working on the problem and I will try to update this post and Twitter whenever I hear anything else.

As always, I apologize for any inconvenience this may have caused and thank you for using YackTrack.

UPDATE: YackTrack is alive once again. It returned to normal around 11AM. I will be reviewing everything over the next few days to determine if there were any issues with the downtime.

As always, if you see anything strange or find any new defects, please let us know.

Thank you for your patience.

One part of the maintenance was running ridiculously slow. At first, I thought it was just taking a bit of time to complete the maintenance, but I noticed that the site was being affected. As I assessed the situation, performance on the site was getting worse. About the same time as I decided to take YackTrack down during the maintenance, the site became unresponsive (how convenient). Once the site was down, I was able to figure out the reason for the slow running maintenance.

What was the real problem? I was trying to cut corners, as most people do for simple things like routine database maintenance. I was trying to implement some performance enhancements, and I figured that some other maintenance could wait until next weekend. If I had thought things through, and did not try to quickly implement my changes, I would have seen that the performance improvements really required some other maintenance. Specifically, I would have seen that the maintenance I postponed until next weekend was really required for my performance improvements. So, I corrected the situation by completing the basic maintenance that I hoped to postpone and then implemented the performance improvements, which ran much quicker this time.

The moral of the story is, there is no such thing as a quick fix and do not try to cut corners when you know better. The database maintenance that I was trying to postpone only required 15 minutes to complete. The performance improvements then required another 30 minutes. However, I wasted almost 5 hours trying to cut corners, and I had to take the site down for a few hours as well.

Again, I apologize for the downtime, the lack of communication prior to the maintenance and for any inconvenience this may have caused. In the future, I will be writing a short blog post to notify everyone of any maintenance, and trying to ensure I do not cut any corners.

So, a better foundation has been laid down. There is still work to do, but there is a much better plan in place. Some things which looked like they would be almost impossible to add, are now possible.

However, when decisions are made, sometimes they do not go in favor of the user. The big change today is that only 13 months of history will be kept for free accounts. This change in data will occur over several days or weeks, depending on how much it affects the live service. Up to this point, I was trying to keep as much history as possible. This is just not cost effective for a free service. You may be asking why I said “free accounts”  or whether there are non-free accounts. Currently, there are no premium accounts, but it they have always been in the plan. There will be premium services in the near future, and longer history could be an option but I am not promising anything yet.

As always, if you have any questions, feedback or feature requests, please let me know.