CompareXY: A Site Experiment

Nifty New Incremental Usability Improvements

2010-06-03T23:38:00.000-07:00

I made several modifications to the comparexy.com site today. Mainly usability tweaks. A little JavaScripting with JQuery, a little CSSing, and a little Grails/Groovying.

(Note: This blog is a continuation of the series that began with http://www.comparexy.com/compare/blog/2010-04-19/The+CompareXY+Site+Experiment)

(Screen shot with some arrows of affected features)

Added a "randomList" action to the Grails site:

Previously, I was making round trips for every random suggestions that faded into the header. Now, I'm getting a batch of them from the server at one time.

Basically, now I take down a list of a hundred random articles or so at a time. Then the JavaScript and Jquery functions that randomly fade in a suggestion pull from the locally cached list until the list is exhausted (then it goes to the server to refresh if need be).

randomList(maxSize, maxState)

* The maxSize argument limits the maximum size of the returned list.

* There is not guarantee on how many records this will return.

* The maxState argument allows for the method to distinguish between states of the articles (official, flagged, suggested, et cetera)

* Generates a list of random longs in between 1 and the maximum record. It will pass the list of random longs as a query param to Gorm's executeQuery() method.

* The controller calls this and turns it into JSON (not shown).

* I'm safe for probably ever, but I do have a TODO item to fix this method to not use integers, since I used the long data type as the tables primary key.

(excuse the formatting)

List randomList(maxSize, maxState){
        int multiplyer = 5 //account for sparse data
   int maxId = getMaxId()
   List ids = new ArrayList(maxSize * multiplyer)
   for(int i = 0; i < maxSize * multiplyer; i++){
             ids << new Random().nextInt(maxId).longValue() + 1 //add 1 to be inclusive
   }
       String hql = """
     SELECT comparison.thing1.name
   comparison.thing2.name,
   comparison.state
     FROM Comparison comparison
     WHERE comparison.id IN (:ids)
     AND comparison.state <= :maxState
     GROUP BY comparison.thing1,
   comparison.thing2
   """
   return Comparison.executeQuery(hql, [ids: ids, maxState: maxState], [max: maxSize])
}

int getMaxId(){
      String maxHql = """
          SELECT max(comparison.id)
   FROM Comparison comparison
    """
return Comparison.executeQuery(maxHql)[0] //returns a single unit list }

--end snippet-

Modified the page branding in the header, "CompareXY: A Comparison Library", so that, it will randomly morph into a clickable function call (click and it links you to the appropriate article). Maybe a little redundant with the other random suggestion box right beneath it. I'll see if it sticks to the wall I guess.

Essentially the normal state of the branding header:
CompareXY: A Comparison Library

may morph into a clickable "function" link like:
CompareXY("Hella", "Giga")

and then back, cute :)
Changed the random suggestion box that fades into the header to be weighted against less unwritten articles. The problem was, it always directing readers to unwritten articles for them to write instead of to articles that were already written. That is fine sometimes, but not most of the time. Hopefully, one day, the balance of unwritten to written articles will change.

I did that by calling the above mentioned randomList controller, putting the data in to a local cache and filtering out unwritten articles if there are too many.

On a similar note, changed the Grails action called "random" to filter out 2/3rds of unwritten documents. Same problem as above, different way of arriving there :). This is actually just the static text link a the top of the header that calls a "random" action on the Grails server (so, no local data pulled here, although, now that I say it, I guess I could pull from the local data, and reduce one unnecessary redirection).
Added a "Close" button to the ads on the right hand side (see the image):

This should allow user that are annoyed by an ad to just close it.
This should allow user that are on unusual windows sizes, or browsers to remove the ads if they don't float correctly. I know, I should really improve my cross browser CSS skills. I've got it working on IE, Firefox, Safari, Chrome, and Opera, but some older versions of IE, and my Palm Pre don't float the ads window. Against the prevailing trends in the industry, I may just put the whole thing in a table to guarantee placement (or talk to a CSS guru). Alignment was so much simpler with table layouts :)

Put a little nifty fish-eye effect on the browser search bar links when they appear in the ads section.
Wrapped the ads in a border.
Removed ad from showing up on the edit page.
Upped the restriction on article size since I began running into it, and it is essentially arbitrary.

If you have any input/suggestions let me know.

Bit.ly Links on Facebook

2010-06-01T11:04:00.001-07:00

I noticed today by analyzing my server logs for comparexy.com that if I place a bit.ly url into a Facebook comment, when people click on the link in Facebook, it does not go to bit.ly. Facebook has already captured the destination URL, and is bypassing bit.ly. This is a bit of bummer because I was curious about real-time analytics on the link. Oh well.

Content is Important: Hella Vs Giga

2010-05-31T23:47:00.000-07:00

The most important part of the CompareXY.com website, even more important than which JavaScript library I chose, or the operating system that it is hosted upon (~sarcasm~), is having actual content. Not only having actual content, but having content that is compelling to read.

With the Rule of Thumb concept, we are hoping to lower the bar for authorship. Not lower the bar in the sense of quality, but lower the bar in the sense of effort. This way we hope to recruit more volunteer authors. We don't want to scare people off from editing. The flip side of that coin is that we also don't want to remove incentive for non-trivial original research. The library won't scale (except for some of the dynamic content that can be generated) without volunteers. Even volunteers that write simple, short, Rules of Thumb, about arcane topics.

Regardless, the beauty of writing is it is fun. Today, I spent a few hours watching the Minnesota Twins beat the Seattle Mariners, and finding illustrative examples for explaining the difference between the SI giga prefix (10^9), and the proposed SI hella prefix (10^27) (see article http://www.comparexy.com/compare/Hella+vs+Giga). When the game was done, I had several good examples. I had skimmed through some books, including one that I had read 20 years ago and always stayed on my mind called Innumeracy by John Allen Paulos (see image on left); Great book. I searched through some science websites. I even pulled out a calculator, and counted a lot of zeros. When the baseball game was finished, I had the article, and references plotted out. I shared a draft copy with my friends so they could have some input on the ordering of the examples for the maximum illustrative impact :) I had a lot of fun writing it. Way more fun than washing a load of clothes.

The point about authors is, it's has to be something someone takes some interest in. Luckily for me, I take interest in a lot of different things (more than some, less than others I suppose), and that is probably why I really see a value in a library like this.

Woo Hoo, Top 200k in the U.S.

2010-05-28T11:52:00.000-07:00

Broke the top 200,000 highest traffic sites in the United States. Also, now in the top 2-million in the world. I figure I will need to get about ten times as much traffic to get the world ranking inside of the top 100,000.

US Ranking

World Ranking

Another Milestone, but Not the Critical Ones

2010-05-27T09:17:00.000-07:00

Another milestone was broke this morning on comparexy.com when we broke the top 200,000 barrier for the highest web site traffic in the U.S.

Other milestones I'd be more interested in seeing:

An Article Written:

I'll be interested in seeing how long it takes for an author whom I do not know to actually write, or edit an article on the site. Perhaps having people sign-up via Facebook connect is too much of a burden.
More Than Pennies Generated:

I'm curious as to what it will take to get a few people to click on some ad support. I'm not expecting much, but I would like to make more than $4.58 a month. If I could get it to $20 a month, I would be happier.
Someone Linking To The SIte as a Resource:

Not a spam link. Not a blog aggregate link. But, a link from somewhere that referred to the site either from curiosity or as a resource.
Top 100,000 Sites in the World

I think I need to increase my traffic by about 10 times get to the top 100,000

5 Trillion Messages on my Facebook Connect Comment Board!

2010-05-25T08:36:00.000-07:00

Oh, hey, sorry, I didn't notice you there. I was just reviewing the five trillion (5,004,072,233,577) comments I got on my website (http://www.comparexy.com/compare/blog/2010-05-03/Facebook,+Continuing+The+CompareXY+Site+Experiment) via Facebook's Connect software. Taking a look back, I was appreciative, and mightily impressed when I was on the verge of being the ten-millionth highest traffic site in the world (see blog http://bit.ly/9utRgU), but five trillion comments will really put me in a different league. A league I didn't dare think about. Yeah, yeah, I know, technically, Facebook is the one that is really the winner on my comment board, but still, of the six-billion people that added a comment, on average, nine-hundred or so times, my site will surely be remembered by one of them, hence, increasing revenue over time. Sure but steady, on the way to #1.

Moved CompareXY to Google Search

2010-05-25T00:48:00.000-07:00

I migrated the search for CompareXY.com to Google's search today. Well, technically, I still am using the Lucene search engine to handle wayward URL requests, but the searches that go through the search box access Google.com. I think it is nice, but it isn't real-time as Lucene is, so I'll have to consider making a hybrid, or going back to Lucene. Oh well, the site is an experiment I guess. If I am not happier with Google's search, I will revert, and spend a day sprucing up the Lucene search to my preferences; it only took me an hour to get Lucene up in the first place, so nothing really lost.

Browser Search Bar From Conduit.com

2010-05-23T10:16:00.000-07:00

Since I am seeking to make CompareXY.com the #1 highest traffic site in the world, I was reviewing the list of the Top One Million sites that Alexa.com makes available on a daily basis. Scrolling through the list, while still in the top 100, I realized that most of the sites that I did not recognize where foreign, but then I came across Conduit.com. It seemed like an English word, I may have even heard of them before, but I did not know what they did, so I researched.

It turns out that Conduit.com is some kind of website with tools for creating gadgets, and such for site promotion. They had a tool bar search option for IE, Safari, and Firefox, and I wanted one, so I built it. It took about five minutes since I already had a couple of images. They host the tool bar downloads at http://comparexy.ourtoolbar.com, but I will make them available for download directly from my site.

Anyway, try one out* (if you use one of these browsers ~I am usually on Chrome:):

* If they don't work, go to the above link, and there is information about the system requirements.

Welcome Me to the 5 Millions Club

2010-05-21T10:21:00.000-07:00

After only two days of officially being ranked on Alexa, CompareXY.com has come hurdling to the Earth by slashing it's original ranking by two.

Changent.com (Grails/Groovy Talk) has made a move over the last day as well, but the heat is on!

CompareXY is also now ranked in the US as well at 546,915: See the report

Shattering the 7 Digit Barrier

2010-05-20T09:52:00.000-07:00

The race to number just got a little bit more interesting. After only one day of being recognized at all, CompareXY.com shatters the 7 digit barrier, and creeps in on 6 millionth place at 6345897.

Ranked 10,219,117th Highest Traffic Site in the World!

2010-05-19T15:16:00.000-07:00

Today, May 19th, 2010, I am proud to announce that after only a month of blogging about my experience developing CompareXY.com with Groovy/Grails, et cetera, I have reached the coveted 10,219,117th ranking for the highest volume website in the world! Only a few hundred-thousand sites to beat before I break the prestigious seven digit barrier. If you are having difficulty believing my good fortune, see for yourself at this report: http://www.alexa.com/siteinfo/comparexy.com. This puts CompareXY squarely between google.com, downlowtooslow.com, changent.com's Groovy/Grails Talk (currently ranked 1st, 2,644,086th and 4,684,476 respectively), and pandaholics.com (no longer ranked).

Read the whole article at: http://bit.ly/dnKE7r
or http://www.comparexy.com/compare/blog/2010-05-19/Ranked+10,219,117th+Highest+Traffic+Site+in+the+World!

Putting Apache 2 In Front for YSLOW, Load Balancing, and Sub-domains

2010-05-19T14:58:00.000-07:00

For my experimental website, http://comparexy.com, I was able to get by just dandy using a Tomcat server as the front-end for CompareXY, but I have a couple of other objectives in mind.
Load balancing, not so much because I need to do this, but because configuring the site to be enabled for load balancing will simplify scaling in the future in case I really need it.

Multiple domains / applications running on the server

I may split up some tasks into different application.

I want a static content domain up there *

This will host static content for my site such as CSS, Scripts, Images, and other artifacts.

As per the suggestions of YSLOW (based upon Yahoo's performance guidelines) and Google's Page Speed I will:

a) Use a static content domain to be as a cookie-free domain (results in less network overhead)

b) Set the headers to have a far ahead expiration date (causes caches to last longer, though you I will need to change the name of the content with a version number to force updates to work)

This should help me score higher on the YSLOW grading chart if I move the static content to the sub-domain.

Happily, I was able to get my YSLOW score up to a B. Not good enough yet, but an improvement:

Also, when I kill my Tomcat server between reloading new versions of the WAR, I would prefer that the site serves a maintenance page to anyone that attempts to connect to my site during the brief downtime, rather just not connecting. I can enable this by adding a “maintenance” site to Apache, that performs a catch-all on requests..

* Though, truly, a domain for static content is probably wholly unnecessary for a low volume web application as mine. This is a good example of Misapplication of a Best Practice on my part :), as I'm not serving millions of people, and no one visitor is likely to have a better experience based upon this. Regardless, this site is largely a vehicle for self-instruction.

Go here to see the rest of the details and instructions

The Beginning with Grails

2010-05-19T14:54:00.000-07:00

I took some time off of work on a self funded sabbatical, so I could, among other things play around with some new technologies. I had experience with various Java based web application frameworks that used the stack including Spring, and Hibernate. Since Grails offered the promise of combining these frameworks plus templating engines, and several other features, I was interested in working with it.

I prefer to monkey around with frameworks a bit before I believe the hype. So, I read the book Grails in Action, and I went at building a non-trivial website.

These linked articles here touch upon several of the elements, and challenges that went into building a Grails site from scratch without having previous Groovy or Grails experience:

2010-05-03: Twitter, Continuing The CompareXY Site Experiement

2010-05-03: Facebook, Continuing The CompareXY Site Experiment

2010-05-03: Amazon WebServices, Continuing The CompareXY Site Experiment

2010-05-03: JQuery, Continuing The CompareXY Site Experiment