danielmclaren.com blogs

Visualizing LinkedIn Connections Using Protovis

daniel — Tue, 08 Feb 2011 17:44:07 +0000

A couple weeks ago, LinkedIn released InMaps, a tool for visualizing your professional network. You can see mine right here.

The layouts are fantastic and they show the clusters well but I wanted a bit more interactivity in the UI. I used Protovis because it seems to be one of the best Javascript visualization packages out there.

Firebug helped me grab the JSON files used by LinkedIn and set about feeding the data to Protovis. A little bit of data-munging and we were good. As usual, though, it took a bit of wrestling to get the force-directed layout to behave.

One of the first things I did was remove the node representing myself. This node was connected to every other node so it didn't communicate anything useful and it was causing the entire network to oscillate and disappear off screen.

So far Protovis has been pretty nice to work with, though I ran into some problems with user interaction. Specifically, I want the user to be able to click a node to open the LinkedIn profile. However, when the user drags a node, the click event fires and opens a new window.

The documentation was also a bit brief but between the example code and the source code it wasn't too hard to figure things out. Looking forward to working more with it!

View the InMaps + Protovis demo.

New Twitter Network Graph Viz

daniel — Wed, 14 Oct 2009 21:03:06 +0000

I just finished up a graph visualization for your Twitter network. It's a dynamic, interactive application that tracks the number of times you mentioned other users or hashtags. You can read more about it on the Asterisq website or, if you have a Twitter account, give it a spin.

Launch Mentionmap

How Google Website Optimizer Encourages Visitors to Act

daniel — Fri, 09 Oct 2009 19:04:37 +0000

I learned a lot about Google Website Optimizer in a presentation at Net Tuesday Vancouver earlier this week. I also have a write-up on Adsense from the same event, if you're interested.

VKI Studios' John Hossack was the speaker. His presentation focused on the "post-click" portion of online advertising. It starts from when a user clicks on an advertisement and ends up in your website.

The goal of this process is to get the user to perform some action, like purchasing your product, making a donation, or telling their friends about your product or service. There are many different ways you can encourage a visitor to take action. Different wording, button styles, product images, and layouts all have an effect.

So how can Google help? One way to decide which button style makes more users want to click it is to use your experience or intuition. Another way is to test, test, test. And that's where Website Optimizer comes in.

What Website Optimizer Does

Website Optimizer a free service from Google that allows you to present different versions of your website to users. Each time somebody visits your website they'll see one of your test cases. If you're testing button styling, one test case might be a yellow button and another case a red button.

Web Optimizer records which version of the website was displayed and then records whether or not the user clicked the button. This button click is your "success event" and should lead to one of the desired actions mentioned above.

Once you get enough of these "success events" you can look at the numbers to determine which button was more likely to be clicked. By testing different improvements all over your website, you can make it more effective at getting visitors to perform the desired action.

Notes on Website Optimizer

There's a caveat to all this. Your site needs to have visitors coming in and triggering "success events" fairly regularly for this to be useful. If only ten people click the button in a month then you don't have very much data to measure. John recommends having at least 100 success events per test case per month.

"Test shouts, not whispers."

John also points out that you should be testing big changes. He likes to use the six-foot rule: you should be testing changes that are visible from six feet away from the monitor.

I have to say I learned a lot from both speakers at Net Tuesday. Many thanks to the speakers! Slides for both presentations are now available to Net Tuesday Meetup members.

Import Drupal Comments into Disqus

daniel — Thu, 08 Oct 2009 20:17:12 +0000

Among the recent updates to this powered-by-Drupal blog is a switch to having Disqus manage comments. The standard comment module has a few shortcomings which are solved by Disqus. However, Disqus isn't without its own problems and the absent "import comments from Drupal" functionality is one of them.

Note: Apparently, this post only covered half the battle. I'm still fighting with Disqus' comment import page which is giving me vague error messages

Reasons for the Switch

My main reason for the change is because Disqus allows users to subscribe to followup comments by email. I was using Drupal's subscriptions module to notify users when a new comment was posted. However, this module processed comments before Akismet and I ended up blasting subscribers with spam.

That's not all, though. are some other advantages of using Disqus over Drupal.

The big problem for me, however, is that Disqus doesn't have an easy way to import comments from Drupal. Since this blog has been up for a while, I would lose a lot of comments if I just let them go. A lot of them provide additional help on tutorial blog posts so there is definitely motivation to keep them.

Getting Drupal's Comments into Disqus

Now Disqus isn't missing import functionality altogether. I'm sure that eventually, the Disqus module for Drupal will include it. WordPress already has a plug-in that can do this and there are options to upload XML comment files exported from JS-Kit and IntenseDebate.

I took the easy route: instead of learning the Disqus API, I learned the IntenseDebate comment export format. I signed up for an IntenseDebate account, created some test comments, and hit export. The format is pretty straightforward, and, since it's XML, self-descriptive.

IntenseDebate Export Sample

<output>
  <blogpost>
    <url>http%3A%2F%2Fdanielmclaren.com%2Fnode%2F4</url>
    <title>How to Use Cross-Domain Shared Objects in Flash</title>
    <guid>http://danielmclaren.com/node/4</guid>
    <comments>
      <comment>
        <isAnon>1</isAnon>
        <name>so-and-so</name>
        <email>so-and-so@example.com</email>
        <url>http://example.com</url>
        <ip/>
        <comment>I like your site!</comment>
        <date>2009-10-08 10:25:25</date>
        <gmt>2009-10-08 10:25:25</gmt>
        <score>1</score>
      </comment>
    </comments>
</blogpost>

Using a PHP script, you can generate the XML output for your own comments. Ideally, the comment info would be gathered using the Drupal API but I just plugged right into the database. Now all you have to do is save the XML to a local file, and upload that file in the Disqus import/export page, using IntenseDebate as the format.

Download the script

The IntenseDebate Export Script

A few notes about the script which translates your Drupal comments to the IntenseDebate format:

The script assumes you're using clean URLs.
Registered users aren't handled any differently from anonymous users. I'm the only registered user on my blog so if your website is any different, you might need to make changes.
The script includes a bunch of entity information in the doctype so you can view the XML properly in Firefox. Comment out the DOCTYPE lines if you don't need it.

Just to be safe, I'd recommend backing up your database. Also, have a good look at the output XML before uploading it to Disqus. You might want to do some tweaking before sending off to their import service.

Let me know if you find this useful or if you run into problems. Good luck!

Getting the Most Out of Google Adsense

daniel — Wed, 07 Oct 2009 19:53:49 +0000

I checked out the Net Tuesday meetup last night. If you're not familiar with the group, they're bring together social change organizations and social/web media types. Last month's meetup showcased some exciting projects going on in Vancouver so I was looking forward to my second time at Net Tuesday.

This month, Omar Al-Hajjar from Magnet Search Marketing and John Hossack of VKI Studios presented on Google Adsense and Google Web Optimizer, respectively. I'll post about John's presentation another time but for now, here are some highlights from what Omar talked about.

Getting More Visitors to Your Website

In a web campaign, users go through a process where they discover your website, visit it, and then perform some desired action. This could be buying your product or making a donation. Omar tackled the "pre-click" portion of this process—that's the bit before they visit your site by clicking on a link or an ad.

Google Adsense can display your message on specific search result pages. For example, your infinite, circular sheet music business might want to target keywords like "sheet music" and "piano music."

The presentation wasn't geared so much on getting started with Adsense but more toward creating better ads through optimization. You see, Adsense allows you to publish multiple ads and compare their performance. Omar tells us to run two ads: the champion and the challenger. If more visitors click the challenger ad during the test period, the previous champion is scrapped and you have a new champion.

I tried testimonials in ad copy... and they worked!"

Omar emphasized that you shouldn't just test once, but iteratively on a regular basis. Keep creating new challenger ads to try to upset your current top-performer and over time you will see a big improvement in click-throughs.

We also heard some great tips on writing good ads, including a clever point about using testimonials in ads.

Be direct. Remember, attention is in short supply on the web.
Get rid of marketing speak. That includes slogans and catch phrases.
Make the ad relevant to the viewer. Since you know what they searched for, mention that.
State your unique value proposition. What sets you apart from competition?
Try using keyword misspellings. These ads are often cheaper because of lower demand.

Many thanks, Omar, for a great presentation. Oh, and if anybody's looking for the presentation slides, they should be posted on the Net Tuesday meetup site in the near future.

Website Back Online With a New Theme

daniel — Tue, 06 Oct 2009 21:33:01 +0000

I started the process of upgrading to Drupal 6. Turned out, I had terrible timing and was suddenly swamped with work so the website has been down for several days now. Happy to say it's back up and running.

Downside is, I still don't have time to properly complete the upgrade so there's going to be a lot of quirky behaviour. If you find broken or strangely-behaving pages, you can help me out by giving me a heads up.

Cheers!

Setting Up OpenFrameworks with Code::Blocks

daniel — Sat, 12 Sep 2009 19:02:48 +0000

I just got setup with OpenFrameworks which is sort of the C++ version of the popular Processing engine. OpenFrameworks launched fairly recently so I had a few kinks getting up and running. Hopefully this helps.

I'm using Code::Blocks as my editor, and OpenFrameworks prerelease v0.06 on Windows XP. At the end of the tutorial you'll have compiled and run an OpenFrameworks application that does nothing. Fortunately, it doesn't die spewing errors either. (-;

Downloads

Here's what you'll need get your first application up and running.

Code::Blocks editor with MinGW. It's important that you get the version that's packaged with the MinGW compiler.
OpenFrameworks. Make sure you get the Code::Blocks version.
OpenFrameworks additions for Code::Blocks. This contains a bunch of files you need to copy into Code::Blocks' "/includes" and "/libs" directories.
A test project. It's based on the "emptyExample" sample app, including some DLLs I had to download separately. The ".cbp" project file assumes you installed OpenFrameworks to "C:/openframeworks" but you can change this using a text editor.

Installing OpenFrameworks and Code::Blocks

Okay, so you can get most of the way there by following the OpenFrameworks for Code::Blocks setup guide. I wish I'd seen this before. (-:

Once you've gone through the guide, make sure Code::Blocks knows where to find the MinGW compiler by opening the "Settings" menu and choosing "Compiler and debugger...". Under the "Toolchain executables" tab you'll find the installation directory. Make sure this points to a folder which contains "bin/mingw32-gcc.exe".

Now you can unzip the test project archive. If you installed OpenFrameworks in a directory other than "c:/openframeworks" you'll need to update "first.cbp" with the correct location. Find and replace in a text editor will do the trick.

You're ready to run the project. Open the workspace file and hit "Build" and you should be good to go!

If that doesn't work, you can always visit the OpenFrameworks forum.

Do You Have a Polling Addiction?

daniel — Tue, 08 Sep 2009 21:30:58 +0000

Here's a visualization of some of the digital distractions we have to deal with by David McCandless at Information Is Beautiful.

My name is Daniel and I'm a polling addict. Anyone who works on a computer connected to the internet knows what I'm talking about. It's those compulsive visits to the likes of Craigslist, eBay, and even services like Google Analytics to see if there's been any change. We're waiting for new job postings or new bids or a slew of new visits from a referring website. Each of these tools definitely has its use but we polling addicts check on them with pointless frequency.

Identifying the Pattern

Targets of polling addiction have some pretty specific characteristics.

Potential for change. Even a miniscule possibility that the price will change on your watched product or some one will comment on your Flickr photo could be enough.
No notifications. If you're already receiving notifications for the website or service, you probably won't feel compelled to check it manually by visiting the website. If the notifications are too infrequent (e.g., weekly) this might not apply.
Easy access. The barrier to entry has to be pretty low for the behaviour to become compulsive. If you need to login with a massive password everytime you check for an update you'll be less likely to poll it.
Relatively infrequent change. If the status of your polling target changes too often, the need to check up on it falls off. However, in some cases, you just refine the criteria that you're checking for in the service so that it occurs less frequently.

There are basically two situations when this behavior kicks in. When you're procrastinating from some big task and when you're bored. Procrastinating isn't so bad because visiting a website to see if anything's changed doesn't take long. However, you might poll the same websites several times while milking other distractions before getting down to business.

The worst trigger for polling addiction is short-term boredom. Whenever you have to wait a few seconds for a code compile or a slow website you might "just quickly check" whether your Flickr stats have changed.

I know that when I see a progress bar for more than a second I impulsively switch tabs or windows to continue with another task. Sometimes, though, it means I'll go check couch prices on Craigslist and spend much more time there than it took for the progress bar to finish.

So why do we do this? It's your basic reward system. If there is a change, that is, if a new couch has appeared or a new comment or a new bid, then we get a reward. Part of that reward comes from the novelty of the new item but it can also be tied to getting attention from others or even financial/success-related.

Countering Polling Addiction

The problem with polling addiction is that, in the long run, a lot of time gets wasted and it adds to distractions from work. Also, research shows that doing too much multi-tasking makes us worse at task-switching.

So here are a few ideas for controlling the compulsive website-visiting:

Make the websites harder to access. By increasing the barrier to access, you reduce the likelihood that you'll try to visit the website. You can do this by deleting bookmarks and forcing yourself to type in the URL. You could also use a Greasemonkey script to prevent you from accessing the website during specific time periods.
Get notifications. If there's a way to get automatic email notifications from the service, you can rely on those instead of visiting the site manually. Just make sure the frequency of these notifications is limited so you don't end up with another distraction.
Exercise some self-control. Easier said than done, but realizing when you're about to check Craigslist for the third time in a day and building the habit of going back to what you were doing before is really the best way to deal with the distraction.

Vancouver Set to Publish Open Datasets

daniel — Thu, 03 Sep 2009 05:34:40 +0000

I attended the Net Tuesday Meetup last night and got to see a variety of people talk about the cool projects they're undertaking in Vancouver. Turns out there's a lot going on, but there was one presentation in particular that struck a chord with me.

Tim Wilson, Advisor to the Mayor's Office, spoke about an initiative called "Open3." This is a motion which was passed by the city back in May and it calls for openness in three major areas:

Open Data: Public data will be made publicly available.
Open Standards: The city will shift to using open, non-proprietary formats.
Open Source: The city will shift to using open source software.

The open source and open standards goals are going to have a huge impact as well, but what I'm really excited about is the open data. By giving "we the people" access to our city's public data we can find additional benefits from it that the city might not have the resources to explore. Even simple things like getting a text message reminder to take out the garbage—an idea Tim mentioned—can help make life easier.

Vancouver's not the first city to take part in this movement. The Obama administration has already launched Data.gov and projects like Oakland Crimespotting show what's possible when a city opens the doors.

Apparently, the first datasets are set to be released in late September. I'm really excited to see what's going to be available and what kind of useful data visualizations might be possible.

These are exciting times we live in!

SketchyD Favourites for August

daniel — Mon, 31 Aug 2009 17:10:23 +0000

Shoot Film, Not Bullets

This one arose from a conversation with friend and Emily Carr film student, Kyla Plewes. We then discovered more English language oddities with the verb "shoot" whose subject can vary between:

the tool used to shoot a projectile,
the projectile itself,
the victim.

Strangely, depending on what you're shooting, not all three work. For instance, you can shoot an arrow but shooting a bow doesn't sound quite right. Neither does, "shoot your camera."

Yoga for Dogs

The little "doggie boutiques" in Yaletown, Vancouver definitely raise eyebrows. Some people love to spoil their pets rotten so I didn't think it would be too surprising to see a "Dog Yoga" session pop up. Well, turns out this idea's been baked. Check out the video:

The Banff Photo-Bombing Squirrel

Earlier this month, a squirrel crashed a photo in Banff and the shot spread across the internet. A couple setup their camera on timer to catch the beautiful Lake Minnewanka in the background. The inquisitive squirrel climbed into the shot and the auto-focus shifted to catch the little creature in a once-in-a-lifetime shot.

Here's CBC's take on the Banff Squirrel.

Using ASDoc with MXML Files and Included Scripts

daniel — Fri, 28 Aug 2009 23:34:55 +0000

The new ASDoc tool in Flex 4 does a great job getting rid of problems documenting with MXML components. You can now document projects that contain references to MXML and you can document the MXML components themselves!

The ASDoc documentation generator tool that ships with Flex is extremely useful but not without its drawbacks. In Flex 3, ASDoc's support for MXML files is missing completely: not only does it produce no documentation for MXML files, it won't even compile if your regular ActionScript classes make any references to MXML components.

A lot of custom Flex components are more efficiently expressed in MXML so I didn't want to give up on MXML components because of a problem with ASDoc. It might be possible to work around this problem using some object-oriented trickery (I think creating Actionscript superclasses might do the trick), however, there's a much better solution.

In Flex 4, Adobe has addressed the limitations with MXML. If you download one of the Flex 4 nightly builds, you'll have the updated ASDoc tool.

The version I used won't choke on source trees with references to MXML and files. Even better, it will produce documentation from comments in the MXML that begin with three hyphens:

<---
      This comment will be found by the new ASDoc tool!
    -->

Check out the MXML spec for more information. Like the Actionscript inline documentation, you have to put these comments in specific places so the documentation generator knows what to do with them.

Unfortunately, there are still some problems with the tool. Scripts that you include with a <mx:Script source="script.as" /> tag aren't handled very well. If you use modifiers/namespaces like public, protected, or static, the compiler's going to complain that they're not members of a class.

C:\src\script.as(288): col: 1 Error: The private attribute may be used
only on class property definitions.
 
private function func():void

There are two ways to get around this. Don't use the "source" attribute to include the script, just paste it directly into your MXML file under a <mx:Script /> tag.

I haven't tried option number two but you should be able to define the members and methods on an Actionscript class, and then create an MXML component as a subclass. Private methods would have to be set to protected.

Still an annoying drawback as I like to separate MXML and AS3 code but this is a huge improvement!

By the way, if you're generating AS3 documentation on windows, this ASDoc batch file example might be of use to you.

Are Situations the Root of All Evil?

daniel — Thu, 27 Aug 2009 08:49:35 +0000

Phillip Zimbardo's talk about how ordinary people can become monsters gives interesting insight into humans and evil. He did some research in a mock prison where he took good, ordinary people and made half of them prisoners and the other half prison guards.

Over the course of two weeks, these guards were made to do increasingly terrible and humiliating things to the prisoners. This was obviously done back when psychology experiments weren't regulated to preserve human rights. Here's the video of his talk at TED, below.

Phillip Zimbardo's talk at TED. His presentation contains very graphic depictions of abuse and violence. Proceed with caution.

Zimbardo's Prison Experiment

The idea of Zimbardo's experiment was to show that he could take regular citizens and get them to do bad things. His success echoes that of the Milgram experiments where people were encouraged to administer an electric shock to another person. There were many variations of the experiment but a surprisingly large number of people were willing to administer a shock at the maximum voltage when encouraged by a person of authority.

And that person of authority was key. There had to be an authoritative figure in a lab coat telling the subject to administer the shock. Zimbardo also points out that anonymity is an important factor: changing your appearance by wearing a uniform, for example, makes it easier to get people to become monsters.

Evil in the Real World

Applying these experiments to real life events, it's easier to understand how German officers—yes, Nazis are people, too—were able to do such terrible things to their fellow humans during the holocaust. Same goes for the Abu Ghraib trials.

It's the situation, then, that allows these people to do evil things. You would have a terrible time convincing a stranger to, say, punch a random person in the arm. But if you set everything up the right way, you get the holocaust.

I think it's also interesting how TV shows like Dexter and Breaking Bad have been able to get viewers to sympathize with serial killers and drug dealers by associating their evil actions with their situation or history and not the characters themselves. I'm not saying this is a new development but these are the two stories that came first to mind.

Breaking the Situations

Once you understand where these "evil actions" are coming from, how do you prevent it? Zimbardo talks briefly about preventing these situations from occurring in the first place. Definitely a good idea, but I think that we should also rise above our situations. We are not the sum of our experiences but something more. I'm not saying it's easy but we can't blame our actions entirely on our history and the situation we find ourselves in.

The Scope of this Blog is Expanding

daniel — Tue, 25 Aug 2009 21:56:44 +0000

Currently, this blog is mostly an outlet for programming experiments and a resource for technical tutorials and source code. Those will continue with their regular irregularity, however, I'm shifting the scope a bit to also include less-technical topics and ideas.

Hopefully the change isn't unwelcome but you can always subscribe to a particular topic rather than the global feed. You can do this by clicking the tags you're interested in and finding the RSS icon at the bottom of the page.

In any case, consider yourself warned. (-;

PayPal Can't Handle Identical Home and Business Addresses

daniel — Wed, 01 Jul 2009 17:30:42 +0000

This one's for people who use PayPal for their home business.

Apparently, the PayPal system isn't able to use the same address for home and business. When I moved, I created a new address, set it as my home address, and updated my credit cards. Problem was, the old address was still listed as my business address.

PayPal wouldn't let me delete the old address because I need to have a business address.
There was no way to set the new address as my business address.
I couldn't edit the old address to match the new address. PayPal would barf back a message saying, "To make a change to your address, please select the address you'd like to update and click the "Edit" button below."

The Workaround for Matching Street Addresses in PayPal

It looks like the only solution is to make the two address different somehow. Adding or omitting the space in the Canadian postal code doesn't help but I added a zero before my suite number and the PayPal website was fine with that.

The second time I phoned customer support they did basically the same thing except they put the suite and street numbers on "Address Line 1" and the street under "Address Line 2."

Another hacky solution but better than the first call: after walking me through the basic steps of changing my address she told me there was a problem with the system and that I should try again tomorrow.

What did we complain about before customer support? (-:

Import Multiple RSS Feeds into Facebook Notes

daniel — Thu, 25 Jun 2009 16:50:05 +0000

Facebook's notes application doesn't have a built-in way to handle multiple streams of RSS input. You only get to choose one URL to import your notes from and that's what appears on your profile. However, if you've got multiple streams of content you can still have it all published on Facebook.

Overview

Here's the basic idea for combining your feeds: Google Reader lets you expose aggregated feeds on a public URL. So we take that URL and hook it up to Facebook notes. Simple!

The Google Reader Side of Things

Firing up Reader, you'll first need to subscribe to your own content. (This is useful for seeing how your stuff looks in feed readers, too!) Next, you add all these feeds to a single folder using the "Feed settings..." drop down menu.

Now if you go to settings in the top right corner, there's a tab labeled, "Folders and Tags." Click the RSS icon beside your folder to make the feed public. Click "View Public Page" to make sure all your content is actually coming through.

Finally, click the "Atom Feed" link under your profile information on the right side of the public page. This is the URL we're going to drop into Facebook.

Importing the Feed in Facebook Notes

It took me a while to find the import page again: click "Applications" in the bottom left corner in Facebook and choose "Notes." Now on the right side under the "Write a new note" button you'll find the link to edit your import settings.

Facebook will probably change the layout again in a few months but for now, that's where you can find it.

All you have to do now is plug in the atom URL we copied from above, follow the confirmation, and you're good to go.

Happy publishing.