every108minutes

Local Testing OAuth Social Signin

2013-06-25T15:59:00.002+10:00

On some recent Grails projects, I have been looking at using the Twitter and Facebook OAuth signin process.

This process allows you to authenticate users based on their Twitter/Facebook logins, without the need for the user to expose their passwords to your site.

When you create your 'application' within Twitter or Facebook, it is necessary to define the URL where the application can be accessed. Twitter and Facebook will only redirect to this URL during the authentication process.

I have tested running some applications on Heroku or Appfog, with Twitter and Facebook happy to redirect to the appropriate URLs with successful authentication.

However, when testing locally, I follow these steps to work through the authentication process.

1. App Context
Ensure that the Grails app context is '/' - as the application is generally deployed this way on Heroku/Appfog:

Config.groovy

grails.app.context = '/'

2. Port Binding:
While the local application will generally run on port 8080, we want it accessible via port 80 - as it is run on Heroku and Appfog. One method of achieving this locally is to bind the ports as follows

sudo ssh -L 80:localhost:8080 root@localhost

Note: if you are working on a Mac - ensure that Remote Login is enabled also.

If you application is running via HTTPS (port 8443), the following port forward command is required:

sudo ssh -L 443:localhost:8443 root@localhost

3. Hosts
Edit the /etc/hosts file in order to redirect calls to your Heroku/Appfog application URL to your localhost:

/etc/hosts

127.0.0.1 murmuring-gorge-6675.herokuapp.com

4. Test the Process

Now, running the application on your local machine, it should be possible to follow the Twitter/Facebook OAuth process, with all interaction taking place with your local server.

Deploying With Git

2013-05-09T09:03:00.001+10:00

I have the ~~mis~~fortune to work on a number of PHP based web applications at work. Previously, the deployment process involved determining which files had changed since the last release and copying them across to the server. Needless to say, this was an error-prone and inefficient way of deploying updates.

Gitobots, Roll Out

We use Git for our version control and, with Heroku's push to deploy in mind, I looked further into the possibilities of using Git for our deployment process. Abhijit Menon-Sen's article details the process very well. With a few slight variations, these are the steps I follow to deploy changes via Git.

Prime Remote

On the remote server for your application (e.g. production, staging or test), create a new, bare Git repository for your codebase:

cd /cygdrive/c/repo
mkdir project.git
cd project.git
git --bare init

Hooks

As this bare repository does not contain a working tree (the actual source), a Git hook is used to checkout the code to a specific location. Git hooks allow custom scripts to be executed when certain important actions occur.

Create the directory from which your application is accessed/executed - e.g. /cygdrive/c/webroot/project
Create the file post-receive in the project.git/hooks directory.

Edit the file to include the following:

#!/bin/sh
GIT_WORK_TREE=/cygdrive/c/webroot/project git checkout -f staging
rm -rf /cygdrive/c/webroot/project/twig_cache/*

Set the file to be executable - e.g.:
```
chmod +x post-receive
```

The main point of this hook is in the configuration of a working directory and the checkout of a specific branch (staging in this case) to this location. I also utilise the hook to clear out a cache directory for a templating engine (Twig) used in the application.

Push It

Within your local development repository, add the bare repository as a remote:

git remote add staging-web ssh://keibro@127.0.0.1/cygdrive/c/repo/project.git

Once you have committed your local changes and are ready to deploy to your server, the server code base can be updated by pushing to the remote repository:

git push staging-web staging

The remote repository should update and also note that it is already on that particular branch. The updated files should now be present in the working directory.

Note: it may be necessary to manually execute the post-receive hook for the first push.

Branching Environment

The above process can be replicated for various environments so that changes can be pushed to these environments as needed (e.g. test, staging, production). Following a similar model as described by Vincent Driessen, I utilise different branches for each environment (e.g. the staging branch is pushed to the remote repo staging-web, the master branch is pushed to the remote repo production-web). However, this generally implies that each environment will require unique configuration settings.

In order to address this issue, I created a new local configuration file to supplement a more generic configuration file.

config.local.php contains configuration settings that are unique to that environment - e.g. database connection details, email settings, etc. Note: This file is not under Git control, as you do not want to commit any changes from this file across all your environments. Changes need to be made to each environment individually - these changes are normally relatively infrequent.
config.php contains generic environment settings that apply across all environments - e.g. global constants, LDAP server connection details, etc. This file is under Git control.

It is necessary to ensure that all environment specific variables are extracted into appropriate configuration files (e.g. REST access points for your Backbone frontend, database connection, etc.)

While not optimal, this solution addressed the problem at hand.

Restricted Access

As all files under Git control are now pushed to the server, you may wantto ensure that some files are not accessible when accessed via the web browser. In this case, I created a new directory restricted within the project and included a .htaccess file to prevent access to the files contained therein.

If This, Then What?

2011-08-01T23:53:00.000+10:00

Imagine that you could combine services (internet and otherwise) together like Lego blocks to create new, personalised services.

If This, Then That (ifttt) is a new service that allows you to do just that.

Through an intuitive interface ifttt puts the power of "Event Driven Programming" at your fingertips, letting you connect services with digital duct tape. Event driven programming can be simply broken into two stages - event selection/detection followed by event handling. ifttt terms these stages as triggers and tasks and provides a comprehensive list of services that you can target as a trigger or task, from Facebook and Twitter through to Google calendar events.

For example, it is possible to build a service that will send a text to your mobile phone (task) if the weather service has forecast rain for the following day (trigger). The mechanism for building such a service is so simple and straight-forward that no programming knowledge is required. Trigger and task construction follows a simple workflow of clicks. In some cases, it is necessary to provide some extra details in order to fine tune the trigger/task or to authorise ifttt to access your online account.

This is a really awesome idea that has been well executed and realised. The simplicity of how these new services can be glued together and the countless possible outcomes from the trigger/task combinations opens the power of the web to everyone.

I realize that ifttt only addresses a small subset of the ways in which you can be creative with digital information, but within that subset there are tons of opportunities to hook together existing services, devices and objects. For instance, you can use Google reader starred items to share images on your Tumblr blog, or customize how and which photos from your Flickr stream show up on your Facebook wall.

Linden Tibbets, ifttt

Farewell to a Friend

2009-09-04T03:06:00.000+10:00

Earlier this week, Jeffrey Walker passed away.

I had the pleasure of meeting and working with Jeffrey when he assumed the role of President of Atlassian in July 2005. In the last few years, Jeffrey inspired us all by publicly announcing his war on cancer, but Jeffrey was so much more than this battle.

He had a passion and zest for life and business that was plainly evident when he walked into a room. He really did have a knowing smile, accompanied by that glint in his eyes - and you knew he was seeing something that you were missing. He was dubbed as the ‘Lance Armstrong of Silicon Valley’ at one point.

I am proud to have known this gentleman and my sympathies go out to the Walker family and the Atlassian staff.

Grey Goose Gibsons, straight up with a twist, all round while we remember Jeffrey like we should.

Brain Error: No space left on device

2008-08-05T23:04:00.000+10:00

I'm not dumb. I just have a command of thoroughly useless information.
Calvin - It's a Magical World, Bill Waterson

Wired's June 2008 edition included an article entitled 'Quiet Please: how Man-made noises may be altering Earth's ecology'.

The article focused on the theory put forward by Bernie Krause, a field recording scientist, that nature's soundtrack (biophony) is being adversely affected by a louder human-made cacophony (anthrophony). Krause postulates that the animal kingdom divides the acoustic spectrum so it's inhabitants do not interfere with each other. However, human-made noise is increasingly disrupting this harmony and intrudes on a piece of the spectrum already in use - drowning out natures voice.

As an example, Krause summizes that the rapidly declining population of the Yosemite spadefoot toad is due to the noise generated from low-flying military aircraft, performing training exercises in the area. Coyotes and owls are able to home in on individual frogs as they have lost the veil of protection provided by the synchronicity of their calls.

The information flow in the jungle is compromised

Mediaophony

Something similar is happening with the explosive adoption of social media tools and the information they generate.

These social media tools now afford the ability to create informational flows of near unbounded proportions from as many sources. So much information threatens to drown out real information of value.

In his book "Here Comes Everybody", Clay Shirky notes that the old process of 'filter-then-publish' has been inverted to 'publish-then-filter' through the adoption of new digital broadcast media by the masses. The inherent simplification and reduction in cost of the publishing process through social media tools has enabled a multitude of 'amatuer' publishers to transmit their work online. The traditional system of publishing only select items has been replaced - a momentous move from where the few decided upon the information that the many should consume. However, with this new freedom, the task of information filtration is now transferred to the consumer.

With this glut of information and the potential for overload, the ability to sift and sort the 'signal' from the 'noise' is key.

Follow the leader

One potential mechanism to address the information overload is to track the digital trail of your peer group. Most social media tools are founded on the very principle of sharing and enable users to see what is of interest to their tribe. A type of group filtering system evolves. The links formed in these social networks imply that information of interest to one of your connections, most probably will also be of interest to you.

Digg is an example of such group filtration in action - content of interest to the general group bubbles to the top. The site further supplements the origial content by facilitating a discussion around it. Delicious allows users to share content they have bookmarked on the web.

Let the Data Talk

In another Wired article, Chris Anderson talks about the Petabyte Age. He notes how large bodies of data can be mined with analytical tools and contextualised. It is

a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear.

Using Google as an example, he notes how this analysis is used in advertising and translation to generate better results - without applying subject knowledege. Further, Google's search results provide a level of filtration through the application of the PageRank algorithm.

As the data cloud and understanding of these concepts grow, these type of tools will help filter informational value and provide relevant data customised for each user.

Appophony

The ill-effects of information overload can also be experienced in software applications and websites. All too often, applications are encumbered with feature bloat - overwhelming the end user with too may options and features.

Matt Linderman suggests that we should be:

the sniper who must pick off every distraction, unneeded feature, and extra word that doesn’t absolutely have to be there.

Some of the latest application success stories include tools that focus primarily on one core task. Dropbox provides a seamless experience in sharing, syncing and sharing data. Skitch makes taking screenshots a pleasure. Twitter, the de-facto micro-blogging tool, concentrates on the fundamental process of posting 140 character status updates. In each case, the principal functionality is the focus - provided with simple, yet intuitive design.

Staying Afloat

The flow of information is only set to increase; as will the need for the skill to sift through this flow. In part, social networks will provide frameworks for data filtration, while new tools will emerge to further help extract the information of value.

Coding Cults & Internet Gods

2008-06-24T18:26:00.000+10:00

Any sufficiently advanced technology is indistinguishable from magic.
Arthur C. Clarke

Its been talked about before, but while reading the 'The God Delusion' by Richard Dawkins (and no he doesn't Twitter!), the meme of the Cargo Cult once again caught my attention.

Cargo Cult: Waiting for Frum

Anthropologists have observed the phenomenon of island inhabitants (most famously of Pacific Melanesia and New Guinea) mimicking and performing rituals of visiting advanced cultures in the belief that these acts will provide them with material goods and wealth. This phenomenon is known as a Cargo Cult.

The islanders observed that the visiting people enjoyed great luxuries, with seemingly little or no useful work required in return - they sat behind desks while new or repaired items appeared on the island as if by magic.

In one instance, on the island of Tanna in the New Hebrides (now known as Vanuata), the islanders awaited the return the messianic Mr John Frum who would bear bountiful cargo. Supposedly arriving by plane from America, the islanders cleared an airstrip and installed a bamboo control tower, replete with wooden headphones. They still await his return.

In our technologically advanced, self-actualising, networked society, cargo cults are still in full effect.

The Cult of the Internet Gods

With new social networking tools emerging on the net every day, users scrabble to adopt these applications in the hope of increasing their social network status. A case in point: Twitter. A few in the industry have risen to deified status, having built large networked tribes with thousands of followers. 'Lesser mortals' reply or re-tweet these Twitter superstars in the hope of building their visibility and profile.

The cult of the social media tool is further expounded with the concept of the invite-only period. The exclusivity built around the new network heightens user desire to be included and follow in the keystrokes of the privileged early adopters. As these tools grow and permeate, the onus is on people to opt in to these new networks - or fear missing this new source of flow information.

That being said, this phenomenon does propel the ideas and tools in that space forward, helping refine processes and opening the network for use in unexpected and innovative ways. The mass take-up of the network helps progress the tool and inevitably leads to a disruptive leap to the next social network tool.

Users also quickly determine if the tool is of real use in their daily workflows and can decide to opt out of that particular network stream.

Cargo cults with less satisfactory results can also be found in coding practices.

The Cult of Copy And Paste

At times, we need to replicate a piece of code from one code-base in another project/application. For the times when the functionality cannot be extracted out to a utility library, the temptation to simply copy the code can be strong. The application may even work as expected during testing and it appears that some time may have been saved. At some point, however, copy'n'paste karma may come calling back with problems.

Time should be taken to understand and verify what each line of the replicated code is doing. Is each line necessary? Is there a subtle change required to achieve the goals in the new code-base? Answering these questions can avoid problems and unexpected behaviour and gives the developer confidence in knowing the flow of the application.

The Cult of Scripting

Build systems and deployment scripts can sometimes be arcane pieces of dark magic - Maven still confounds me! Often written in languages that require real expertise to understand subtle nuances, developers can be left executing scripts which they know little about other than the final (hopefully successful!) result. This lack of understanding can lead to problems.

On one banking application I worked on, a set of build scripts was employed to deploy an application into the target environment - configuring the database and the application server. The development team was given little information on the intention of the scripts, with only sample scripts from another project as a guide.

New scripts were developed (copied!) and executed. However, soon after application initialisation, problems with database connections were reported and the application would need to be restarted. For each deployment, the release engineer was requested to run all the scripts. Upon further investigation into the nature of the scripts, it transpired that one of the scripts was only required to be executed once at the outset of the project, in order to configure the database and connections. Thereafter, only the application configuration scripts were required.

Build and deployment scripts can be difficult and time-consuming to deconstruct. In this respect, breaking the task into discrete parts can help, by aiming to understand small sections of the script at a time. Learning to understand the intricacies of the process can lead to resolution of issues as described above, or improvements in the overall process workflow.

So, is your code expecting the return of Mr Frum anytime soon?

Explore. Dream. Discover. Be Interesting.

2008-06-24T04:23:00.000+10:00

Interesting 2008

Bunting and all ...

Twenty years from now you will be more disappointed by the things that you didn’t do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover.

Mark Twain

Over the weekend, my wife and I attended Interesting 2008. The concept of the event - to gather a number of speakers to talk about something they are interested in - was intriguing and all reports suggested it would be a Saturday well spent. Comparisons with the TED talks certainly helped to pique my interest and we were most definitely not disappointed.

Russell Davies, the coordinator of the day, has this to say about being 'interesting':

The way to be interesting is to be interested. You’ve got to find what’s interesting in everything, you’ve got to be good at noticing things, you’ve got to be good at listening. If you find people (and things) interesting, they’ll find you interesting.

Attended by designers, developers, social media consultants, teachers and more, the speakers came from a similar varied background of skills and interests. The day was extremely entertaining and enjoyable. I believe video casts of most of the talks will be available on the Guardian website soon.

For me, some of the highlights of the day included:

Daniel Raven-Ellison, a teacher (with such energy and passion, he is the kind of teacher you wish you had had back in school!) spoke about Geography. Daniel looked at how we accept scrutiny under CCTV due to its inconspicuous nature, the diminishing level of designated play areas for children and the energy footprint of Salisbury. Seemingly, we need another 2.5 planets to keep going at the average rate of energy consumption of your typical Salisburian.

Michael Johnson gave a lively talk on the pattern of convergence of good music and design over the last century, interspersed with guitar examples performed by Michael himself. Maybe you can tell a good album by the cover?

Steve Hardy looked at what, specifically, generalists do. Steve spoke about the phenomenon of how niche interest has taken over. While being a good thing overall, Steve noted that we also needed to embrace broad thinking ideas; giving ideas space to run around and occasionally bump into strangers, connecting people in different areas of specialism. This is a talk I would like to revisit.

Jenny Owen presented a 7:42 minute synopsis (a personal public speaking record!) on Winston Churchill and his qualities.

Andrew Dick sought to enlighten the audience on how to combat chronic insomia. The answer, according to Andrew, lies in listening to badly written, and equally badly narrated, audio books. The plot excitement level should lie somewhere between extreme boredom (forcing you to think about all the world ills as reported on the BBC World Service) and high octane addiction (and thereby keeping you awake to hear what happens next). Andrew has found iTunes to be a great source of prime material - 50's BBC produced story lines.

Matt Dent introduced us to his winning design for the new UK coinage and some fascinating (he assured us!) facts on the minting process and their coin use.

Phil Gyford spoke about the art of mask making and its use in theatre, harking back to La Commedia dell'Arte. Phil showed the power of the medium with the application of a simple red nose and some quirky acting.

Unfortunately, I have forgotten the name of the speaker (mixed up in the line of Matts and Andrews!) who presented a tale of Patagonian physics, conquistadors and ancient libraries. This is another talk I would like to review.

Max Gadney discussed his passion behind World War II history and the need for accurate representations of this information. He presented some superb examples of how he has applied his form of data visualisation on the topic. A striking example took shape of graphically representing a soldier in a platoon, in a battalion and a division - finally displaying the number of dead per country involved. Stark data to behold but illuminating facts in an understandable manner.

Younghee Jung ventured into the world of the toilet with images taken from around the world looking at the different cultural and social mores around this everyday activity (seemingly 6 times a day is the healthy recommended number of visits).

James Wallis led the audience on a geophysical survey of the World of Warcraft and the planet Azeroth. Using novel methods of throwing characters off buildings and traversing straight line sections of the planet, James has determined such data as the force of gravity and the diameter of planet. Based on this data, it appears that Azeroth would be roughly the same size as another well know spherical object.

Be Interested. Be Interesting.

This list is certainly not exhaustive and the speakers all presented very thought-provoking talks. I would urge you to review some of the talks once they come online (or indeed head over to TED for some further enlightenment). As Russell noted, it is wonderful to think that so many would turn out to hear 'interesting' talks.

The day struck a chord with me in that we should all be motivated to look at things anew and find what is interesting about it - no matter what that object, subject or experience be. The sense of enthusiasm generated by someone finding something interesting can, quite often, be contagious.

Explore. Dream. Discover. Be Interesting.

Let your Confluence Tweet!

2008-06-12T10:15:00.000+10:00

The single biggest problem in communication is the illusion that it has taken place.
George Bernard Shaw

The recent Atlassian Codegeist competition afforded me an opportune moment to further explore new areas of Confluence plugin development. My current contracting assignment involves heavily customising both the theme and functionality of the Confluence platform - but Codegeist offered a chance to really experiment!

With the explosion of social networking applications and the rise of the flow, I decided to integrate the functionality of one of the most profilic flow tools in the netsphere, Twitter, with Confluence.

Twitter + wiki = Twikkir
Posing the simple question 'What are you doing now?", millions have taken to tweeting their response on Twitter to anybody who will listen. From a simple status update to a possible support circle to a tribe building ecosystem, Twitter has become another pillar of the fast-evolving web. In his blog, Rands notes:

I want to see how they see the world. This is why I follow people on Twitter. This is why they follow me.

Enterprise Flow
As I commented on the Workstreamer blog, I think enterprise companies have witnessed social networks flourish outside of their control and domain. They are now looking to replicate these networks in their own space in order to captialise on the perceived benefits, create structure/control mechanisms and expose the data for processing. Granted, some of these integrations may be exploratory - but the enterprise is definitely looking to keep apace with these developments - and a Twitter-like tool is a main contender.

Twikkir

The actual plugin development proffered some interesting issues to solve - persistence of posted messages, an update mechanism for diffusing new messages and the display of messages amongst others.

Persistence

The issue of persisting user posts was one of the first obstacles to tackle. Confluence offers various persistence mechanisms for storing data - in brief:

Hibernate - Confluence uses the Hibernate framework to persist its objects. However, it is generally not recommended to modify the Hibernate files to incorporate persistence of custom objects.

Content Properties - a mechanism to store a key value pair with an associated object (content, space, etc.) within Confluence

Bandana - this Atlassian framework uses XStream to convert arbitrary Java objects into XML for storage.

I selected the Bandana framework as the solution - allowing content-unrelated data (Twikkir posts and users) to be stored in the Confluence database. Extremely simple to use, Bandana allows the persistence and retrieval of objects through keys in a global or space related context. In order to avoid class loader issues, as noted by David Peterson, it is necessary for plugins to set the classloader reference in the XStream object correctly:

[java]
// Create xstream reference
if(xstream == null)
{
xstream = new XStream();
xstream.setClassLoader(getClass().getClassLoader());
}
...
// Persist data within the global Bandana context
String xml = getXStream().toXML(userSet);
bandanaManager.setValue(new ConfluenceBandanaContext(), Constants.BANDANA_CTX_KEY_TWIKKIR_USERS, xml);
...
// Retrieve
String xmlString = (String) bandanaManager.getValue(new ConfluenceBandanaContext(), Constants.BANDANA_CTX_KEY_TWIKKIR_USERS);
[/java]

AJAX

I was eager to add some AJAX functionality to this plugin and, in doing so, explore some of the JS libraries offering some 'Web 2.0' goodness. My goal was to enable the asynchronous ability to post and receive new twikkir posts and avoid a total page reload.

jQuery is an extremely powerful library and affords the developer some neat tricks with minimal fuss. In order to enable the jQuery library for use within the plugin JavaScript files, I found it necessary to define an alternate name for the jQuery variable through the noConflict() function in the main Velocity template.
[js]
var $jq = jQuery.noConflict();
$jq("document").ready(function(){ ...
[/js]

All subsequent calls to jQuery were made through the variable $jq. jQuery really impressed me - the animated tab panels were generated by one line of code! I look forward to exploring jQuery further.

jQuery can also simplify the construction and management of AJAX calls, but as I was interested in understanding the inner details of the application flow, I coded these manually in JavaScript. All calls followed this format - with the response callback using an anonymous function:

[js]
var request = createRequest();
var url = "/plugin/twikkir/postit.action";
request.open("POST", url, true);
request.onreadystatechange = function()
{
if(request.readyState == 4)
{
resp = request.responseXML;
document.getElementById("poststatus").innerHTML = getTagContent(resp, "poststatus");
postStatusImg.src = restImage.src;
document.getElementById("twikkirpost").value = null;
document.getElementById("charsleft").innerHTML = maxPostLen;
$jq("#poststatus").fadeIn(2000).fadeOut(3000);
}
};
request.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
request.send("twikkirPost="+escape(twikkir)+"&username="+escape(username));
[/js]

It should be noted that it is necessary to set the appropriate request type header in the request parameter (line 16 above). It is also necessary to set the appropriate response type in the associated action class - for example:

[java]
ServletActionContext.getResponse().setContentType("text/xml");
[/java]

Further, the altassian-plugin.xml should also reflect the return type of the action - in this instance "velocity-xml":


 <action name="postit" class="com.sidus.confluence.twikkir.action.PostItAction">
  <result name="success" type="velocity-xml">/templates/sidus/twikkir/posted.vm</result>
 </action>

Another experiment took shape in the version control system used - Github - where you can find the source code behind the plugin.

Future

Building the plugin proved to be an extremely useful learning exercise. The plugin certainly could be improved and there are many extension points:

Send/receive tweets from Twitter

Protect your twikkir feed

Enable Confluence entities (pages/ blogposts, etc.) to send tweets

Enable a command line interface to execute Confluence actions

Build twikkir clouds

Build twikkir connection data - inferring/suggesting potential relationships between users based on who is following who

The flexibility of the Confluence plugin system creates countless possibilities for extending the wiki platform with social networking integration and Twikkir is just one of these possibilities.

... and of course you can follow me @ keibro on Twitter.