“a technology test-bed to help us crack a new design and data problem: how do you make it as easy as possible for users to pinpoint a specific bus stop, or train route, or a ferry port, as easily as possible? There are over 300,000 such beasties, and nobody has ever really tried to build an interface that makes it easy to find each one quickly and reliably.”

As Tom outlines on a recent blog post how Brief Encounters has been developed to provide design input to one of the more challenging projects MySociety has undertaken – FixMyTransport.

Brief Encounters provides a cost effective, fun and engaging way to crowdsource user requirements and user testing. I gave it a whirl and had a few thoughts on how to improve the mini site.

• The site is using Google maps but it could include Google Street View so you can check if you’ve selected the right bus stop.
• Somehow it wasn’t entirely clear when I’d selected my stop on the map and that I had to complete another form – perhaps a progress indicator would be helpful.
• I wasn’t asked to provide feedback after submitting my story. I noticed a feedback link in the footer, but it would have been good to get my feedback while I was in the flow of things.

I’m wondering whether the developers will get enough input from the types of users who are most likely to be the target audience of a service like FixMyTransport. Especially people who don’t have particularly good skills in using interactive mapping and web interfaces.

I was also reminded of the interesting discussion back at UX Camp London a few months back about how UX professionals could (and should) get more involved in projects like this.

Anyway, regardless of who you are, go try it out and give MySociety some feedback!

I’m making fairly good progress with my MSc research, having completed a few interviews and some of my literature review. Every so often I take stock of the approach I’m taking and re-evaluate my research question and approach. I decided that writing about the issues I come across would be a good form of therapy and also a good way to reflect on my situation and decisions.

“[We] become reflective researchers in situations of uncertainty, instability, uniqueness, and conflict.”
Donald Schön, The Reflective Practitioner, 1983

I have chosen to research attitudes and perceptions of council officers and citizens to community engagement via social media. Because this is a potentially vast and complex area I have chosen to focus my research around a project I am working on in Fenland, Cambridgeshire. I decided to limit my research to council officers and citizens. Because within the scope of an MSc project I didn’t have sufficient capacity or time to also interview colleagues working in the Police and Fire services or elected members.

Because there is not much existing research or theory in this area of investigation, I’ve chosen to use a Grounded Theory method (Glaser and Strauss, 1967; Corbin and Strauss, 1990; Charmaz, 2006) . I’m carrying out semi-structured interviews to collect my data. Once I have transcribed the interviews I am carrying out a detailed analysis of the data to identify emergent concepts or themes. As I identify concepts I start to categorise them and compare them across all the data. As the process continues I am starting to develop theories which I can in turn test out by collecting and analysing more data. And so on until saturation point, or until I need to write up my dissertation and hand it in. Whichever comes first.

So far I’ve interviewed four colleagues and am lining up several more interviews. My sample is probably what you would describe as ‘selective’ rather than ‘representative’ or ‘purposeful’ (Coyne, 1997). Simply because of the constraints of doing research within your own organisation and in the timescales of an MSc.

I’ve encountered a few issues with the approach and method I’m taking so far. Using social media for community engagement is a new phenomenon, certainly within the organisation I work for. The colleagues I have interviewed so far do not have hands on experience of using social media for community engagement. So their stated intentions may well not reflect their actual behaviour (Ajzen and Fishbein, 1980).

Now for me using social media has been experiential. I personally believe that you can’t really understand the possibilities and impact of the social web without experiencing it yourself. Hence the JFDI mantra which is so often mentioned by those who are active in the local social media or digital engagement field (see Dan Slee, Sarah Lay,  Dave Briggs and Steph Gray to mention a few). To get round this I have shown my participants a range of sites which I feel represent how social media can be (and is being) used for community engagement.

Whilst initially I found it problematic that I was not interviewing participants about their actual behaviour and experiences, I feel there is still merit in my research approach. Many local authorities have been reluctant to adopt social media. Some are blocking staff access for fear of time wasting or the risks of security breaches or damage to reputation (Socitm, 2010). Other authorities are cautious of the benefits of allocating resource time to monitor social media spaces and interact with citizens.

By researching the attitudes and perceptions of authorities and citizens I hope to gain a better understanding of perceived barriers, threats and opportunities of using social media for community engagement. I hope my findings can be shared with other authorities, organisations and researchers. I am also hoping my research could be a useful reference to anyone researching attitudes and perceptions of council officers and citizens who are actively using social media for community engagement.

At this point in time I don’t want to have too many lofty ideas about the impact of my tiny microcosm of research. But even as a student researcher I have to consider the value of my research to my employer and the wider research community.

I’ve also had concerns about interviewing colleagues who are unlikely to be using social media in a very hands on way for community engagement. This is primarily because they are at a more strategic level and not what you would describe as ‘front-line’ staff. But in reality they are quite likely to be the people who make the policy level decisions about how social media is used and incorporated within community engagement activities. So understanding more about perceptions and attitudes is important, to be able to sell the benefits and persuade senior managers to take risks and innovate.

So far the data I’ve collected has been fascinating and has led me to reflect on how we are approaching our project in Fenland. I’m really looking forward to interviewing some residents in Wisbech in the next couple of months to get their perspective.

References:

Ajzen, I. and Fishbein, M. (1980). Understanding Attitudes and Predicting Social Behavior. Prentice Hall, facsimile edition.

Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide through Qualitative Analysis. Sage Publications Ltd, 1 edition.

Corbin, J. and Strauss, A. (2008). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Inc, third edition edition.

Coyne, I. T. (2007). Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? Journal of Advanced Nursing, 26, 623–630, Blackwell Science Ltd.

Glaser, B. G. and Strauss, A. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine Transaction.

Schon, D. A. (1984). The Reflective Practitioner: How Professionals Think In Action. Basic Books, 1 edition.

Socitm (2010). Social media – why ICT management should lead their organisations to embrace it.

We’ve now set up a project blog so we can share what we learn with other local authorities and public sector agencies. Or anyone that’s interested really. I’ve written a bit of blurb about the project.

It’s also an opportunity to share some ideas about community engagement using social media and reflect on how social media is changing local government and local democracy.

There are some other really interesting projects going on that I’m keeping an eye on:

• Talk About Local
• Local 2.0
• Citizenscape – documented in an action research style by Catherine Howe of Public-i
• IZWE
• Kirklees Council’s work with Councillors
• Research on the impact of local citizen-led websites by Networked Neighbourhoods
• MySociety

I’m fascinated by top down vs. bottom up approaches to using social media to improve local democracy. I’m also researching the perceptions and attitudes of council officers and citizens towards the use of social media for community engagement.

If you know of any other projects in this area please let me know by leaving a comment.

I’ve decided to focus on 4 social media case studies within Cambridgeshire County Council, one of which is our Chief Executive’s internal blog. Dave Briggs has recently interviewed our Chief Exec Mark Lloyd about his blog, in his role as Community Evangelist at Learning Pool.

At the recent Localgovcamp in London I ran a session on embedding social media in your organisation. One of the outcomes of the session was that the participants agreed that we need more stories (case studies) about how social media has been used – good and bad. We agreed that publishing our social media stories on the (forthcoming) IDeA Knowledge Hub would be the best way to convey that the potential uses of social media by local authorities can vary considerably, dependent on context, purpose and target audiences. A number of these stories are now being developed at Work Together.

Updated 19/03/10 – My slides are below:

Many thanks to Anke Holst and Hadley Beeman for organising and for all the sponsors and participants for making it such an inspiring and enjoyable day. Check out the Twitter feed from the day and also the timetable of sessions and photos.

Because my time is at a premium at the moment (and I really need to spend the weekend working on my dissertation), I decided to do some live blogging during the day. It was really a spur of the moment decision when the first session started and I’ve never tried it before. I set up a Posterous account there and then and it was pretty straightforward. Luckily I can type fast so hopefully I have captured the flow of the conversation, however I may have misrepresented some of the points made. If you have any comments/inputs do add comments.

I ran two sessions in the morning. One on embedding social media in your organisation which turned out to be a very lively and useful discussion. I hope someone has some notes as I didn’t make any! However we did note that embedding social media is a full-time role (and some councils have taken on social media officers). We also agreed that more social media case studies are needed as the use of social media is very contextual and depends on service area, service users and circumstances. It was suggested that we use the IDeA Knowledge Hub for sharing stories.

The other session was on usability testing tools and there was a good discussion around usability testing in councils. I’ve added the slides with links to the usability tools below for information.

A number of blog posts have pointed to the fact that the two main parties have not yet managed to generate much positive user generated content online or truly harness the power of social media to build networks. There have been many reports that unofficial blogs are ranking highly in Google search results for terms like ‘David Cameron’ or ‘Gordon Brown’. I can vouch for this fact, as a friend’s rather excellent ‘unofficial site’ was ranking highly in Google search results for ‘David Cameron’ a few weeks ago. However within a day or two it suddenly dropped down in the rankings (bah!).

But at a local campaigning level there is still a huge disparity between those councillors who blog and tweet and those who don’t. Just recently I did a quick audit of the number of blogging county councillors in Cambridgeshire and found 17 blogs out of a possible 69 county council members, just under 25%. Now I may have missed a few, but those who do blog and tweet will definitely have the edge if they can communicate well online and use it to their party’s advantage, with the potential to reach a far wider group of constituents and potential voters.

It will also be interesting to see how the main parties deal with hyperlocal bloggers. How will they respond to stories at a local level that could potentially have a significant impact on the party’s reputation at a national level?

One thing is for sure, the 2010 election will bring the use of social media in UK politics into sharp focus. There will be lots to learn about how national and local politicians adapt to using social media and how voters are influenced by social media.

Firstly I evaluated the participation model provided by David Wilcox in his 1994 participation framework, which was based on Sherry Arnstein’s ‘ladder of participation’ from 1969.

Wilcox’s participation levels

Arstein’s ladder of participation

I noticed some similarities with Charlene Li’s and Josh Bernoff’s ladder of Social Technograph profiles. The profiles are based on survey research into consumer participation in social technologies. The ladder was recently updated to include a category for Twitter users!

The Groundswell site provides an interactive profiling tool which is based on demographic and behavioural data, to help companies define their commercial social technology strategies. However I think the tool has some transferable relevance for defining citizen participation profiles and assessing the propensity of certain age groups and genders to engage.

What would be really useful would be to overlay this behavioural data with the type of profile data that some local authorities have access to, either through OAC or Mosaic, along with other data layers, e.g. Council survey data, Place Survey data.

Li and Bernoff suggest various activities which the Social Technographs participate in. I have adapted these along the lines of Catherine Howe’s recent ‘long list’.

Lastly I thought about what type of roles might be involved at each level and drew inspiration from Steph Gray’s digital engagement roles. I have added other roles which are more relevant to local government, where communities are likely to play a more active role in engagement. Particularly where there are active hyperlocal sites in existence.

So, voilà! An adapted digital engagement framework for communities and local government. It’s a first draft so any comments welcome.

At last I’ve made it to a gov barcamp! Yesterday I attended UK GovCamp 10 (or #ukgc10 on Twitter) which was organised brilliantly by Dave Briggs and generously hosted by Google at their HQ in London. As a barcamp newbie I really enjoyed myself and talked far too much. The best thing about this type of event has to be the diverse range of interesting and enthusiastic people from the UK gov webbie community who attend (on a Saturday). And I really appreciated the opportunity to meet and chat to the people I know through Twitter and those I hadn’t come across before.

It was hard to pick and choose which sessions to go to from so many good ones, but here’s a quick round up of those I attended.

Localgov group hug

The group hug never quite materialised, but this was an opportunity to meet up with fellow local gov webbies and find out how different authorities are using social media and tackle how you get Senior Managers on side (which I have blogged about previously). Some of the things we discussed included:

• An example of using social media in Adult Social Care in Stockport in a secure area of the intranet (not sure I noted that bit correctly). Updated 10/02/10 – Mark Watson has added a comment on this post to note that the Stockport content resides on a secure area of http://www.careknowledge.com, which enables external stakeholders to access it as well as Council staff. The project used blogs and video ‘stories’ with service users as a way to engage staff and facilitate internal culture change.
• Sometimes it’s better to ask for forgiveness not permission – try something out and demonstrate the success to senior managers using statistics to back you up, rather than trying to explain the tools themselves. E.g. Derbyshire’s very successful social media campaign for the 2009 elections. We debated that not everyone felt in a position to innovate and take this kind of risk.
• The importance of showing financial implications, where there might be savings in traditional print and marketing budgets.
• Getting a collection of good quality case studies together – Liz Azyan noted how her PhD research had resulted in being able to collate several good examples from different authorities.

Socialising Internal Comms and Change

This was a brilliant session. I was surprised at how few central or local government organisations are adopting social media for internal comms. We discussed various barriers to adopting social media for staff engagement including fear of the unknown, loss of control, security and information governance risks and blocking staff access. Whereas actually implementing the technology seems to pose less of a problem. We discussed continuing the work on tackling these barriers by providing good case studies and the potential for updating Tim Davies’s Social Strategy 50 hurdles wiki. The discussion included the following points:

• Social media provides a way to share knowledge between old timers and new starters.
• Internal social networks can enable staff to network, find out who has the knowledge/skills/influence to help you get things done.
• It is unclear how to link social media with existing knowledge management systems, but it was generally felt to be worthwhile.
• What are the measurable benefits? Carl Haggerty talked about an internal social networking pilot in Devon County Council using Blue Kiwi. They have been able to estimate considerable non-cashable savings in terms of productivity. Carl also mentioned that whilst they had been disciplined in their team, the volume of emails being sent had reduced significantly and they were no longer solving organisational problems via email.
• Carl also mentioned that they quickly learnt to change the terminology to ‘business networking’ not ‘ social networking’. In management’s view social = Facebook = time wasting.
• James Gardner talked about the Ideas forum at the DWP. There had been massive take up internally. They identified a risk that people would spend too much time on the Ideas forum (which is the case!) so it has proved important to work with middle managers so they understand the benefits of staff using the forum. Staff vote using an internal trading ‘currency’ and the best/most popular ideas are taken to internal review boards.
• It’s important to have a very clear policy for staff conduct.
• Identity can be an issue. Anonymous forums can encourage staff to make negative comments, but when using real names staff need the confidence to be able to find their voice but with safety.
• If using internal social networks to share information there must be security around safeguarding service user details (either by using a secure area of the network or having a clear code of conduct).
• It can be challenging to get internal networking and social media adopted. It was suggested that the way forward was to start with a small pilot (like Devon CC), although interestingly this contradicts what was discussed at a later session led by Steph Gray.
• There are pro’s and con’s toYammer and in some organisations it hadn’t taken off due to concerns about security, regulation and FOI, whereas other organisations were using it successfully.
• It was suggested that tracking conversations with social media would actually make FOI requests easier because finding conversations in email is time consuming and difficult.
• Staff are using social media outside of work, which raises their expectations about technology at work.
• It is essential to identify the business need rather than focusing on tools and technology (a recurrent theme!).
• Whilst many organisations have HR policies they are not being enforced by Managers. Instead there are still knee-jerk decisions to block staff access to social media sites, without understanding that social media can be accessed on iPhones.

Because of the huge interest in this area Phil McAllister proposed a barcamp focused on social media and internal comms.

User Involvement in Service Design

After lunch I attended a session led by David Wilcox on user involvement in service design in relation to a project he has been working on for Consumer Focus. David proposed a framework for involving users in the design of digital public services.

• The group had quite diverse involvement in co-production and user involvement within the public and private sector and we wrangle with whether we were dealing with ‘user’ or ‘citizen’ involvement.
• Some of the participants in the group were interested in the way service users can have greater involvement in the way services are delivered, participate in the democratic process or have more specific involvement in web usability.
• We talked about various models of user involvement e.g. involvement up front (initial research), at the design stage only, at the end of the process to get feedback or iteratively throughout.
• Barriers we discussed ranged from cultural to financial.
• There were anectodal reports of dealing with service managers who have been in public service for many years, think they know the service best and won’t involve users in service re-design.
• User involvement methods such as focus groups, interviews and usability testing can be costly and time consuming, so resources and funding may be an issue for local and central gov.
• Social media can be used to incorporate user voices/stories in service design e.g. Patients Opinion.
• How do we identify which groups to engage? And how do we recruit them? I gave an example of tapping into existing user groups which voluntary sector organisations consult with.
• Are there low cost methods/tools we can use? I suggested a remote usability testing tool Loop11 which we have been using for usability testing our website.
• We discussed some existing services in Health like Patients Know Best which enables patients to have more control over their health care provision by giving them access to their patient records and discussing their care with the doctor.
• David Wilcox proposed a further workshop to discuss user involvement in service design.

After the session I had a really interesting chat with Adam Bailin about the potential for developing an affordable panel of user testers for central government agencies.

At this point my computer battery ran out of power, so I stopped taking notes temporarily and skipped between the Data.gov.uk session and the session on the Future of Journalism which was rather oversubscribed for such a small room.

Localgov Digital Engagement with Social Media and Business Change

The last session I attended was the one I proposed with Paul Henderson where we did a show and tell about a project we are working on in Fenland, Cambridgeshire. In summary we are using social media to engage communities and in particular hard-to-reach groups such as migrant workers and we’ve done quite a bit of research with communities to find out their perceptions towards social media.

I’ve put our slides up here:

David Wilcox has posted a video interview with Paul and I here:

The session rounded off with an interesting discussion about approaches to governance and moderation. We also discussed how local gov community engagement relates to hyperlocal activity (local blogs etc.). The key points I noted were:

• It takes patience to build hyperlocal involvement. Nick Booth and Paul related their experiences of running social media surgeries for local communities in Birmingham.
• It may not be appropriate for local gov to try to manage/control online spaces. But having a central space where discussions can occur does mean it is easier to monitor conversations.
• Those managing and responding to online communities need to distinguish between constructive criticism / suggestions and complaints.
• You can encourage positive feedback by responding quickly and sorting peoples’ problems out.
• There are existing examples of best practise in moderation and how to manage conversations without damaging the organisational ‘brand’.

So that’s my round up. All in all a truly inspiring day and I also enjoyed the pub session afterwards (thanks to Hadley Beeman for organising) which provided further opportunities to catch up over a pint or two.

Many thanks again to Dave, hosts Google and sponsors 4IP, Boilerhouse, Timetric, Learning Pool, Opportunity Links and everyone who attended for making #ukgc10 happen!

Promote the Vote is a fantastic site promoting the vote (obviously) for people with learning disabilities which has been led by Speaking Up in my home town Cambridge. Great design, content and videos. Designed by Easy Read Online.

On a similar theme of accessibility, it’s good to know that Google are sorting out automatic captioning for videos on YouTube.

Broadband NotSpots let’s you check broadband availability in your area. Not sure how accurate it is, but useful for a rural project I am working on at the mo.

Carl Haggerty wrote a very interesting post about whether Council websites should be more community based:

My key point here is, without any external pressure, would councils have taken a more community based approach to their websites instead of being forced to deliver services online that offered no value initially.

Philip John blogged a request asking for ideas about widgets for hyperlocal sites. This is a heroic effort to make democracy more accessible, which I take my hat off to (well I would if I was wearing one). I’m looking forward to seeing what gets developed.

Ok it’s not new, but I found some great articles on sketching on the Adaptive Path site.

Phew that’s enough for now.

Before we started our usability testing we did quite a bit of analysis of our Google Analytics and customer feedback data, to find out what the top customer tasks are on the site (i.e. the things our customers do most frequently). We’ve based our user testing around a selection of the top 20 customer tasks.

Preparation

Writing the test plan and preparing materials is important for any research, because you need to be clear about your objectives for the testing. It’s a good opportunity to think about exactly why you are doing the user testing and what you hope to find out about your users.

For example I wanted to know how our users perceive our web site on a first visit. So we asked participants to spend the first minute or two of the test looking at our home page and telling us what type of things they thought they could do on the site. This was very revealing and made us realise that our current home page really doesn’t say what it does “on the tin”, so to speak.

From our Google Analytics data we knew that around 50% of our website traffic comes via search engines. I wanted to know how findable some of our top tasks are whether users are searching in Google and deep linking into the site, or accessing our home page and browsing from there. So in the test plan I proposed that 50% of participants start the tasks from Google and 50% start from our home page.

I also wanted to collect some demographic information and assess user satisfaction, so I decided to create a pre-test and post-test questionnaire.

As well as the plan we had to:

• create a profile for our participants – to ensure a representative sample of our main web site audiences;
• create various materials – participant information sheet, test script, questionnaires, task list etc.;
• book facilities – we opted to book rooms in our new library building which was easily accessible.

Recruiting participants

I’d forgotten how much admin is involved in user testing and if you plan to recruit your own participants, don’t under estimate the effort involved. You can pay a specialist company to recruit participants who match your participant profile, but obviously you pay a fee to the Recruiter.

Our first round of recruitment was pretty unsuccessful. We had a link to a sign up form linked from the home page of our site and advertised for participants in libraries.  We are offering participants an incentive (some tokens for a high street store) but even that wasn’t sufficient to entice residents to sign up! So we resorted to recruiting participants face-to-face, one at a time. That approach has been far more successful.

Ideally participants should be contacted by phone a week before and then a day before the session so you don’t run the risk of having ‘no shows’.

User testing software

Face-to-face testing with Morae

We finally invested in a copy of TechSmith’s Morae for our team laptop which enables us to record user tests.  The software records real-time screen capture (e.g. mouse clicks and cursor movements), audio and video (a small video of the participant’s face is shown in the bottom right corner of the screen). You could use Silverbackapp to do the same job (and it’s cheaper) but we are PC based at work.

So far we’ve done some user testing sessions and have more planned for this week. First impressions of Morae are very good. As a moderator you can facilitate the session far more easily, without worrying about having to write notes. The note taker was able to focus on logging the times when users experienced problems using the site during the tests, so we could add markers into the Morae timeline afterwards. I have since found out that we could have done this automatically using a Wii remote, so perhaps we can try that next time.

We ran the tests in the local library on a laptop with Morae Recorder installed, connected to the Council network. Using Morae’s Observer software, colleagues back at the office were able to login and observe the tests in real time from a PC. This is a real breakthrough and there is great potential for getting Council services to observe tests remotely, without putting off participants by being in the same room and wincing visibly or, worse still, offering to help when users struggle with their section of the site. We can take the laptop to any Council offices or premises that have a Council network connection, so effectively we now have a mobile testing lab on our laptop!

Remote, unmoderated testing with Loop11

Another tool we have tried using is Loop11, a DIY remote, unmoderated user testing tool. By setting up tests in Loop11 our participants can test the site from the comfort of their own home in a more relaxed environment. We set up a number of information seeking tasks in Loop11. The interface was very easy to use and setting up the tests was straightforward.

The task instruction is in the top green bar (as shown below) and the participant carries out the task on your site in the main frame of the browser window.

We set up the test so that participants have to answer a multiple choice question about the task they have just completed and then indicate how easy or difficult they found the task with a scalar question. Participants then click a ‘task complete’ or ‘abandon task’  button to proceed to the next task.

Loop11 records the time taken to complete a task and success or failure rate. We opted for Loop11 because, realistically, it was the only software we could afford to use. You can run the same test more than once, so for us it’s an excellent way to benchmark user task performance and measure improvements we make to the web site.

Next steps

We haven’t analysed data from our user testing yet. So we’ve yet to discover how much time that part of the process takes. I’m also conscious that we need to be aware that we’re relatively novice researchers and we have a vested interest in the site, which may bias our findings.

The final step of this research phase will be to create a usability issue log, prioritise the usability ‘problems’ and document the key findings in a presentation for Council services.

Then we’ll progress to the design phase, which is where it gets really fun!

References

Some books I referred to when planning the user testing (and as reference for document templates) included:

Handbook of Usability Testing and templates here

Observing the User Experience

Understanding your Users