Blended Learning Benchmarking - Home

Benchmarking Project Reports

2007-09-02T16:17:00Z

The final reports for the Benchmarking project are now available. Apologies for the delay in them appearing here. They were formally submitted back in April. But I have only noticed that they had not been added here.

The first is the Summary Report submitted to the Higher Education Academy

The second is the Final Report submitted to the University Steering Group.

Blended Learning Survey Results - Part 2

2007-03-28T13:34:59Z

Many thanks to those who responded to the previous post on the Blended Learning Survey. In this post, I would like to share the findings that relate to the perceived development needs of teaching staff.

We asked staff about their present developmental needs and offered them the opportunity to cite up to three needs from a drop-down menu of choices. (Note: Staff were also invited to list needs not already included in the drop-down menu.)

In total 84 staff responded to this question: not all staff cited three needs. The key needs were:

More time to prepare lessons - [frequency] 46
Work alongside enthusiasts (mentors) - 31
Access to good examples of good practice in my subject area - 29
More training in using the VLE - 28
More training in using IT in teaching and learning - 27
More, or better, IT equipment - 22
Specially configured rooms to enable me to teach effectively with technology - 18
More training in general IT - 2

Time is a pressing issue. In the main, some staff were concerned about the amount of time required "to get [blended learning] right" (respondent from HeSAS). A bold solution was put forward by a member of staff at the Business School:

"I have been critical for some time that there is much talk but little action when it comes down to e-pedagogies. I now realise why - there is little time available for lecturers to make that change. The working week for lecturers should be cut by 4 hours/week in all cases to allow lecturers time to adapt. Of course, the 4 hours cut for each lecturer is subject to making efforts to apply e-technologies to the teaching programmes."

As you can imagine, there were a number of quotes I could have used on the lack of time, but the above offered a solution. Any thoughts to this?

There were variations in faculty needs. For instance, staff at Advanced Technology were more discerning about the physical 'kit' they were using (so 'more, or better, equipment' was ranked higher); at HASS, there was a greater focus on training; and at HeSAS working with enthusiasts (mentors) was ranked higher, compared to other faculties - "we need champions in departments that can spread the word" (HeSAS).

The underlying message from many of the accounts was that technology does not change practice, people do. Staff need the space, time and confidence to develop effective teaching plans or, to take forward the skills that have already been learned:

"I need to develop my skills in using a more blended approach. It's taken a while to get colleagues to use Blackboard for information purposes, but we now need to go up a level and develop more interactive approaches for students. I probably need to attend some staff development activity for this, time is a massive barrier to undertaking additional training though." (HeSAS)

Misleading Myths about Technology-enhanced learning?

2007-03-12T14:54:38Z

As noted in a previous message, upon embarking on this exercise, each institution was challenged to think about their notions of technology in teaching. I came across some notes I made when I did some research in the US in 2001. I wondered what you think of these? How do they stand up in 2007? Are they as 'misleading' in the UK as they are in the US?:

Technology saves money
Technology increases revenues
Technology cannot save money
Lifelong learning is for "them" not for us
Everyone can learn new uses of technology in the same way at the same pace
Most academics, administrators... will, on their own initiative, soon take advantage of most of the capabilities of new uses of technology
Technology can fully record and reproduce what happens f2f
Running a college or university is just like running a business
Running a college or university is nothing like running a business
The purpose of HE is solely to deliver knowledge to students
The purpose of HE is loftier than delivery of knowledge
The "digital divide" doesn't apply to "us". All our academics and students arrive equally well-prepared to use computers and related information resources
Support staff can be ignored. They don't want to be included in new initaitives involving technology
Most academics will resist accepting technical help from students
Effective collaboration is easy and everyone knows how
Institution-wide standards (picking one type of computer, one software, platform, one Web-based course management system) are easy to develop and enforce
All important decisions must be made solely or primarily on the basis of solid data
Most important decisions must be made solely or primarily on the basis of local politics and personal judgement based on experience and wisdom
Someone will enjoy the transition from current operations to a new institution-wide information system that seamlessly integrates administrative and academic uses of technology.

Blended Learning Survey Results - part 1

2007-03-05T14:32:55Z

During this Benchmarking Exercise, we administered a teaching staff experience survey at the end of last year. The headline data has been presented to the
Benchmarking Steering Group - I now look forward to sharing some of the findings with you.

Up to 118 questionnaires were received from staff across all faculties - most were full-time members of staff (88%).

1. Awareness of "the Vice-Chancellor's vision for Blended Learning" was high - only 13.5% were not aware of the vision.

As far as technology in the curriculum is concerned:

2. 69% of staff use Powerpoint "most of the time", whilst a further 21% use it occasionally.
3. Over 58% of staff used a data projector attached to a PC most of the time.
4. Less than 10% used an OHP, although over 61% used it occasionally.
5. Blackboard was used by nearly 74% of staff most of the time. It's use appears to be more widespread/embedded in the Business School.
6. Small pockets of Staff are taking advantage of other 'open source' VLEs (such as Moodle) - about 13% reported using this most of the time.
7. Over 70% of staff use video, DVDs or digital cameras in their teaching.
8. As many as 35% of staff have used other mobile devices, such as Personal Digital Assistants or iPods and MP3 players (it is embedded in 4% of practice)
9. Over 35% of staff have also used blogs and wikis (embedded in 3% of teaching practice)
10. As many as 43% of staff have used games and simulations in their teaching (embedded in 6% of teaching practice)
11. Video-conferencing is less embedded - over 72% have never used it in their teaching.

As far as the VLE is concerned:

12. 67% use it as a noticeboard most of the time, whilst a further 27% have used it occasionally this way
13. 49% have used it to post seminar themes and questions, whilst a further 32% have used it on occasions to post themes and questions
14. A majority of staff use the VLE to post lecture notes - over 79% use it this way: only 2% have NEVER used the VLE to post lecture notes.

15. 45% have used the VLE as a chatroom for discussion with or between students
16. About 48% have used the VLE to send feedback back to their learners.
17. 56% have used the VLE to post tests and conduct assessments.
18. 62% have used the VLE to track an individual student's progress.

There is a lot of food for thought - which figures catch your eye?

Organic Embedding

2007-02-28T14:51:23Z

This may seem a strange post but I have wondered if we are trying too hard to engage a majority in blended learning.

I am reminded that nature does not start large .... acorns and oak trees, single bulbs and clumps of daffs, puppies to dogs. Could it be that we are attempting to force-feed people into embracing concepts and technologies that it has taken a number of years to evolve.

Would it be better to concentrate on a number of key projects and utilisation within each of the faculties and allow student demands and academic research to provide the impetus for engagement?

I remember one of the Benchmarking team commenting in casual conversation a number of months ago that we cannot cajole people to take part in activities. The desire has to be there to both investigate and then adopt new ways of delivering learning and teaching, even a top-down institutional approach can be met with resistance.

I believe that it is through our own experiences, and the encouragement from others to explore new and interesting facets of existing roles, that we develop an interest in utilising new tools and technologies and this has to be considered to be evolutionary in nature.

But maybe, like nature, we need to bifurcate more often?

Transformation through staff development?

2007-02-28T14:09:58Z

First of all, it is good to be back from my hols! I look forward to answering some intriguing comments posted since I've been away. The benchmarking project is coming into the final straight and I hope this piece, along with others, offers the first of many reflective accounts of our experiences and ideas we have picked up along the way.

I noted with interest, the latest Investors in People review - this is a three-year rolling revirew. Within the scope of the benchmarking project, it shows that there is a strategic view of HR and staff development - how we make sense of reaccreditation and our attempts to embed blended learning is an interesting issue. One of the benefits of the MIT90s model is that it encourages institutions to benchmark themselves against institutional strategy (and sub-ordinate strategies) rather than against other institutions per se (which in my book is a good thing). MIT90s is a transformational model, rather than a crude cookbook. With this in mind, we have learned, that in embedding blended learning, for both teaching and support staff, transformation presents many challenges - it is uneven, continuous, it has to be endured and, one can even argue, in view of the latest technological developments (e.g. Web 2.0) transformation could be viewed to be gathering pace. I think, as we embrace transformation, faculty heads need to encourage staff to understand, create and cope with transformation as tensions, stresses and strains may become more evident: we may be able to project trends into the future, but we cannot predict the form of the challenges at the chalk face (sic.).

Staff development is an oft-repeated panacea to effective transformation. Here, at this university, staff can attend Blackboard and/or e-Moderator courses. Perhaps the IiP review is reflective of the broader strategic appraoch that has been favoured by this University? (Believe me, it was not to long ago that I was visiting institutions who seemed driven by short-term training events). The MIT90s framework implies that training provision ahould be determined by the short and long-term institutional goals and, upon reflection of our accompanying 'measures of success', training provision should be determined by standards for their achievement. The MIT90s model is allowing us to align staff development with strategic goals, but our staff survey has alerted us to keep analysing learning needs on a much more systematic basis and prioritise these within the wider faculty/institutional plan.

In my experience, staff often warm to initiatives and courses that are imbued with an explicit and meaningful purpose, have status, are co-ordinated and resourced adequately. It is propitious that the IiP review is being undertaken but I also think that the MIT90s framework has provided an extra impetus in this regard.

7 challenges - answers sought

2007-02-13T11:05:38Z

Derek Morrison of the Academy challenged each of the benchmarking institutions to embrace the exercise in a manner that stretched their existing ‘ideologies' of e-learning. I'm happy to say that I think the MIT90s framework did, in part, lead to discourse and debate (and one that I think can be extended to the end of the blended learning project next year). However, It has occurred to me that this exercise - or any other exercise that seeks to implement large-scale ‘e' or blended learning will need to counter a set of views (some, not all, have re-surfaced during this exercise). So, in the tradition of Radio 4's ‘Any Answers?' They are:

Lecturers don't have the time to take on anything new.
Lecturers don't have data projectors in all classrooms (or other appropriate learning spaces).
Senior managers fail to grasp the potential of e-learning or the VLE.
There is a new (hidden) cost to the institution for no immediate return.
We surmise that it may improve motivation but can't necessarily prove that it improves retention and/or achievement.
Not all learners have access to PCs/laptops or broadband at home.
It detracts from our main business - improving face-to-face teaching.

Reflecting on Smooth(ies) Operations

2007-02-07T11:27:39Z

Reflections on the Blended Learning Roadshow

I have been asked by many people over the last few days about the Roadshow and how I felt it went. I realise I had high expectations in terms of numbers attending (and attendance increased exponentially after the first day) but overall I was very encouraged by the variety of people that attended, from both academic and support areas and at a many different levels within the Institution.

Feedback from some senior managers was that the Roadshow would have most impact if we took it to each faculty in turn and created a more bespoke offering. I hope this is something we will be able to accommodate over the next few months.

For me some of the Roadshow highlights were;

Seeing every area within the Showcase engaged with a Roadshow visitor, answering questions or demonstrating developments on the last day.

A steady stream of questions within seminar sessions, that ranged from basic ‘how-to's' to requests for system amendments and technology-enhanced learning developments.

A large pile of follow-up forms with individual's requests for support.

The way the Smoothies folder (http://celt.glam.ac.uk/Support-Resources/?c=Smoothies ) was embraced and utilised - one attendee was seen to be making notes in it all the way through every seminar they attended.

Observing the effectiveness of a cross-departmental collaborative team approach working so well, with everyone in the Roadshow teaming demonstrating their commitment to the project.

Watching a steady stream of post-Roadshow meetings taking place where individuals have captured the opportunities that blended learning offers and want to develop and deploy this quickly within their own faculties.

I'm now excited for the possibilities of our second Roadshow on 7^th /8^th March on our Glyn Taff site. We will be able to reflect on what went well first time round, on what could be improved and deploy these changes to improve the experience for those who attend.

Welcome to E-xcellence!

2007-02-02T19:24:02Z

Two colleagues in our Benchmarking Steering Group alerted me to the EADTU's E-xcellence Model for benchmarking e-learning.

There are some established European distance learning providers nvolved in this, including the Open University. As Professor Terry Mayes asserted at last week's Academy Phase 2 Town Meeting, benchmarking e-learning is now beginning to take off internationally: it's quite rewarding to be part of an exercise that is prioneering this ‘scholarship.'

Enduring Change: Technological Management in Education

2007-01-31T15:18:06Z

Some years ago Bruce Carter (now a BELA consultant in the Benchmarking Exercise) and I produced a paper for ASCILITE. We questioned the veracity of tools and methodologies (especially those developed outside the realms of education). Going back to that article, we asked three questions, which I think have a bearing on our project - I'd be interested in any further ideas or thoughts:

(1) can methodologies developed for one education system ever be transferred to another?

(2) can prescriptive methods secure palatable and enduring change? and, having identified these
two questions,

(3) where does one start the process of change?

Answers anyone?

Towards an e-Learning Benchmarking Club for Wales?

2007-01-31T14:33:55Z

Last week, on the 25 January, Haydn and I attended a Town Meeting for ‘Phase 2' institutions. Phase 2 institutions are expected to commence the project in May 2007 and, according to the Academy, "is the final iteration of the Benchmarking Exercise within the 2005-2008 programme ."

The format of the day largely mirrored the event I attended in London last August: delegates had the opportunity to visit various stations to listen to the experiences of institutions applying the various methodologies. It was a great opportunity to share our experiences with institutions. I had a sense that staff were concerned by the resource implications and the strict Academy timescale. As we noted on several occasions we were very lucky to devote staff time in this activity. We cited the key benefits of MIT90s - from our point of view, it's flexible and focuses on interconnections of activity and strategy (and therefore stimulates an institution into dialogue and discourse). It's a transformation model, rather than a measuring model. To be honest, it's hard going to an event like this without meaning to do a ‘hard sell' on the methodology - which we didn't do. We even advised institutions that it may be worthwhile examining MIT90s before trialling any of the other methodologies (especially Pick & Mix, ELTI or the e-Learning Maturity Model). I think MIT90s will appeal to institutions who wish to establish their own benchmarking criteria - without grappling with interpreting criteria developed elsewhere and out of the context of their institutional culture.

I was extremely pleased to see interest from our fellow Welsh institutions: Newport, Trinity College Carmarthen, Bangor, Lampeter and Aberystwyth. Given UWIC's previous engagement in the Pilot phase, there are legitimate grounds for a Welsh e-Benchmarking Club. It's been on the agenda for some time now, but in the context of the ‘imminent' e-learning strategy for Wales, I think it would be propitious for such a club to take root.

Glenaffric and Us

2007-01-30T15:43:15Z

In my ‘reflections on Slough' you will have noted that the MIT90s institutions had the privilege to listen to the thoughts of Veronica Adamson and Dr. Jane Plenderleith from Genaffric Ltd. They, together with Professor Terry Mayes, are involved in something called EDSuT (I know - what a cumbersome acronym!) - they are now the "Education and Dissemination Support Advisers". In this capacity, Veronica and Jane will (and this is extracted from the paper they circulated to us in Slough):

(1) "Extract lessons for the sector at Programme level" - on this point, I have a lot more confidence in the benchmarking exercise. Veronica and Jane have been involved with some high-profile evaluations in e-learning, such as the JISC MLE Programme. Senior individuals don't want to be wading through piles of ‘evidence' (I've yet to meet a PVC who has read beyond an executive summary of a research report - no matter how erudite that report actually is): so I hope the lessons are extracted in a format that will allow, e.g. the likes of me, to present to senior individuals and committees.

(2) "Help institutions enhance their own capacity to evaluate themselves" - I think we are still coming to grips with mainstreaming the key elements and thinking behind e-College Wales into our existing blended learning plans. In many ways the focus has changed to a more neutral perspective of technology, but it would be useful to have a ‘critical friend's' perspective (especially as Veronica and Jane did visit Glamorgan during the ECW days)

(3) "disseminate their outcomes" - at the Phase 2 Town Meeting, Terry referred to the "scholarship (sic.) of benchmarking." I'm not sure what that is, or what it might look like. After Veronica and Jane left the 2nd MIT90s Cohort meeting, we talked about a symposium at this year's ALT-C. It might be useful to get their ideas on this and any other dissemination activities.

(4) "Complement the work of the support consultants" (in our case the BELA group of consultants)

(5) "Collaborate with the consultancies supporting the benchmarking methodologies and, when relevant, contribute to or attend the support events and activities" (see my comment relating to point 3)

(6) Enhance the capacity of the Phase 1 institutions (including us) "for dynamic reflective dissemination to the sector" - via the weblog network or other "institutionally-defined approaches to dissemination." As Haydn mentioned previously at the Phase 2 Town Meeting, we were lucky at Glamorgan because we already had a blogging culture. Veronica gave a useful insight of the different types of blogs last week - I wonder how they can help us improve the reflective component of this exercise.

(7) "Support and encourage a culture of reflection and quality postings in the different projects' blogs" (see above)

(8) "Help institutions wishing to proceed to Phase 1 of the funded Pathfinder Programme: manage their expectations; refine their initial ideas; focus on the transformative nature of the Programme; recognise the importance of outcomes which are of benefit to the sector as well as the institution; and consider how institutions can best enhance their capacity for participation in the Programme." We are not eligible for Pathfinder funding (as we are a Welsh institution), but we need to keep lobbying HEFCW and the Academy (Wales): it will be a shame to lose the momentum at this stage, when so much has been achieved. I know both Veronica and Jane have studied and scanned the e-learning policy developments in all four countries (they were also deeply involved in the HEFCE e-learning strategy), and I also know they are well-connected in Welsh policy circles - so any advice would be most welcomed by the team. By the way, I noted with interest, Veronica and Jane's use of the word ‘transformative' - it is a term favoured by the Scottish Funding Councils in their e-learning Transformation Projects. At last week's meeting at the Academy, Veronica noted that the sectors have diluted their vision of ‘embedded practice' to a (more realistic) ‘transformative' perspective. Perhaps we have all had a rose-tinted view of developments so far?

I look forward to seeing both Veronica and Jane soon and also to their advice as we seek to build on what we have achieved.

Are you a smooth(ie) operator?

2007-01-30T13:11:43Z

If you haven't been to the Blended Learning Roadshow DO GO. It's a great opportunity to see what is available ‘on show' - and developed by our extremely talented ISeLS e-Support Team. When I attended the Phase 2 Town Meeting at the Academy last week, I took with me a couple of the Smoothie Folders. The Academy and other institutions spent ages poring through the guide. It was interesting to note that many institutions do not have the capacity to develop what we have done at Glamorgan: i.e. they may have technical support staff, but they haven't invested in people with a creative flair (as we have done). It's interesting that many institutions just don't have teams with the same skills mix as we do.

Disseminating and communicating (in a creative way) is the precursor to embedding technology in teaching. It is only by seeing (and playing) that you really get a sense of what might be achieved for your course and your learners. On show are the projects developed within Glamorgan but there's also an opportunity to see developments elsewhere, which really fascinated me yesterday (see Jacqui Neale's presentation on "Typologies of Learning").

The Roadshow is critical to the benchmarking exercise. One of our criteria focuses on the way we disseminate and communicate with staff. If you haven't had the opportunity to go - take the plunge and have a smoothie! (It's on in the LRC today and tomorrow)

A Postcard from the Edge: Reflections of the 2nd MIT90s Cohort Meeting

2007-01-29T15:34:20Z

Haydn and I attended the 2^nd MIT90s cohort meeting hosted by Thames Valley University in Slough. It was a well attended meeting with colleagues from all universities, including a large delegation from TVU (it was good to see so many senior staff engaging with the benchmarking activity).

As we discovered at the first meeting, at Glamorgan, the most positive aspect of the cohort meetings is that they allow each of the institutions to ‘touch base'. So here's a quick resume of an update provided by each of the institutions.

TVU activity has progressed quickly since we last met. Andy, Tad and the team have really impressed me both in the quantity of evidence they are collecting and the fact that they have ‘secured' people to work on the project. I know, from my experience, that this is an extremely testing and time-consuming activity - so getting bodies TO DO the work is critical. It was interesting to note, too, the administration of the TVU staff survey: we informed our colleagues at TVU that it was important for ‘other' KEY staff (e.g. heads of learning and teaching, to take ownership of the survey). In our case our Heads were responsible for distributing the URL which, I think led to a higher than ‘normal' response rate. TVU do plan to administer a paper-based version of the survey and conduct focus groups with staff which will be extremely challenging to analyse within the timescale. For me, the most critical issue was raised by Tad who cited the difficulty of "mapping" information to the criteria created and ‘over-generalisation'. For me, the MIT90s model allure was that it allows us to create an impressionistic picture of the development at the University, rather than deriving a ‘picture' from criteria developed elsewhere with metrics which would mean very little to us. However, whilst I have been putting together the Glamorgan benchmark, I must admit, I am having concerns with the content and tone, rather than the shape of the report.

Perhaps the greatest leap forward was taken by Bradford. As you may have noted from a previous message, a blog has been established by them (see the Important Links section to the right). Peter's tactic is an effective one - i.e. make sure benchmarking is on the agenda of the key committees. I was intrigued by the type of discussions and discourse taking place - what is learning? What is appropriate for the learner? What is appropriate for the lecturer? If using ‘blended learning' - what is the blend? I am pleased that our initial criteria were of help to Bradford as Peter circulated a draft set of criteria to the group.

Mark O'Toole is leading the benchmarking activity at Brighton. He informed the group that his main efforts had been diverted towards defining the criteria (which we have read with much interest). Mark's tactic has been interesting. He is certainly very practical in his approach - e.g. being focused on issues such as what ‘x' measures? And ‘what ‘x' actually tells us. He is looking very much from the perspective of using existing data sets and "naturally recurring data" and, especially, synergy with their JISC-funded IT Governance study and also HEFCE's ‘Measures of Success'.

After the round-ups, we were treated to a presentation by Veronica Adamson and Dr. Jane Plenderleith from Glenaffric Ltd. I wish to deal with Glenaffric's contribution to the Benchmarking Exercise separately (in another blog posting - so please bear with me).

I thought the afternoon session would give us an opportunity to compare criteria (as billed on the agenda), but it didn't quite live up to that promise. I must admit, and not for the first time, I was a little confused by the focus of the afternoon activities. We seemed to be talking about the structure of the final report and how each report would ‘look like' against each other. For me, this wasn't a pressing, or insurmountable, issue which seemed to take an inordinately long time to resolve - if indeed it was resolved? I think the consultants were looking for commonality in the final reports and, if so, I wasn't too comfortable with this. As Tad pointed out, one of the best features of MIT90s is that it doesn't tie you down to pre-existing criteria: given that we interpret and develop criteria in different ways (to reflect individual institutional culture and practice) I felt that what (I think) we were asked to do, impossible.

Thankfully, Peter saved the day! He stepped in and offered a very simple report structure which I, and others, seemed happy with: INTRO - Context, Reason for MIT90s, How we Did it; MAIN BODY - Experience of using MIT90s; NEXT STEPS - implications; where we are going; future processes; APPENDICES - Criteria, Results, Calendar, Mini Case Studies.

A large portion of the day was spent on the imminent Pathfinder Expression of Interest. Of course, Wales is, as I've mentioned previously, not eligible for Pathfinder funding but it was interesting to note the very tight timescales - 2nd for the EOI and, just as we get ready to go to the 3rd MIT90s cohort meeting at Bradford on the 16 February - a full proposal! (WOW!)

We briefly talked about raising the profile of the activity we are currently engaged in and noted the looming deadline for ALT-C papers. Colleagues warmed to the idea of a symposium on the MIT90s use in this, and other, e-learning related projects.

So, I drove back from windy Slough with further thoughts and ideas. I definitely see the 'end' in sight for this project but wonder how the project will look after April 2007?

York - home of the Vikings, the Terry's Chocolate Orange, the LNER 4468 Mallard... and The Academy!

2007-01-24T20:01:17Z

Haydn and I will be going to York tomorrow to attend the Academy's Town Meeting for institutions interested in joining Phase 2 of the Benchmarking Exercise

Tomorrow's programme mirrors the programme of the London meeting in August 2006. The Town Meeting is billed as an opportunity for institutions to probe Pilot and Phase 1 institutions' experiences with their benchmarking methodologies.

I've produced a briefing paper, which is actually a reflective account of our journey since September. I'll ensure that these are in the Project Resources. It will be a fascinating, but challenging, day and it will be interesting to ascertain the priorities of the ‘interested institutions' and also their general impressions of the MIT90s model.

We will be fielding questions in a series of 'carousel' sessions - but what can you leave out in 10 minutes? As Mark Twain said "All generalisations are wrong, including this one!"