• Firstly for JISC, were we are co-authoring a Digital Infrastructure Directions Report on activity data for senior decision makers in the HE sector.
• And secondly, we are collaborating with Sero to contribute to the CETIS Analytics Landscape Study.

More about these two projects as we start to develop our content.

Events today made me realise that we never blogged about FishDelish here, a project that we performed June 2010 to July 2011, in partnership with the University of Manchester’s School of Computer Science (UMCS) and the FishBase Information and Research Group Inc (FIN), a not-for-profit NGO.

Rather than repeat ourselves, the project aims, technologies and results appear on the FishDelish blog in the UMCS domain. Please head over there to check it out, but warning, the blog is hosted on a virtual machine that may at any time be rolled out from the fishdelish server, so the first page load may take a few seconds.

In that project we learned a whole lot about triples, linked data, triple stores and SPARQL while polishing our skills in various LD-related technologies. There remain several innovative ideas that we want to exploit in the future, including the FishOMatic architecture and advances in the area of active essays [1] which we would, sometime in the future, like to use in improved pedagogic practice.

We could not have done this project without help from Rainer Froese (IFM-GEOMAR and FIN) and Maan Bimbao (FIN) and their bravely open licensed data in FishBase, and our good friends Bijan Parsia (UMCS) and Sean Bechhofer (UMCS), both of whom know approximately a million times more about linked data and semantic technologies than we will ever know!

[1] cf Alan Kay, to whom, together with the rest of the PARQ Smalltalk group, a huge hat tip from Smalltalk programmers everywhere….

As the major part of that work we designed and developed a very clean user interface to replace Jorum’s rather arcane and frustrating user interface. The new user interface was implemented by the provision of a new front-end webservice to handle all user interaction including new navigation and discovery features. The new front end was  implemented in Ruby on Rails, and used the DSpace REST and SWORD APIs to communicate with Jorum’s DSpace core.

Our work also entailed considerable improvement of the open source DSpace REST API, where we performed testing and refactoring to support our new front end functionality. This involved complex work in Java. Other Java-specific work performed for Jorum involved modifying DSpace’s OAI-PMH interface to allow ingest of Open Educational Resources with specific kinds of open licences from other repositories.

We have some regrets about this contract for Jorum because, due to a need to perform unscheduled work on the REST API, Jorum’s funding  ran out before we could transform our clean user experience into the phenomenal end-user experience that we wanted to implement. This early end was always acknowledged as a possibility, but nonetheless we are pleased to note that, when rolled out, a clean user interface and accompanying user experience will be a vast improvement for Jorum.

Despite the contract ending, we did some pro bono work for the service post-contract purely because we like Jorum and want to see it succeed. This extra work was in four areas: (i) a final refactor of code and tests to ensure clarity (ii) Rails skills transfer, (iii) a bit of tricky Rails development in a separate branch of the front end code so Jorum could later rapidly proceed with the development of some more advanced features they want, and (iv) performance and load testing of the new composite system (report to be published in a later post). As a result of the latter we confidently predict that the new Jorum will handle many times its current load.

We look forward to seeing Jorum rolling out our work in the near future.

Announcing that we’ve started a developer blog for notes on the technologies we use, experiments we have performed, etc.

One post is on Ember.js , which we are using in a current development project.  As part of that project, we have recently open sourced four gems that we are using to help us deploy Ember in our Ruby on Rails technology stack.

To date, these gems have been downloaded at an average rate of 35/day, with, to date, 467 downloads. That’s good .

I enjoy my role as a member of the Advisory Board for the HEA funded Enhancing Fieldwork Learning project. Today was no exception; our most recent meeting was held at the Field Studies Center at Preston Montford in Shropshire, and offered the opportunity of a post-prandial stroll round the Center’s 12ha grounds.

The project is concerned with discovering and promoting improvements in fieldwork-based pedagogy, in particular as enabled by relatively cheap and available technology. Today the use of mobile phones, Livescribes, iPads and others were mentioned. While I’m not sure about iPads as being relatively cheap, cheap tablets will be upon us soon enough.

So how might technology transform fieldwork? An example is provided by Welsh, France, Whalley and Park who discuss the transformational advatages of geotagging photographs: Geotagging helps to provide an extended (post-fieldwork) space that can be used as the basis for reflection and, through that, more effective learning:

“Taking photographs also allows a student to revisit areas of the landscape post-fieldwork, thereby extending their learning space and allowing for further reflection outside of the landscape. Geotagging photographs can add an enhancement to this as students can associate the photographs with different parts of the landscape and potentially identify spatial patterns as they have the location information associated with that photograph. This is particularly useful for post-fieldwork reflection as the spatial information can act as a reminder of where the photograph was taken which may help to further interpret the landscape. If students annotate their geotagged photographs and add further reflection on the photographs post-fieldwork, this technique could compliment reflective fieldwork diaries (e.g. McGuinness & Simm, 2005) which themselves encourage critical reflection and can facilitate deep learning.”

The context in which this kind of learning occurs is interesting, since fieldwork is concerned with, inter alia, the following

Development of observational, classificatory and analytic skills
Development of teamwork skills
Facilitation of experiential learning
Encouragement of student responsibility for their own learning
Provision of a taste of ‘real’ research and, for some, a taste of future work
Kindling of a respect for the environment
Development of personal skills
Increased interaction between staff and students
[drawn, with two very slight adaptations, from Park]

Of course, this is all very interesting from a Personal Learning Environment perspective, where there are common aims, to help enable reflective and peer-assisted self-directed learning.

As always, a valid question is ‘how do students acquire the self-awareness, analytic skills and knowledge to become self-directed learners who refine and improve their learning practices?’

Today, fieldwork practitioners were mentioned as change agents. Thinking about this, it is apparent that fieldwork provides a great environment to practice the mantra of change agents, “live the change”, and engage with students to assist them to become self-directed learners who increasingly, as they learn, adopt the practices and identity of practitioners in their domain.

We also spent time discussing how to gather evidence of the impact of new practices on learning and on student satisfaction. I look forward to seeing the results of this new phase of activities in the project.

Thanks then, to the project team for organising today.

But quietly and unannounced, Hedtek has been expanding over previous months.

First we employed Ashish Sehra and more often than not, contractor Danny Ghilea. They both contribute agile developer skills in the Rails and Java areas. For our future development we also look to Ash’s past mobile experience. More recently Apache Software Foundation Member and friend Robert Burrell Donkin joined us. Robert brings twelve years of heavy-duty concurrent Java skills and contributes a very strong agile focus to our already very agile team. At the same time we appointed Alex Tse as a salaried Intern.

So now together with Hedtek staffer David Workman, we have a superb team of developers. A dream team, in fact.

All to often sudden growth like this destroys development companies, or leads to gross inefficiencies in development practices as the new staff members dilute the good habits that the company previously possessed.

We have been very careful about sudden growth, to the extent of waiting more than two years for the people who would fit with Hedtek’s development culture. As such, we were looking to employ people (i) who we already knew and had worked with and (ii) who have the key skills and the right kind of enthusiastic team-oriented personality to fit with us.

With the exception of our Intern, Alex, who comes well recommended, we have all worked together previously. We already know what to expect of each other; we all subscribe to very similar and compatible ways of working; and we all strive to help each other and apply continuous improvement practices.

It is this mutuality of experience and practice that has enabled smooth, effective and rapid growth.

Of course, we would be nowhere without our administrator and office manager, Renata Vitovjakova. Renata gets the prize for being the world’s most efficient person. Particularly, we suspect that she concurrently runs several kanban boards dedicated to administration and finance in her head.

Finally, we have Anita Workman working part time as a user-level and acceptance tester, and use Anita’s carefully honed skills to good effect in providing an independent opinion on the usability of our software.

It feels like Hedtek has been blessed

Past contracts have been

1. A small contract to provide assistance migrating the Jorum from Edina to Mimas.
2. A larger contract to provide temporary technical direction, diverse technical support, and to identify and start work on a way to be able to add front-end services to Jorum and upgrade Jorum’s DSpace core.

The third contract is

3. To supply technical direction (now handed over to a new member of Jorum staff, Ben Ryan), to supply diverse DSpace technical support (e.g. we are deep in DSpace’s OAI-PMH import code right now), and most importantly, to build a new front end for Jorum, while also ensuring sustainability, thatJorum technical staff have worked on and can continue to work on the new front end.

We hope to demonstrate this front end at the next (early Feb) Jorum Steering Committee Meeting; demonstrating resource discovery by browse and search facilities, resource pages, resource content download, and deposit of new resources.

Having Jorum as a client is great: The work is interesting and enjoyable, and Jorum staff are great people to work with.

Also, for me (Mark) it feels a little like coming home; some of the work that we are doing feels like a continuation of things I identified in a previous small consultative report for Jorum.

Achieving some rapid gains in user experience

We’ve been doing some mods to the existing Jorum/DSpace user interface to tidy up some aspects of that interface. This, however, can only be only a partial fix: Beyond making a few simple but effective mods to the existing user interface, it rapidly becomes becomes too expensive to modify further.

Providing a new Jorum user experience

The most cost effective way of providing a better user experience is to start again.  The preferable approach is to use the DSpace REST API to enable the use of a new front-end service for Jorum. This architecture is shown in the first diagram.

So far, Jorum’s user-knowledgeable staff have been acting as user representatives in collaborative evaluation of incremental deliveries with co-design of improvements for the next incremental delivery. Our plans are then to provide a public beta, enlist test users, and provide a web-based user feedback mechanism to involve beta users in progressive enhancements.

As it stands, the new front end already offers more useful functionality than is available via the current version of Jorum.

Opening up the Jorum architecture

Proving an API also allows the use of specialist front ends with Jorum, illustrated in the second diagram.

This is great, because Jorum does get approached for and wants to support this kind of (specialist) functionality. In effect, the API opens up the use of Jorum at an architectural level, enabling others to create their own front ends to access and contribute to Jorum’s content.

Notes 

Under the hood

The work here has been to extend the Jorum/DSpace architecture.

In order for a front end to obtain data from Jorum, the DSpace REST API was retrofitted to to the version of DSpace used by Jorum. The read side of the API was unit tested to ensure correct operation of the API.

Deposit into Jorum  is via DSpace’s SWORD 1.3 API, authenticating using Shibboleth.

The front end service is being built using Ruby on Rails and client-side technologies that include HTML, CSS3 and JavaScript.

Jorum is built on the DSpace repository, and part of our work involves building a new front end to Jorum. For this we need an API to DSpace, and while, conveniently, there is the DSpace REST API [1] module available for DSpace, it has not been used with the version of DSpace that Jorum uses, and until recently, it had no automated tests available for it. Given the centrality of this API for Jorum’s future development, we have started developing a suite of automated tests for the API. This post discusses progress, and mentions where our tests can be found on github.

Because the first front-end work we intend to perform is for OER resource discovery and download, so far we have only focussed on the ‘read’ functionality of the API. For this we have tested endpoints (URLs) that focus on read capabilities for:

• communities
• collections
• items
• bitstreams
• searching
• metadata harvest

For all of these, we wrote integration tests that tested the API endpoints running on DSpace,and then re-ran the tests on a full Jorum build.

The test development process

To implement the tests, we had to create a framework that would run the DSpace REST API using Jetty and control this programmatically.

We also needed to load database fixtures in order to have a known DSpace state to test against. When we started writing tests, loading SQL fixture files was problematic. We went through several iterations where, using a given means of loading our fixtures,  all our existing tests would run, but frustratingly, for the very next test we tried, our fixture load mechanism would fail. Eventually we discovered that the appropriate way of loading a fixture is via the DatabaseManager class in DSpace.

We also needed to trace a bug in the DSpace REST API code where database connections weren’t being closed. This bug caused intermittent freezes during our test runs, which in turn made a stable test framework impossible while the bug existed.

Later, fixture creation became quite intricate when we needed to test file downloads were correct, as this involved finding and editing download directories in the data. This was solved neatly by the creation of a command line script that generates a database dump from a DSpace database, processes it, and replaces all the download paths with a Maven token that can be replaced with the actual file path location when the tests are run.

To test search, we also needed to create and load a Lucene index for DSpace. This was primarily done by using DSpace to construct a more intricate test fixture for the search actions and then taking a copy of the Lucene index from the DSpace installation. The index is then copied into place for use by the embedded Jetty server during the test run.

The tests themselves do a lot of validation of JSON structure. This is done with the aid of a simple JSON library which processes the JSON into JSONObject and JSONArray instances. This was a good starting point and was used to write the first few tests. As more tests were written, there was a lot of messy test fixture matching going on. To reduce this, we refactored our tests and started building up a set of matchers and matcher methods that allowed us to write tests much more quickly. The end result of this process was the beginning of a DSpace-specific test matcher library. This process could be taken further, but the stage we got to was enough to enable us to write tests quickly and with a minimum of effort.

Running the tests

The tests are built and run using Maven, so if you want to run the tests yourself (e.g. on a new DSpace release), you just need to get a copy of the REST API with our tests added, available at github.

Once you have the code, you will need to create a postgres database for the test run. This is aided with the script ‘create_integration_test_db’. Once the database is set up, you can run the tests with the command ‘mvn test’.

Here is the end of a successful test run (please click on this thumbnail image to see the output detail):

There is a lot of ‘noise’ on the output of a test run. This is unfortunately caused by various parts of the REST module that are logging directly to the console, rather than using a logging framework. Hopefully this will be fixed in future development on the module.

[1] We would prefer to call the API restful, since it does not follow the HATEOAS principle. For further details see any good reference on REST.

So as a record, simply a list of highlights of some of the things that came out of the sessions:

Learning

Group work as learning

Content creation as a means of learning

Active learning

Learning through teaching

Experiential learning through problem finding and problem solving

Processes

Process creation for learning processes

Self-organised structure

Peer support

Curriculum

Self-designed curriculum

Community-responsive curriculum

Accreditation

Self accreditation in a public manner