Metadata, Not E-Books, Can Save Publishing

O’Reilly TOC—Tools of Change for Publishing   •  July 29, 2010

Needles in haystacks. “E-books will not revolutionize reading, nor will they change the content,” says metadata enthusiast Nick Ruffilo. Reading on a screen rather than paper will not create new markets, but better metadata on each book could allow readers to find more of what they like, regardless of format: “If every book had this data, you could essentially have an eHarmony for books. You fill out a small profile of your likes and dislikes and now are shown a much smaller set of books to choose from.”

Not surprising but interesting to hear coming from somebody who was an early innovator in recommendation systems. It also confirms what we’ve known for a long-time about library descriptive data—it’s a crude tool for the task of getting people more of what they want. Our “tags” are tough to mobilize for that purpose. For a really good effort in this regard check out WorldCat Genres from my colleague, Diane Vizine-Goetz and her team in OCLC Research.

(Michalko)

Mao, King Kong, and the Future of the Book

Triple Canopy   •  Issue #9

A trip down memory lane. Bob Stein, founder of the Institute for the Future of the Book, reminisces about his days with LaserDiscs, HyperCard and CD-ROMs.

Read this. If you weren’t around when these were happening then you’ll be amused and amazed at what Stein and others were doing (did Alan Kay really draw those pictures of an “iPad” in 1967!?) and the extent to which we stand on their shoulders. The reminder about Laurie Anderson’s “Puppet Motel” was particularly laden. I remember having that and thinking it was brilliant. Of course, it’s now unobtainable and likely unplayable. The music without the immersive, weirdly unsettling environment can be purchased, and you can see the Voyager demo for the CD-ROM.

(Michalko)

At the Spring CNI Taskforce meeting last April, Karen Wetzel (Standards Program Manager at NISO) announced a new piece of work related to “supplemental materials” in journal articles. In the scientific literature, it is not uncommon for articles to be accompanied by a secondary set of figures, data, documentation of experimental protocols that aren’t considered part of the core content. Karen reported that thought-leaders from a variety of sectors had expressed concerns about the expense that publishers incur in managing this material, as well as the additional work that it creates for editorial staff and authors. Libraries were included in a long list of potential stakeholders, as potential curators of this supplemental material.

A central concern is that scholarly citation and reuse of this kind of supporting material is limited by the absence of identifiers, bibliographic metadata etc. This is especially true for disciplines that lack official data centers that might provide DOI registration etc. to support discovery, preservation and even re-discovery services. As Sasha Schwarzman of the American Geophysical Union observes, this has important consequences for the longevity of the scholarly record:

while the main article is going to enjoy eternal life with many reincarnations along the way, supporting material is likely to rot and die, with very little possibility of resuscitation

[Alexander (Sasha) Schwarzman. Supporting Material. 2 November 2009]

There are some equally important operational and economic considerations. From an editorial perspective, there is concern that the “core” of an article may be buried in supplemental material, which is not always subject to rigorous peer review. From a publisher perspective, there is a reluctance to assume the costs of managing, marking up etc this content. From the researcher/author perspective, there is concern about how contributions are credited (e.g. first author on supplemental material v. fourth on the article) and confusion about standard formats for submission of supplemental materials.

In spite of all this, the trend is reportedly toward an increasing burden of supplemental materials in the journal literature. The Journal of Clinical Investigation (0021-9738) — which charges authors a supplemental fee for supplemental materials –- reports that the percentage of article submissions accompanied by secondary figures, data etc has been rising steadily in recent years:

[Report of the Roundtable on Best Practices for Supplemental Journal Article Materials, Co-Sponsored by NFAIS and NISO, January 22, 2010]

I do not know what the outcomes of the NISO/NFAIS effort in this area have been thus far. Several working groups were to have been convened this summer, so presumably the work is still in course. Meanwhile, I was interested to see that the top-ranked Journal of Neursocience (1529-2401) has formally announced that it will no longer accept supplemental materials with article submissions. Henceforth, non-essential figures, tables, code libraries etc will be referenced in a footnote linked to an external Web site. Data sets and multimedia files will be accepted as part of the article content, but everything else will have to be hosted externally.

The Journal of Neuroscience points to the rapid increase in the size of supplemental materials submitted with manuscripts as a factor in its decision.

[”Announcement Regarding Supplemental Material” The Journal of Neuroscience, August 11, 2010, 30(32):10599-10600]

Optimistically, one might say the problem will resolve itself: aspiring authors will be forced — by editorial boards — to do a better job of writing articles that include all the necessary data and figures without recourse to voluminous supplements. Elegance and concision will be restored in the scientific literature, etc. But I would think this might still have knock-on effects for research institutions and perhaps libraries, as responsibility for managing those external websites will likely devolve to place where the work was performed.

As publishers like the J of Neuroscience renegotiate their role in scholarly communication and retrench their investment in the downstream products of research, universities will need to allocate more resource to managing the upstream processes and by-products of scientific/research practice.

This struck me as an instructive example of the “attention switch” that Lorcan Dempsey and Jim Michalko have talked about recently (here and there), viz., a change in the traditional library effort to bring the outside in by acquiring published content, to moving the inside out — taking greater responsibility for the stewardship of locally created research, teaching and learning materials.

In 2007-2008, the Digital Library Federation (DLF) convened a Task Group to recommend standard interfaces for integrating the data and services of the Integrated Library System (ILS) with new applications supporting user discovery. The group produced a report with recommendations in December 2008. After that not much happened.

In February 2010, at the Code4Lib Conference, Karen Coombs (the OCLC Developer Network manager) and I brought together some of the people who had been on that task group as well as other interested parties who were at the conference to take this work to the next stage. At this ad hoc meeting we agreed that we were ready to take this work to the next stage. The next stage, we felt, was to actually create a middleware layer that we could collaboratively maintain.

This middleware layer would allow discovery interface systems like VUFind, Blacklight, WorldCat Local and others to write standard code for interacting with an ILS. The middleware would translate that standard request into what a particular ILS would understand, send off the request, parse the result and pass it back through to the requesting application. As you might imagine, this is very much more difficult than it sounds. While some ILSs support a standard interface such as NCIP, others do not. In some cases, “screen scraping” (extracting meaning from an HTML page) may be required.

Also, whenever a vendor changes their ILS it may break the connector that interacts with that ILS. Therefore, the work of creating (and more importantly maintaining) these connectors is best spread around a large community of developers. Thus, the collaboration. The present collaborators (anyone can join us) include: OCLC (WorldCat Local), Proquest (Summon), the eXtensible Catalog (XC) Project, Talis, VUFind, Blacklight, and several university libraries. Emily Lynema from NCSU, a member of the group, kindly offered the Google Groups site that had supported the DLF ILS-DI work as the working platform for this further work, which we gladly accepted.

Over the intervening months we discussed how to proceed. The primary options on the table were to build out Jangle or adopt the eXtensible Catalog (XC) NCIP Toolkit. Eventually part of the group moved to adopt the XC NCIP Toolkit as the infrastructure we would work with. Just yesterday we announced that OCLC would contribute NCIP 2.0 code from our Web-scale Management Services codebase to expand the capabilities of the XC NCIP Toolkit and bring it into compliance with the emerging 2.0 NCIP standard.

Meanwhile, we have submitted a proposal to talk about these developments as the Digital Library Federation Fall Forum. If you want to hear about this work, make your wishes known via the DLF Community Discussion Forum. You can request access to this by sending an email request of this nature to DLF at clir dot org.

Those of us working on this effort feel like we may be close to busting open our ILS silos — not just for us, but for everyone who can write to one standard interface. If that is you, then join us. We can use the help, although we will keeping plugging away at it with or without you.

Image courtesy of erikref, Creative Commons Attribution 2.0 Generic.

This is the last in a series of posts on the Leadership through Collaboration Forum and the thinking that went into structuring the agenda. Before I conclude, I’d like to acknowledge that creating the forum agenda was a collaborative activity in and of itself - we’re grateful in particular to our host (The Smithsonian Institution), and to all of the RLG Partners on the planning group who contributed (you’ll see them listed at the bottom of this page). Additional support for the event came from the Institute of Museum and Library Services and the Gladys Krieble Delmas Foundation - thank you!

Some final words on the collaboration contexts: It is important to remember that no one of the collaboration contexts (local, group, or global) is inherently better than the other. Each provides the appropriate framing for solving different types of issues. Within any of these three contexts, the collaboration can be very shallow or very deep.

Figure 1: The Collaboration Continuum (introduced in “Beyond the Silos of the LAMs: Collaboration Among Libraries, Archives and Museums” [pdf])

Moving along the collaboration continuum, collaborations which express themselves as contact, cooperation or coordination are additive; they foster a working relationship among partners, yet remain distinct projects easily separable from the core functions and workings of the institution. Such collaborations do not impact how an institution organizes itself and its workforce. Deeper collaborations trend toward convergence, a transformative process that eventually will change behaviors, processes and organizational structures, and leads to a fundamental interconnectedness and interdependence among the partners. In transformative collaborations, participants find efficiencies that free up time and resources to focus on the things they do best. At the extreme end of the continuum, convergence in a specific area may turn into infrastructure: a service that is so deeply embedded into our everyday life that it becomes visible only when it breaks down. You only think about who hosts your e-mail, or where your electricity comes from, when the service is interrupted.

The stages of contact, cooperation and coordination on the collaboration continuum are likely the prerequisites for reaping the benefits of deep collaboration and convergence. Within each of the local, group, and global collaboration contexts, additive or transformative relationships can emerge. For both the collaboration contexts and the stages of the collaboration continuum, each setting provides unique benefits and drawbacks. Finding the appropriate collaboration context for a given challenge, and building relationships along the continuum so all parties derive the maximum benefit, are hallmarks of successful long-term collaborations.

Our last panel of speakers during “Yours, Mine, Ours: Leadership through Collaboration“ focuses on global collaborations:

In this segment of the forum, we acknowledge all of the activity which has already gone into collaborations benefiting the entire community. However, we also feel it is time to take a step back and re-assess whether our current behaviors in creating shared aggregations, standards and tools are serving us well in meeting user expectations at the network level.

Here’s the background:

Global Solutions - Common Values
“Things work at scale because the community subscribes to the same values.”

In local and group collaborations, institutions and their interests are at the forefront, and the collaborative activity is predicated on the direct local benefit reaped. A collaboration guided by common values introduces a notable paradigm shift. It does not put the institutions first, but rather focuses on the intended audience and what that audience expects us to deliver.

While any type of collaboration can be fueled by common values, including those circumscribed by institutional boundaries or specific group interests, value-based collaboration emerges as a survival strategy in the global networked environment. Ultimately, we all serve those who want access to our information, increasingly in digital form. Collaboration around values is driven by a shared vision which allows an entire community to respond to challenges in a consistent manner, and invisibly aligns all of us in an effort to realize a shared vision. In this context, the emphasis shifts from managing the collaboration to addressing the shared values. The sphere of common values collaboration includes standards, policies for copyright and data aggregation, the commons and open data movements, and the vision of Linked Data.

While common value collaborations may have the lowest direct overhead (parties do not have to remain in constant and carefully orchestrated communication to remain in sync), they may also be the most difficult to sell to parent institutions, which generally pay their employees to work on local issues. The institutional benefit of such collaborations is less tangible since they raise all ships. As a matter of fact, in some instances common value behaviors may be perceived as threatening local goals, such as policies and technical protocols for making institutional content freely and openly available in many different venues.

At its best, applying global values that make things work in a larger context in group and local settings ultimately prepares those institutions for the opportunities of the networked environment. There’s benefit in thinking globally and acting locally.

In the next (and final) post in this series on the collaboration contexts and how they’ve shaped the overall structure of the forum, we’ll revisit the popular collaboration continuum, first introduced in the Beyond the Silos of the LAMs [pdf] report.

A few weeks ago I listened in on Karen Coombs TAI-CHI webinar “OCLC Web Services in action” (you can find the slides here, and watch the webinar here). Even though I work at OCLC, I’m not always as up to speed on what’s going on as I should be, so I was quite amazed at just how many web services are available, and the variety of use cases.

I was particularly gratified to see the Terminology Services being put to use in several different prototypes, since Günter, Diane, Andy and I pulled together our Strawman document and held a meeting on uses of termonolgies way back when. All of the prototype work has been done around our top vote getter, which was “leveraging terminologies for search optimization.”

I particularly like this example, from Demian Katz (via Karen Coombs) which shows how VuFind could be used with WorldCat Identities and OCLC Terminology Services to provide users with suggested terms.

Terminology Services is quite experimental at this point — Karen and others who work with our Developers’ Network are looking for feedback in order to gauge the community’s level of interest in the Terminology Services before committing the resources to make it a production service. So if you have ideas about how to put the power of terminologies to work, I hope you will give it a whirl!

Other examples of use of the Terminologies Service are featured in the Application Gallery. You can also view other offerings in our TAI CHI webinar series.

Over the past few years, OCLC Research has done quite a bit of analytic work based on what my colleague Brian Lavoie refers to as “supply-side” data. Examples include the well-known Google 5 study, as well as a variety of projects examining the library long tail, several of them summarized in an article Lorcan published some time ago. Much of this work has been based on data aggregated in the WorldCat bibliographic database. These data have been contributed over many years by OCLC members to support a variety of shared library services, including cooperative cataloging and inter-lending operations; as a secondary effect, the aggregation has provided a rich source of information about the system-wide library collection that is regularly mined in both internal and extra-mural research projects.

More recently, we have begun to think about how we might make better use of the demand-side data that is generated by a variety of routine library operations, especially circulation and inter-lending.  Lorcan in particular has given thought to how “intentional data” might usefully shape library service provision.

Inter-library loan transactions are a particularly interesting example of intentional data, I think. First, because while local circulation of academic print collections in North America is generally observed to be on the decline (a trend is borne out by a decade of ARL statistics), inter-lending traffic in this same population is increasing.  We discussed these trends some years ago at a program organized by OCLC Research.

Secondly, aggregated ILL data provide us with a view into the system-wide demand for titles for which local fulfillment options are inadequate. This is important in the context of local and system-wide planning, since it reinforces the value of locally managed inventory in the larger library network: local demand is not the sole measure of a collection’s function or worth.  Finally, inter-library lending is a heavily subsidized activity for which the direct costs are rarely appreciated or quantified. As the volume of ILL transactions increases, and local circulation diminishes, we can expect to see closer scrutiny of the “ROI” on operations that support an external rather than internal constituency.

There is also a pragmatic reason to be interested in ILL data, and that is that a considerably quantity of it is readily at hand as a byproduct of the millions of ILL requests handled each year by WorldCat Resource Sharing. I will confess that I do not know what proportion of library inter-lending traffic passes through WCRS compared to other channels; presumably the rise of direct consortial borrowing arrangements and reciprocal borrowing agreements has meant that some part of the inter-lending economy is managed regionally or within institutional peer groups. The Association of Research Libraries reported a total of some 4.5 million items loaned and 3.5 million items borrowed by 113 university libraries in 2007/2008, which provides at least some measure of the activity in this sector. A recent article describing the highly successful BorrowDirect lending partnership, which includes 7 ARL members, reported that participating libraries collectively received and filled more than 140,000 loan requests in 2008.  These figures provide a backdrop against which the portion of inter-lending activity reflected in WorldCat Resource Sharing may be assessed.

As an experiment, we extracted from the WorldCat Resource Sharing data repository a record of inter-lending requests placed by 68 research institutions between 1 October - 30 November 2009. Late autumn is generally a period of high activity in university libraries, as students race to complete (and sometimes to begin) writing and research projects assigned during the first part of the academic year. The cohort of institutions included represents a subset of the total ARL membership. Our initial data-set amounted to more than one hundred thousand transactions; by limiting to transactions for “returnables” (i.e. books and other materials that must be returned to the lending library), we reduced this approximately 74,000 transactions representing about 71,000 unique titles. Most of the titles were requested only once; about 3% were requested more than once and a small number of titles (6) were each requested more than 5 times by members of the research library cohort between October and November 2009.

It is useful, I think, to think of the titles represented in our resulting data-set as titles for which extant local and group inventory is inadequate — that is, local demand can’t be met with local supply, because the title either isn’t owned or isn’t currently available (is checked out, on reserve, has been lost, etc) during a period of relatively intense demand. An obvious question one might ask is whether these requests might be fulfilled through alternative means, at a lower per unit cost. ILL has traditionally functioned as a means of lowering the transaction costs of sourcing material for which a long-term local investment in ownership is not feasible or desirable, sometimes referred to as a opting for “access over ownership.” In the age of mass-digitization, however, one might reasonably ask if content previously sourced from partner libraries might instead be delivered directly from large-scale digital aggregations in the cloud.

As a side note, I will observe that inter-lending partnerships like Borrow Direct were explicitly intended to reduce the transaction costs associated with traditional ILL.   Borrow Direct has successfully reduced the per-transaction cost for its members to less than half the average and is generally (and rightly) regarded as a model to be emulated.  The question, I think, is whether a model of collaborative resource sharing based on a very large and distributed regional inventory with considerable redundancy in local holdings is scalable in today’s library environment. Especially as local circulation rates fall, and as individual libraries seek to move collections into lower-cost high density facilities from which on-demand print delivery comes at a relatively high price, it will be necessary to re-evaluate what part of the aggregate print collection is amenable to the informal governance and distributed management model that underpins many resource-sharing partnerships, even highly successful ones like Borrow Direct.  Going forward, cost-effective management of legacy print collections will likely require new organizational structures that are accountable and responsive to system-wide demand dynamics, including the progressive shift toward elastic, scalable digital provisioning options.

To understand what part of current academic inter-lending demand, i.e., demand for print books that cannot be met with local inventory, might one day be directed to large scale digital providers, we evaluated the match rate between our ILL data and the mass-digitized book corpus.  We tested the 71,000 titles in our set against a June 2010 snapshot of the HathiTrust digital library and found that 17% of the titles requested via ILL (during the Oct-Nov 2009 period we examined) are represented in the mass-digitized corpus. This strikes me as a reasonable, and even impressive figure. [Our previous investigations have suggested that about 30% of the titles owned in individual academic libraries are represented in the digitized Hathi Library collection.]  Of course, titles represented in the mass-digitized book corpus are not always available for download or onscreen reading; our current estimate is that 16% of titles in the Hathi collection are in “full view.” A little more than 500 (~1%) of the titles in our ILL sample are currently available as full-view content in the Hathi repository. If you figure that each inter-lending transaction costs an average of $20 (this estimate is based on 1992 survey data; costs are significantly lower in partnerships like BorrowDirect), one could say that in the two months from 1 October to 30 November 2009, research libraries collectively paid more than $10,000 to borrow books that might be sourced in digital format at little or no cost, directly from Hathi or Google.

This is, admittedly, a pretty small number when placed in the context of total library spending; in 2007/2008, aggregate ARL library expenditures topped $3BN. Nevertheless, as the mass digitized corpus continues to grow and as the relative proportion of public domain titles increases (up from 12% to 16% of titles in Hathi quite recently), it will surely become more important to weigh the costs and benefits of traditional print based inter-lending operations. As a thought experiment, one might extrapolate from our initial (and very preliminary) data and speculate that if 1% of all ARL borrowing were available as public domain digitized content, the total savings in inter-lending expenditures could be quite significant.

Let’s do the math. ARL inter-lending figures do not distinguish between returnables and non-returnables, but we might estimate (based on our WCRS sample) that 70% of the reported requests in 2007/2008 were for books rather than journal articles.

3.5M requests * .70 = 2.45M requests

Now let’s assume that 1% of those requests are available as public domain digitized content, and apply the $20 per transaction cost estimate:

(2.45M  requests * .01) * $20 = $490,000

Nearly half a million dollars in research library expenditures per year for a traditional print based operation that might be saved. Of course, not every demand for print can be met with a digital surrogate; but in many cases, I would argue, providing a digital-first option could help mitigate the increasing demand and cost of meeting extra-mural demand for locally-owned inventory. As universities look to contain costs and libraries seek to embrace more locally responsive service portfolios, it will likely become harder to reconcile the costs of subsidizing print delivery services for an external clientele.

Further study of aggregate demand data will help to identify which parts of the locally-organized print management operation in libraries are amenable to externalization. Our initial examination of WCRS transaction data suggests that emerging network service providers like the HathiTrust will play an increasing important role in a progressive out-sorting (sic) of library operations.

The following panelists will help us explore the ins and outs of group collaborations during “Yours, Mine, Ours: Leadership through Collaboration“:

As with our previous panel on local solutions, the specific projects serve as exemplars for collaboration strategies which the audience will be able to apply to realizing their own ambitions. Speaking of which, we’ve made sure to have some time on the agenda where attendees can explore the implications of what they’ve heard in smaller group settings (see the Birds-of-a-Feather slots on Day 2). During online registration, people vote for specific topics they’d like to see covered in these facilitated discussion settings, such as single search (local), digital preservation (group) or open access (global).

Here’s some background on group collaborations:

Group Solutions - Common Interest
“We work together because we have common interests.”

Moving beyond the single institution, collaboration across organizational boundaries occurs when there is a common interest. A group of motivated individuals or institutions bands together to work on an issue they would have found difficult or impossible to solve in isolation. Many collaborative grant-funded projects fall into this category: a finite number of players tackle an issue that vexes participants in their own local contexts. Because the local benefit of this type of collaboration can be readily perceived, common interest collaborations are generally accepted as a way to achieve broad outcomes. In the sphere of group collaborations, we see activities such as open-source software development, subject-based data aggregations, and shared technological platforms such as HathiTrust.

On the other hand, group collaborations around a common interest have a high management overhead for setting and managing expectations, dividing up the work, coordinating outputs from different groups, and staying on track. Different work cultures among a group’s participants can pose a serious threat to the most rationally conceived projects. Furthermore, participants’ interests may evolve in different directions; commonalities may dissipate over time.

Since common interest collaborations rely on direct contact, meetings and constant negotiation, it is challenging to mount and manage them at scale. Once these collaborations mature, they often require the creation of new organizational structures such as governing boards or foundations.

In the next post in this series, we’ll look at common value collaborations as a strategy for aligning the entire community.

“Local” is the first Collaboration Context we’ll explore at “Yours, Mine, Ours: Leadership through Collaboration.” Our panelists will be:

We’ve instructed all of our speakers to spend the bulk of their time on strategies for creating and deepening collaborations, and to focus both on successes and failures. (All of the presentations in this section are hence titled “Collaboration Trials and Triumphs”.) Here’s the background:

Local Solutions - Common Administration
“We work together because we have the same employer.”

From the perspective of a large institution (e.g., a university campus) with many units (e.g., libraries, archives and museums), incorporating collaboration into the underlying work culture is foundational to realizing that institution’s potential and achieving its mission. When ideas, data and services flow freely, new solutions emerge, and new knowledge is created. From the perspective of individual units, collaboration allows them to thrive when times are good and survive when times are bad. Deep and pervasive service and data relationships with other units provide a compelling argument for continued or increased funding, whereas isolation calls into question the value provided to the institution as a whole.

In highly distributed environments, deep collaboration requires conscious effort and leadership. Since both the institution and its constituent units directly reap the benefits of local collaboration, the context of common administration offers a straightforward environment for engaging in joint work. In the sphere of local solutions, we currently see activities such as cross-collection search, shared digitization and digital asset management, and shared conservation facilities.

On the other hand, contemplating collaboration solely within the boundaries of your own institution is arbitrarily self-limiting. While there is no shortage of issues that beg to be addressed at the local level, some aspirations are simply beyond the reach of individual institutions acting alone.
Group collaborations try to address that which transcends any single institution. Don’t try it abroad if you haven’t done it at home: in many instances, collaboration at the local level is likely to be a prerequisite for entering into meaningful collaborative activities centered on common interest.

We’ll take a look at common interest collaborations in the next blog post. (Also take a peek at the initial post in this series if you haven’t already.) Stay tuned!