DeepDyve appears to have adopted an iTunes-like buisness model ….$0.99 rentals for scientific research articles!

I like many things about the search engine – the way one can enter entire paragraphs of text as a query block, for instance.  You can use that feature in PLoSONE (Public Library of Science), though it is not obviously available on the DeepDyve site itself.

I don’t think that knowledge can be commodified in this way, though. It doesn’t look like it is going to be sustainable business model.

I think I now understand why David Weinberger’s book “Everything is Miscellaneous” is so provocative and sometimes enraging.  It often sounds like he’s claiming that there is no point at all in classifing / categorizing information.  No matter what you do, you’re going to get the category “wrong” because there is no such thing as a “right” category. Ergo, don’t even try – everything belongs in the category “Misc”.

I think Weinberger’s emperor has no clothes – in fact, he is asserting that nothing is “Miscellanous”. Everything belongs to some category for someone, it’s just that it may not be the same category for everyone. A banana is likely to be a fruit for most people, but also a weapon for John Cleese.  The point is: a banana is always a kind of something in every context.

So isn’t there is a middle ground between banishing the Dewey decimal system (or indeed any other library classification system) and dumping every digital object into an undifferentiated pile.  Indeed, there’s a lot to be said for a thoroughly well-understood standard, albeit a dated and even a bad, system of classification: at the very least, it is predictable.  If you know how the meta-data was generated (e.g. call-number, subject category, keywords), for a given item, you’ll be better able to retrieve it.

Furthermore, I expect there are some unforseen problems with the democratization of knowledge generated by social tagging and recommender systems.  Who’s doing the tagging?  Who’s doing the bookmarking? High school students?

This is of particular concern to me in the context of scholarly articles. Are the numbers of co-downloads in a digital library primarily due to professors’ undergraduate course syllabi?  Would professors’ syllabi be influenced by scholarly recommender systems?  I expect that the recommender-effect studied in Daniel Fleder’s “Blockbuster Culture’s Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity” and which shows that recommenders decrease aggregate diversity would be an especially accute problem when sources of co-download behaviour are (relatively) few (e.g. professors’ course syllabi).

Conclusion? I think it matters what population you are drawing from for your metadata – be it social tagging or collaborative filtering recommendations.  There is a point in relying on experts and big thinkers.  They are more knowledgeable and credible than even the collective intelligence of the masses.

The recommender system that Toine Bogers experimented on a few years ago with CiteUlike data and which is the subject of a very interesting poster given at Recomender Systems 2008 is now on-line at CiteUlike.

Paradoxically, my personal CiteUlike library of (only) 22 articles (mostly on recommender systems) isn’t sufficient to generate any recommendations. Probably there aren’t enough people who have similar collections.

Personally, I am glad for book reviews at the New York Times.  Without them, it would no doubt have taken me much longer to discover LogicoMix, the comic book version of “the epic story of the quest for the Foundations of Mathematics”.

The concept and story comes from Christos H. Papadimitriou who teaches theoretical computer science at UC Berkely and wrote the erudite text book “Elements of the Theory of Computation”.

One anonymous reviewer of LogicoMix on Amazon says:

…the book tries to be too many things at once, and succeeds as none of them. It is neither a strong introduction to Russell’s ideas, nor a worthwhile biography in condensed form, nor a successful piece of historical comic art. It’s a pleasant enough read, but considering its ambition ultimately a disappointing one.

So… my expectations from the NYTimes (mostly) positive review are dampened somewhat.  Still – how often does this golden age of logic get the attention of the graphic novelist?  Maybe they scooped Art Spiegelman.

In his recent Essay in the Globe and Mail, Peter Nicholson makes some interesting observations about the relationship between information and attention.

He observes that thanks to computing technology and crowdsourcing, there is an abundance of low-cost information and that the scarce resource now is attention (whose correlate resource is our time). We waste information because it’s free and we favour superficial coverage versus depth of thought.

We may think metaphorically of the production of knowledge as a function of “information” and “attention,” with attention understood as the set of activities by which information is ultimately transformed into various forms of knowledge.

I don’t think I’ve heard this non-cognitive definition of attention before: that which transforms information into knowledge.  It doesn’t quite work as a definition but attention is no doubt one of the ingredients required for this transformation. Intelligence is probably helpful as well.

Nicholson also observes that knowledge has changed recently from “stock” to “flow”. Knowledge-as-”thing”, an object to be to be accumulated and stored (stock), belongs to 20th century libraries.  21st century knowledge is more a “process” that changes and is updated all the time (flow).

There are two reasons for this, Nicholson says. One is that electronic information “permit[s] it to be changed continuously and almost at no cost.” The other is the “shift of intellectual authority from producers of depth – the traditional “expert” – to the broader public.” The result of this shift

….is the growing disintermediation of experts and gatekeepers of virtually all kinds.

This increasing “disintermediation” means we no longer need to think deeply for ourselves – we can rely on the wisdom of the crowds for just-in-time consumption (viz. Wikipedia).

This can’t last, he argues.  The buck (the deep thinking and attending) has to be done somewhere by someone.

What is apparently being eroded is the deep, integrative mode of knowledge generation that can come only from the “10,000 hours” of individual intellectual focus.

Maggie Jackson in her book “Distracted“ goes further and suggests that our individual and collective inability to focus threatens the very fabric of civil society.  Our inability to pay attention makes us unable to distinguish between the trivial and the important.

A recent article entitled “Cognitive control in media multitaskers” published in the Proceedings of the National Academy of Sciences provides some evidence to support this.  In the abstract, the authors say:

….heavy media multitaskers performed worse on a test of task-switching ability, likely due to reduced ability to filter out interference from the irrelevant task set.

Which is consistent with the view that multitaskers are less able to discriminate between tasks that are important and those that are trivial.

Maggie Jackson isn’t alone in being concerned about our collective attention deficit disorder. Ottawa’s Heather Menzies concurs in “No Time“.

I was re-reading 1995 (i.e. pre commercial web) Unte Reader article the other day in which Pico Iyre was quoted as saying:

I worry about the relentless acceleration of the world, the dramatic shortening of our attention spans and the temptation [...] to value information before knowledge and knowledge before wisdom.

A few months ago the irish sociology student Shane Fitzgerald perpetrated a Wikipedia hoax that led to a mis-attributed quote by the composer Maurice Jarre in the Guardian’s obiturary‘ about him.  This led pundits to reflect on what counts as an authoritative trustworthy sources of knowledge.  They concluded that

• Wikipedia isn’t authoritative;
• journalists are lazy and don’t check their facts and
• If such simple mis-attributions can be printed in a Guardian obituary, what information sources can we trust?

These observations, however true they may be,  miss an important point – the need for authoritative sources depends how likely or unlikely any bit of information is a priori. If I told you that Aung San Suu Kyi has decided to support the Burmese Military Junta, you would have every right to consider me a crackpot and demand that I document my claim, given what we know about her political history. That Maurice Jarre might have said (as was asserted in the Wikipedia hoax)

“My life has been one long soundtrack. Music was my life, music brought me to life.”

is not entirely unbelievable (unless, perhaps, you knew Jarre personally and were quite sure it wasn’t in his character to say such a thing.)  How important should it have been for the journalist to substantiate the attribution of this quote? Not as important, I submit, as if the journalist had reported that Jarre had spent the first 5 years of his infancy being raised by wolves in Siberia – well, perhaps it would have been important to get that right, given the low probability that this is true.

In practice, academic peer-reviewing also depends on the a priori probability of a paper’s claim. Despite the methdologically sound call for repeatable experiments, documented procedures, public dataset etc.  there just aren’t enough hours in the day to comb through a paper’s claim in detail.  Unless the claim is unlikely to be true, given what we already know.

The now 20 year old Cold Fusion debacle is a good example of that.  Given the extraordinary claim that nuclear fusion can happen at room temperatures, it’s obviously critical that the experiment that demonstrates this phenomenon be both repeatable and repeated.  But I think it’s unreasonable to expect the same level of scrutiny to hold for what Thomas Kuhn called “Normal Science”.

In his March article for CACM, Greg Linden opines that RMSE (Root Mean Square Error) and similar measures of recommender acuracy are not necessarily the best ways to assess their value to users.  He suggests that Top-N measures may be preferable if the problem is to predict what someone will really like.

“A recommender that does a good job predicting across all movies might not do the best job predicting the TopN movies.  RMSE equally penalizes errors on movies you do not care about seeing as it does errors on great movies, but perhaps what we really care about is minimizing the error when predicting great movies.”

This problem is compounded when it isn’t even possible to measure errors of any kind. Suppose you have an item-based recommender for journal articles in a digital library and recommendations are restricted to items in the collection owned by the library.  These recommendations are then restricted to a certain set which may be incommensurable with recommendations generated from a different collection. So any quality measure would depend on the size of the collection.

How then would one go about evaluating recommendations in this circumstance?  One way is for an expert to inspect the results and judge them for relevance or quality.  Another is to measure some meta-properties of the recommendations, such as their semantic distance from one another or from the item they are being recommended from.  At least y0u would be able to say that one recommender offers greater novelty or diversity than another.

This is the kind of approach taken by Òscar Celma and Perfecto Herrera in a paper delivered at Recommender Systems 2008. They concluded that content-based recommendations for music that are less biased by popularity (i.e. more biased toward content-similarity) produced less novelty in recommendations and also less user-satisfaction.

While music listeners may appreciate novelty and diversity, my expectation is that users of recommenders for scholarly articles actually want something closer to “more like this” (content similarity) than “other users who looked at this looked that” (collaborative filtering).

At least that’s the conclusion (not yet scientifically corroborated) that I came to when I compared a usage-only recommender (‘bX’ from Ex Libris) to a citation-only recommender for scholarly articles (Synthese). At first blush ‘bX’ produces more “interesting” recommendations (greater diversity) whereas Synthese (in citation-only mode anyway) generates more “similar” recommendations.

Perhaps what the user needs is both kinds of recommenders – depending on thier information retrieval needs.

Yesterday was a triple birthday celebration at Canada Institute for Scientific and Technical Information (CISTI). It began as the NRC Library 85 years ago and became CISTI and Canada’s National Science Library 35 years ago.

Yesterday was also the 80th birthday for the NRC Research Press which marked the day by unveiling a truely spectacular monograph of annotated plant and bird portraits by John Audubon taken from the Canadian Library of Parliament’s copy of Birds of America.

Audubon: Beyond Birds

A note in an appendix values the Canadian copy of Birds of America at $15M dollars! At $49.95 this new NRC Press book is a steal.

One of the authors is Ernie Small, who was at the birthday celebration to autograph copies of his book. Ernie is a Principal Scientist at NRC and his scholarly erudition about plants is phenomenal. His 1000+ page monograph on Culinary Herbs is hardly what you would call a “popular classic” but it is definitely a classsic to anyone who knows and cares about herbs. Meeting people like that really makes being a research scientist at NRC mean something to me.

Ernie told me that, of the 22 books he has authored, the Audubon book is the one he is the most proud of.  It is a tragedy of the privatization of NRC Press that it will no longer have a monographs program for the publication of such superlative books.

The Google Wave communication architecture makes a lot of sense: integrate all your interpersonal communications applications (currently served by Email, IM, Blogs, Wikis, Twitter, FriendFeed) into one application hosted in the cloud and every correspondant / commentator / friend becomes a “contributor” to a common collaborative application (see this overview if you don’t want to spend 80 minutes watching the painful video).

While it’s a fine idea whose time has no doubt come, it appears to me to be simply a natural evolution of Facebook with finer grained access-control rather than a truely revolutionary platform.

I’m guessing that Wave’s achilles heel might be the current ubiquity of good old fashioned store-and-forward e-mail.  If Wave is going to succeed in supplanting e-mail, it’s going to have to co-exist with it somehow and provide an interface to SMTP …. which opens the door to spoofing and spam.  

On the other hand, Google may be able to induce corporations and governments to throw away their piles of Microsoft Exchange servers and replace them with open-source Wave servers. Who knows, miracles sometimes happen.

The Structure of Scientific Revolutions is perhaps the best known book by Thomas Kuhn. But I think his most interesting book is The Copernican Revolution.  In it Kuhn defends the thesis that Copernicus was more of a Platonist about the importance of circular motion in celestial bodies than Greek astronomers were. 

The Greeks also believed that circular motion was the only way to explain celestial motion, but in order to accomodate the additional principle that the earth is at the center of the universe, they had to explain the movement of heavenly bodies in terms of circles moving around circles (epicycles) without worrying too much about whether the physical entities themselves were actually moving incircles.

Copernicus, on the other hand, held the view that the bodies themselves had to be moving in circles.  And if that becomes the core of your theory of the heavens, then explaining the phenomena (e.g. the retrograde motion of mars) is “merely” a matter of  ”shifting paradigms”, i.e. setting the sun at the center of the universe and putting the earth in motion. So the core of Khun’s thesis is that Copernicus was more of an “absolutist” about circular motion than ptolemaic astronomers.

Similarly, I would argue that Einstien too was more of an “absolutist” about the laws of nature than his predecessors and that the theory of  ”relativity” is a misnomer.  In fact, it is a theory about the “absoluteness” of the laws of nature. Einstien’s insight was that all the laws of nature are the same in all frames of reference. For instance, no matter how fast you are moving, the speed of light is a constant.  And for Einstein there nothing that is exempt from being subject to laws of nature, not even “space” or “time” (which are thereby relinquish their role as “absolutes”).

Microsoft Research Asia appears to have taken over where Microsoft Live Academic left off with Libra Academic Search.  Libra’s collection is limited to computer science (although, with 1.8M articles to data-mine, it appears to be quite comprehensive) it proves that one can do better than Google Scholar by implementing simple facets (Papers / Authors / Conferences / Journals / Communities) that cluster or order results according to different criteria.

I don’t think Libra is new (it appears to have started in April 2007) and it may be that no on is working on it actively any more – perhaps because CiteSeerX (also supported by Microsoft!) dominates the (limited) market.  But I hope it’s core features are not forgoten.

A recent paper in Accountability in Research by Richard Gordon and Bryan Poulin, entitled “Cost of the NSERC Science Grant Peer Review System Exceeds the Cost of Giving Every Qualified Researcher a Baseline Grant” has the following abstract:

Using Natural Science and Engineering Research Council Canada (NSERC) statistics, we show that the $40,000 (Canadian) cost of preparation for a grant application and rejection by peer review in 2007 exceeded that of giving every qualified investigator a direct baseline discovery grant of $30,000 (average grant). This means the Canadian Federal Government could institute direct grants for 100% of qualified applicants for the same money. We anticipate that the net result would be more and better research since more research would be conducted at the critical idea or discovery stage. Control of quality is assured through university hiring, promotion and tenure proceedings, journal reviews of submitted work, and the patent process, whose collective scrutiny far exceeds that of grant peer review. The greater efficiency in use of grant funds and increased innovation with baseline funding would provide a means of achieving the goals of the recent Canadian Value for Money and Accountability Review. We suggest that developing countries could leapfrog ahead by adopting from the start science grant systems that encourage innovation.

The new “search engine” Wolfram Alpha by Stephen Wolfram is interesting.  It’s neither a typical query-based search engine nor a question answering system.  But it also isn’t (yet) a “computational knowledge engine” as the web site would have us believe. It’s something in between perhaps.

There’s no question that Wolfram Alpha’s goals for the future are lofty:

Wolfram|Alpha’s long-term goal is to make all systematic knowledge immediately computable and accessible to everyone. We aim to collect and curate all objective data; implement every known model, method, and algorithm; and make it possible to compute whatever can be computed about anything. Our goal is to build on the achievements of science and other systematizations of knowledge to provide a single source that can be relied on by everyone for definitive answers to factual queries.

But we’re not quite there yet – not in May 2009 anyway.

I was interested by the results for “Multiple Sclerosis”, even though what I wanted to know were it’s known causes:

But if you try ” collaborative filtering” or “statistical semantics” or “demdemyelinating disease”, Wolfram Alpha is stumped and you are given subject areas (that it knows about) to browse.

Within a subject area that it does know something about (e.g “Quantum Physics”) you are presented with template question-types for whichl Wolfram Alpha will produce answers:

Which is quite educational, as far as it goes.

All of this uses what they call “curated data” – which presumably means that lots of formulas and equations have been entered into a web-based version of Mathematica and annotated with subject-area metadata.  Is this enough, though?  And can we trust the “objectivity” of the knowledge (e.g what Wolfram Alpha knows about cellular automata)?

To be a really useful tool, it sounds like a lot of people are going to have to contribute a lot of information. And even then that information will only be retrievable in a very particular way.

This effort seems more likely to succeed at codifying all human knowledge than Cyc, but it still seems like an impossible task.

Yesterday’s NYTimes Op. Ed. “The End of the University as We Know It” by professor Mark C. Taylor is quickly making the rounds in academic circles.  

There are two elements in this article that concerned me.  One is the dismissive criticism of one of his colleague’s student thesis topic as trivial:

Each academic becomes the trustee not of a branch of the sciences, but of limited knowledge that all too often is irrelevant for genuinely important problems. A colleague recently boasted to me that his best student was doing his dissertation on how the medieval theologian Duns Scotus used citations.

Presumably professor Taylor is not bothered that a Ph.D. student in a religion department is studying Duns Scotus – one of the most important philosophers of the middle ages.  It must be, then, that he thinks it is not important to be studying how Scotus is using citations.

I know enough about citation analysis to be confident that professor Taylor is being dismissive too hastily. I wish this graduate student well in his or her use of a 21st century tool to discover new things about Scotus that were heretofore unknown about his thinking. 

The other remark which I thought was ill conceived is the argument that Universities should:

Abolish permanent departments, even for undergraduate education, and create problem-focused programs. These constantly evolving programs would have sunset clauses, and every seven years each one should be evaluated and either abolished, continued or significantly changed. It is possible to imagine a broad range of topics around which such zones of inquiry could be organized: Mind, Body, Law, Information, Networks, Language, Space, Time, Media, Money, Life and Water.

Professor Taylor appears not to have headed the advice about categories in David Weinberger’s book Everything is Miscellaneous. It is indeed possible to imagine not just a “broad range” of topics, but a virtually infinite range of “zones of inquiry”, each equally worthy of consideration.

Furthermore, getting academics to agree on even on one set of such topics within which to fit their work would be an interminable excercise. Take the citation analysis of Duns Scotus’ work, for instance.  It  arguably belongs equally to “Information” or “Networks” or “Language” or, in categories not yet mentioned, such as the more traditional “Mathematics” or “Philosophy” or “Library Science”.  

Besides, who would decide which categories are the relavent ones?  The government of the day? In that case would professor Taylor have wanted to apply for grant funding under the topic “Anti-Terrorism” in the past 8 years?

 Congratulations to NextBio, which has just launched its recommender for science articles (and data and clinical studies).  And they have a new and improved UI as well!  We need more portals like that.

Unlike my prototype Synthese Recommender on CISTI Lab, NextBio offers up recommendations in a piecemeal way.  It looks like the recommendations depends on an overall usage profile (downloads or clickstream – I’m not sure) and there do not appear to be many ways to control what gets recommended.

In the 5 minutes that I played with it, I searched for Kawasaki’s disease (my next door neighbour’s daughter has it) and Multiple Sclerosis (several friends have it) and I was given recommendations for literature and studies on Cancer. But there isn’t an explanation for why I was given those recommendations, so I don’t know why I should trust them.  Explanation is a critical feature for recommenders and I need to start working on explanations for my recommender too. 

But really, it’s much too early to assess the quality of NextBio’s recommender. I expect it suffers from the cold start problem endemic to all recommender systems and acutely endemic to digital library recommenders. NextBio’s recommender will no doubt get better as its 1.6M visitors (so says their stats counter) to generate usage data!