Reuben@AU Libraries

2011/01/31-02/03 - Goodbye, Auburn!

2011-02-03T09:53:00.000-08:00

The big news this week is that I just turned in my resignation from the Auburn libraries. It's time to move on to other things.

Otherwise it's been a frustrating week. I'm porting our ETD site to a new server. The ETD site currently runs DSpace-1.5 in a glassfish v2 (Sun Appserver) container on an old Sun Solaris box, and we're moving to an Ubuntu VM and glassfish v3.1. I was able to copy all the data over, and boot up the server, but there's some subtle bug where DSpace-1.5 does some invalid redirects at login time that glassfish-v3 complains about. Now I'm trying to upgrade to DSpace-1.6.2 which fixes the problem, and working through the little problems that keep tripping me up.

I also spent some time preparing for next Monday's minnows project training session with Prof. Armbruster's lab.

Anyway, I have two weeks left, so I'm going to try to finish the server migrations we have in process for ETD, repo, and vufind, and make sure the minnows project is in a sane state before I leave.

2011/01/24-27 Viva Tunisia!

2011-01-31T08:38:00.000-08:00

It was a quiet week at the library. I brought up vufind on the new devcat64 server that we want to move to, and I worked on the Minnows project site. I think I've got the different parts of the site tied together in a reasonable way, and Tony and Clint plan to do a pass over the site to improve it. Jon is also going to move the server to a production environment, so we'll be all set for some training with Prof. Armbruster's team after that.

Next week I hope to continue working on the Minnows site, prepare for the training class, and try to bring up a copy of the ETD repository on a new server.

2010/01/18-20 Hurray 4 MLK

2011-01-23T12:57:00.000-08:00

I love short weeks, and MLK day is my favorite holiday, so it was a double good week. I worked on a few tasks this week. First, I began building the new vufind dev server. Solr is up and running fine, but I ran into some problems trying to upgrade to Smarty 3 - I'll work on that next week.

I met with Jon Armbruster on Thursday to review the minnows project. The meeting went well. I'm going to try to tie up some loose ends next week, and arrange a training session with Jon and his students the week after, so they can begin testing the data-submission and approval workflows. We're shooting for an April 1 release for internal review.

I fixed a couple bugs in a library that underlies the circulation-request project. I also want to work on circ-req next week. I need to add e-mail support, enable staff-group based security, flesh out the request metadata, show notes in the UI, and add support for "lost trace" requests.

Finally, Aaron is my new boss this year as I've been moved into the systems group as part of the tech service reorganization. I attended my first "systems department" meeting on Wednesday. I'll try to remember to get an update on Aaron's request for clearance to open source some of Clint's code. If the lawyers don't reply to Aaron by next week, then I'll suggest that we send an e-mail to tell the lawyers what we're doing, and to let us know if they have a problem with it. I'm sure they don't care.

2010/01/12-13 - Auburn wins BCS

2011-01-18T08:43:00.000-08:00

Auburn shut down Monday and Tuesday for bad weather, so it was another short week. I got some good work done on the minnows repository code. I'll try to arrange a meeting with Armbruster next week after talking things over with Clint and Tony.

2010/01/03-06 back to the grind!

2011-01-08T11:41:00.000-08:00

It was a good first week back after winter break. I deployed an early beta version of a request tracker for circulation. J.P. likes what I showed him, and recommended that we extend the tool to also track missing-trace requests. I'll pickup work on that tool in a couple weeks.

I released a vufind patch to fix a bug in the link-resolver integration.

I wrote a javascript module that integrates the new webcat Voyager OPAC with the click tracker.

I normalized the advisor names in our ETD collection. Scanning through the spreadsheet of 7000 names was annoying, but it will be easier next time. I also want to add some pick-list support to the form that the graduate student fills in to encourage the student to pick the variation of an advisor's name that is already in the system, but that's a project for another day. The e-mail excerpt below has more details.

Jon is going to build us a clean 64-bit devcat virtual machine for us to build up a fresh vufind install. Clint and I will probably work on that next week.

Finally, we ended the week with an e-mail exchange with University of Alabama and UA Birmingham libraries - which was unusual, but cool. One of UA's digital projects librarians asked us about our ETD server, so I asked about how they managed their ETDs. It turns out that UA's setup is a little different than what I expected. I knew that UA had signed a deal with UMI Proquest to manage ETDs, but it turns out Proquest only manages the student submission and graduate school review processes. After the grad school approves a pdf, then Proquest adds the pdf to Proquest's ETD collection, and Proquest FTP's a copy to UA (or UAB) along with a metadata file. From there UA and UAB have processes to publish the pdf to their own digital collection servers.

Subject: Re: Thesis/Diss Repository - Spelling of my name
Hi Professor,

I just ran an update against the ETD server that standardizes the
advisor names for the theses already in the system:
  http://etd.auburn.edu/etd/browse?value=VAZSONYI,+ALEXANDER&type=author
Let me know if you spot any problems.

New incoming theses will introduce name variations depending
on what the students enter, so we'll go back and update the
data again every semester or so.
I also want to patch the server to allow a student to chose
his advisors from a list already in the database,
but it will take me a while to work that code up.

Cheers,
Reuben

--- Alexander Vazsonyi 12/13/2010 12:48 PM ---
Thanks Reuben - that sounds great.

Alex

____________________________________________


--- Reuben Pasquini 12/13/2010 10:15 AM ---
Hi Professor,

Thanks for reminding me about this problem.
I'll try to work out a solution over the next couple weeks -
have something in place before spring semester.

Cheers,
Reuben


--- "Alexander Vazsonyi"  12/12/2010 3:22 PM ---
Hi - 

I was wondering whether it would be possible to consolidate the 6
different variants of my name in the thesis/dissertation repository.

Here are the variants I readily found . . .  

Vaszonyi, Alex
Vazsonsy, Alexander
Vazsonyi, Alex T.
Vazsonyi, Alexander
Vazsonyi, Alexander T
Vazsonyi, Alexander T.

Thanks,

Alexander Vazsonyi

2010/12/6-9,13-16 - Goodbye 2010!

2010-12-16T12:41:00.000-08:00

It's been a good couple weeks at the library. I managed to get a couple things done, and try a couple new things too. First, I helped Jack setup his new laptop with the Library Csharp code I wrote when I first started at the library. Jack wants to move away from writing VB6 code, and start up with .NET, so hopefully the Csharp code helps him out. We managed to install the code's dependencies, and build a few of the applications. We'll see how it goes.

Next, I setup a spreadsheet with Google Apps to illustrate how we could begin tracking library statistics online. I think I'm the only one excited about the idea, but I'll try to get something off the ground in 2011.

Kathie at the Honors College got in touch with me. It looks like we might have our first student submit a honors thesis to our new online collection. Hopefully that comes off without a hitch.

I did a little vufind work - adding a "collection" sub-facet under the "Digital Collections" location facet. Clint and I enabled full-collection browse by simply handling an empty search string as "*:*", and it works surprising well. I went back and patched the vufind import to setup a location facet for government documents, and to index online theses and dissertations as an ETD collection under the "Digital Collections" facet. A full Voyager import to vufind is running on devcat - hopefully that will finish in the next few days. Since I was working on the import tool, I went ahead and added a "delete records" tab to the GUI version.

Finally, today I spent some time setting up a script on Yahoo Pipes that merges several library RSS feeds into a unified feed. I think Clint and Tony like the idea of publishing a unified feed for the library; we'll see what happens.

I'm looking forward to two weeks off over the holiday break. I'll be back in a couple weeks.

2010/11/29-12/2 - all the little things

2010-12-03T08:59:00.000-08:00

It was a good week for accomplishing little things. First, Clint and I wired up webcat (our Voyager Tomcat OPAC under development by Clint and Denise) with support for Auburn-ID authentication via the AuCataloging webapp.

Next, we rolled out some patches to the production vufind server that incorporate hierarchical call-number facets, an AUBIExpress link in the record detail view, and an updated "no results" page.

I added the libsharp code to littleware's google code project. Jack wants to move away from developing VB6 code, and move toward .NET - hopefully libsharp will ease his migration.

Finally, we interviewed the last two candidates for the systems department position. They were all good candidates.

2010/11/15-18,22-23 - Happy Turkey Day!

2010-11-23T11:03:00.000-08:00

It has been a good couple weeks. I've managed to stay out of trouble, and everyone is looking forward to Thanksgiving. I also managed to get a few things done. First, we imported Claudine's pamphlet collection into our DSpace repository. Jon installed the Solaris freeware xpdf package on repo, which allowed me to enable the dspace support for pdf-thumbnails, and Tony helped fix the layout problems that popped up when the thumbnails came online. I also imported the collection metadata into the vufind index on devcat with our vufind-OAI harvest tool and this xsl.

We interviewed a couple good candidates for the systems department position. The poor candidates each endured three hours of meetings, and we interviewed two candidates in a day, so it was exhausting for everyone. Two more candidates to go next week.

I checked in a couple vufind patches. One patch adds a link to a record's holdings screen that auto-populates the AUBIExpress (illiad) request form. The other patch fixes the call-number refinement in the search facets that I broke a while ago replacing AJAX facets. I think Clint will probably get the ok to release the patches next week.

Finally, I was able to setup authentication with dspace credentials for the minnows data-submission webapp. This bLog post has more details.

2010/11/8-11 - whoops!

2010-11-14T13:48:00.000-08:00

This week had good news and bad news - bad news first.

There was a little excitement this week as we accidentally released a bug to our live vufind server. On Monday we flipped the switch on a VM Clint had prepared for release that included a bug (that I wrote - ugh!) in the AJAX handler that marries a call-number to its proper item in a search result. The bug had gone unnoticed on our test server for over a week, and didn't lead to serious problems till Monday night, when search results started popping up with wacky call numbers. The reference desk paged systems, but the page went unnoticed till Tuesday morning (before I came in), when Denise and Jon called Clint at home on vacation. Clint quickly found the bad commit I had checked in, and reverted the bug back out. I keep expecting someone to throw a fit, but people have been cool, so maybe I'm too cynical.

The good news was that Claudine signed off on the metadata massaging that my import program runs through to marry her collection of scans with catalog MARC records for import into d-space. This project has been lingering since June - hopefully the end is in sight. I'll import the rest of the collection on Monday, and setup some initial XSL for Marliese to index the collection into vufind on devcat.

Finally, the vufind bug inspired me to setup some vufind regressions. The following e-mail excerpt explains what I did.



--- Reuben Pasquini 11/13/10 8:52 PM ---
Hi Clint!

I setup a framework for vufind regression tests that we can
add to over time.
    http://code.google.com/p/littleware/source/browse/?repo=catalog#hg/VufindRegression/src/main/scala/edu/auburn/library/tool/vufindTest

You can download a build of the test runner at:
    http://devcat.lib.auburn.edu/tests/vufindTest.zip
- just run 'vufindTest.bat'.

The test suite currently just runs through 3 sets of tests against devcat.
The first test runs a "Harry Potter" search (
    using HtmlUnit http://htmlunit.sourceforge.net/
), and verifies that the results include a couple expected records,
and that the call-numbers are correct on those 2 records.

The second test loads a specific Harry Potter record, then
checks the title,
verifies that the record has some holdings information,
and checks that the "campus delivery" cookies are set.

Finally, the third test accesses
    http://devcat.lib.auburn.edu/vufind/AuburnTestSuite.php
which runs through a suite of PHP unit tests
    http://devcat.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/file/tip/web/AuburnTestSuite.php

Anyway, we can start running these regressions before releasing new code.
I can set you up with a copy of the code if you want to add some tests.
I want to add tests that directly access some of the AJAX services, 
add a test fixture for Solr.php,
and setup a Hudson server (http://hudson-ci.org/) to
automatically run the tests whenever we commit some code to the vufind repository,
but I probably won't get to that for a while ...

Let me know if you run into problems with the test runner ...

Cheers,
Reuben

2010/11/1-4 - election day!

2010-11-07T19:44:00.000-08:00

It was another quiet week at the library. I spent some time updating the littleware build configs to simplify setting up the build for a new project. The build.xml files for each project now share a master build-template that defines some common cross-project build rules, and automatically installs ivy. I also updated the ivy.xml files to define compile, client, server, ee_client, ee_server, and test standard configurations; and to reference latest.integration dependency versions where appropriate. I hope to setup a hudson continuous integration server in the next few months to automate the test, artifact release, and server deploy processes.

I put most of the code in place that integrates the Google spread sheet data with the ACES collection d-space import tool. I hope to have a version of the collection on the server next week.

Finally, I helped with a couple small vufind bugs. Clint worked hard this week to release a few patches to the production server. There's some kind of big "vufind, love it or hate it" meeting next week. There are plenty of flaws with our vufind setup, but it has a lot of advantages over our only alternative.

2010/10/25-28 - Happy Halloween

2010-11-02T12:43:00.000-07:00

I got a few nice things working this week. First, the multi-step minnows submission process is running. I scheduled a meeting for Thursday, but Jon forgot about it, so we'll try again next week.

Next, I integrated Barbara Bishop's gifs of the library maps into our devcat vufind install. I used ImageMagic to down-rez Barbara's images, and convert them to PNG, and wired up vufind to popup a YUI-2 panel to display the map. Barbara and Marliese gave some good feedback, but we haven't released the code to catalog yet - bla.

When working on the maps thing I stumbled across the fact that our "retrieve X from stacks" request system consists of a patron writing down what he/she wants and handing it to circulation. I'd like to try to setup an online system - submit a request, and get an e-mail or text when the item is at circulation kind of thing.

Finally I implemented a MultiVoyager.php vufind driver for some people on the vufind e-mail list. I sent out the following e-mail, but haven't heard back from anyone. Bla.


--- Reuben Pasquini 10/31/2010 4:57 PM ---
Hi David,

I was able to get a simple 2-ILS system working at Auburn on our dev server.
Here's what I did.

*. Setup a solrmarc import that prepends an ILS-specific prefix to each record.
    You can get something working quickly by modifying the 'id' rule in
    the solrmarc marc.properties to:

      id = script(customId.bsh),getCustomId("PREFIX")

    where customId.bsh has:

import org.marc4j.marc.Record;
import org.marc4j.marc.ControlField;


/**
 * Generate a custom id as prefix + 001
 */
public String getCustomId( Record record, String prefix ) {
    return prefix + ((ControlField) record.getVariableField( "001" )).getData();
}

.........................................

*. I wrote a 
      MultiVoyager.php
driver that reads
     MultiVoyager.ini
and maintains a list of Voyager.php driver instances.
I attached a copy of MultiVoyager.ini, or
you can clone a copy out of our Mercurial repository:
     http://catalog.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/ 

The MultiVoyager.ini file maps a prefix to and ILS config -
an example follows.
The "EMPTY" keyword indicates no prefix.
This example works for records that look like
     12345 (no prefix)
and 
     AUM12345 (AUM prefix).

MultiVoyager.ini:

[Catalog]
configList = EMPTY,AUM
defaultDriver = EMPTY
[EMPTY]
host        = bla
port        = 1521
service     = VGER
user        = bla
password    = bla
database    = bla
pwebrecon   = http://pooh.lib.auburn.edu:7008/vwebv/holdingsInfo 
[AUM]
host        = bla
port        = 1521
service     = VGER
user        = bla
password    = bla
database    = bla
pwebrecon   = http://pooh.lib.auburn.edu:7008/vwebv/holdingsInfo 

........................................

*. I had to patch a few other php lines here and there too
    to get the ajax load of holding status working right,
    modify Voyager.php to accept a config in the constructor
    rather than load Voyager.ini, ... - stuff like that.
    The complete patch is here, but our vufind code is
    pretty far out of date compard to vufind.org:
       http://catalog.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/rev/9e94166d5e0b 


........................

Hope this code works for you - let me know how it goes.

Cheers,
Reuben


-----Original Message-----
From: Reuben Pasquini [mailto:rdp0004@auburn.edu] 
Sent: Monday, October 18, 2010 1:20 PM
To: Harmon, Kelly; vufind-general@lists.sourceforge.net; Osullivan L.
Subject: Re: [VuFind-General] Anyone merging more than one
VoyagerDatabaseintoVuFind?

At Auburn we currently only index one Voyager database, but
we also OAI harvest other collections.
An easy trick is to just prepend a unique prefix before the
normal bib-id.
For example - we have 'SLEDGE143' and '143' records:
   http://catalog.lib.auburn.edu/vufind/Record/SLEDGE143 
   http://catalog.lib.auburn.edu/vufind/Record/143 

I think you'll have to hack at least 2 things to get this scheme to
work:

  *. Modify Drivers/Voyager.php to manage connections
    to multiple Oracle servers, and choose a connection
    based on the id-prefix.

  *. Modify the solrmarc vufind.properties file, and replace
          id = 001, first
    with something like
          id = my001WithPrefixFunction
    ... something like that.

I'd like to do something like this myself, but it keeps falling off
the end of my TODO list.
Let me know if you need help implementing this, and I'll set aside a 
day or two to give it a try.

Cheers,
Reuben

2010/10/18-21 - fish data

2010-10-24T19:32:00.000-07:00

It was a quiet week for me at the library. I spent almost all my time working on the multi-step zip-submission forms for the minnows morphology project. The code is coming along well - it's just taking some time. I'll keep at it next week.

2010/10/04-07,11-14 - lazy blogger

2010-10-17T13:38:00.000-07:00

I haven't kept up with this bLog the last couple weeks, but there's not much to tell anyway. I've mostly just been working on my usual assortment of projects.

I finished a v2v refactor moving the CLI interface over to littleware's lgo infrastructure, and adding Clint's -halt and -continue flags. Clint kicked off an import last week, and it looks like it runs like a charm.
I sent Jon Armbruster a link to a prototype data-submission form for the cyprinella morphology repository. After a little back and forth we added a few more requirements for the submission to support. I'll work on that next week.
After Claudine finishes her pass over the ACES data I think we'll be set to import that collection into the repository.
We met with Troy from fisheries who has a document collection he'd like us to index into vufind. It looks like that will be easy for us to do - Troy will send us an XML data file, and we'll run that through our XSLT pipeline. There was a little bit of "should we do this" goofiness that the librarians had to talk themselves through, but I was able to avoid most of that.
Along the same lines - there was some nervous collapse in the vufind committee, who decided it's outside their "charter" to test vufind integration with article level search. The librarians will probably form yet another committee to consider article level search.
I'm going to check with Tony next week about setting up podcast feeds for some of the video he's posting to vimeo. That should be easy to do, but Tony might prefer to only stream the content rather than make files available for download. We'll see.
I also need to touch base with Kathie Mattox at the honors college next week. It's getting close to the end of the semester - does she still plan to allow honors students to deposit into the online thesis repository ?
We reviewed the 20 submissions for the library programmer position, and narrowed the list down to 10 phone interviews. They're all good candidates, but I'm not excited about hiring more staff at the library.

2010/09/27-30 - moving through jello

2010-10-02T19:14:00.000-07:00

I spent most of this week working on a submission tool for the minnows morphology project. The following e-mail describes what I'm implementing.

... Reuben Pasquini 09/30/10 1:01 PM ...
Hi Jon,

I've been looking at how to manage submissions to the morphology repository,
and I have some ideas to bounce off you.
I'd like to implement the following process.

  *. We'll setup a custom submission form that only allows a user
    to upload a single .zip file.
    The .zip file may contain an arbitrary number of .tps files,
    and the .jpg images that go with them.

  *. A user uploads the zip file, and fills out a form of metadata.

  *. The server automatically unzips the zip file,
    and breaks it down into individual items like
       http://131.204.172.126:8080/minnows/handle/123456789/10 
   , and submits the items for review after the user
   verifies that everything looks ok.

  *. The reviewer approves or rejects each individual item.     

That's the direction I'm working in now - sound ok ?

Cheers,
Reuben

BTW - If you want to allow excel spread-sheeting in addition to .tps files, 
    then send me instructions on how the data in a spread sheet
    maps to the data in a .tps file, and I'll code that in too.

I felt like I was moving through jello this week - writing code and doing little things, but nothing released to users. I did contribute (see below) to an e-mail exchange about testing article-level search in our vufind catalog.

... Reuben Pasquini 09/30/10 10:36 AM ...
Hi Nancy!

Thanks for taking the time to write down your thoughts - you have
great ideas. I'll help stir the pot a little - see if we can get her to boil!

*. Your argument about wanting students
"
to think more critically about
where and what they're searching for information
"
made me think that I actually want the opposite.
I don't want to think at all - just give me what I want.

*. Your point about the apparent overlap between the vufind, voyager-2,
and EBSCO efforts is well taken.
It makes sense to have vufind, voyager-2, and WAG
under one umbrella.
I don't think it's a big problem though.
Clint or Tony are in every meeting, so they keep things connected.
I think they like meetings.

* Google Scholar:
http://scholar.google.com/intl/en/scholar/about.html
"
Google Scholar aims to rank documents the way researchers do, weighing the full text of each document, where it was published, who it was written by, as well as how often and how recently it has been cited in other scholarly literature.
"
Sounds awesome.

*. "Are we using our technical and human resources correctly?"
Are you having doubts only now ?
You probably still leave cookies out for Santa ...

Cheers,
Reuben

... "Nancy Noe" 9/29/2010 8:56 PM ...
Hi Marliese

Working tonight at the Village. I've been able to play around with the
Villanova site a bit, but haven't given it the time or consideration it
deserves. These are my initial impressions. In the spirit of
full-disclosure, I should probably say that my comments come from a very
specific information literacy philosophy.

One of the concerns I have with this kind of searching is the same issue
I have with federated searching. While I understand the 'google-like'
feel of such a search, I want students to think more critically about
where and what they're searching for information. There was a recent
discussion on the instruction list serv - are people teaching federated
searching/discovery services? While a couple of people said 'yes', the
majority replied in the negative. Again, it comes back to helping
students to learn to move away from the 'everything possible' approach,
to one that asks them to consider what disciple/subject specific
resources might provide them with higher quality academic/scholarly
material. I want them to think critically about their information need
before they start typing in poorly constructed, broad-based keyword
searches, especially when looking for articles. For instance, I taught
an honors ENGL1127 today, working on literacy issues. Within the EBSCO
suite, I was able to have them select ASP and three education related
databases. That's the kind of thing I'm going to teach in class.
That's where I'm going to fall on the issue.

Technically, I found that in the searches I conducted, there were a lot
of 'clicks' one had to go through before you got to the actual article.
In two instances, there was a link to full-text that failed (and that
was prior to asking for log-in information.)

Vendors are creating systems that in a way compete with Google Scholar,
yet I suppose there's a 'cost' for these systems. Perhaps I don't fully
understand, but won't they be charging us for the privilege of using
metadata when we already pay for access to the info through our database
subscriptions? Does this give us permanent access to the articles?
Faculty tell me that they now use Google Scholar as the starting point
for their research. Many schools have a link to Google Sciholar off
their library homepage. Would this serve us just as well? I may not
be as informed as I need to be on cost and ownership issues.

Should the VuFind cmte investigate? Maybe. We're still working on
trying to make the catalog work (advanced searching?) and trying to
determine if we should recommend as primary. To be honest, the past
couple of weeks I've found myself reverting back to the 'classic'
catalog to answer journal title questions. I haven't settled on
questions I have about VuFind as a catalog. I'm not saying that we
shouldn't be exploring new systems/new technologies. We should. Is
this the right cmte to take this forward? We have the new voyager cmte,
and the EDS cmte, and the vufind cmte? Are we using our technical and
human resources correctly? Is there a way to better optimize time and
talents, especially since it seems that all of these are converging.

So, those are my two cents this evening. Again, I'll miss the meeting
tomorrow.

Thanks

Nancy

... "Marliese Thomas" 09/29/10 2:52 PM ...
Hi everyone,

Here's some background about the opportunity I mentioned at our last
meeting.

We've been talking for a while about taking VuFind to the next level by
including article-level metadata in it. Indeed, at our August 9th
meeting, Bonnie volunteered to contact vendors about getting a batch of
article-level metadata that we could run through Vufind in order to see
how this might work.

There have been a couple of new developments since that meeting.
First, Bonnie, Marcia, Aaron, and I met with Jane Burke and Mary Miller
of Serials Solutions a couple of weeks ago. In the course of that
meeting, Jane told us that Serials Solutions has developed a programming
interface that allows other discovery services, including VuFind, to
search Summon and display results from it. This functionality allows
VuFind to do article-level searches in Summon without the host library
having to ingest, store, and index large article-level metadata sets
provided by vendors.

To see how the VuFind-Summon combination works in practice, go to
Villanova's VuFind catalog at:

https://library.villanova.edu/Find/Search/Home

Do a search. In addition to the familiar-looking VuFind results,
you'll see a box in the upper right-hand corner called "Top results from
Articles & more", with links. "Articles & more" = Summon. One example:
https://library.villanova.edu/Find/Search/Results?lookfor=cat&type=&search=catalog+AllFields&submit=Find

The day after we met with the Summon folks, EBSCO announced that EBSCO
Discovery Service has a similar functionality. Since we're a full-level
development partner for EDS, we can test this feature with VuFind at no
cost to us. (Of course, we'd have to buy EDS, or Summon, or a similar
product, in order to offer article-level content to our users as part of
our regular services.) Since then, I have also heard that Ex Libris is
offering this same functionality for their discovery product, Primo
Central. Obviously, this is becoming a common and viable way for
libraries to leverage use of their software contracts.

I believe it would be worth exploring how VuFind works with a
full-featured commercial discovery service like EDS. In fact, Bonnie
gave Aaron and me the go-ahead to take a shot at getting a VuFind-EDS
test installation working in time for my presentation this weekend at
LITA. I'd be interested in hearing your thoughts on these developments
at our meeting tomorrow, or when you have had a chance to explore the
Villanova catalog.

Thanks,

Marliese

Marliese Thomas
Database Enhancement Librarian
RBD Library
Auburn University

231 Mell Street
Auburn, AL 36849

334.844.8171
mst0001@auburn.edu

Finally, Thursday was Penny's last day. Enjoy your retirement, Penny!

2010/09/20-23 - ouch!

2010-09-26T11:43:00.000-07:00

I sprained my ankle Wednesday morning when I planted my foot on some uneven ground as my three dogs took after a rabbit, so I spent Thursday at home on the couch, but it was still a productive week. Monday morning I presented the OAI-to-vufind tool I've been working on to several people. Marliese will now take care of maintaining our Content-DM digital library collections in Vufind. Liza heard about the OAI tool on Tuesday, and we were able to harvest records into Vufind from the NASA server she's interested in. She'll get back to me eventually to let me know if she wants to begin setting up some harvests of government documents into our Vufind install or not. I'll need to enable command-line options for OAI harvest, so we can setup cron jobs to automatically harvest new records every week or so.

Otherwise I did various small things this week as usual: patched a bug in Vufind to work aound some bad data in Serial Solution's link-resolver, setup a test for EBSCO's article-level search web services for Marliese, worked through some more of the ACES collection spread sheet, argued with Aaron over Institutional Repository plans, and did a little work on for the Minnows morphology project.

2010/09/13-16 - fish stories

2010-09-16T11:06:00.000-07:00

A lot of small things were going on in parallel this week. I spent some time testing the OAI-to-Vufind tool. I exchanged a few e-mails to help a guy at Purdue giving OAI2Vufind a try. Hope it works for him!

We met with Jon Armbruster on Wednesday to discuss the Minnows Project. I really like Google sites - we'll see if that project page helps us stay organized or not.

Finally, I finished my first pass over my chunk of Claudine's ACES-collection metadata spread sheet. The data is better than I thought it would be. We should be in good shape to import Claudine's collection into d-space with some Voyager metadata next month.

I also got an e-mail from the Tuskegee archive guys. Their DSpace repository is online (or it was yesterday) - looks good!

2010/09/07-09 - happy labor day

2010-09-09T20:50:00.000-07:00

It was a short week for me at the library, but I managed to get a couple things done. First, we setup a spread sheet in Google docs for the ACES collection project where we can manually check that each scan is matched with the right MARC record. I have some code that tried to match scan to MARC by title, but it had a lot of misses that we'll clean up with the spread sheet.

I was also able to use the vufind-import tool to OAI harvest records from ContentDM into vufind's Solr index. I wrote up the details here.

2010/08/23-26,08/30-09/02 - chugging away

2010-09-05T10:11:00.000-07:00

I was absent minded the last couple weeks, and completely forgot to update my bLog. Looking back through my e-mail it looks like I spent most of my time on a few things. First, I helped Clint implement a few vufind patches before he kicked off a new full copy from voyager to vufind.

On Thursday 08/26 Aaron, Bonnie, Midge, and I met with Drew and Iryana from the assessment office to further discuss their plans to implement digital measures to standardize the faculty annual review and assessment process. Auburn University will require her faculty to submit material for annual review to our digital measures server from which department heads, deans, and university management may generate reports and metrics for teaching and research performance for individual faculty on up to the university level.

At the meeting we discussed how assessment's digital measures project might dovetail with the library's nascent plans to implement an institutional repository through which faculty may publish research artifacts for open access by the general public. It turns out that the university's new guidelines for university funded research requests that faculty make their funded research available for open access either through Auburn's own "scholarly repository" (which the library supposedly is developing) or some other online repository.

Anyway, the digital measures project and the open access clause in the funding guidelines have combined to make Bonnie and Aaron very excited that the library quickly establish a repository for faculty research. Some library staff will join Drew and Iryana's group of digital measures beta testers, and I'll setup some collections in our DSpace server on repo that will be ready to accept faculty research papers.

Although I spent a lot of words describing the digital measures effort above, I actually spent much more time over the last two weeks on the tool to marry Claudine's scans of ACES documents with the appropriate MARC records in Voyager from which she wants to pull metadata. Each metadata file from the scanner specifies a title that includes series information like, "Bulletin No. 5 of the Alabama Soils Bla bla: Peanut Performance in Alabama" while the matching MARC record stores the "Peanut Performance in Alabama" part as a title, and stores the series information in other parts of the MARC record. Anyway, from the start I should have just used Lucene or tapped into the Sorl server that backs our Vufind install at http://catalog.lib.auburn.edu, but I thought I already had the set of 1200 or so MARC records to draw from, so I wound up implementing my own simple title-search algorithm. Everything looked good for my set of 4 test records, but when I ran a sanity check against the full set of over 1000 scans, it turned out that many of the MARC records I had pulled based on an earlier exchange with Claudine were not needed, and many needed MARC records were missing. I eventually collected a larger set of MARC records to draw from, but the amount of data that I'm working with now is large enough that my simple title-searcher takes a long time to run over the full record set. Next week I'll run some more tests over a subset of the data, then run the full data set into a Google docs spread sheet, and ask Claudine to go through the title matches, and manually correct the MARC bib-id's for titles that the code does not correctly match. Once we have that spread sheet, I'll feed it to the code I have to generate the dublin core for import into d-space. Anyway - the code is cool and fun, but it's freaking frustrating to trip over so many problems for this stupid little project that could have been done by hand by now.

Finally, I made some good progress on the minnows morphology database project we're working on with Professor Jon Ambruster in biology. I read over an introduction to "geometric morphometrics", and I think I came up with a good way of tricking our Minnows DSpace server to store morphometric data in a dublin core field attached to an item record for a minnow sample's reference photograph. We'll see what Jon thinks next week.

2010/08/18-19 - goodbye cataloging, hello serials & monographs

2010-08-21T09:39:00.000-07:00

The staff meeting Wednesday morning went very well. Aaron, Paula, and Helen presented their plans to merge the acquisitions and cataloging groups into a single unit. Those of us in cataloging will move to new cubes in the acquisitions area of the building by the end of the year. The new unit will be managed as two teams - a monographs team will report to Helen, and Paula will lead the serials and electronic resources team. We all expect our monographs work will continue to shrink over time, but combining acquisitions and cataloging is a good step in streamlining library technical services.

Otherwise I've spent most of the last couple days working on a few patches to the vufind tools. I spent most of the time tracking down the reason why our vufind server does not properly filter connecting diacritics so that a search for vtoriaa finds Russian language records like this. I eventually discovered that vufind's underlying Solr server has a filter system that handles that kind of thing, and that the unicode filter that indexes versions of unicode tokens without diacritics was not properly handling combining halfmark characters. I built an AuUnicodeFilter copy of the unicode filter that just adds a switch-statement block that checks for the combining half marks; problem solved!

2010/08/16-17 - patching code

2010-08-17T18:00:00.000-07:00

I've spent the last several days on a series of small tasks that added up to take all my time. I checked in several patches to small bugs in the AuCataloging tools, worked on the ACES-project metadata import tool, setup a d-space collection on repo for a professor in computer science to experiment with, installed a valid SSL certificate on repo, .... stuff like that.

Tomorrow there's a big meeting for everyone in the systems, acquisitions, and cataloging groups where Aaron will present some vision of the future of technical services. I'm looking forward to it - technical services can use an overhaul. The cataloging group shrank a lot over the last year with Harriet retiring, Henry's retirement and death, Tom's unexpected death, and Lori leaving for a new job. Then a few weeks ago three women in cataloging were told out of the blue that they would begin reporting to the circulation group. The move was perceived by staff as a ham handed decree from library leadership, and one of the transferred women was angry enough to go ahead and retire rather than accept the change. I'm sure more personnel moves are in store in the near future considering the libraries' budget constraints, but a meeting to present and exchange ideas with staff will help avoid bad feelings.

2010/08/09-10 tweaking vufind

2010-08-10T12:35:00.000-07:00

I'm finally back at the library after three weeks away. Three weeks sounds like a long time, but it went fast.

I spent most of today and yesterday experimenting with the Solr index that backs our vufind catalog. Last week the library finally decided to make vufind our default online catalog rather than the older Aubiecat Voyager OPAC.

A few small vufind bugs have popped up that we can deal with, but one big problem we've had is that vufind's Solr server would periodically run out of memory, and require a restart. We've tried several things over the past week (I lent some help from home last week) including moving Solr to a 64 bit Solr and allocating a 5 GB heap, and we also tweaked the Solr cache configuration, but the memory problem persisted.

Last night Clint noticed that the Solr memory use spiked when he ran a title-sort on a search result, so we've been looking at sorting since then. It turns out that Solr's lucene index engine uses a "field cache" to implement sorting, and the cache size is proportional to the number of unique entries in the sort field and the size of each entry. Our catalog has over three million unique titles, so it's very expensive to process a title search. I experimented with a solution that just post-sorts the first part of a relevance ordered search outside Solr, but Clint decided to just disable the title-sort as it's not a critical feature. Problem solved - hopefully!

2010/07/13-15 - ACES Voyager merge pain

2010-07-15T12:49:00.000-07:00

I've been working on the tool to decorate the dublin-core for each pdf in the ACES scanning project with metadata from our Voyager catalog. I've got code running that extracts each record's metadata from the mets files the scanning contractor delivered, and also have the list of Voyager ids, so now I need a strategy to match A with B, and glue it all together.

I'm making the best of this ACES project by taking the opportunity to update some shared code and trying some new things, but I believe that the index of the pdf's full-text makes the meta-data from the catalog superfluous for discovery - especially after Google and Bing scan our sever. The librarians refuse to believe that though!

2010/07/08,12 - Honors Theses are Go!

2010-07-13T08:45:00.000-07:00

I think the new repo server is nearly ready to start accepting Honors College theses this fall. I updated the site to include links to Midge's final liability release form; submitted a couple test records; spoke with Tony about the web design; sent the following e-mail to Kathie at the Honors College, and finally submitted a purchase order for a new SSL certificate for the server. Tony had the good idea to setup a rule that routes http://repo.lib.auburn.edu/honors/ to the collection too.


--- Reuben Pasquini 7/8/2010 4:58 PM ---
Hi Kathie!

I just want to give you an update on the 
online collection for honors theses.
I think we're on track for a fall release.

*. The attached Word document has the final legal
    disclaimer approved by the lawyers.
    The form is also online:
        http://repo.lib.auburn.edu/HonorsLiabilityForm.pdf 
    and
        http://repo.lib.auburn.edu/HonorsLiabilityForm.doc 
   Let us know if you have questions or see problems.
    
*. The theses collection is at:
         http://repo.lib.auburn.edu/honors/ 
    You can add that link to the Honors College web site
    once we go live.
    We'll remove the 2 test records before then.

*. The library web designer, Tony Oravet, plans to update the
    site design in the next month.
    Please let us know if you have any requests.

*. I'll be out of town July 19 until August 9.
    Can we plan to meet sometime the week of August 9
    to review the site, and to give you some training
    on how to add a student to the system, and how
    to review a student's submission ?
    Monday to Thursday 
    starting 9:30am to 2:00pm is best for me ...

I think that's all I wanted to report.
What do you think ?

Cheers,
Reuben

2010/07/06-07 - vufind redux

2010-07-07T13:18:00.000-07:00

I spent most of my time the last couple days patching a couple search related bugs that pop up in corner cases - one with "author" search and another with very long search strings. I think the patches are ok, but there's no regression test suite, so we'll see.

I need to get back to work on the AU-repo ACES import and configuration for the Honors College collection. I made a few small changes yesterday, but I'll do more tomorrow.

06/29-30,07/01/2010 - respect cataloging!

2010-07-02T07:55:00.000-07:00

Over the last few days I managed to stage Claudine's ACES collection scans on our server, configure an XSL transform to translate each scan's title, author, date metadata file to the format Dspace wants for bulk import. I imported some records into a test collection , and sent an e-mail out to let several people take a look. The librarians want to enrich the Dspace records with metadata from our Voyager catalog. For example, this Dspace record corresponds to this Voyager record. I pointed out that the DSpace server indexes the OCR full-text of each record's PDF content, so the extra metadata won't help much with discovery. Librarians don't like it when you say things like that!

I spent most of Thursday preparing a patch that fixed a bug Julie at the graduate school found in the ETD proquest-export tool.