Some of the quotes I highlighted in John’s post:

• “First test on a sample bank statement: clean, accurate text in 2.1 seconds. Tesseract was faster but dramatically noisier. Second test on a tall Fred Meyer receipt: disaster. The model entered a repetition loop, hallucinating “TILL YGRT” endlessly.”
• “The models love regex.”
• “On the fourth attempt I said “I would have expected we start a new process per batch.” That was the fix… Codex patched it, launched it in a tmux session, and the ETA dropped from 12 hours to 3. Not a hard fix. Just the kind of thing you know after you’ve watched enough overnight jobs die at 3 AM.”
• “I asked Claude to build me a labeling tool – keyboard-first, receipt image on the left, classification data on the right, arrow keys to navigate, single keypress to verdict. It built the whole Flask app in 22 minutes. I sat down and hand-labeled 375 receipts.”
• “Real-world data is messy.”
• “Here’s what made the quality good: every time I caught something, I could show the agents what to look for and they’d go fix it everywhere.”

This reminds me, for the umpteenth time, of the aphorism: “AI won’t take your job, but someone using AI might.”

Here’s a link to the example in question (now up to 56 citations!), and here are the sections being discussed:

My solution is to pay librarians to flag these suckers. Someone at Google (does anyone know if there’s someone doing QC on this project?) would then have to investigate and then remove the ghost citation. Let’s make it $10/USD for each one that gets removed – Google can afford it. While it’d be a game of whack-a-mole, surely it would help?! Others have proposed the same thing, so it must be a good idea! The biggest issue is that someone would have to vet and manage the list of librarians for whom the Dispute button would appear.

One of the links in Aaron’s post goes to a project called LLM Citation Verifier, which might be worth checking out further. It seems more technical than most people would want, but I love that it verifies citations produced by LLMs in real time. One drawback is that it seems to work only if there’s a DOI, which would rule out older stuff.

Maybe this will serve as my kick in the pants to finish the hallucitation project I started about this time last year!

We have also refined the scope of the tracker. While our tracker previously sought to include all products that might reasonably be used by instructors, researchers, and students at the postsecondary level, the current version focuses on products that are marketed specifically towards these same users for teaching, learning, and research activities. We are no longer including general purpose AI products (e.g., ChatGPT, image or code generation products for various contexts, etc.) with use cases that can extend significantly beyond higher education in the tracker.

Sadly, one of the features missing from the original tool is not addressed by moving to Airtable. There’s still no way to receive updates when anything changes on the site Airtable doesn’t offer an RSS feed, and the one tool they appear to have partnered with that does, miniExtensions, doesn’t offer a free tier. USD$50/month is too much for me to eat! I spent a LOT of time yesterday with ChatGPT trying to create a workaround, but so far, the best option, Visualping, only offers an email option. That’s fine, and is what I’ll use for now, but it sure would be nice to have that feed, so I’m still poking around.

The other update I’m happy to see is how easy it now is to suggest a change to a particular product (or an entirely new tool). I submitted a tweak yesterday, and it’s already been added.

She goes on to suggest that libraries might consider subsidizing reviewers rather than covering article processing fees for open-access titles. The idea is that if professionals were paid rather than volunteering, the whole peer-review process might move a little faster. I like that idea

But what I *really* took away from her post was the reminder of something those outside the profession probably don’t know about us; how strongly we take patron privacy, and how, in going directly to the consumer (student, faculty), publishers can learn an awful lot more about users than they ever could when users were consistently anonymized behind a proxy server.

Ehh, I’m sure that’s just fine.

There, I find that I can personally subscribe to ScienceDirect AI for US $25 / month or US $249 / year. There’s also a link to get in touch with sales for institutional access; I wonder how many hits that gets.

The FAQs go on to explain that purchasing SDAI won’t actually get you access to any additional content you don’t currently have. “Access to underlying content depends on your institution’s existing ScienceDirect subscriptions or any individual content purchases you make.” And, “…you can use ScienceDirect AI’s features regardless of your institution’s content access. However, the Reading Assistant tool is only available on documents you are entitled to access.” So that’s a strange potential mish-mash of institutional and personal accounts.

Do any other academic vendors market services directly to the end user? I can’t recall seeing anything like this before!

Artificial Intelligence is rapidly transforming the landscape of data and information services—and librarians are at the forefront of this change. That’s why we created AI for Librarians: a practical, hands-on course designed to help you build AI competencies in ways that directly support library services and workflows. No prior RDMLA coursework is required—this course is free and open to all learners around the world!

What’s Inside?
RDMLA: AI for Librarians introduces you to the fundamentals of AI while emphasizing ethical, responsible, and library-focused applications. The content is rooted in real-world scenarios you’ll encounter in library practice.

We’re launching today with multiple brand-new units:

●      AI Tools for Library Research – Explores how generative AI tools can support library research questions in innovative ways.

●      AI Ethics – Examines key ethical challenges, applies frameworks for responsible AI use, and outlines policy recommendations for AI integration in libraries.

●      AI Use Cases – Showcases how AI can streamline library workflows and enhance user services.

And this is just the beginning, stay tuned for more AI Use Cases to be added in 2026!

Thanks to the generous sponsorship of Elsevier, RDMLA: AI for Librarians is completely free and available worldwide. All materials are hosted on the Canvas Network under a CC-BY-NC-SA license.

To access the course, please enroll via: https://www.canvas.net/browse/simmonsu/courses/rdmla-ai-for-librarians

Note: RDMLA is the Research Data Management Librarian Academy. More info at https://rdmla.github.io/

Here’s your jaw-dropper:

Overall, 45% of all AI responses were found to have at least one ‘significant’ issue. When including ‘some issues’, 81% of responses have an issue of some form.

I have no reason to doubt the findings, but do note the caveats that appear in the paper itself, that,

This does not mean that the LLM has no potential value as a tool for other science communication
outlets. The findings of this project are specific to ChatGPT Plus’ adherence to SciPak style and
standards. Moreover, this assessment could not account for human biases…

Regarding that last point, Ars Technica points out,

…which we’d argue might be significant among journalists evaluating a tool that was threatening to take over one of their core job functions.

The actual prompts used by the evaluators are listed in the appendix of the paper (pg. 9), and are a nice illustration of how one should write a prompt if one is looking for a specific type of response. Sadly, the paper doesn’t indicate whether the results of that most-specific prompt were generally better than the less-specific ones:

In early April 2024, the team revised the writer survey to include more specificity. Before then, each
writer who nominated a paper reviewed the overall ability of ChatGPT Plus, assessing its collective
performance across the three generated summaries. After the revision, writers evaluated the LLM’s
performance for each individual summary instead. This led to a more detailed interpretation of the
LLM’s skills. Because this data is qualitative and anecdotal, it does not lend itself to graphs.

It’s important to do your own testing, I think, because one of the ways we’re seeing students, especially, use LLMs is for exactly this purpose – summarizing longer and more difficult papers. If the summarizations are wrong, that’s obviously concerning, but if the summarizations are right, but don’t conform to a particular style, that’s much less concerning, IMHO, and could possibly be corrected through better prompting.

I found that the results closely mirrored what I’m seeing at MPOW and in North America, except for reference chatbots, which are an important thing in my library, but apparently not so much over there or over here, generally.

I always like to hear specifically what tools others are using, and here the top three were ChatGPT, Copilot and Claude, but very closely followed by Gemini.

Two quotes that stuck out:

The commonest activity that there is in the area of AI literacy, is training users to understand AI as an aspect of information literacy, rather than direct uses on AI services.

Again, that’s just like here, and:

It may be significant that fears about job displacement among librarians did not appear frequently in comments. There was no direct question in the survey about this but it did not appear as an issue in open text questions.

Yikes! I can't say I'm surprised, but I just received the following email from Typepad support (this blog is hosted on Typepad):

We want to inform you that we have made the difficult decision to discontinue Typepad, effective September 30, 2025. 

What Does This Mean for You? 

After September 30, 2025, access to Typepad – including account management, blogs, and all associated content – will no longer be available. Your account and all related services will be permanently deactivated.   

Please note that after this date, you will no longer be able to access or export any blog content. 

What Do You Need to Do? 

If you need to retain your content, please export your content before September 30, 2025. After this date, your content will no longer be accessible to you and will not be available for export. 

• You can find more information on exporting here.

Refunds & Final Billing 

• Effective August 31, 2025, we will no longer charge you for services. 

• If you have made a recent payment, we will attempt to issue a prorated refund to the payment method on file. 

• Please verify that your payment method on file is up to date to ensure successful refund processing. 

Have Questions or Need Assistance? 

If you have any questions, please refer to our Frequently Asked Questions page here. 

If you have any additional questions or need help, please open a ticket at Help > New Ticket from your Typepad account.  

We truly appreciate your business and apologize for any inconvenience this may cause. Thank you for being a valued customer. 

 

Sincerely, 

Typepad 

Right now, there's no confirmation on the actual site, but I guess I'll be looking for a new home, and possibly a new domain! Stay tuned.