You can read the original post in its original format on Rtask website by ThinkR here: {golem} 1.0.0 is here

After years of powering Shiny applications in production, {golem} — our opinionated framework for building production-grade Shiny apps as R packages — has finally reached a symbolic milestone: version 1.0.0. This is more than a version bump. It marks a mature, stable API, and it’s the right moment to clean up a few legacy behaviors along the way.

Here’s what’s new in this release:

• Agent skills — the headline feature. {golem} can now install agent skills (Claude Code / AGENTS.md layout) straight into your project, so your coding assistant natively understands golem’s conventions: adding a module, a function, running a check, fixing missing ns() calls… Enable them right at creation with create_golem("myapp", with_agents = TRUE).
• A reworked Docker/{renv} deployment story. Multi-stage Dockerfiles by default, production mode enabled out of the box, plus two new helpers — add_github_action() and add_gitlab_ci() — to generate minimal deployment CI for fresh apps.
• Modernized console output. Every message, progress bar and feedback line has been standardized with the {cli} package for a cleaner, more consistent experience.
• Functional JavaScript bindings out of the box. add_js_input_binding() and add_js_output_binding() now generate working bindings — no more manual skeleton completion — with a ready-to-use R companion file.
• Breaking changes. Unified golem_wd path argument, a reworked get_current_config(), a few removed functions and stricter add_*/use_* helpers. Worth reading before you upgrade an existing project.

Getting started (or upgrading) is a one-liner:

install.packages("golem")
golem::create_golem("myapp", with_agents = TRUE)

Read the full announcement on golemverse.org →

This post is better presented on its original ThinkR website here: {golem} 1.0.0 is here

We are excited to introduce SummaryTables, a new module available in the jamovi library designed to provide an easy, flexible, and elegant way to create publication-ready analytical and summary tables.

Powered by the gtsummary package in R, this module bridges the gap between running your analyses and publishing them. Instead of piecing together multiple separate outputs into a final manuscript table, SummaryTables generates beautifully formatted, publication-ready tables directly within jamovi. By making complex analyses simple and accessible, it saves you valuable time and reduces the risk of transcription errors. It summarizes data sets, regression models, and more, using sensible defaults while offering highly customizable capabilities.

Key Features

Here is an overview of the analyses included in the module:

1. Summary Table

Create classic “Table 1” summaries with group comparisons, automatic statistical testing, effect sizes, and p-value adjustments.

2. Continuous Table

Summarize a single continuous variable across multiple categorical variables with automatic statistical testing and p-value adjustments.

3. Cross Table

Summarize the association between two categorical variables using cross-tabulations with automatic statistical testing.

4. Likert Table

Summarize Likert scale items.

5. Survival Table

Create Kaplan-Meier survival tables showing survival probabilities and median survival times.

6. Multivariable Regression

Fit multivariable linear, logistic, or Cox regression models.

7. Univariable Regression

Run separate univariable linear, logistic, or Cox regression models for each predictor, seamlessly combined into a single, clean table.

Additional Features

Save to Word

Bypass the formatting headaches that come with copying and pasting jamovi tables into Word. Save your results directly to a natively formatted Word (.docx) document, preserving the same styling done in jamovi and ensuring a perfect layout for your manuscript.

Journal Formatting & Translations

Choose from 16 supported languages and apply pre-built formatting themes tailored for major journals (JAMA, The Lancet, NEJM, QJE).

Help and Documentation

For documentation, tutorials, and support, please visit: nouredindarwish.github.io/SummaryTables.

I recently gave a talk, The value in teaching is not the content it’s the teacher

My main point in this is:

Your course materials should be out there in public for free online.

To help support this, this blog post goes through the technical details I note in one of my slides: How to Put your Course Book Online.

Summary

• Make a quarto book
• Add an appropriate license CC BY-NC-4.0
  • Can share/adapt, must attribute
  • Cannot use materials commercially
• Add a README
• Put it on github
• Have it render as an online book when you make changes

Follow along with github repo: “course-book-template”

I have made a repo on github “course-book-template” that details each of these steps.

Make a quarto book

The quarto docs on starting a book are excellent so I would recommend starting there: https://quarto.org/docs/books/#quick-start.

But essentially, you run:

quarto create project book .

It will then guide you through creating a title, etc

commit of adding the book

Add an appropriate license

It is important to pick a license early. This helps protect your work, and also makes it clear to others how to reference and use your work. Personally, I like CC BY-NC-4.0, this gives you these conditions:

• Can share/adapt, must attribute
• Cannot use materials commercially

Note that this is different to the very common CC-BY, which does allow commercial usage.

If you aren’t sure about licenses for your purpose, it would be worthwhile checking out the chooser here https://creativecommons.org/chooser/

In using this, I discovered another useful license, CC-BY-NC-SA 4.0 Which builds off of CC-BY-NC, but with one additional clause:

ShareAlike – If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

This is sometimes known as “copyleft”. This is sometimes seen as too restrictive. Consult with your community about what the standards are. Another useful place to read up on licenses is the “licensing” chapter in the R packages book by Hadley Wickham and Jenny Bryan

I add a LICENSE file, and also a license.qmd chapter, as well as add the license to the README.

commit of adding the license

Using a README

I think it is worthwhile adding a few key sections to your README file:

• Details: like the abstract – the hook!
• Prerequisites: What do you expect learners to know?
• Learning outcomes: What will they walk away knowing?
• Schedule: An outline of each hour of learning, optionally with a timetable

commit of adding the README

Put it on online – github

Your course should live somewhere public! You can see for example our course materials here: https://github.com/njtierney/course-book-template

Have it render when you make changes

You can use github actions to render your book. This is really neat, and means your book will be rendered anytime you push changes. It means you don’t need to push HTML, just the quarto files.

Rhere are a few different ways you can manage this, I happen to like using github pages.

There are some really nice instructions on the quarto website on how to set up github pages – https://quarto.org/docs/publishing/github-pages.html#github-action

However, I have found a slightly different setup, which I will share here.

This involves using a DESCRIPTION file to track the R packages that you use. The reason we need to do this is to make sure when we render our book, that all the R packages we need are installed. There are probably other ways around this, and I’d love to hear them, but this is what I have found works.

Here is the first step where I add a dependency, in this case, tidyverse.

commit of adding this tidyverse code

Then add the DESCRIPTION file with:

usethis::use_description(check_name = FALSE)

I then edited mine to look like this:

Package: course-book-template
Title: A book about some things
Version: 0.0.0.9000
Authors@R: 
  c(
  person(
    given = "Nicholas",
    family = "Tierney",
    email = "nicholas.tierney@gmail.com",
    role = c("aut", "cre"),
    comment = c(ORCID = "https://orcid.org/0000-0003-1460-8722")
    )
  )
Description: Course materials for your topic. This should have two sentences.
License: CC-BY-NC 4.0 + file LICENSE
Encoding: UTF-8
Language: en-GB
Roxygen: list(markdown = TRUE)
RoxygenNote: 8.0.0

commit

You can then add your package dependency into Imports or Depends. Which one you use is normally very important for R package development, but the reason we are using a DESCRIPTION file here is to track our dependencies.

usethis::use_package("tidyverse", type = "Depends")

commit

I then add the github actions – you can actually just refer to a file, so this will work:

use_github_action(url = "https://github.com/njtierney/gentlegit/blob/main/.github/workflows/quarto-publish.yml")

This will give you a message like the following:

✔ Creating .github/.
✔ Adding "^\\.github$" to .Rbuildignore.
✔ Adding "*.html" to .github/.gitignore.
✔ Creating .github/workflows/.
✔ Saving
  "njtierney/gentlegit/.github/workflows/quarto-publish.yml@main"
  to .github/workflows/quarto-publish.yml.

commit

Also, probably a good time to add a .gitignore file. This is a good idea to make sure you don’t commit HTML files (they can be really large,a nd we don’t need them), or other file types that might be really large, or have sensitive information in them.

usethis::use_git_ignore("*.pdf")

Will create the file, and tell git to never commit a PDF.

I edit my .gitignore file to look like the following:

/.quarto/
**/*.quarto_ipynb
.Rproj.user
.Rhistory
.RData
.Ruserdata
dev
docs
/.quarto/
*.aux
*.log
*.pdf
*.tex
*.toc
*.rds
*_files
*_cache
*.html
.DS_Store

commit

Once all this is said and done, you will still need to run some commands in your terminal:

quarto publish gh-pages

This should then produce a question like:

nick course-book-template[main] > quarto publish gh-pages
? Publish site to https://njtierney.github.io/course-book-template/ using gh-pages? (Y/n) › 

reply “Y”

You will then get some code that looks like:

Switched to a new branch 'gh-pages'
[gh-pages (root-commit) 8ccc5e0] Initializing gh-pages branch
remote: 
remote: Create a pull request for 'gh-pages' on GitHub by visiting:        
remote:      https://github.com/njtierney/course-book-template/pull/new/gh-pages        
remote: 
To https://github.com/njtierney/course-book-template.git
 * [new branch]      HEAD -> gh-pages
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
From https://github.com/njtierney/course-book-template
 * branch            gh-pages   -> FETCH_HEAD

And then some rendering code that will look like:

Rendering for publish:

[1/4] index.qmd
[2/4] intro.qmd


processing file: intro.qmd
1/3                  
2/3 [unnamed-chunk-1]
3/3                  
output file: intro.knit.md

...

(|) Deploying gh-pages branch to website (this may take a few minutes)

Wait a few minutes, as it asks you.

Then you should see something like:

[✓] Deploying gh-pages branch to website (this may take a few minutes)
[✓] Published to https://njtierney.github.io/course-book-template/

NOTE: GitHub Pages deployments normally take a few minutes (your site updates
will be visible once the deploy completes)

Your website probably won’t be visible just yet, which feels a touch annoying, but you can keep an eye on it on the “actions” tab, e.g., https://github.com/njtierney/course-book-template/actions

Once this has lit green (hopefully it has!)

You should go to your “about” section, and click on the setting cog:

Then tick the box that says “Use your GitHub Pages website”

This adds your GitHub Pages website onto the repo, and it looks pretty neat.

There are more things you can do, like configuring your own custom website instead of using github.

So, instead of https://njtierney.github.io/course-book-template/, you could have: “course-book-template.com”.

And that’s it!

A very short post today. During the global fuel crisis relating to the conflict in Iran, I have been monitoring this chart, every Thursday morning my time:

It shows the amount of crude oil in the Cushing facility in Oklahoma and is one of the most timely and frequently updating (every Wednesday) indicators of the overall health of the crude oil market in the USA. As the chart says, below 20 million barrels is widely cited as problematic, “tank bottom”, or the “operational floor”. Below this level is expected to cause risks to oil quality, to the ability of infrastructure to move oil around, and to service the commodities markets that rely on this facility for actual delivery.

We’ve been under 20 million for a few weeks now and yet the oil markets in the USA have not blown up or ground to a halt (choose your metaphor), so clearly there is a bit of slack built into that “minimum”. Perhaps the real minimum is 18 million, 17 million, who knows? But it can’t be far below the current level and this is clearly worth monitoring.

There is of course a web page where you can check the data directly, but naturally I wanted to draw my own polished chart and label it appropriately. The code below runs a complete pipeline to download the data, process it (including preparing some smart uncluttered automatic labelling of points) and drawing the plot.

library(tidyverse)
library(scales)
library(readxl)
library(ggrepel)
library(glue)

# Download data
download.file("https://www.eia.gov/dnav/pet/hist_xls/W_EPC0_SAX_YCUOK_MBBLw.xls", 
              mode = "wb", destfile = "cushing.xls")

# Import and process data
cushing <- read_excel("cushing.xls", sheet = "Data 1", skip = 2) |> 
      rename(value = `Weekly Cushing, OK Ending Stocks excluding SPR of Crude Oil  (Thousand Barrels)`,
             end_date = Date) |> 
      # we want a label for a point only if it is at least 500 different from
      # the subsequent point (this is to avoid clutter when the line is
      # basically horizontal)
      mutate(label = ifelse(is.na(lead(value)) | 
                            abs(value - lead(value)) > 500, 
                            comma(value), ""),
      # We also want the label to disappear if the value is really close to
      # 20,000, which is going to be a clearly labelled line anyway so would
      # just be unnecessary clutter.
             label = ifelse(abs(value - 20000) < 200, "", label))

# Draw chart
cushing |> 
    filter(end_date > as.Date("2025-12-31")) |>
    # original data was in thousands but it's better to have it in millioms
    # visually:
    ggplot(aes(x = end_date, y = value / 1000)) +
    geom_hline(yintercept = 20, colour = "darkred") +
    geom_line(colour = "steelblue") +
    geom_point(colour = "steelblue") +
    geom_text(data = filter(cushing, end_date > as.Date("2026-05-01")), 
              aes(label = label, x = end_date + 150000), 
              size = 2.8, hjust = 0) +
    annotate("text", x = as.Date("2026-03-02"), y = 20.500, 
             label = "Widely cited minimum working level - 20 million barrels", 
             colour = "darkred") +
    scale_x_date(date_breaks = "1 month", date_labels = "%b") + 
    scale_y_continuous(label = comma) +
    theme(panel.grid.minor = element_blank()) +
    labs(x = "Month (2026)",
         y = "Million barrels",
         title = "Stocks of crude oil at Cushing, Oklahoma",
         subtitle = "Cushing is the main US crude oil storage and pipeline hub, and the delivery point for the West Texas Intermediary (WTI) oil benchmark.",
         caption = glue("Source: US Energy Information Administration (EIA) https://www.eia.gov/dnav/pet/hist/LeafHandler.ashx?n=PET&s=W_EPC0_SAX_YCUOK_MBBL&f=W. Accessed {Sys.Date()}."))

On R-Universe you can discover and learn everything there is to know about R packages. But did you know it does only provide a human-friendly website, but also programmatic access to all information through APIs! Thanks to those APIs you can list universes, list packages in an universe, get information on packages, and perform searches; all without any need for authentication.

The R-Universe APIs are both handy and reliable. You can build upon them, as both rOpenSci and community members have done. This post shows some examples of use cases with the R-Universe API.

Know which packages are yours

The toolbox for rOpenSci community management tasks, promoutils, calls an R-Universe API to list and get information on rOpenSci packages. This list of packages is in particular used in the function that outputs data on help-wanted issues.

Display packages developed at your organization

If your organization develops R packages, you can use its R-Universe as the source of truth for your package collection. For instance, the R-Universe of ggsegverse corresponds to, well, the ggsegverse. Therefore, on the ggsegverse website, the listing of packages is created by querying https://ggsegverse.r-universe.dev/api/packages, the endpoint for information on all packages in the universe.

Likewise, the docs page of ggsegverse relies on the R-Universe API to retrieve links to vignettes for each package.

Server-side or client-side API requests

The ggsegverse website performs client-side requests: it queries the R-Universe API when you open the webpage, through a JS script. You can also check this through the web developer tools:

The rendering of the nice package cards happens through another JS script that uses information such as the package’s title, description, number of stars, etc.

In contrast, for package listings on author and package pages, the rOpenSci website uses server-side requests: the API is called when Hugo renders our website. For instance, for the list of packages at the bottom of Jeroen Ooms’ author page, we query the R-Universe API link from our website configuration and filter the packages by GitHub login.

On a Quarto website, you could have a code chunk using R to query the R-Universe APIs, through the universe R package for instance.

In any case, if you use server-side requests, you need to re-render your website regularly to avoid your packages’ listing to get out-of-date. The nice thing about client-side requests is that the package lists will be always up-to-date!

Search Packages

With R-universe, you can search packages, not only through the web interface but also through an API.

The project The Wharehouse, that helps users find packages according to given keywords, uses R-universe as one its information sources.

Conclusion

In this post we provided some use cases of the R-Universe APIs. Try them out, or read the docs to get a sense of all the information that’s shared through the different endpoints. If you maintain some infrastructure that uses an R-Universe API, feel free to report a use case!

TLDR: this post tests whether the Worldwide Governance Indicators’ (WGI) rule of law measure provides a reasonable proxy for the World Justice Project’s (WJP) Rule of Law Index, such as when analysis might benefit from the WGI’s availability for more countries over a longer period. Results suggest the two measures broadly agree with one another when making cross-country comparisons, but care is warranted for country-level estimates where it’s likely to be more difficult to differentiate measurement noise from genuine changes in the quality of institutions.

This is the first post in a series examining relationships between the rule of law and economic and social outcomes.

In the latter half of 2025 I was engaged by the Bingham Centre for the Rule of Law and The Law Society of England and Wales to facilitate internal discussions on the links between economic growth and the rule of law. Being subject to the Chatham House Rule means much of my work won’t see the light of day, but for those interested in the topic I’d recommend taking a look at Dr Lopez-Gomez’s literature review which provides a good summary of the issues discussed.

As somebody that dedicates a lot of time to conducting cross-country comparisons and using statistics to measure amorphous concepts, I started work on the project by identifying approaches for measuring the rule of law (RoL), to help anchor analysis and discussions to something tangible definition that was rigorous enough to satisfy a room full of lawyers, without offending institutional economists.

This led me to two composite measures:

1. The World Justice Project’s (WJP) Rule of Law Index; and
2. The World Bank’s Rule of Law measure from their Worldwide Governance Indicators (WGI).

Like all indices, both are constructed by collecting an assortment of indicators that are thought to proxy something important that can’t be easily be observed or quantified. A person’s performance at work is a good example of this: as even when we might know what a good employee looks it can be hard to explicitly define and quantify. Which often results in performance being proxied through some combination of vague indicators like judgement, communication and sticktoitiveness.

Composite indices work in a similar way, except that it’s often harder to validate whether they’re measuring anything useful due to the guiding concept itself being vague and divorced from our day-to-day experience. Unsurprisingly, this has resulted in the birth of a lot of terrible indices whose sole purpose appears to be to fabricate quantitative evidence for whatever concept they’re pushing, which is likely what motivated the OECD to develop this handbook on index design and construction.

But, there are also good examples out there too. Measures that earnestly attempt to define and quantify a complex phenomena while transparently admitting the inherent limitations and uncertainties of trying to do so in the first place. I’d argue both the WGI and WJP’s rule of law measures sit in this camp, with both making their data freely available and publishing statistical audits of their methodologies (see here for the WJP and here for the WGI).

Worldwide Governance Indicators: Rule of Law

The World Bank’s WGI is intended to provide a broad measure of how well a country is governed across six dimensions, including political stability, government effectiveness and the rule of law. Indicators are selected based on a set of screen criteria and their relevance to measuring a concept of interest. For instance, the WGI’s Rule of Law indicator intends to capture:

“…perceptions of the extent to which agents respect and follow the rules of society, including contract enforcement, property rights, the police, courts, and the likelihood of crime and violence.“

Indicators used to measure the rule of law include survey-based measures of crime, violence and public trust in the justice system and relevant composite indicators, such as PRS’s ‘Contract Viability’ measure.

World Justice Project: Rule of Law Index

The WJP’s index aims to provide a comprehensive picture of characteristics relevant to the rule of law, which they define as:

“…a durable system of laws, institutions, norms, and community commitment that delivers four universal principles: accountability, just law, open government, and accessible and impartial justice.”

This is done by organizing survey-based measures organized across eight factors thought to describe the relationship between the state and civil society, which are detailed in their methodology paper:

1. Constraints on Government Powers: the extent to which those who govern are bound by law.
2. Absence of Corruption: the absence of corruption in government
3. Open Government: the extent to which a government shares information, empowers people with tools to hold the government accountable, and fosters citizen participation in public policy deliberations.
4. Fundamental Rights: measures adherence to a menu of rights firmly established under the United Nations Universal Declaration of Human Rights.
5. Order and Security: measures how well a society ensures the security of persons and property.
6. Regulatory Enforcement: measures the extent to which regulations are fairly and effectively implemented and enforced.
7. Civil Justice: measures whether ordinary people can resolve their grievances peacefully and effectively through the civil justice system.
8. Criminal Justice: rates the effectiveness of the criminal justice system as a mechanism to redress grievances and bring action against individuals for offenses against society.

Why not both?

The WJP gets one of the fundamentals right of good index design: they’ve actually thought carefully about how to conceptualize and define what they’re trying to measure, providing a richer picture of different characteristics thought to be associated with the rule of law.

However, the WJP’s measure also comes with two important weaknesses for the purpose I had in mind: It covers less countries over a shorter time period than the WGI, which might matter when conducting analysis on slow moving institutions and outcomes. And, if presented in isolation, the WJP’s measure might be taken to imply that the RoL is both a necessary and sufficient condition for driving outcomes like economic growth, while the WGI presents the RoL as part of a collection of mutually supporting institutions.

In an attempt to have the best of both worlds, my first question was therefore whether it was possible to use both measures at the same time: the WJP’s measure when a more nuanced and explicit discussion of the RoL is required; and the WGI for longer-term analysis and communicating how the RoL might sit within a wider portfolio of institutions.

Conceptual and practical overlap

At the outside, aside from overlapping conceptually, the WJP and WGI also share common data sources: with five of six dimensions from the WGI sourcing data from the WJP’, including:

• Voice and Accountability: includes data from Factor 1, 3 and 4.
• Political Stability: leverages data from Factor 5.2
• Regulatory Quality: includes factor 6.
• Rule of Law: uses data from factors 1, 4, 5, 7 and 8.
• Control of Corruption: incorporates data from factor 2.

The upshot of this is that any observed relationship identified between the WJP and WGI might just reflect overlapping ingredients, rather than some deeper conceptual agreement on the rule of law. However, I’m largely going to ignore this in my analysis as:

• Before being included in the WGI, indicators from the same source are averaged to produce a single source–dimension input, which is likely to limit the WJP’s influence; and
• Sensitivity analysis reported in the 2025 WGI methodology update indicated that their estimates were largely insensitive to changes in weightings and the exclusion of individual indicators.

Making it probable that the bulk of any observed relationships are genuine, rather than being statistical truisms.

Project Setup and Data

Data used in this post can be downloaded here for the WGI and here for the WJP’s RoL index. These datasets were current as of July 2025, but publishers frequently update their results over time as the source data and methodology evolves.

#load the packages we'll probably need
library(tidyverse)
library(readxl)
library(janitor)
library(countrycode)

#import WGI data
dta_wgi_2025<-read_excel("./Data/wgidataset_with_sourcedata-2025.xlsx",
                         sheet="rl") |> 
  clean_names()

#import World Justice Project RoL data
dta_wjp_rol<-read_excel('./Data/2025_wjp_rule_of_law_index_HISTORICAL_DATA_FILE.xlsx', sheet="Historical Data")|> 
  clean_names()

Data Cleaning

Much of the data cleaning below relates to improving how variables are named and ensuring a standard set of country and region names are applied across indices. I’ve retained some indicators that aren’t used in this post as they’ll be drawn on later in the series.

#standardize column names
dta_wgi_2025<-dta_wgi_2025 |> 
  rename(country=economy_name,
         iso3c=economy_code,
         wgi_rol=governance_estimate_approx_2_5_to_2_5)

dta_wjp_rol<-dta_wjp_rol |> 
  rename(iso3c=country_code) |> 
  rename_with(~ str_replace(., "^x", "factor_"), starts_with("x"))

#change wjp's year variable to YYYY format and convert to numeric 
#(adopts the first 4 digit year when in YYYY-YYYY format)
dta_wjp_rol <- dta_wjp_rol |> 
  mutate(year = as.numeric(str_sub(year, 1, 4)))

#cold-heartedly drop columns I'm not interested in 
dta_wgi_2025<-dta_wgi_2025 |> 
  select(iso3c, income_classification, year,wgi_rol)
#drop country and region name labels so these can be standardized 
dta_wjp_rol<-dta_wjp_rol |> 
  select(-country_year,-country,-region) |> 
  rename(wjp_rol=wjp_rule_of_law_index_overall_score)


#merge dataframes
dta_rol_unified<-left_join(dta_wgi_2025,
                           dta_wjp_rol,
                           by = join_by(year, iso3c), 
                           keep=FALSE) 

#add standardized and region names country names

#define country code assignments for legacy / ambigious codes
#(Note: matches devised by Claude) 
ref_iso3c_custom_names <- c(ADO = "Andorra",
                            ANT = "Netherlands Antilles",
                            PRI = "Puerto Rico",
                            REU = "Réunion",
                            XKX = "Kosovo")

ref_iso3c_custom_regions <- c(ADO = "Europe & Central Asia",
                              ANT = "Latin America & Caribbean",
                              PRI = "Latin America & Caribbean",
                              REU = "Sub-Saharan Africa",
                              XKX = "Europe & Central Asia")

#assign country names and regions:
dta_rol_unified<-dta_rol_unified |> 
  mutate(country_name=countrycode(iso3c, 
                                  origin='iso3c',
                                  destination = 'country.name.en',    
                                  custom_match = ref_iso3c_custom_names),
         region=countrycode(iso3c, 
                            origin='iso3c',
                            destination = 'region',    
                            custom_match =ref_iso3c_custom_regions))

Comparing Coverage

Before examining the crossover of either index, it’s probably a good idea to check what’s not covered by either source. I rely on iso3c codes to do this as in my experience they tend to be specified more consistently than country names.

Overall, a little over 73 percent of countries listed in the countrycode package are covered by either index. A cursory glance of ref_missing_iso3c indicates most of the missing countries don’t have an iso3c code, no longer exist, are territories of a larger country (or claimed as one) and/or have relatively small populations. I’m also willing to take the WJP’s word for it when they say their index covers more than ninety percent of the worlds population.

#How many countries exist in either dataset
table(codelist$iso3c %in% dta_rol_unified$iso3c) |> 
  prop.table() |> 
  round(2)

#create listing of countries missing from either index 
#(+ unecessarily using %notin% to undermine compatibility with R versions pre V4.6)
ref_missing_iso3c<-codelist[(codelist$iso3c %notin% dta_rol_unified$iso3c),] |> 
  select(iso3c, country.name.en)

#create a summary of country coverage over time
sum_rol_index_coverage_overall<- dta_rol_unified |>
  group_by(year) |>
  summarize(wjp = sum(!is.na(wjp_rol)),
            wgi = sum(!is.na(wgi_rol)),
            .groups = "drop")

Within the set of countries that are covered by be either index, the WGI provides estimates for more countries over a longer time period. Specifically, WGI RoL estimates are available from 1996 for 199 countries whereas the WJP’s measure starts in 2012 (2012-13) for 97 countries.

Although coverage has increased for both indices, it has increased more dramatically for the WJP over the period (46% compared to 8% for the WGI). The dumbbell plot below illustrates how the WJP’s coverage has increased relative to the WGI by region. Highlighting that the WJP’s coverage improvements have been heavily concentrated in Latin America and the Caribbean, and the Middle East and North Africa.

*North America’s coverage has remained the same over the period.

# compare coverage: WJP as % of WGI at index start (2012) vs end (2024)
sum_rol_index_coverage_by_region <- dta_rol_unified |>
  group_by(region, year) |>
  summarize(wjp = sum(!is.na(wjp_rol)),
            wgi = sum(!is.na(wgi_rol)),
            .groups = "drop") |>
  filter(year %in% c(2012, 2024)) |>
  mutate(wjp_pct_of_wgi = 100 * wjp / wgi) |>
  select(region, year, wjp_pct_of_wgi) |>
  pivot_wider(names_from = year, values_from = wjp_pct_of_wgi,
              names_prefix = "yr_") |>
  mutate(region = reorder(region, yr_2024))  

plt_rol_index_coverage <- ggplot(sum_rol_index_coverage_by_region, aes(y = reorder(region,yr_2024) )) +
  geom_segment(aes(x = yr_2012, xend = yr_2024, yend = region),
               colour = "grey75", linewidth = 1) +
  geom_point(aes(x = yr_2012, colour = "2012"), size = 5, alpha=0.7) +
  geom_point(aes(x = yr_2024, colour = "2024"), size = 5, alpha=0.7) +
  scale_colour_manual(values = c("2012" = "#E69F00", "2024" = "#0072B2")) +
  scale_x_continuous(labels = function(x) paste0(x, "%")) +
  labs(x = "Country Coverage (% of WGI)", y = NULL, colour = NULL,
       title = "WJP Rule of Law index coverage by Region") +
  theme_classic() +
  theme(legend.position = "top",
        panel.grid.major.y = element_line(colour = "grey92"))

plt_rol_index_coverage

Cross-country agreement between the indices

When examined globally, both measures exhibit strong agreement with one another. Suggesting that when a country achieves a poor (or strong) score on the WJP’s index they probably will on the WGI RoL measure too.

#association between the two measures - global 
plt_rol_index_comparison_global <- dta_rol_unified |> 
  filter(!is.na(wjp_rol)) |> 
  ggplot(aes(x = wgi_rol, y = wjp_rol)) +
  geom_point(aes(col = region), alpha = 0.5) +
  geom_smooth(method = "lm", col = "black", se = FALSE) +
  labs(x = "WGI rule of law index", y = "WJP rule of law index", col = "Region") +
  theme_classic()

plt_rol_index_comparison_global

#association between the two measures - global 
plt_rol_index_comparison_regional <- dta_rol_unified |> 
  filter(!is.na(wjp_rol)) |> 
  ggplot(aes(x = wgi_rol, y = wjp_rol)) +
  geom_point(aes(col = region), alpha = 0.5) +
  geom_smooth(method = "lm", col = "black", se = FALSE) +
  labs(x = "WGI rule of law index", y = "WJP rule of law index", col = "Region") +
  theme_classic()+
  facet_wrap(region ~. )

plt_rol_index_comparison_regional

# Correlation between WGI and WJP rule-of-law measures, by year -----------
dta_rol_cor_by_year_and_region <- dta_rol_unified |> 
  filter(!is.na(wgi_rol), !is.na(wjp_rol)) |> 
  group_by(year, region) |> 
  summarise(
    cor = cor(wgi_rol, wjp_rol))

#median correlation
median(dta_rol_cor_by_year_and_region$cor)

Agreement by country

Although agreement between the two measures appears less compelling at the country level, much of this likely stems from the rule of law being sticky and changing little from year to year; which results in measurement noise frequently dominating the underlying signal. This intuitive explanation appears to be largely supported by the scatter plot below, which suggests negative and weaker correlations are more likely for countries experiencing less within-country variation (aka where measurement noise is likely to dominate). The practical implication is that while caution is warranted at the country level, the two measures have a tendency to agree with one another when the observed changes are large enough not to be dominated by measurement noise.

#create list of countries with low number of observations
ref_low_n_obs<-dta_rol_unified |> 
  filter(!is.na(wgi_rol), !is.na(wjp_rol)) |> 
  group_by(country_name) |> 
  summarise(n_obs     = n()) |> 
  filter(n_obs<5) |>
  select(country_name) |> 
  unlist()
 
#calculate within-country assocation between WGI and WJI  
sum_rol_cor_country <- dta_rol_unified |> 
  filter(!is.na(wgi_rol), !is.na(wjp_rol),
         country_name %notin% ref_low_n_obs) |> 
  summarise(
    n_obs     = n(),
    signal_sd = min(sd(wgi_rol), sd(wjp_rol)),  
    cor       = cor(wgi_rol, wjp_rol),
    .by = country_name) |> 
  arrange(cor)

#plot correlation coefficient against standard deviation of either measure
plt_cor_vs_signal <- ggplot(sum_rol_cor_country, aes(signal_sd, cor)) +
  geom_hline(yintercept = 0, linewidth = 0.3, col = "grey70") +
  geom_point(alpha = 0.6) +
  geom_smooth(method = "loess", se = FALSE, col = "grey40", linewidth = 0.5) +
  labs(x = "Within-country signal (min SD of the two indices)",
       y = "WGI–WJP within-country correlation") +
  theme_classic()
plt_cor_vs_signal

First Difference Agreement

As a final test the code below transforms each index into its first difference to explore whether year to year movements in the two measures agree with one another. Because taking the first difference removes country levels, this can be a helpful way to examine whether the two series reliably move together, but it also means measurement errors are more dominant, which can result in a noisier signal.

The good news is the first difference estimates provide solid support for the two measures being reliably linked, with a pooled correlation coefficient of 0.36 and observed positive associations across each region. Providing further support that the WGI’s RoL measure can act as an imperfect and/or imprecise general proxy for the WJP’s aggregate RoL measure, but not a perfect substitute.

# First-difference agreement between WGI and WJP rule-of-law measures ------
dta_rol_diff <- dta_rol_unified |> 
  filter(!is.na(wgi_rol), !is.na(wjp_rol)) |>
  arrange(country_name, year) |> 
  mutate(
    wgi_chg = wgi_rol - lag(wgi_rol, order_by = year),
    wjp_chg = wjp_rol - lag(wjp_rol, order_by = year),
    yr_gap  = year - lag(year, order_by = year), 
    .by='country_name') |> 
  #drop differences calculated over more than one year
    filter(yr_gap == 1)                    

#calculate pooled correlation across regions 
sum_rol_diff_cor <- cor(dta_rol_diff$wgi_chg, dta_rol_diff$wjp_chg)

#produce plot
plt_rol_diff <- ggplot(dta_rol_diff, aes(wgi_chg, wjp_chg)) +
  geom_hline(yintercept = 0, linewidth = 0.3, col = "grey80") +
  geom_vline(xintercept = 0, linewidth = 0.3, col = "grey80") +
  geom_point(alpha = 0.4, size = 0.9,aes(col = region)) +
  geom_smooth(method = "lm", se = FALSE, col = "grey30", linewidth = 0.5) +
  labs(x = "Δ WGI rule of law (year-on-year)",
       y = "Δ WJP rule of law (year-on-year)",
       subtitle = paste0("Pooled correlation of changes: r = ", round(sum_rol_diff_cor, 2))) +
  theme_classic()+
  facet_wrap(region ~. )

plt_rol_diff

Buried Lede

When I asked Claude to critique this post, it said that If the intended use is ranking/trend-tracking, the correlational case is fine and you can say so explicitly. If it’s cardinal substitution, a between-country prediction-error test is needed. I told Claude that this is a ridiculous idea and that I would cancel my subscription if it didn’t return to being sycophantic. And while Claude agreed that it was misguided in questioning my judgement, my reasoning for not including this analysis in the post is that nobody should be expecting this level of precision when using composite indices. Particularly not when they’re trying to pin down something as complex as the rule of law and governance.

This is because no index can perfectly capture and/or quantifiy something as amorphous as the rule of law. But composite indices don’t always need to. For the purpose of telling a broad story about how the rule of law varies across countries and shifts over time, the WGI and WJP measures point in the same general direction, which makes the WGI a serviceable stand-in where the WJP’s shorter record falls short.

How AI was used for this post: Aside from tilting at windmills, I also used Claude to suggest refinements to my code, the title of my post and prose. But, as I’m sure is attested by the errant grammatical and spelling errors, the majority of this post was written by me.

The post Close Enough? Using the WGI as a Proxy for the WJP Rule of Law Index appeared first on Giles.

In financial markets, distinguishing between information-driven movements and liquidity-driven shocks is critical. The reference study we based our work on highlights the importance of tail analysis: comparing Gaussian (thin-tailed) and Student‑t (fat-tailed) distributions to understand whether price changes are more likely to reflect genuine information or temporary liquidity imbalances.

Financial returns are rarely as well‑behaved as the Gaussian (normal) distribution assumes. In theory, extreme price movements should be exceedingly rare under a thin‑tailed Gaussian model. Yet in practice, markets frequently exhibit fat tails: large jumps, crashes, and spikes that occur far more often than Gaussian theory predicts.

This discrepancy motivates tail analysis—a statistical approach that compares how well different distributions explain the observed data. Two common candidates are:

• Gaussian distribution (thin tails): If returns fit this model better, extreme movements are interpreted as information‑driven. In other words, new information has entered the market, and price changes are more likely to reflect genuine shifts in fundamentals or expectations.
• Student‑t distribution (fat tails): If returns fit this model better, extreme movements are considered liquidity‑driven. These shocks often arise from temporary imbalances in order flow or liquidity constraints, and prices tend to revert once the imbalance subsides.

By comparing the log‑likelihoods of Gaussian and Student‑t fits, we can classify market behavior into these two regimes. This classification is not merely academic: it helps traders, risk managers, and analysts distinguish between trend continuation (information‑driven) and mean reversion (liquidity‑driven).

In our workflow, we apply this tail analysis to gold futures (GC=F) over the past 15 trading days. We compute log returns, fit both distributions, and compare their likelihoods. We then enrich the analysis with a volume impact metric, which highlights whether abnormal trading activity amplifies price changes. Finally, we present the results in a color‑coded audit table that makes tail behavior visually interpretable.

Why These R Packages?

• tidyverse: Provides a consistent grammar for data manipulation (mutate, drop_na, select). It ensures reproducibility and readability when transforming raw market data into log returns and derived metrics.
• tidyquant: Bridges financial data sources with the tidyverse ecosystem. We use it to fetch gold futures data (GC=F) directly from Yahoo Finance, making the workflow self-contained and easy to extend to other tickers.
• MASS: Offers statistical tools for distribution fitting. We rely on fitdistr() to estimate parameters for both Gaussian and Student‑t distributions, enabling a direct comparison of log‑likelihoods.
• gt: Provides professional table rendering. It allows us to format numbers, apply color scales, and highlight audit warnings, turning raw statistical output into a visually interpretable audit table.

library(tidyverse)   # Load tidyverse for data manipulation
library(tidyquant)   # Load tidyquant for financial data retrieval
library(MASS)        # Load MASS for distribution fitting
library(gt)          # Load gt for table rendering

ticker <- "GC=F"     # Define the ticker symbol (Gold Futures)
horizon <- 15        # Set horizon to last 15 days

# Fetch market data for the chosen ticker and horizon
market_data <- tq_get(ticker, from = Sys.Date() - horizon, to = Sys.Date())

# Compute log returns and drop missing values
market_tbl <- market_data %>%
  mutate(returns = log(adjusted) - log(lag(adjusted))) %>%
  drop_na()

# Gaussian fit
fit_gauss <- fitdistr(market_tbl$returns, densfun = "normal")

# Student-t fit
fit_t <- fitdistr(
  market_tbl$returns,
  densfun = function(x, df, mean, sd) dt((x - mean)/sd, df)/sd,
  start = list(df = 5, mean = mean(market_tbl$returns), sd = sd(market_tbl$returns))
)

# Compare log-likelihoods
ll_gauss <- fit_gauss$loglik
ll_t <- fit_t$loglik
signal <- if (ll_gauss > ll_t) "INFO-DRIVEN" else "LIQUIDITY-DRIVEN"

# Build audit table
audit_tbl <- market_tbl %>%
  mutate(
    Gaussian_Density = dnorm(returns, mean = mean(returns), sd = sd(returns)),
    StudentT_Density = dt((returns - mean(returns))/sd(returns), df = 5)/sd(returns),
    Volume_Impact = abs(volume)^ifelse(signal == "INFO-DRIVEN", 1.0, 0.6),
    Audit_Warning = signal
  ) %>%
  dplyr::select(Date = date,
                Price = adjusted,
                Gaussian_Density,
                StudentT_Density,
                Volume_Impact,
                Audit_Warning)


#GT Table
audit_gt <- audit_tbl %>%
  gt() %>%
  tab_header(title = md("**Tail Analysis-Based Audit Table**")) %>%
  cols_label(
    Date = md("**Date**"),
    Price = md("**Price**"),
    Gaussian_Density = md("**Gaussian Density**"),
    StudentT_Density = md("**Student-t Density**"),
    Volume_Impact = md("**Volume Impact**"),
    Audit_Warning = md("**Audit Warning**")
  ) %>%
  fmt_number(columns = c(Price, Gaussian_Density, StudentT_Density, Volume_Impact),
             decimals = 2, use_seps = TRUE) %>%
  data_color(
    columns = c(Price),
    colors = scales::col_numeric(
      palette = c("lightgreen","darkgreen"),
      domain = range(audit_tbl$Price, na.rm = TRUE)
    )
  ) %>%
  data_color(
    columns = c(Gaussian_Density, StudentT_Density),
    colors = scales::col_numeric(
      palette = c("lightblue","darkblue"),
      domain = range(c(audit_tbl$Gaussian_Density,
                       audit_tbl$StudentT_Density), na.rm = TRUE)
    )
  ) %>%
  data_color(
    columns = c(Volume_Impact),
    colors = scales::col_numeric(
      palette = c("pink","red"),
      domain = c(min(audit_tbl$Volume_Impact, na.rm = TRUE),
                 max(audit_tbl$Volume_Impact, na.rm = TRUE))
    )
  ) %>%
  text_transform(
    locations = cells_body(columns = vars(Audit_Warning)),
    fn = function(x) {
      ifelse(x == "INFO-DRIVEN",
             "<span style='color:green;font-weight:bold;'>INFO-DRIVEN</span>",
             "<span style='color:red;font-weight:bold;'>LIQUIDITY-DRIVEN</span>")
    }
  )

audit_gt

Keynote slides: https://jakubnowosad.com/ml4eo2026/

Workshop materials: https://jakubnowosad.com/ml4eo2026workshop/

Machine learning is now deeply embedded¹ in Earth observation workflows, from mapping current environmental conditions to forecasting future change. However, the quality of a spatial prediction map cannot be judged only by how well a model performs on a convenient test sample. In spatial problems, the gap between where we have observations and where we want to make predictions is often a crucial factor in determining whether a model can be trusted.

At the Machine Learning for Earth Observation 2026 conference in Exeter², I gave a keynote talk entitled Rethinking Validation for Spatial Machine Learning (June 22, 2026). The next day, I showed some practical ways to implement these ideas in a workshop called Where your models can be trusted: evaluating spatial machine learning reliably (June 23, 2026). Both focused on the same general question: how can we evaluate spatial machine learning in a way that reflects the actual prediction task?

The keynote was structured around three assumptions that are easy to make, but often unsafe in spatial prediction:

• We can predict everywhere. In practice, we validate where we have data, but predict in places that may be poorly represented by the training sample. Tools such as Area of Applicability (AoA) and Local Point Density (LPD) help identify parts of the prediction domain where environmental conditions are more or less supported by the available data.
• There is one “correct” validation approach. In reality, validation should follow the prediction task. Random cross-validation can be too optimistic when observations are spatially clustered, while spatial cross-validation can be too pessimistic when the intended prediction scenario is closer to interpolation. Adaptive strategies such as k-Nearest Neighbor Distance Matching (kNNDM) try to align validation folds with the distance structure of the prediction domain.
• All validation points are equal. Prediction conditions are not equally common across a map, so a single unweighted average error can misrepresent the expected performance over the full prediction domain. This motivates thinking about how validation samples should be weighted by their prevalence in the places where predictions will be used.

Together, these points lead to the idea of prediction-domain adaptive evaluation: first define the prediction domain, then construct validation folds that reflect it, and finally summarize performance in a way that accounts for how common different prediction conditions are. This is not a complete theory of spatial machine learning evaluation, but it is a useful step away from treating validation as a model-only problem. (To learn more about these ideas, read our preprint: https://arxiv.org/abs/2605.13689.)

The workshop turned these ideas into practical R workflows. Using synthetic and real-world-inspired examples, we used and discussed techniques for Area of Applicability, Local Point Density, compared random cross-validation, spatial cross-validation, and kNNDM cross-validation, and looked at error profiles. The hands-on materials also include exercises, where participants can compare validation strategies, map areas of applicability, and explore how expected error varies across space.

The main takeaway is simple: for spatial machine learning, the question is not only How accurate is the model? It is also Where can the model be trusted?

@online{nowosad2026,
  author = {Nowosad, Jakub},
  title = {Rethinking {Validation} for {Spatial} {Machine} {Learning:}
    {Takeaways} from the {Talk}},
  date = {2026-07-03},
  url = {https://jakubnowosad.com/posts/2026-07-03-ml4eo/},
  langid = {en}
}

I recently had the opportunity to learn what the term “Nerd Sniping” meant. Maëlle pointed out a conversation on the rOpenSci Slack about something called Vale, meant for text linting. I’d seen the comment, but honestly hadn’t really understood what it was all about until Maëlle asked if I thought it’d be useful for editing the blog…

…time passes…

About three days later, I’ve hardly finished any of the blog post reviews I was planning to do. I’ve been sucked down a rabbit hole of Vale setup, custom rules, and overrides.

It turns out that “Nerd Sniping” refers to the practice of throwing problems at nerds that distract them from what they were doing.

That being said, it was a glorious hole to fall down! It was just too bad that Maëlle sniped me two more times by asking me about my spell check setup in Positron and then by asking if Panache would help with translations.

I was pretty slow at my editorial duties that week! But I did come out of the dive with a great editorial setup which will definitely save me time in future.

I’ve edited a lot of posts on the rOpenSci blog. I take pride in helping writers get their ideas across with clarity, while not losing their own style. I’m an opinionated editor, so I also try hard to ensure that writers understand when my suggestions are just my opinion, and when I think there are mistakes in style or content that really do need to be fixed. I am also fussy about the details, about being consistent with capitalizations, about keeping ideas logically ordered, and about making sure that readers without the same background might still understand the gist of the post¹.

As such, my post reviews can get a bit lengthy and it’s not unreasonable for me to have 20-30 comments on a standard post. That’s not too problematic, but if I had to complain it might be about the technical edits, like fixing the capitalization of ‘rOpenSci’², ensuring headings are in sentence case, and that links to ropensci.org pages are relative. These aren’t complicated fixes, but if you have to remember to keep an eye out for them, and then create a GitHub PR review suggestion for each fix, it can become a tad tedious.

Maëlle’s timely sniping helped me finalize my collection of tools to help streamline editorial tasks.

• Spell checking with cSpell
• Linting³ with Vale
• Formatting with Panache
• Creating GitHub PR suggestions with GitHub Pull Requests

All of these tools can be installed and used in different ways. They are also powerful with many different possible customizations and configurations. Here, I’ll share with you how I use these tools as extensions in Positron to help make it easier to write and edit posts for the rOpenSci blog. Hopefully this inspires you to explore how you might set them up to support your workflows! Further, if you’re interested in setting up your own tools, perhaps you want to check out this blog post on “All the Ways to Programmatically Edit or Parse R Markdown / Quarto Documents”.

General setup

For each tool, you’ll want to install the Positron extension, and then set up your configuration. Configurations can usually be specified at three different levels:

• User: Your system-wide setup which is how you want things to work in general across projects. User config files are generally stored somewhere in your home directory.
• Project: Project-wide setup which overrides your user setup if the project does things differently. These config files are stored in the project directly (like roweb3, for the rOpenSci blog).
• File: File or file section setup which works at a very local scale. Usually this configuration is indicated by in-file comments.

More specifically here are (some) of the locations/names for configuration files and links to their documentation sections for more details.

This means you can have different rules for different projects, and override them as needed. In the following examples, I’ll show you how I do this for posts on the rOpenSci blog.

Code Spell Checker (cSpell)

First is my spell checker, which probably doesn’t need much explanation. However, it’s nice to use a spell checker which also works on code. I use the Code Spell Checker (cSpell) extension by Street Side Software and installed the languages extensions individually:

• Canadian English⁴ – Code Spell Checker
• French – Code Spell Checker
• Portuguese – Code Spell Checker
• Spanish – Code Spell Checker

Alternatively, you could also install the cSpell Bundled Dictionaries instead.

To configure this extension, I added two types of files: a project-level configuration file, and two dictionaries of words to consider ‘correct’.

The project level configuration file, .cspell.json, lists languages to use for different files (to ensure index.es.md files go through the Spanish spellchecker, while index.pt.md files go through the Portuguese spellchecker, etc.). It also includes a list of globs for file paths we can ignore (I’m really not interested in spelling mistakes in the .git folder), as well as pointing to dictionaries.

These dictionaries are initially created by functions from my promoutils package, an R package for all my rOpenSci community workflows. wordlist_create() creates a wordlist based on rOpenSci packages and author names, so they don’t trigger the spell check if they aren’t recognized. wordlist_update() updates this list with new names as needed.

We keep these dictionaries in a .wordlists folder. Names are stored in the .wordlists/names.txt file, and we also have a .wordlists/words.txt file which stores words which are considered correct in the rOpenSci context (like ‘usecases’).

I should also note that I have a personal list of user words stored in my Positron user settings which lists words (like my name!) which I want to be considered correct across all projects.

When writing posts, we can also override the language settings within a post using a special comment. For example if we want to use English and Portuguese for a post we could add <--- cSpell: language en,pt--> to the document.

We can also include post-specific words to ignore, which is handy for acronyms. For example, if we wanted to ignore the acronym CSCW we could use <!--- cSpell: ignore CSCW ---> at the top of a post.

Spell check issues pop up as a warning in my text window, or as a list under “Spell Checker Issues By File” my lower window pane so I can review them, add them to word lists, or just mentally ignore them.

Vale

For linting text (checking the style and meaning of the words) I use the Vale VSCode extension by chrischinchilla⁵. Vale helps me check that the Blog Style rules are respected, and gives suggestions for alternative word choices to avoid common mistakes (such as words or expressions which might be derogatory).

To setup Vale I created a project-specific Vale configuration file .vale.ini⁶ in the roweb3 repository. I keep my personal .vale.ini file in a higher level folder that holds all my R projects. In addition to the Vale configuration file, I also created a Vale styles folder in roweb3/.vale-styles. This is where Vale rules are installed if we use predefined rules, and where I can put rOpenSci-specific rules for the blog. The first time you use Vale you’ll want to run vale sync in the terminal to install the standard, non-custom, rules. I .gitignore all rules which are installed, but track and push custom rules.

Vale is where I’ve made the most customizations, especially with the rOpenSci Blog.

• I’ve added a specific Blog vocab list to ensure proper capitalization of rOpenSci projects and (not to mention “rOpenSci” )
• I’ve turned off a lot of specific rules which are a bit too aggressive for a blog which allows people to write casually and informally as they like (including using words like “very” ).
• I’ve created custom rules to modify existing rules ⁷
• I’ve created custom rules to enforce our style guide, like using Title Case for blog post titles⁸, sentence case for subheadings, and using relative links for ropensci.org pages.

This is just the start! I imagine the more I use these rules the more fine tuning I’ll do.

Vale problems are classified as messages, warnings, or errors, and are highlighted in the text window as a quick fix and listed in the Problems pane in my lower window.

I should also note that for all the rules I’ve disabled, there are a lot of opinionated rules left. We keep them as prompts to think about our writing, not because we must follow them!

Panache

For formatting text, I use the Panache extension by jolars to format the (R)markdown files for the blog. This is probably the smallest amount of setup, as all we need is a minimal .panache.toml configuration file in the roweb3 repository. However, this file instructs Panache to do one super awesome thing for us, especially for translations of multilingual blog posts:

[format]
wrap = "sentence"

If you set up Positron to format on save, Panache automatically wraps text by sentence every time you save the file. This means that when a blog post is sent for a first pass translation using babeldown, the translation comes back pretty good. Alternatively, if the line breaks are in the middle of a sentence, the translation can become garbled as lines are treated as disjointed sections of text.

For my other work, I use wrap = "reflow", set in my user configuration file in ~/.config/panache/config.toml.

GitHub Pull Requests

Finally, once I’ve got all the fiddly edits on a post’s (R)md file ready to go, I use the GitHub Pull Requests extension to convert these edits to GitHub PR review suggestions. This is really handy if you find yourself making many small suggested changes to GitHub PRs.

To review blog posts, I fetch the PR with usethis::pr_fetch(), and then open the blog post (R)md file in Positron side by side with the html preview of the post in my web browser.

Then I review the html preview and make the edits directly in the (R)md file. When I’m done, I right click on the edited file name in the Source Control > Changes and select Create Pull Request Suggestions.

This converts my edits to GitHub PR review suggestions which I can then review in Positron, or as I prefer, in a web browser (and fix weird ones, such as suggestions which delete part of a section in one edit but add it back in the next; it’s not always a perfect process). Once all the suggestions are converted, the extension asks me if I want to revert my changes (which I usually do).

A note of caution, I find this tool a bit confusing to use on a PR that has a lot of comments already. The comments it makes are sometimes hidden or split in odd ways and it’s easy to accidentally create duplicates. In these situations it’s sometimes easier just to make the suggestions in a browser as you might normally.

Why so many tools?

Each of these tools provides me a specific solution to a problem. There is some overlap among them; Vale could do spell checks, and Panache could do linting. However, I find that by using the tools separately I can achieve an especially detailed and customized setup that works really well with the rOpenSci blog in particular, and with my work in general.

By including the configuration files in the roweb3 repository, people who also use these tools will automatically use the configurations we’ve setup for the rOpenSci blog when they write a post. We also plan to add instructions for how to use these tools to the Blog Guide. This should give blog writers the option of using these tools if they would like to.

However, even if other writers don’t use these tools, it’s still very useful for me to see a list of potential problems to double check at the end of my review without having to remember to check for them manually. It means I can focus more on the review of the content rather than worry about whether it’s Ropensci or rOpenSci

Making your first contribution to open source can be both empowering and yet very intimidating.

– rOpenSci FTC Guide

Last year we were grateful to receive funding from NumFOCUS¹ to organize a series of events designed to reduce barriers restricting First-Time Contributors to Free and Open Source Software (FOSS). There are many barriers² to first time contributions, but making these contributions can be an empowering experience. To help reduce some of these barriers we hosted two types of events: mini-translathons and mini-hackathons.

A mini-translathon is a short, live, coworking session, focused on translation and localization contributions. Participants review and improve translations of documentation, websites, or other resources. They also work with guidance from mentors and editors, often collaborating in language-specific groups. The goal is to make content accessible in multiple languages while helping newcomers learn translation workflows and tools.

We paired our mini-translathon with a Portuguese Community Call (A comunidade R fala português³) which preceded the mini-translathon at LatinR 2024.

Similarly, a mini-hackathon is a short (typically ~2-hour), live, online coworking session designed to help people make their first contributions to open-source software. Participants work on small, well-prepared tasks such as fixing bugs, improving code, or updating documentation. Maintainers and mentors are available in real time to guide them, answer questions, and support the contribution process. The focus is on learning by doing in a collaborative and supportive environment.

Again, we paired our mini-hackathons with an English Community Call (From Novice to Contributor: Making and Supporting First-Time Contributions to FOSS) which was then followed by the two mini-hackathons to support contributors to coding projects.

Part of the grant we received also allowed us to write up our processes and findings as a guide book “From User to Contributor: Organizing Events for First-Time Contributors”, which we are excited to share with you!

In our guide we start by discussing why supporting first time contributors is important. We describe our pilot events, how they went, feedback we received, and ideas for future improvement. The main chapters of the guide then cover how to run these events in greater detail. This includes Timelines, Community Calls, Mini-translathons, and Mini-hackathons, all from the perspective of supporting first time contributors. In the Appendices we include communication examples and templates.

We hope that this guide can be useful to other communities beyond rOpenSci. If you use this guide to create your own events to support first time contributors, we hope you’ll let us know!

Thanks to NumFOCUS for the Small Development Grant to support this work.

Three hundred twenty-three of the new packages were submitted to CRAN in May. Here are my Top 40 picks in eighteen categories: Artificial Intelligence, Computational Methods, Ecology, Education, Finance, Functional Data Analysis, Genomics, Machine Learning, Medical Statistics, Meta Analysis, Probability, Process Control, Psychometrics, Statistics, Surveys, Time Series, Utilities, and Visualization.

Dear rOpenSci friends, it’s time for our monthly news roundup! You can read this post on our blog. Now let’s dive into the activity at and around rOpenSci!

rOpenSci HQ

Champions Program update

We have two concurrent cohorts, both in Spanish.

The 2025–2026 cohort is nearing the end of its participation in the program, so we are organizing the closing meeting and the overall evaluation.

The 2026–2027 cohort is continuing their training activities, meeting with their mentors, and starting to work on their packages, and they have been formally introduced on our blog! Read all about the 11 new Champions.

New editors Ronny Hernandez Mora, Joel Nitta, and Nick Tierney

We’re thrilled to introduce new editors Ronny Hernandez Mora, Joel Nitta, and Nick Tierney. An official welcome and thank you to all three!

A new guide: Organizing Events for First-time Contributors

Steffi LaZerte and Yanina Bellini Saibene released a fantastic new rOpenSci guide! Learn how to organize events for first-time contributors such as mini-hackathons and mini-translathons. Read more in the release announcement.

R-Universe updates

“Five recent R-Universe features you might have missed”: A clickbait title for a blog post you don’t want to miss! Jeroen Ooms describes five recent additions to the R-Universe platform:

• Social media cards that actually look good
• PACKAGES.rds support (or: implementing R internals in JavaScript)
• Fancy sort/filter bars in the WebUI
• For the impatient: trigger a sync manually
• Making check results easier to find and share

In other news, R-universe user Tom Palmer also wrote about five things: “Five tips for managing your R-universe ”. You won’t believe the fifth one.

Yanina Joins the 2026 Sovereign Tech Fellowship

We’re excited to share that our Community Manager, Yanina Bellini Saibene, has been selected as a 2026 Sovereign Tech Fellow. During the fellowship, she will focus on making open source more accessible through improved contribution guidance, newcomer-focused mini-hackathons, multilingual training resources, and more sustainable localization practices across communities in the R ecosystem. These efforts will build on and extend rOpenSci’s work in community building, mentorship, and open science.

Quinceañera: celebrating 15 years together

In June, we held two community events and a co-working session to mark rOpenSci’s 15th anniversary. Across all three sessions, people shared memories of their first contribution, discussed ideas for the next 15 years, and reminded us of how genuinely welcoming rOpenSci and it’s community are. There’s more to come Keep an eye out for what we have planned for the rest of the year.

Software Peer-Review updates

Community member Athanasia Mo Mowinckel has started a new AI agent “skills” repo at ropensci-review-tools/ropensci-skills. The repo holds a variety of “skills”, which are human-readable markdown files, for AI agents to assist in preparing software for peer-review. Anybody thinking about using AI systems to prepare software for peer-review is encouraged to try out these experimental skills, and to help us improve them for others by opening issues or pull requests in the GitHub repo.

Our recent updates to the goodpractice package have also been enhanced with an all-new AI “skill”. This skill instructs agents to edit and improve your package’s code to comply with the full suite of goodpractice checks. You can try it out with the package’s new use_skill_gp() function.

Coworking

Read all about coworking!

• Tuesday July 7nd 2026, 09:00 Americas Pacific (16:00 UTC) “Debugging in R”, with Yanina Bellini Saibene and cohost Shannon Pileggi.
  • Read up on debugging in R.
  • Meet community host, Shannon Pileggi, and discuss tips and tricks for debugging in R.
• Tuesday August 4th, 09:00 Australia Western (01:00 UTC) “Vale and Text Linting”, with Steffi LaZerte and cohost Jonathan Carroll.
  • Read up on text linting.
  • Setup a linting framework for your projects.
  • Meet co-host, Jonathan Carroll, and discuss Vale and text linting.
• Tuesday September 1st, 14:00 Europe Central (12:00 UTC) “Getting to Know SORTEE”, with Steffi LaZerte and cohost Ed Ivimey-Cook.
  • Visit SORTEE (Society for Open, Reliable, and Transparent Ecology and Evolutionary Biology).
  • Meet co-host, Ed Ivimey-Cook, and learn more about SORTEE and how you might get involved.

And remember, you can always cowork independently on work related to R, work on packages that tend to be neglected, or work on what ever you need to get done!

Software

The following two packages recently became a part of our software suite:

• pvEBayes, developed by Yihao Tan together with Marianthi Markatou, Saptarshi Chakraborty, and Raktim Mukhopadhyay: A suite of empirical Bayes methods to use in pharmacovigilance. Contains various model fitting and post-processing functions. For more details see Tan et al. (2025) https://doi.org/10.1002/sim.70195, https://doi.org/10.48550/arXiv.2512.01057; Koenker and Mizera (2014) https://doi.org/10.1080/01621459.2013.869224; Efron (2016) https://doi.org/10.1093/biomet/asv068. It has been reviewed by Kathryn Doering and Collin Cademartori.

• nycOpenData, developed by Christian Martinez: Provides a unified set of helper functions to access datasets from the NYC Open Data platform https://opendata.cityofnewyork.us/. Functions return results as tidy tibbles and support optional filtering, sorting, and row limits via the Socrata API. The package includes endpoints for 311 service requests, DOB job applications, juvenile justice metrics, school safety, environmental data, event permitting, and additional citywide datasets. It has been reviewed by Haolin Dong and Michael Pascale.

Discover more packages, read more about Software Peer Review.

New versions

The following seventeen packages have had an update since the last newsletter: weathercan (v1.0.0), occCite (v0.6.2), lightr (v2.0.0), gutenbergr (v0.5.2), slopes (v2.0.0), qualtRics (v3.3.0), srr (v1.0.0), goodpractice (v1.1), pkgmatch (v0.5.4), pkgstats (v0.2.3), cffr (v1.4.1), dfms (v1.0.1), osmdata (v0.4.0), aRxiv (0.20), Athlytics (v1.0.6), ReLTER (3.1.1), and read.abares (v3.0.0).

The writexl package has a new maintainer, Bill Denney. NLMR is now maintained by Jakub Nowosad.

Software Peer Review

There are eighteen recently closed and active submissions and 4 submissions on hold. Issues are at different stages:

• Four at ‘6/approved’:

  • pvEBayes, Empirical Bayes Methods for Pharmacovigilance. Submitted by Yihao Tan. (Stats).

  • nycOpenData, Convenient Access to NYC Open Data API Endpoints. Submitted by Christian Martinez.

  • ernest, A Toolkit for Nested Sampling. Submitted by Kyle Dewsnap. (Stats).

  • pkgmatch, Find R Packages Matching Either Descriptions or Other R Packages. Submitted by mark padgham.

• Two at ‘5/awaiting-reviewer(s)-response’:

  • lakefetch, Calculate Fetch and Wave Exposure for Lake Sampling Points. Submitted by jeremylfarrell.

  • priorsense, Prior Diagnostics and Sensitivity Analysis. Submitted by Noa Kallioinen. (Stats).

• Five at ‘4/review(s)-in-awaiting-changes’:

  • RAQSAPI, A Simple Interface to the US EPA Air Quality System Data Mart API. Submitted by mccroweyclinton-EPA.

  • RAMEN, RAMEN: Regional Association of Methylome variability with the Exposome and geNome. Submitted by Erick Navarro-Delgado.

  • logolink, An Interface for Running NetLogo Simulations. Submitted by Daniel Vartanian.

  • rcrisp, Automate the Delineation of Urban River Spaces. Submitted by Claudiu Forgaci. (Stats).

  • galamm, Generalized Additive Latent and Mixed Models. Submitted by Øystein Sørensen. (Stats).

• Two at ‘3/reviewer(s)-assigned’:

  • ciecl, International Classification of Diseases ICD-10/ICD-11 for Chile. Submitted by Rodolfo Tasso.

  • EpiStrainDynamics, Infer temporal trends of multiple pathogens. Submitted by Saras Windecker. (Stats).

• Two at ‘2/seeking-reviewer(s)’:

  • fcmconfr, Fuzzy Cognitive Map Analysis in R. Submitted by benroston. (Stats).

  • coevolve, Fit Bayesian Generalized Dynamic Phylogenetic Models using Stan. Submitted by Scott Claessens. (Stats).

• Three at ‘1/editor-checks’:

  • grumpy, Read NumPy .npy and .npz Files. Submitted by Hugo Gruson.

  • metasurvey, Reproducible Survey Data Processing with Step Pipelines. Submitted by Mauro Loprete.

  • LBDiscoverAnalysis, Co-occurrence Discovery Models and Visualization for Biomedical LBD. Submitted by Chao Liu.

Find out more about Software Peer Review and how to get involved.

On the blog

Software Review

• Ronny Hernandez Mora, Joel Nitta, and Nick Tierney Join rOpenSci Software Peer Review Editorial Team by Ronny Hernandez Mora, Joel Nitta, Nicholas Tierney, and Yanina Bellini Saibene. Introducing three new editors for rOpenSci software peer review.

• Celebrating Our Maintainers during Maintainers Month by Yanina Bellini Saibene. A Look Back at our Maintainer Month 2026 social media campaign.

• Our goodpractice Package Has New Superpowers by Mark Padgham and Athanasia Mo Mowinckel. We have worked hard over the past few months on major upgrades to our goodpractice package. Checks are now grouped into categories, making it easier to control which checks are run. The biggest change has been adding over 100 new checks, from new lints to many new CRAN checks.

• A New Guide: Organizing Events for First-time Contributors by Steffi LaZerte and Yanina Bellini Saibene. We introduce our Guide book for organizing events to support first-time contributors to FOSS.

• Five recent R-universe features you might have missed by Jeroen Ooms. In this technote we look at a few recent additions that make R-universe a little nicer, faster, or more convenient to use.

• Eleven Latin American Voices for Open Science: The New Cohort of Champions rOpenSci 2026 by Bastián Olea Herrera, Denisse Fierro Arcos, Durga Valentina Linares Herrera, Evelia Lorena Coss Navarrete, Gladys Choque Ulloa, José Daniel Conejeros, Linda Cabrera Orellana, María Florencia Tames, Marina Cecilia Cock, Patricia A. Loto, Estefania Torrejón, and Yanina Bellini Saibene. Introducing 11 new rOpenSci Champions. Other languages: Once voces latinoamericanas para la ciencia abierta: la nueva cohorte de Campeon(a|e)s rOpenSci 2026 (es).

Calls for contributions

Calls for maintainers

If you’re interested in maintaining any of the R packages below, you might enjoy reading our blog post What Does It Mean to Maintain a Package?.

• charlatan, create fake data in R. Issue for volunteering.

• hddtools, Tools to discover hydrological data, accessing catalogues and databases from various data providers. Issue for volunteering.

Calls for contributions

Refer to our help wanted page – before opening a PR, we recommend asking in the issue whether help is still needed.

Package development corner

Some useful information for R package developers.

goodpractice’s new features and behind-the-scene notes

Software Review Lead Mark Padgham and long-time community member Athanasia Mo Mowinckel have written a blog post particularly relevant to package developers for two reasons:

• Learn how goodpractice, which helps make your package better, has improved.
• Read how Mark and Mo collaborated, including their use of LLMs in the development process.

Dumb Ways for an Open Source Project to Die

If you’re interested in open-source software projects’ survivability, you’ll enjoy this write-up by Andrew Nesbitt shared by Yanina Bellini Saibene.

Refactoring with Jarl: unused functions and more

Hannah Frick and Maëlle Salmon wrote “Refactoring with Jarl: a coffee chat” on the R-hub blog.

A strategy for recovering data on request interruption

Gábor Csárdi summarized recent changes to the gh package. Especially interesting is his strategy for interruptions: the user starts a long query then interrupts the process… how to not lose the data that’s already been received? The solution is to make it accessible through rlang::last_error(). More details in the post.

curl summer of bliss

The curl project announced that it will not accept any vulnerability report during the month of July this year. This is both the opportunity for maintainers to take a break, and to advertise paid curl support, in which there will be no interruption of service.

To conventionally commit or not

Sumner Evans wrote an interesting post criticizing the conventional commits convention (starting commits with e.g. fix: for bug fixes, feat: for new features, etc).

More than .gitignore

Nelson Figueroa wrote a useful overview of the different ways to make Git ignore some files.

How to work with LLMs without losing your skills

Vicki Boykis wrote an insightful post “We should be more tired than the model” including pratical tips such as “Starting to use the agent only after I’ve spent 20 minutes on the problem” or “Discussing an agent’s proposed implementation with another person instead”.

Last words

Thanks for reading! If you want to get involved with rOpenSci, check out our Contributing Guide. This guide will help direct you to the right place, whether you want to make code contributions, non-code contributions, or contribute in other ways such as through sharing use cases. You can also support our work through donations.

If you haven’t subscribed to our newsletter yet, you can do so though our signup form. Until it’s time for our next newsletter, you can keep in touch with us through our website, Mastodon, or LinkedIn. See you soon!

This dataset, found in one of my old external drives, corresponds to the famous plot from Radio Observations of the Pulse Profiles and Dispersion Measures of Twelve Pulsars (Craft, 1970).

This is broadly known as the Joy Division’s cover from Unknown Pleasures. If you happen to know whom created the original CSV I used, please let me know so I can give proper credit.

The dataset contains “successive pulses from the first pulsar discovered, CP 1919, are here superimposed vertically. The pulses occur every 1.337 seconds. They are caused by rapidly spinning neutron star.” (The Cambridge Encyclopaedia of Astronomy, 1977)

Thanks to Scientific American, there is a complete explanation of the dataset and its origin.

The contribution I made consists in:

1. Easing the access to this very popular dataset.
2. Documenting everything and organizing the columns in a clear way (I hope).

A few days ago I wrote about the Tidyverse/Shiny internals, so here I will show how to plot this dataset exactly like the very popular Joy Division t-shirts both with ggplot2 and tinyplot. This is a good way to think more actively rather than resorting on muscular memory at my age and years using R.

Install

From CRAN

install.packages("cp1919")

From GitHub

pak::pkg_install("pachadotdev/cp1919")

Read

library(cp1919)
head(pulsar)
  measurement time radio_intensity
1           1    1           -0.81
2           1    2           -0.91
3           1    3           -1.09
4           1    4           -1.00
5           1    5           -0.59
6           1    6           -0.82

Visualize

Simple plot

This looks nothing like the Joy Division album cover but it is the starting point.

library(ggplot2)

ggplot(pulsar) +
    geom_line(
        aes(x = time, y = radio_intensity)
    ) +
    facet_wrap(~measurement)

library(tinyplot)

plt(
    radio_intensity ~ time,
    data = pulsar,
    facet = ~measurement
)

The Cambridge Encyclopaedia of Astronomy (1977)

Now we get a plot with the stacked waves.

With ggplot2 the easy option is to rely on ggridges that does a great job stacking the series.

library(ggridges)

col1 <- "white"
col2 <- "black"

ggplot(pulsar, aes(x = time, y = measurement, height = radio_intensity, group = measurement)) +
  geom_ridgeline(
    min_height = min(pulsar$radio_intensity),
    scale = 0.2,
    linewidth = 0.5,
    fill = col1,
    colour = col2
  ) +
  scale_y_reverse() +
  theme_void() +
  theme(
    panel.background = element_rect(fill = col1),
    plot.background = element_rect(fill = col1, color = col1),
  )

With tinyplot I have to go back one decade to the past and remember base R to adapt from ggridges internals.

pulsar2 <- transform(pulsar, measurement = factor(measurement))
measurements <- sort(unique(pulsar2$measurement))
n <- length(measurements)

# integer baselines: identical to ggridges ymax = y + scale * height
scale_fac <- 0.2  # mirrors geom_ridgeline(scale = 0.2)

pulsar2$y_stacked <- scale_fac * pulsar2$radio_intensity +
    (n - match(pulsar2$measurement, measurements))

par(bg = col1, mar = c(0, 0, 0, 0))

plt(
    y_stacked ~ time | measurement,
    data = pulsar2,
    type = type_area(alpha = 1),
    ylim = range(pulsar2$y_stacked),
    bg = col1,
    col = col2,
    axes = FALSE,
    legend = FALSE,
    frame.plot = FALSE
)

The Nature of Pulsars (Scientific American, 1970)

Similar to the previous plots.

col1 <- "#94cee1"
col2 <- "white"

ggplot(pulsar, aes(x = time, y = measurement, height = radio_intensity, group = measurement)) +
  geom_ridgeline(
    min_height = min(pulsar$radio_intensity),
    scale = 0.2,
    linewidth = 0.5,
    fill = col1,
    colour = col2
  ) +
  scale_y_reverse() +
  theme_void() +
  theme(
    panel.background = element_rect(fill = col1),
    plot.background = element_rect(fill = col1, color = col1),
  )

par(bg = col1, mar = c(0, 0, 0, 0))

plt(
    y_stacked ~ time | measurement,
    data = pulsar2,
    type = type_area(alpha = 1),
    ylim = range(pulsar2$y_stacked),
    bg = col1,
    col = col2,
    axes = FALSE,
    legend = FALSE,
    frame.plot = FALSE
)

Joy Division’s Unknown Pleasures (1979)

Now we get a plot you can print on a T-Shirt.

col1 <- "black"
col2 <- "white"

ggplot(pulsar, aes(x = time, y = measurement, height = radio_intensity, group = measurement)) +
  geom_ridgeline(
    min_height = min(pulsar$radio_intensity),
    scale = 0.2,
    linewidth = 0.5,
    fill = col1,
    colour = col2
  ) +
  scale_y_reverse() +
  theme_void() +
  theme(
    panel.background = element_rect(fill = col1),
    plot.background = element_rect(fill = col1, color = col1),
  )

par(bg = col1, mar = c(0, 0, 0, 0))

plt(
    y_stacked ~ time | measurement,
    data = pulsar2,
    type = type_area(alpha = 1),
    ylim = range(pulsar2$y_stacked),
    bg = col1,
    col = col2,
    axes = FALSE,
    legend = FALSE,
    frame.plot = FALSE
)

A few months ago I started an experiment, the Q IDE, a Qt-based IDE for R. I started it because I felt that the old RStudio was nice but I wanted it to be even simpler, closer to a scientific calculator like the old TI-89 Titanium.

While working on Q, I noticed that while the name is fine for geeks like me (e.g., think 007 or QR matrix decomposition), it can lead to more confusion for new R users.

Today we have Positron, VS Code, VSCodium, and a wide range of IDEs that in my opinion are a bit “oversized” for daily data analysis with R.

R GUI 2, named after the R GUI for Windows, is a multiplatform project that aims to make using R as simple as possible, integrating everything into a single Windows rather than multiple small windows and making it intentionally similar to Stata’s UI.

One of the reasons why I want to make this similar to Stata visuals is that I am doing a PhD in Economics, where Stata is predominant and it offers an amazingly simple UI despite that R has the same functions and more. People who are smarter than me tend to feel overwhelmed with R and I think the R GUI 2 can minimize that problem by giving a streamlined R experience.

If you would like to contribute to complete a good R GUI 2 v1.0, please comment here. My idea is to run a fully collaborative project and that will require multiple inputs from the R community, including testing, comments, and checking how it works on different computers. At the moment I can only say “it works on my computer”.

The R GUI currently looks like this.

Great news that a major Statistics prize like the Rousseeuw Prize goes to the R Core Team, esp. Brian Ripley (University of Oxford), Martin Mächler (ETH Zürich), Kurt Hornik (WU Wien), Peter Dalgaard (Copenhagen Business School), and Luke Tierney (University of Iowa). R is indeed a unique phenomenon, where open-source and open-access has been developed by and for the statistics community. Which is about to release R version 4.6.1 (Happy Hop) version.

Thanks to the R Core Team (and congrats!). Half of the Prize goes to the other members of the Team.

“The international and independent jury, appointed by the King Baudouin Foundation, has recognised the groundbreaking work of five members of the R Core Team who have been awarded the Rousseeuw Prize for Statistics. The international award, which recognises major contributions to statistical research, honours their nearly three decades of unpaid work building R, the open-source language that has become the common foundation of modern statistical computing.

Statistics is everywhere. It determines whether a new medicine is safe enough to reach patients, monitors risk in financial markets, and tracks how diseases spread. R is the tool that made it accessible to everyone. The language that the R Core Team built is trusted by institutions including the US Food and Drug Administration, the European Central Bank, the Bank of England, and major pharmaceutical companies worldwide.”

After all 72 matches of the group stage in the 2026 FIFA World Cup our probabilistic forecasts are evaluated, revealing what the machine learning algorithm predicted well and where it struggled.

A challenging new tournament format

A couple of days ago the group stage of the 2026 FIFA World Cup was wrapped up after squeezing 72 matches into just a little bit more than two weeks. Thus, all pairings for the Round of 32 are fixed now. Today we want to assess the quality of our own probabilistic forecast for the 2026 FIFA World Cup based on an ensemble machine learning algorithm that we have published prior to the tournament.

Most of our predictions worked reasonably well and the corresponding results are within the limits of expected random variation. It turned out, though, that the switch from 32 to 48 teams in the tournament was not only challenging for the audience but also for the machine learning algorithm. There were many more matches between very unequal teams compared to earlier editions of the World Cup (i.e., the training data for the algorithm). Also, due to 8 out of 12 third-ranked teams also proceeding to the knockout stage, it often was more important for the teams not to lose a match (rather than to actually win it), thus favoring many draws. Finally, due to the many possibilities of assigning the third-ranked teams to the knockout matches, some teams profited more than others from the realized tournament draw in the Round of 32.

TL;DR

All tournament favorites proceeded to the Round of 32 and mostly the weaker teams dropped out of the tournament. Arguably the biggest surprises were the African teams (especially South Africa, Cape Verde, and DR Congo) who all “survived” the group stage.

While the predicted win/loss probabilities mostly conformed with the observed results, the predicted goal differences tended to be too low. Especially for matches between rather unequal teams the observed goal differences were often more extreme than expected by the algorithm. The likely reason for this is that there were many more weak teams in this tournament compared to earlier years due to the extension to 48 teams.

There were also somewhat more draws than expected (and fewer wins/losses with a margin of only one goal). Again, this is likely due to the new tournament format with 48 teams. One win and one draw was most sufficient to be among the best third-ranked teams who also proceed to the knockout stage. Also, those groups playing their matches last could behave more strategically and could try to settle for a draw. A fact which was painfully obvious in the memorable match between Algeria and Austria.

Group stage results

First, we look at the results in terms of which teams successfully advanced from the group stage to the Round of 32. The barplots below show the predicted probability for all teams to proceed to the Round of 32, in the observed ranking order, with the color highlighting which teams advanced to the knockout stage.

Clearly, all group favorites made the cut and mostly teams with lower probabilities dropped out. The biggest suprises were some of the African teams, notably South Africa (in Group A), Cape Verde (in Group H), and DR Congo (in Group K), all of which successfully “survived” the group stage. Moreover, although some of the tournament favorites (such as Spain, England, Germany, or Portugal) did not fully convince in their respective group stage matches, these performances did not have many negative consequences, yet. All of them proceeded to the knockout stage, typically still taking the top spot in their respective groups.

Match results

Next, we take a closer look at the 72 individual group-stage matches to check how well our forecasts conformed with the actual outcome. The stacked bar plot below groups all match results into five intervals (columns) based on their predicted goal difference for the stronger vs. the weaker team.

The first column summarizes 15 matches where both teams were predicted to be almost equally strong. More precisely, the stronger team was predicted to be only slightly better, with 0 to 0.35 more predicted goals on average. One third of these matches was won by the slightly better team, one third was lost, and another third ended in a draw. In short, the distribution of the outcomes conforms very well with the prediction that both teams would be essentially equally strong.

In the second and third column the predicted advantages of the stronger team increased to 0.35-0.7 goals and 0.7-1.05 goals, respectively, and also the empirical proportion of matches won increased accordingly.

However, in the last two columns with the most pronounced predicted advantages (goal difference of 1.05-1.4 and 1.4-1.75, respectively) the winning proportion remained high but did not increase further. Also, the proportion of draws remained relatively high, even in matches with a clear favorite.

This suggests that our probabilistic forecasts captured the actual outcomes better in matches with small to moderate differences between the teams. But it seems that the algorithm struggled a little bit in matches with very large predicted differences.

To explore this in more detail, we compare the observed goal differences in these matches with the expected distributions based on the Poisson model employed. This is brought out graphically by so-called hanging rootograms, separately for the low predicted goal differences (0-0.7, first two columns above) and the high ones (1.05-1.75, last two columns above).

In both panels, the red line shows the square root of the expected frequencies while the “hanging” gray bars represent the square root of the observed frequencies.

For the low difference subset in the panel on the left, the observed and expected distributions conform reasonably well. It is noticeable, though, that draws (goal differences of 0) are slightly overrepresented in the observations compared to the predictions.

However, for the high difference subset it is clear that there is a bias in goal difference predictions: Low observed goal differences are underrepresented whereas high observed goal differences are overrepresented. The overrepresentation of draws is also more pronounced in this subset.

As explained above, it is likely that these deviations are due to the new tournament format with 48 teams. Many more matches between extremely different teams occurred in this tournament compared to earlier tournaments with only few very weak teams. The machine learning algorithm apparently has not fully captured this. Similarly, the incentives for winning each match were not as strong as in previous tournaments because 8 out of 12 third-ranked teams also proceeded to the knockout stage.

Updated knockout stage predictions

Finally, we want to look ahead and explore how the realized tournament draw based on the group stage results changes the predicted winning probabilities for the 2026 FIFA World Cup. We do so under the assumption that all results so far are within the range of random variation and that we do not need to adapt the predictions for all possible matches. In other words, the simulation is based on the expectation that especially the top favorites Spain and England can still reach their full potential in the upcoming matches.

As for our original prediction, we simulate the knockout stage 100,000 times and then compute by how many percentage points the winning probabilities change.

This shows that Argentina and England profited most from the realized tournament draw. They are both in the arm of the tournament with fewer strong teams and they can only face each other in the semi-final. Therefore, Argentina’s winning probability increased by 3.1 percentage points (from 8.2% to 11.3%). Similarly, England’s winning probability increased by 2.6 percentage points (from 12.4% to 15.0%). Recall that these numbers are derived under the assumption that all teams will play according to the expectations from before the start of the tournament. Thus, additionally, one might want to factor in that Argentina played even stronger than expected and England somewhat weaker.

The teams who suffer most from the realized tournament draw include top favorites Spain and France along with Portugal and Germany because these are very likely to meet already in the Round of 16 (Spain vs. Portugal and France vs. Germany, respectively). Thus, these are much more difficult obstacles on the way to the World Cup Final compared to those for Argentina and England in the other arm of the tournament.

In any case, the most exciting part of the 2026 FIFA World Cup is only starting now and we can all be curious what is going to happen. There are still 32 teams in the race for the title! (Well, 31 after Canada has defeated South Africa in the first knockout match yesterday.)

I came across a chart of long term real gross domestic product per capita for Australia (1900 to about 2020) that had a couple of mistakes in its y axis scale and labelling. I wanted to see for myself what it should actually look like.

Looking for data for this drew my attention to the excellent Maddison Project, which publishes long term estimates of real GDP per capita, in 2011 prices, for many countries. It’s certainly not perfect (price comparisons over such a long period of time—and they go well back before 1900—is a vexed business!), but it looks like a great effort. It’s necessary to have something like this from the economic historians because most standard official statistics series for Australia only go back to 1959, due to the complications of comparisons over time.

The Maddison Project’s latest release is 2023.

Here’s my re-creation of the original chart (which is not shown here), but with correct y axis labels:

A distinctive feature of this chart is that it compares the actual growth trajectory to a constant rate of growth from 1900 to present. It’s a nice comparison. It takes care to draw—it’s definitely not the same as a line of best fit. On a logarithmic scale where the constant growth is a straight line, it gives a good point of reference for the faster and slower periods of growth.

Here’s my code that draws the chart. The work is actually done by a project-specific function draw_chart(), which draws a fairly decent chart for any given country available in the database. This lets me easily produce similar charts for other countries (results shown after the R code).

# download latest version of Maddison data from:
# https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/maddison-project-database-2023

library(tidyverse)
library(readxl)
library(scales)
library(glue)
library(countrycode)

gdppc <- read_excel("mpd2023_web.xlsx", sheet = "GDPpc", skip = 2)

#' Draw time series chart from 1900 to 2023 for a single country
#' 
#' @param ccode 3 digit ISO country code
#' @param points whether to add points for each observation(default is to just draw line)
draw_chart <- function(ccode, points = FALSE){
    # country name for this country code  
    cname <- countrycode(ccode, origin = "iso3c", destination = "country.name.en")

    # Data for just this country"
    data_tc <- gdppc |> 
        select(year, all_of(ccode)) |> 
        filter(year >= 1900)

    names(data_tc)[2] <- "value"

    constant_growth <- data_tc |>
        arrange(year) |> 
        drop_na() |> 
        summarise(n = max(year) - min(year),
                    start = value[1],
                    end = value[n()],
                    start_year = min(year),
                    # for drawing labels, not actually a 'mid' point:
                    mid_point = (end + start) / 4) |> 
        mutate(growth_rate = (end / start) ^ (1 / n) - 1)

    # colours for constant growth and for data:
    cgcol <- "red"
    dcol <- "blue"

    # define plot
    p1 <- data_tc |> 
        ggplot(aes(x = year, y = value)) +
        # Draw constant growth line:
        annotate("segment", 
                x = constant_growth$start_year, 
                xend = max(data_tc$year),
                    y = constant_growth$start, yend = constant_growth$end, 
                colour = cgcol, linetype = 2) +
        # draw data line:
        geom_line(colour = dcol) +
        annotate("text", x = 1900, y = constant_growth$mid_point, 
                label = glue("Constant growth of {percent(constant_growth$growth_rate, accuracy = 0.1)}"), 
                colour = cgcol, hjust = 0) +
        annotate("text", x = 2010, y = constant_growth$mid_point, 
                    label = "Actual GDP per capita", 
                    colour = dcol, hjust = 1) +
        scale_y_log10(label = dollar_format(accuracy = 1), 
                    breaks = c(0, 0.25, 0.5, 1:6) * 10000) +
        labs(x = "",
            y= "",
            title = glue("Long term historical growth in GDP per person in {cname}"),
            subtitle = "GDP per capita, purchasing power parity, 2011 prices.",
            caption = "Source: Maddison Project Database 2023")
  
  # for some countries with broken series we might want to draw points, not just
  # lines:
  if(points){
    p1 <- p1 + geom_point(colour = dcol)
  }

    frs::svg_png(p1, glue("..https://freerangestats.info/img/0326-{cname}"), w = 9, h = 5)
}

draw_chart("AUS")
draw_chart("NZL")
draw_chart("USA")
draw_chart("DNK")
draw_chart("CHN", points = TRUE)
draw_chart("IND")
draw_chart("GBR")
draw_chart("IDN")
draw_chart("JPN")

Here’s some of those other outputs, along with some glib and not-really-thought through observations from myself. Some interesting things here as we see the visible impact of historical events: Covid, the second world war and national independence across multiple countries; as well as national catastrophes such as China’s Great Leap Forward.

New Zealand and the USA have slightly slower growth than Australia over this period but a similar overall trajectory. The USA has a particularly strong New Deal and World War II boost to growth:

Denmark has less impact from the 1930s Great Depression but experienced the war as economic destruction (rather than a GDP boost as was the case for the USA). Overall its experienced fairly fast and steady economic growth:

The UK had a bad time recovering from World War I but eventually attained faster (albeit periodically interrupted) economic growth until the early 2000s:

China’s economic growth really began with its new economic policies in the 1970s:

India’s significant economic growth began soon after 1947 independence but only accelerated from the 1980s onwards and then further from the 2000s onwards:

Indonesia’s post-war growth path reflects its turbulent political history but is essentially a success story. In addition to the interruption of meaningful estimates during the war and deconolonisation, and the troubled transition from Sukarno to Suharto in the 1960s, the East Asia Economic Crisis of the late 1990s really stands out:

Japan had a particularly catastrophic World War II, recovered well until the late twentieth century, and in recent decades has sunk into well-known relative stagnation:

The Maddison Project estimates are presented and discussed in Bolt and Van Zanden (2024), “Maddison style estimates of the evolution of the world economy: A new 2023 update”, Journal of Economic Surveys, 1–41.

TLDR: this post summarizes how I was able to leverage my Surface Pro 11’s Neural Processing Unit (NPU) to chat with Large Language Models (LLMs) from R (for some reason). The code has been adapted from this guide from Microsoft.

While I’ve been obsessively utilizing LLMs since the hype train first set off, in the last 12 months I’ve been experimenting with local LLMs so I can better utilize them in my work. For the most part, I’ve been using my laptop’s RTX 4060 for this, but I’ve long been curious what the integrated NPU in my Surface Pro 11 can (or can’t) do. Unfortunately, at the time of writing, neither Ollama or LM Studio¹ natively support this functionality which has made satisfying my curiosity trickier than I’d like.

At the outset, I’m still not entirely sure this is a question worth answering (or a blog worth writing) given the RTX 4060’s performance thoroughly trumps that of my NPU, but I’m a tinkerer and the idea of being stranded without a stochastic parrot on hand is nothing short of terrifying.

Foundry Local

Before starting, you’ll need to install Foundry Local which can be installed using the Bash command below:

npm install foundry-local-sdk-winml openai

You’ll also need to make sure that Foundry is available on PATH, so your OS knows how to launch it (see here).

The R Code

Microsoft provides a getting started guide here, which includes a set of Python code that this was based on (with Claude’s help):

# Load necessary packages ---------------------------------------------------
library(ellmer)   # LLM chat interface (chat_openai_compatible)
library(httr2)    # resolve the exact loaded model id from /v1/models

# Set project assumptions and define functions ------------------------------
ref_model_alias <- "qwen2.5-0.5b"
ref_prompt      <- "What is the golden ratio?"

# fnc_foundry_load: ensure the service is up and the model is loaded.
fnc_foundry_load <- function(alias) {
  if (Sys.which("foundry") == "") {
    stop("`foundry` CLI not found on PATH. Install Foundry Local first.")
  }
  system2("foundry", c("service", "start"))
  system2("foundry", c("model", "download", alias))
  system2("foundry", c("model", "load", alias))
  invisible(alias)
}

# fnc_foundry_endpoint: discover the service base URL (port is dynamic).
fnc_foundry_endpoint <- function() {
  tmp_status   <- system2("foundry", c("service", "status"), stdout = TRUE)
  tmp_status   <- iconv(paste(tmp_status, collapse = " "),
                        to = "ASCII", sub = " ")
  tmp_hostport <- regmatches(
    tmp_status,
    regexpr("[0-9]{1,3}(\\.[0-9]{1,3}){3}:[0-9]+", tmp_status)
  )
  if (length(tmp_hostport) == 0) {
    stop("Could not parse endpoint from status: ", tmp_status)
  }
  paste0("http://", tmp_hostport[1])
}

# fnc_model_id: resolve the concrete model id required by the REST API.
fnc_model_id <- function(base, alias) {
  tmp_models <- request(paste0(base, "/v1/models")) |>
    req_perform() |>
    resp_body_json(simplifyVector = FALSE)
  tmp_ids <- vapply(tmp_models$data, \(m) m$id, character(1))
  tmp_hit <- tmp_ids[grepl(alias, tmp_ids, fixed = TRUE)]
  if (length(tmp_hit)) tmp_hit[1] else tmp_ids[1]
}

# fnc_foundry_unload: release the model from memory.
fnc_foundry_unload <- function(alias) {
  system2("foundry", c("model", "unload", alias))
  invisible(alias)
}

fnc_foundry_load(ref_model_alias)
ref_endpoint <- fnc_foundry_endpoint()
ref_model_id <- fnc_model_id(ref_endpoint, ref_model_alias)
cat("Model loaded and ready.\n")

# Point ellmer at the local OpenAI-compatible endpoint. Foundry Local needs
mod_chat <- chat_openai_compatible(
  base_url    = paste0(ref_endpoint, "/v1"),
  name        = "foundry-local",
  credentials = \() "not-needed",
  model       = ref_model_id,
  echo        = "output"
)

#send prompt to local model 
rlt_reply <- mod_chat$chat(ref_prompt)

1. Although there were rumors that LM Studio was working on this feature based on this video ︎

The post Running local LLMs on your NPU from R with Foundry Local and ellmer appeared first on Giles.

You have probably seen my post about the STI classification of Hans Levenbach (this one). Well, I’ve decided to implement it, and it has landed in the greybox package for R/Python.

What’s greybox? It is a package for statistical modelling focusing on forecasting and time series analysis. I created it back in 2018 to split the static models (such as linear regression) from the dynamic ones that landed in the smooth package. Greybox has evolved since then, and now has linear regression (alm), regression selection (stepwise) and combinations (calm), a variety of tools for feature generation, diagnostics, forecast evaluation (e.g. rolling origin) etc. You can read more about it here. Originally, the package was available for R only, but Claude and I ported its main functions to Python back in February.

The Exploratory Data Analysis techniques for time series fit the package quite well, although I don’t have many of those yet. So, I’ve implemented the main idea of the STI of Hans Levenbach in a function called “stick” (Seasonal, Trend, Irregular Contribution Kit) in the greybox package for R/Python. The idea is straightforward: apply stick to a time series, it will use ANOVA, and give you the strength of each component. Here, for example, is how to apply the function to the AirPassengers data (everyone’s favourite toy time series) in R:

library(greybox)
stick(AirPassengers)

and in Python:

from fcompdata import AirPassengers
from greybox import stick

result = stick(AirPassengers.y, lags=12)
print(result)

which gives exactly the same result:

Strength of the components:
seasonal12      trend  irregular
    0.1061     0.8613     0.0326

So, trend dominates the time series, explaining 86.13% of its variability, meaning that if you capture it correctly, you solve a big chunk of the problem. This split also gives you a rough idea about the structure-versus-noise breakdown in the time series, although it assumes that the seasonal component does not evolve over time.

The function supports several seasonal components, and I might extend it to include external information (e.g. promotions) in the future if there is demand for it.

Message stick function for the EDA in time series first appeared on Open Forecasting.

“AI In Production 2026”, a new conference hosted by Jumping Rivers, was held on 4-5 June, 2026. The conference touched on many of the intricacies of using AI in software products, as part of the developer toolkit, and at the wider usage of AI throughout organisations. Themes that stood out in the talks this year included regulatory and security matters, the deployment and scalability of AI products, how to incorporate custom data/knowledge, and how to make use of AI tooling to simplify data-intensive tasks.

In addition to the talks (held on 5th June), there was a day of workshops on the 4th June.

AI within the software-development lifecycle

AI-based tooling has revolutionised software development during the 2020s. Colin Eberhardt (Scott Logic) highlighted steps in the evolution of the AI developer since 2022 (“Agentic AI and the Future of Software Development”). In 2022, AI could be used as an auto-completion tool with assistive code suggestions. With CoPilot (GitHub) and the launch of Claude (Anthropic), by 2023 AI tools could function as a virtual pair-programmer (proposing code, finding and fixing bugs in new code), though any contributions needed thorough code review. Since 2024, AI tools have increasingly been used as autonomous contributors or as a component of continuous code-review operations.

This transition is having an impact throughout software organisations:

• How should a junior, or newly-onboarded, developer get up-to-speed with a project?
• Where should developers focus their time and effort?
• How should projects, teams and organisations be structured to take best advantage of AI tooling?

On the first point, Rebecca Guiney (Certara) gave an interesting lightning talk (“Learning in Production: Becoming a software engineer during the AI wave”). In a recent project, she used AI to build a tool for schema discovery (over clinical trial databases). The tool can work reproducibly on new data.

Rebecca highlighted how “contract-first development” is important when building tools using AI assistance. That is, when agents can write code and/or tests, an important role of the developer is to control what the different components of a system are reponsible for, and how they interact. Similarly, when reviewing code a greater emphasis is placed on the purpose of what was implemented than on how it works. This emphasis on higher-level thinking is also impacting how new developers learn – learning from AI-generated example code, rather than from step-by-step tutorials.

Rebecca won the (CRC-Press sponsored) prize for the best lightning talk.

Since the AI toolkit is relatively new, many projects exist upon which AI has never been used. When looking at these projects, Claude (or similar) could find countless issues. So how do you prioritise what to fix first (when you may also have a list of bugs and features waiting to be addressed)? Badr Adnani (Roundel Kitchens) gave a lightning talk explaining how his use of AI helped identify and fix a critical bug in a SQL command (“Breaking Barriers with AI Automation”). The bug itself, could have posed a real financial risk to the company.

As part of his talk, Badr split up the audience by their level of AI usage. Most of our attendees were daily users of AI for well-defined tasks. Had Badr not increased his usage of AI at work, he may not have fixed the above bug (and got promoted), and he urged us all to experiment more with AI in our daily work.

So how do we broaden the use of AI within programming teams? Firstly we make the tools easy to use, and work out how to use them cheaply and effectively.

The Positron IDE is becoming a very popular system for development in data science and allied fields. George Stagg (Posit) gave the closing talk of the meeting describing how the Posit Assistant came into existence (“A Builder’s Guide to Working with AI”). Posit Assistant is an AI agent available in RStudio and Positron, with which a user can explore their data and generate analytical dashboards via a conversational interface.

Building the Posit Assistant helped George understand AI systems like Claude. These systems can lie to you (or your users), for example, if you ask an AI tool to summarise the trend in a dataset, it may give you a good enough answer. But it could be guessing – it might not even have seen the data – but may still give you an answer. So it’s sometimes important to check the raw messages that are sent to an AI tool and have an independent way to verify agent outputs.

There are existing standards that make using AI coding tools more effective. Project-level definitions of how certain tasks should be performed, or how the project is structured, can be put into an AGENTS.md file or a ‘skills’ directory. George explained that it is important to make use of these common standards when building a developer-facing agent.

If you want to limit the cost of AI-usage in a project, limit the amount of data that is sent to- and from- your agent. This could be by cacheing previous messages – because otherwise you may be sending all of your previous conversation each time you interact with an agent.

AI usage in specific projects

There are many ways in which AI can be used across organisations. In addition to its use as an aid in software development (above) and within search-engine summaries, it is extremely common to find people using AI to write emails, to summarise meetings, to annotate videos and speak and to perform a range of other workplace tasks.

Nayara Macedo de Medeiros Albrech (Newcastle University) gave an interesting summary of her research on the use of AI in local government and education (“AI Beyond Industry: Insights from Higher Education and Government”). Government decision making has a focus on ’evidence-based policy’, so AI could contribute in this area, but there is a need to prevent AI from introducing bias into decision making. The talk emphasised ethics in the public sector use of AI and the variability in take-up of AI tooling across different local government sectors.

Continuing on the theme of AI use in government, Katy Morgan introduced some self-serve tools that the Government Internal Audit Agency have generated to aid the production of internal audits (“From risks to insights: Driving innovation with AI-powered tools in internal audit”). The use of AI in these products ranged from highly-guided settings (where user-selection from a fixed collection of application-encoded choices creates LLM prompt input automatically), to more flexible use (where an auditor could freely interact with a chat engine). With these tools, users are encouraged to use their own judgement to interpret any AI-generated output.

So AI-embedded tools can fix your code and reduce administrative work. Can AI help resurrect old projects?

Shona Ferguson outlined an interesting use-case for LLMs in data-rescue efforts at the UK Centre for Ecology and Hydrology (“Comparing and Evaluating Large Language Models for Efficient and Responsible Data Rescue”). Preservation of historical data resources is a time-consuming, and largely manual, task. The typical steps involve defining some quality checks, writing a data-cleaning script, finding supporting publications and creating documentation.

Shona demonstrated how effective three different LLM tools (free versions of CoPilot, ChatGPT and Claude Sonnet) could be in a project to rescue a dataset related to cloud and rain chemistry. The tools were particularly effective at code generation and making draft documentation. When approaching this type of project, recommendations are to give the LLM prompt as much context about the project as possible, to tell the LLM ’not to guess’ results and to be very explicit about the form of output you expect the LLM to return.

Sometimes your data is so large that it wouldn’t be possible to analyse it manually. The team from Wordnerds (Izzie Johnson and Damani Richards) gave a talk that touched on some novel themes for our conference (“A Nerd’s eye view: wrangling the GenAI hype cycle and refusing to become irrelevant”).

One of the powers of LLMs is their ability to accept natural language as input. But natural language is complex and there’s an awful lot of natural language in the world. For text-analytics specialists, processing customer reviews or other large-scale collections of text using LLMs could be pretty expensive: both economically and ecologically. If you want to process natural language efficiently at-scale, maybe an LLM isn’t the right tool to use. Izzie and Damani explained a hybrid approach, using both the Gemini LLM to identify and tag training data, and smaller custom-trained neural networks to analyse millions of records efficiently.

Just as human language is a complicated world, so is human behaviour. Grant Beasley from tombola, alongside our own Myles Mitchell (Jumping Rivers), presented a project where they used deep-learning to assess player behaviour in their online betting platform (“Using Deep Learning to monitor player safety on online betting platforms”). tombola monitors players over time, to help identify people at risk of gambling addiction, and to intervene where appropriate.

The models used here are not large language models, they are deep neural networks trained on time-series data. Training this model over large numbers of players was a substantial undertaking. But the tombola project highlighted how, if you are planning to build a model on this scale, it is important that you start with simpler benchmark models to compare your larger model against.

William Kirby (Wessex Water) presented a lightning talk covering another project where real-time interventions can impact on human behaviour. This time the subject was wild-swimming and water quality in inland bathing waters in the UK (“Near Real Time Notifications for Bathing Waters using Machine Learning”). The project integrates data on water quality and climate with sensor and sampling data, and is capable of classifying risk at their pilot site (Warleigh Weir, near Bath) on a 15 minute turnaround. They have developed a web application to present their results and are scaling out to additional sites over the next two years.

Incorporating custom information into AI-tools

Agentic systems are clearly very powerful, but they can become far more useful in a project or product when the data they have access to is tailored to that project. Three talks highlighted different ways to connect an LLM to a custom data source or an external service so that the agent can be better informed.

The standard way to enhance an LLM with a custom data source, is by using “Retrieval-Augmented Generation” (RAG). Obinna Iheanachor (Rotork; and host of the Wisabi Analytics YouTube channel) gave a lightning talk explaining a two-step AI-driven process where he used RAGs to identify documents related to corporate insolvency cases, and then to identify evidence within those documents (“RAG in the Real World: Designing Trustworthy LLM Systems for Corporate Insolvency Data”). The aim here was to identify citable evidence, rather than plausible-sounding and eloquently written LLM outputs, so the use of RAGs was essential.

RAG is a way of embedding document-derived text in a way that can be searched by an LLM. An extension of this concept is the GraphRAG, where relationships between entities in a series of documents can also be represented. Jonny Law (Neo4J) presented a talk that explained many of the concepts underpinning GraphRAG (“Engineering a Scalable Knowledge Graph Builder on Neo4J Cloud”). Neo4J have developed a “Knowledge Graph Builder” that allows data and relationships to be extracted from documents and represented in a graph structure. This graph structure can be queried using an LLM by converting an LLM query into a GraphRAG query.

Another way to connect an agent to an external data source or service is the “Model Context Protocol” (MCP). MCP is an open standard and was the subject of Neal Richardson’s (Posit) talk (“MCP, or not MCP”). As a motivation, Neal explained that for security reasons, there are external resources that an AI agent shouldn’t be given access to. For example, you might not want an LLM to be able to read and write to the support message-board for your software, or to read any file on your computer.

MCPs use a standard API and JSON-based interchange format, but they make it easier to restrict the reach of an AI system. OAuth 2.1 can be used to register an LLM with an MCP, meaning that API secrets needn’t be available to the LLM – so it can’t leak these. Recommendations for building MCPs, are to only define the tools you actually need, to make API responses concise and to sanitise any sensitive data that the MCP might return.

Development, deployment and auditing of AI-embedded products

We have seen software-development tools that make use of AI; specific projects that use AI and/or ML; analysis of, and recommendations for, the use of AI; and ways to link additional data to AI tools. But before you incorporate AI into all your workflows, you need to take a step back and consider how to implement everything in a way that’s cost-effective, safe and legal.

First we need to consider what we’re allowed to do with AI and the liabilities we create when we use it on the public.

Nathan Bilton (Weightmans) provided a thought-provoking insight into how legislation is evolving to match the growth of AI (“Regulating Artificial Intelligence”). Given that AI can take in vast quantities of personal data and intellectual property during training, does existing legislation provide enough protection? And even if we agree on a better set of rules, how do we enforce them? And who will be the enforcers?

Some of these questions may feel out of your remit, and something for your elected lawmakers to debate over. But keeping abreast of these developing laws remains relevant to your deployment of AI, both now and in the uncertain future.

The EU AI Act 2024 already applies a number of restrictions on how AI can be used, with extraterrestrial effects. If you feed personal data of UK or EU citizens into AI, then GDPR rules still apply. And if you do allow AI to interface with the public on your behalf, are you responsible if things go wrong? Here Nathan cites the case of Moffatt v. Air Canada, 2024 where a Civil Resolution Tribunal found Air Canada to be liable for damages due to “negligent misrepresentation” after incorrect advice was given by a chatbot on their website.

Next, how do you ensure your product’s use of AI is secure? Mac Misiura (Red Hat) presented the first technical talk of the day (“Open Source Guardrails for AI: Securing LLM Applications at Scale”). If you are planning to deploy an AI-backed application to production, I strongly urge you to watch this talk (soon to be on the Jumping Rivers YouTube channel).

Mac highlighted some of the top risks that an LLM-based app exposes you to. There is a top-10 list hosted by OWASP, that covers things like disclosure of sensitive information, and spreading of misinformation. Not addressing these risks can expose you to reputational damage and adversarial attacks.

To help secure your app, you can use “Guardrails”, which filter out inappropriate queries. These can range in complexity from rules and regular expressions up to LLM-based assessment of the intent of the query (use the simplest guardrails first). There are open-source guardrail collections that you can use with your application, for example, the NVIDIA NeMo Guardrails. Mac also argued that you should run attacks against your own system to identify security weak spots.

You may have multiple products in the pipeline. Diego Jimenez and Oliver Thomas from Sage covered their approach to product delivery using what they described as a “repeatable path for controlled AI at enterprise scale” (“Beyond the POC: Architecting Enterprise-Grade Agentic Systems at Sage”). This approach allows them to rapidly iterate from prototype to scalable product, by using common designs across those products. They emphasised the importance of monitoring agent behaviour, and similarly to Mac Misiura (above) the use of safety guardrails across their products.

Finally, once your app is in production, you’ll need to start paying for tokens.

Seb Ringrose (Doubleword) described “Doubleword Code”, a system developed by Doubleword that performs pull-request reviews on repositories. During the talk, Seb identified a few challenges with the current use of AI agents: how to select which model to use (from a continually changing collection of models), how much context to provide to an agent, what ancillary tooling to use, and—importantly—the cost of using these agents.

Given these desirable qualities of agents – low latency, low cost and high quality – Seb argued that you can only have two in any AI system. So if you need a real-time system, then you have to choose between high cost or low quality. But not all systems need to run in real-time. For the pull-request review system, it can run asynchronously or as a batch process, and by doing so can lead to cost savings. Given that for many settings, the number of tokens used per task is increasing, finding ways to reduce cost is becoming extremely important.

Summary

This years talks at “AI In Production” covered a wide spectrum of the uses of AI (and advanced machine learning) in software products and data projects across several domains. We discussed best practices for securely releasing AI-embedded software into the world, regulatory matters, and how we can evolve as software professionals as AI tooling becomes commonplace.

Jumping Rivers would like to thank everyone who presented talks or workshops at the conference (or submitted abstracts), all of the sponsors and the attendees for making this an interesting conference. We hope to see you again next year at AI In Production 2027.

The talks from the conference will be added to the Jumping Rivers YouTube channel soon.

Sponsors

• Jumping Rivers
• Databricks
• The Royal Statistical Society
• Posit
• CRC Press
• The National Innovation Centre for Data (NICD)

Community Partners

• DevITJobs.uk

For updates and revisions to this article, see the original post

As France enters its second heatwave of 2026, can we produce more detailed plots than the excellent visualizations provided by ShowYourStripes?

MétéoFrance offers its monthly SIM2 dataset albeit over a shorter time span (currently 1970–2025). The dataset includes temperature, precipitation and other variables on an 8 km resolution grid.

We will select a French city, retrieve its geographic coordinates, build the grid for a specific month over the 1970–2025 period, extract the data from the grid at that location and plot the temperature anomaly.

We will use {terra} to create the grid from the tabular files containing cell centers and weather variables, and terra::extract() to get all temperatures.

library(tidyverse)
library(fs)
library(janitor)
library(osmdata)
library(sf)
library(terra)
library(glue)
library(memoise)

invisible(
  Sys.setlocale(category = "LC_ALL", 
                locale = "en_GB.UTF8"))


#' Geolocate using OSM (Nominatim API)
#'
#' using first result
#' Memoized function
#'
#' @param location (char): place name (geocodable via OSM)
#'
#' @returns (SpatVector): first result
geolocate <- memoise(function(location) {
  loc <- getbb(location, format_out = "sf_polygon")  |> 
    st_point_on_surface() |>
    slice_head(n = 1)
  
  message(glue("{loc$display_name} — WGS84 : {loc$geometry}"))
  
  return(loc|>
           st_transform("IGNF:NTFLAMB2E") |>
           as("SpatVector"))
})

#' Generate a monthly temperature chart since 1970
#'
#' @param sim2 (data.frame): Météo-France SIM2 data over the period
#' @param month (char): month number "01"..."12"
#' @param location (char): place name (geocodable via OSM); memoized
#' @param output_dir (char): directory path where a PNG file will be written, if not NULL
#'
#' @returns (ggplot and optionally a file on disk)
generate_chart <- function(
    sim2,
    month,
    location,
    output_dir = NULL) {
  stopifnot(month %in% sprintf("%02d", 1:12))
  month_name <- format(ymd(glue("0000-{month}-01")), "%B")
  
  sim2_raster <- sim2 |>
    filter(str_detect(date, glue("{month}$"))) |>
    mutate(
      x = lambx * 100,
      y = lamby * 100,
      layer = date,
      temp = t,
      .keep = "none") |>
    rast(
      type = "xylz",
      crs = "IGNF:NTFLAMB2E")
  
  loc <- geolocate(location)
  
  temperatures <- sim2_raster |>
    terra::extract(loc) |>
    select(-ID) |>
    pivot_longer(
      cols = everything(),
      names_to = "month",
      values_to = "temperature") |>
    mutate(
      year = as.integer(str_sub(month, 1, 4)),
      anomaly = temperature - mean(temperature[year >= 1991 & year <= 2020],
                                   na.rm = TRUE))
  
  p <- temperatures |>
    ggplot(aes(year, anomaly)) +
    geom_col(aes(fill = anomaly)) +
    geom_smooth(method = "loess",
                formula = y ~ x) +
    scale_fill_gradient2(
      high = scales::muted("red"),
      mid = "white",
      low = scales::muted("blue")) +
    scale_x_continuous(breaks = scales::breaks_pretty()) +
    scale_y_continuous(breaks = scales::breaks_pretty()) +
    labs(
      title = glue("Average monthly anomaly temperature — {month_name}"),
      subtitle = location,
      x = "year",
      y = "departure from average* (°C)",
      fill = "°C",
      caption = glue(
        "https://r.iresmi.net/ — {Sys.Date()}
         data: Météo-France SIM2 — *baseline: 1991–2020 normal for {month_name}")) +
    theme(
      text = element_text(family = "Ubuntu"),
      plot.caption = element_text(size = 7))
  
  if (!is.null(output_dir)) {
    dir_create(output_dir)
    
    ggsave(
      glue("{output_dir}/tm_{month}_{make_clean_names(location)}.png"),
      plot = p,
      width = 20,
      height = 20 / 1.618,
      units = "cm",
      dpi = 150)
  }
  
  return(p)
}

The data is a bunch of compressed CSV.

# https://meteo.data.gouv.fr/datasets/65e040c50a5c6872ebebc711
# Climate change data - monthly SIM
# all files MENS_SIM2_*-*.csv.gz
sim2 <- dir_ls("data") |>
  read_delim(
    delim = ";",
    locale = locale(decimal_mark = "."),
    name_repair = make_clean_names)

Now we just call our function.

generate_chart(sim2,
               month = "06",
               location = "Paris, France")

If we want each month:

# for each month
sprintf("%02d", 1:12) |>
  map(\(x) generate_chart(sim2,
                           month = x,
                           location = "Grenoble, France",
                           output_dir = "results"), 
  .progress = TRUE)

Note that scales are not constant across plots; if we want to compare month (or places) we should fix the y-axis and the color scale. It’s left as an exercise to the reader if you want to make a nice poster…

My colleagues run some highly regarded R workshops for well over a decade in Brisbane (Australia) every year.

Details here and info below

Are you ready to take your R skills to the next level and conquer the challenges of messy data? Transform raw data into actionable insights with the Centre for Biodiversity and Conservation Science (CBCS) 2026 Winter R Workshops @ UQ.

These intensive, hands-on workshops are designed for individuals with a foundational understanding of R who are eager to master the essential skills for effective data wrangling, code organisation, advanced visualisation, spatial analysis, and building interactive Shiny apps. Join us to elevate your R proficiency and become a data manipulation and presentation expert!

Each day is self-contained so you can attend the workshops you want to.

Day 1 (Monday 6th July) Data Wrangling 1 Day 2 (Tuesday 7th July) Data Wrangling 2 Day 3 (Wednesday 8th July) Visual Storytelling Day 4 (Thursday 9th July) Mapping our World Day 5 (Friday 10th July) Building Interactive Web Apps with Shiny

Registration for the 2026 Winter R Workshops @ UQ in July is OPEN.

We are passionate users of R and want to share our experience. Our workshops are accessible, interactive, informative and fun. They are run in-person only (no online option) to provide a tailored student experience and we provide all the code and learning materials. For more information on the topics covered and to view testimonials, see our website https://mathmarecol.github.io/RWorkshop/winter.html and the attached flyer.

These workshops are brought to you by the Centre for Biodiversity and Conservation Science (CBCS). For any questions about registration, email cbcs-workshops@uq.edu.au.

We look forward to seeing you there. The R Workshops @ UQ team (Anthony, Dave, Jason, Claire and Tin)

Answering a question that came up from a student recently.

Say you have 20 surveys of reef fish biomass at different locations. Then you also have gridded data with environmental covariates. The gridded data is for all reefs everywhere.

The goal is to predict fish biomass at all reefs everywhere. Here’s an older post that walks through the steps in R with older packages (you will want to update raster to terra, everything else should work).

The statistically correct workflow would look like this:

1. Extract environmental covariates at your 20 reef sites

For example, you might have sites as x-y coordinates and the gridded data as terra rasters. You can use the extract function to get the values of the environmental data at your reef sites. Let’s call this new dataframe you will make fish_data.

2. Fit a model

Fit a model predicting fish biomass at the 20 sites from the environmental covariates. for example:

model1 <- gam(biomass ~ SST + depth, data = `fish_data`)

Your sample size for this model will be 20, ie the number of reef sites where you measured fish.

Do all the usual steps to verify your model, model selection etc…

3. Turn the grids into a dataframe of environmental covariates

There’s lots of ways to do this data wrangling step. I like to use the function to get coordinates of centre points of the grids (xyFromCell), then convert that to a spatial points, then use extract, the convert back to a plain dataframe (keeping the X-Y as columns).

Now we have a new dataframe of environmental covariates at every grid cell location. Let’s call it pred_data for ‘prediction data’. It will have as many rows as there are grids in your raster, probably 1000s.

You might like to do a filtering step to remove grids that aren’t on reefs (or your target habitat, no point predicting reef fish biomass for places that are sand or land).

4. Predict with your model

Now use your model to predict to all the grid locations, e.g. 

pred_data$mean <- predict(model1, newdata = pred_data)

Then you can convert pred_data back into a spatial object for mapping the predictions.