In the process I’ve learned a few more things about GPG keys, subkeys, signatures, UIDs, etc., which I thought I might dump here in case it’s useful for someone else (or me, in the future). I don’t claim to be an expert, so I may still be misunderstanding some bits. GPG is complex. One thing which has helped ground things is finding the documentation in RFC 4880 (and related) which defines the GPG packet format.

One thing which keeps flatpak’s use of GPG simple is that it doesn’t use any of the web-of-trust features or trust-on-first-use (TOFU). Its use of GPG is limited to the keyring format (essentially pubring.gpg), for storing and publishing public and private keys, subkeys, signatures, revocations, UIDs; and using them to sign and verify OSTree commits and various repository files (summary, summary.idx, etc.).

Quick primer on the parts of GPG

GPG has keyrings: collections of primary keys. A primary key has public and private/secret parts. The way flatpak uses it, the secret part of a key always stays on the server, and is used to generate signatures. The public part is published by the server and configured in every client using that repository, used to verify the commit signatures. Each client has one keyring per remote it has configured. Typically this contains one primary key, but it could contain several, any of which can be used to verify signatures from that remote.

A key is identified using a fingerprint, a 40-character hex string. It can also be identified using a key ID, which is a substring of the fingerprint. There’s also keygrips, but let’s ignore those.

You can see fingerprints for keys using gpg --list-keys --fingerprint.

A primary key has one or more subkeys (at least in flatpak’s usage). All of these keys have public and private/secret parts as above. Each key has a usage which indicates what GPG will let you use it for, such as certifying, signing, authenticating, encrypting. The primary key is typically used to certify its subkeys by creating cross-certification signatures which bind them to the primary key, forming a short chain of trust — anyone who trusts the primary key should trust a subkey which it cross-certifies.

You can see subkeys as the sub lines using gpg --list-keys --with-subkey-fingerprint. The [SCE], [S], [E] (etc.) fields show the usage flags for a key.

The other usage we care about is signing (flatpak doesn’t use authentication or encryption usages). GPG separates keys by usage to prevent attacks where a key is used for a purpose it’s not intended for, and because keys used for different purposes often need to be treated with different levels of care.

In particular, separating by usage means the private/secret part of the primary key can be kept completely offline, and only brought out in a special key signing ceremony when a new subkey needs to be generated and cross-certified. This reduces the risk of the very valuable primary key, which is the root of every client’s trust in the repository, being leaked.

So, we use a signing subkey for day-to-day signing of OSTree commits. For a big flatpak repository, the private/secret part of this subkey might be kept in a hardware security module, so it can’t be exfiltrated from the server if the server were compromised. But there’s still the risk of a compromised server being used to sign things it shouldn’t (such as malicious apps).

That’s a matter for server security, but we can somewhat mitigate against the possibility of the signing subkey being leaked by setting an expiration date on it. Clients might choose not to trust signatures made by it after that date; and gpg certainly wouldn’t allow it to be used to create new signatures.

The expiry date of a key is shown as an expires field in the gpg --list-keys --with-subkey-fingerprint output.

What happens when the subkey expires? By that point, the administrators should have generated another subkey, cross-certified by the primary key in a key signing ceremony (I assume the ceremony involves cake). The private/secret part of the new subkey needs to stay secret, as before; but the public part needs to be distributed to every client’s keyring, along with the new cross-certification signature from the primary key, so the clients know they can trust signatures made by that subkey.

That’s the bit which flatpak is currently lacking.

So in summary: GPG has keyrings. Keyrings have primary keys. Primary keys have one or more subkeys and cross-certification signatures from the primary key on those subkeys. Each subkey has a usage, but flatpak only uses certify (for the primary key) and sign (for the subkeys). Keys can have expiration dates.

And if you want to see the full contents of a keyring, run gpg --list-keys --with-colons. It’ll output everything (no filtering) in a machine readable format described here (best reference I’ve been able to find), which is sometimes easier to use than remembering which --with-blah option to pass to GPG to get it to show the information you want.

What else does GPG have?

Quite a few things. We’ll ignore the big things which are not relevant to flatpak.

Each primary key also has one or more UIDs. These are like subkeys in that they are cross-certified by the primary key. Each UID is a user identity — typically a name and email address. If you were using GPG in a web of trust, the binding between the primary key and a UID is what you sign that you trust when you sign someone’s key in a key signing party.

The UIDs are listed below each primary key in gpg --list-keys.

Flatpak doesn’t need UIDs, but they are an unavoidable part of GPG — each primary key must have at least one. A flatpak repository will typically put a server contact email address in the UID and then everyone will ignore it.

UIDs can be revoked; for example if someone loses control of the email address in it and wants their friends to no longer trust emails from it. Flatpak currently doesn’t use this.

What else can be revoked? The cross-certification signatures! You may have heard of a GPG revocation certificate. This is a way of revoking an entire primary key. But there’s also a way of revoking a particular cross-certification signature, meaning that the primary key is still valid/trusted, but the owner of the primary key has lost control of one of the subkeys, and that subkey should no longer be trusted. This is different from key expiration, as it’s a statement that something has explicitly gone wrong.

Because of how GPG is built up as a series of packets of different types, a signature revocation is actually a revocation packet appended to the primary key. This means you can re-cross-certify a subkey after revoking it, by appending another cross-certification packet. And even revoke it again after that. Not sure if there’s a use case for this or if it’s just a consequence of the packet format, but this behaviour does play havoc with working out whether to trust a subkey.

Cross-certification signatures can also have an expiration date built into them, separate from the expiration date of the subkey. I’m not sure of the use case for this either, but there must be one.

Some notes on running GPG on the command line

GPG is historically famously hard to use. I feel this has got better in recent years, particularly for scripting it. In particular it’s added a whole load of --quick-blah commands to generate keys, set expiries, etc. from scripts.

One thing which repeatedly tripped me up before I stopped trying to fight it was its concept of a ’homedir’. GPG needs to look for its keyring (and trust database, and various other files) somewhere, and will not run without them, so you always need to pass it a ‘homedir’ to look for them in. By default, this will be ~/.gnupg, so it’s very easy to accidentally end up operating on your personal GPG keyring when you’re trying to do something in a project.

If using GPG as a tool or in a script, I think you should always create a temporary homedir, pass it as gpg --homedir=/path/to/temp and explicitly import whatever keys or context you need into this homedir before doing whatever operation you need.

This is necessary even if ‘all’ you want to do is view a downloaded .gpg keyring, because what GPG displays may be affected by the trust database in its homedir. So to view a downloaded keyring you should still do something like mkdir temp; gpg --homedir=./temp ./path/to/download.gpg.

If you are trying to sign something, you will typically pass the fingerprint or key ID of a primary key to GPG; for example as gpg --local-user 0xfingerprint --sign ./path/to/file. GPG will helpfully use the usage flags of the subkey of that primary key to choose which subkey to sign with. If you want to sign with a specific subkey, you need to suffix the fingerprint with an exclamation mark (!) otherwise GPG will still choose what it thinks is the most appropriate subkey, which might not align with the subkey you carefully chose. This ! suffix format is common throughout the GPG command line interface for when you want to specify a specific subkey.

Sorry

That was more of a braindump than I imagined when I set out to write this. I hope some of it is useful; feedback welcome if I’ve got anything wrong. If any GPG experts fancy reviewing key rotation support in flatpak, the draft implementation is here.

We haven’t said much about web filtering so far, because the user interface for it isn’t finished yet. The backend is, though, and it will get plumbed up eventually. Currently we don’t have a GNOME release targeted for it yet.

When is web filterings? What is web filtering?

(Apologies to Radio 4 Friday Night Comedy.)

Firstly, what is the aim of web filtering? As with screen time limits, we’ve written a design document which (hopefully) covers everything. But the summary is that it should allow parents to filter out age-inappropriate content on the web when it’s accessed by child accounts, while not breaking the web (for example, by breaking TLS for websites) and not requiring us (as a project) to become curators of filter lists. It needs to work for all apps on the system (lots of apps other than web browsers can show web content), and needs to be able to filter things differently for different users (two different children of different ages might use the same computer, as well as the parents themselves).

After looking at various different possible ways of implementing it, the best solution seemed to be to write an NSS module to respond to name resolution (i.e. DNS) requests and potentially block them according to a per-user filter list.

A brief introduction to NSS

NSS (Name Service Switch) is a standardised name lookup API in libc. It’s used for hostname resolution, but also for user accounts and various other things. Names are resolved by various modules which are dlopen()ed into your process by libc and queried in the order given in /etc/nsswitch.conf. So for hostname resolution, a typical configuration in nsswitch.conf would cause libc to query the module which looks at /etc/hosts first, then the module which checks your machine’s hostname, then the mDNS module, then systemd-resolved.

So, we can insert our NSS module into /etc/nsswitch.conf, have it run somewhere before systemd-resolved (which in this example does the actual DNS resolution), and have it return a sinkhole address for blocked domains. Because /etc/nsswitch.conf is read by libc within your process, this means that the configuration needs to be modified for containers (flatpak) as well as on the host system.

Because the filter module is loaded into the name lookup layer, this means that content filtering (as opposed to domain name filtering) is not possible with this approach. That’s fine — content filtering is hard, I’m not sure it gives better results overall than domain name filtering, and means we can’t rely on existing domain name filter lists which are well maintained and regularly updated. We’re not planning on adding content filtering.

It also means that DNS-over-HTTPS/-TLS can be supported, as long as the app doesn’t implement it natively (i.e. by talking HTTPS over a socket itself). Some browsers do that, so the module needs to set a canary to tell them to disable it. DNS-over-HTTPS/-TLS can still be used if it’s implemented by one of the NSS modules, like systemd-resolved.

Nothing here stops apps from deliberately bypassing the filtering if they want, perhaps by talking DNS over UDP directly, or by calling secret internal glibc functions to override nsswitch.conf. In the future, we’d have to implement per-app network sandboxing to prevent bypasses. But for the moment, trusting the apps to cooperate with parental controls is fine.

Filter update daemon

So we have a way of blocking things; but how does it know what to block? There are a lot of filter lists out there on the internet, targeted at existing web filtering software. Basically, a filter list is a list of domain names to block. Some filter lists allow wildcards and regexps, others just allow plain strings. For simplicity, we’ve gone with plain strings.

We allow the parent to choose zero or more filter lists to build a web filtering policy for a child. Typically, these filter lists will correspond to categories of content, so the parent could choose a filter list for advertising, and another for violent content, for example. The web filtering policy is basically the set of these filter lists, plus some options like “do you want to enforce safe search”. This policy is, like all other parental controls policies, stored against the child user in accounts-service.

Combine these filter lists, and you have the filter list to give to NSS in the child’s session, right? Not quite — because the internet unfortunately keeps changing, filter lists need to be updated regularly. So actually what we need is a system daemon which can regularly check the filter lists for updates, combine them, and make them available as a compiled file to the child’s NSS module — for each user on the system.

This daemon is malcontent-webd. It has a D-Bus interface to allow the parent to trigger compiling the filter for a child when changing the parental controls policy for that child in the UI, and to get detailed feedback on any errors. Since the filter lists come from third parties on the internet, there are various ways they could have an error.

It also has a timer unit trigger, malcontent-webd-update, which is what triggers it to periodically check the filter lists for all users for updates.

And that’s it! Hopefully it’ll be available in a GNOME release once we’ve implemented the user interface for it and done some more end-to-end testing, but the screen time limits work is taking priority over it.

High level design

There’s a design document which is the canonical reference for the design of the backend, but to summarise it at a high level: there’s a stateless daemon, malcontent-timerd, which receives logs of the child user’s time usage of the computer from gnome-shell in the child’s session. For example, when the child stops using the computer, gnome-shell will send the start and end times of the most recent period of usage. The daemon deduplicates/merges and stores them. The parent has set a screen time policy for the child, which says how much time they’re allowed on the computer per day (for example, 4h at most; or only allowed to use the computer between 15:00 and 17:00). The policy is stored against the child user in accounts-service.

malcontent-timerd applies this policy to the child’s usage information to calculate an ‘estimated end time’ for the child’s current session, assuming that they continue to use the computer without taking a break. If they stop or take a break, their usage – and hence the estimated end time – is updated.

The child’s gnome-shell is notified of changes to the estimated end time and, once it’s reached, locks the child’s session (with appropriate advance warning).

Meanwhile, the parent can query the child’s computer usage via a separate API to malcontent-timerd. This returns the child’s total screen time usage per day, which allows the usage chart to be shown to the parent in the parental controls user interface (malcontent-control). The daemon imposes access controls on which users can query for usage information. Because the daemon can be accessed by the child and by the parent, and needs to be write-only for the child and read-only for the parent, it has to be a system daemon.

There’s a third API flow which allows the child to request an extension to their screen time for the day, but that’s perhaps a topic for a separate post.

So, at its core, malcontent-timerd is a time range store with some policy and a couple of D-Bus interfaces built on top.

Per-app time limits

Currently it only supports time limits for login sessions, but it is built in such a way that adding support for time limits for specific apps would be straightforward to add to malcontent-timerd in future. The main work required for that would be in gnome-shell — recording usage on a per-app basis (for apps which have limits applied), and enforcing those limits by freezing or blocking access to apps once the time runs out. There are some interesting user experience questions to think about there before anyone can implement it — how do you prevent a user from continuing to use an app without risking data loss (for example, by killing it)? How do you unambiguously remind the user they’re running out of time for a specific app? Can we reliably find all the windows associated with a certain app? Can we reliably instruct apps to save their state when they run out of time, to reduce the risk of data loss? There are a number of bits of architecture we’d need to get in place before per-app limits could happen.

Wrapping up

As it stands though, the grant funding for parental controls is coming to an end. Ignacy will be continuing to work on the UI for some more weeks, but my time on it is basically up. With the funding, we’ve managed to implement digital wellbeing (screen time limits and break reminders for adults) including a whole UI for it in gnome-control-center and a fairly complex state machine for tracking your usage in gnome-shell; a refreshed UI for parental controls; parental controls screen time limits as described above; the backend for web filtering (but more on that in a future post); and everything is structured so that the extra features we want in future should bolt on nicely.

While the features may be simple to describe, the implementation spans four projects, two buses, contains three new system daemons, two new system data stores, and three fairly unique new widgets. It’s tackled all sorts of interesting user design questions (and continues to do so). It’s fully documented, has some unit tests (but not as many as I’d like), and can be integration tested using sysexts. The new widgets are localisable, accessible, and work in dark and light mode. There are even man pages. I’m quite pleased with how it’s all come together.

It’s been a team effort from a lot of people! Code, design, input and review (in no particular order): Ignacy, Allan, Sam, Florian, Sebastian, Matthijs, Felipe, Rob. Thank you Endless for the grant and the original work on parental controls. Administratively, thank you to everyone at the GNOME Foundation for handling the grant and paperwork; and thank you to the freedesktop.org admins for providing project hosting for malcontent!

Ignacy is at GUADEC, so please say hi to him! The next phase of parental controls work will involve changes to gnome-control-center and gnome-shell, so he’ll be popping up all over the stack.

I’ll try and blog more soon about the upcoming features and how they’re implemented, because there are necessarily quite a few moving parts to them.

I’m not organising GUADEC, so this is about as far as I can push this proto-handbook, but if anyone wants to fork it and build on it then please feel free.

Most of the notes so far relate to A/V things, remote participation, and some climate considerations. Obviously a lot more about on-the-ground organisation would have to be added to make it a full handbook, but it’s a start.

An update on parental controls and digital wellbeing for GNOME 47

This was my first talk at GUADEC this year, serving as a little teaser of the work I’ve been doing recently (sponsored by Endless Network via the GNOME Foundation) to add features to parental controls and digital wellbeing.

Thank you to Allan Day for fitting in work on the design for digital wellbeing this cycle, to Florian Müllner and Felipe Borges for reviewing all the code I’ve thrown at them, and to Dylan McCall for feedback on earlier versions of break reminders.

Here’s the recording, the slides, slide notes and source.

Somewhat merging gobject-introspection into GLib

This was the second talk, and a companion to Emmanuele’s talk about changes in introspection in GNOME 46. It gives an overview of how we’ve merged half of gobject-introspection into GLib recently, and what this means for app authors (basically nothing), binding developers (something, on a timeline of your choosing) and distributions (some packaging rework, for the GLib 2.78 and 2.80 releases).

Thank you to Emmanuele for spearheading this work in GLib and doing the gobject-introspection side of it, and the many GLib and gobject-introspection contributors for helping us stabilise this (in particular with build system improvements) after it landed.

Here’s the recording, the slides, slide notes and source.

The quality of the talks this year was really high. I don’t think there was a single talk slot I skipped. As a result, I didn’t get much hacking done! But there were good hallway conversations and catch ups.

I gave two talks, one on some simple improvements people can make to their apps to reduce internet data use and power use when doing so would be beneficial to the user (when on a metered network or in power-saver mode).
The aim was to remind people how easy it is to do this, and provide some examples of how different apps present these states/events in the UI, since the best way to do that can differ between apps.

You can find the slides to that talk here (notes here), and a video of it is on YouTube.

The talk has resulted in three GNOME initiatives. Each corresponds to a different state your app can pay attention to — some apps should pay attention to all of them, some apps only some of them. Please take 10 minutes to
check your app against the initiatives and make updates if necessary. I’m in the gnome-hackers and gnome-circle Matrix rooms if you have any questions about them.

• https://gitlab.gnome.org/GNOME/Initiatives/-/issues/42
• https://gitlab.gnome.org/GNOME/Initiatives/-/issues/43
• https://gitlab.gnome.org/GNOME/Initiatives/-/issues/44

My second talk was an overview of work I’ve been doing on-and-off over the past couple of years (based on work by others) to allow apps to save and restore their state when the user logs out or shuts down the computer. The idea is that the user can restore
the set of apps they were using, in the state and configuration they were left in, when next starting the computer or logging in. It’s a big project, and there’s a long way to go, but it felt like the right time to present what
we’ve got so far, let more people know how the project is structured, and get feedback from toolkit developers and app authors about the whole idea.

You can find the slides to that talk here (notes here), and a video of it is also on YouTube.

Thankfully there was 5 minutes for questions at the end of the talk, and people used them well to raise some good points and ask some good questions. I’m still in the process of factoring all that feedback into the plan, but
should hopefully have an update to give on the project in a future blog post soon.

Interesting talks I attended included Peter Hutterer’s talk about sending and capturing input in Wayland, which I think distilled his blogposts about the topic very clearly.
Allan Day’s talk about communication, which was an excellent summary of a rather long list of books on the subject, and it was very clearly presented. I feel like I could do with a cheatsheet list of his recommendations to stick next to my computer sometimes.
Evan Welsh, Philip Chimento, Nasah Kuma and Sonny Piers’s talks about JavaScript, TypeScript and the latest changes in GJS. These provided a good update on a lot that has been happening in GJS, a useful overview of TypeScript, and were really clearly presented.
Jussi Pakkanen’s talk about circles, or was it about settings? It was a good example of a lightning talk which draws you in then spins you around.

In the hallway track, I had some interesting chats with Bartłomiej about the infrastructure of ODRS, which is used to provide ratings/review data to gnome-software.
I also had some positive conversation with Cassidy about plans for GUADEC 2024 in Denver.
And at the event dinner, a really energising chat with Scott about canyoning, hiking, skiing and caving in the US and the UK.

• Having an online component to the conference increased the audience by a factor of 10: there were around 120 in-person attendees in Mexico, but there were an average of 1300 people using Big Blue Button.
• The carbon emissions from providing the remote infrastructure were around 2.8tCO₂e, or about 2kgCO₂e per remote attendee.
• Having a remote attendance party in Berlin allowed around one tenth of the attendees to attend with a factor of 10 lower transport emissions than those who attended in-person in Mexico. The average transport emissions for those who went to Berlin were 88kgCO₂e, whereas they were around 1tCO₂e for those going to Mexico.
• For context, the annual emissions per person can be at most 2.3tCO₂e by 2030 in order to limit global warming to 1.5C by the end of the century. That covers all food, travel, heating, purchases, etc. So travel to Mexico was 40% of the average attendee’s annual target.
• Half of the in-person attendees travelled from within Mexico, which will have skewed the mean transport emissions downwards. The distribution is more likely bimodal between around 50kgCO₂e for locals and more like 2-3tCO₂e for those coming from outside Mexico.
• As I wrote at the time, the remote attendance party was fun to attend and felt like it worked as a way to attend the conference (there were no A/V problems, we had some nice local socials, etc.). I would do it again.
• The post-conference survey had a low response rate of about 19% of registered attendees, which made some of this analysis hard. Please always fill in the post-conference survey! It doesn’t take long, and aside from any of this analysis, it helps the organisers plan conferences better in future.
• Many thanks to Kristi, Caroline and Bartłomiej for collecting the data needed for this analysis.
• If anyone spots problems in the analysis please say! This is not the kind of thing I practice doing very often and I’ve had to make a number of assumptions.

Other conferences

A post-conference survey has been done for other big GNOME events, such as GNOME.Asia and LAS. Would anyone be interested in doing a similar analysis for those events? Perhaps we can get a semi-automated pipeline in place for it.

I am not flying, because planes are too polluting. Instead, I am taking the train to Lübeck in Germany, then an overnight ferry to Liepāja, and then a bus the following morning to Riga. It’s a bit slower, but means it’s a bit easier to get some hacking done, stretch my legs and move around, and not fund fossil fuel companies as much. I’ll get enough stopover time in Köln, Hamburg and Lübeck to quickly look round, and a night in Liepāja to see it.

Overall the travel time is just over 2 days, with half of that spent on trains, and half on a ferry. By comparison, a flight is about 7 hours (5 hours flying, 2 hours faffing in airports) plus travel time to the airport.

The carbon emissions (140kgCO2e return) are roughly a quarter of those from flying (520kgCO2e), and interestingly a significant part of those emissions (46kgCO2e) is the 3 hour bus journey to get from Liepāja to Riga, though that’s quite sensitive to the occupancy level of the bus.

The financial cost (£800 return) is about two times that of flying (£380), though I have not factored in the costs of getting to/from airports and have not fully explored the hidden fees for baggage and other essentials so the ratio might be a little lower. This is quite upsetting. A disproportionate part of the cost (£178 return) is the Eurostar, because it’s oversubscribed and I missed the early ticket releases due to waiting for grant approval. Perhaps I should not wait next time.

The journey

On 2023-07-22:

• Eurostar from London to Brussels-Midi, departing 07:04
• Train from Brussels-Midi to Lübeck-Travemünde Skandinavienkai, departing 10:25 (ICE 15, ICE 200, RE 11428, RB 11528)
• Nice walk from there to the ferry terminal for half an hour
• Overnight ferry from Lübeck/Travemünde to Liepāja, departing 23:30

On 2023-07-23: On the ferry all day, then stay overnight in Liepāja

On 2023-07-24: Bus from Liepāja to Riga, departing early morning

Alternatives

I strongly looked at taking the train from Hamburg to Stockholm, and then the ferry from there to Ventspils. Unfortunately, it has limited capacity and there is track maintenance planned for around my travel dates, so I could not get suitable tickets. It would have made the timings a little more convenient overall, for about the same overall carbon emissions and cost.

Join me

If anybody else is going overland from the UK or far-western Europe, this is hopefully a sensible route for you to take, and it would be lovely if you wanted to join me. I will be arriving 2 days early for GUADEC (as we’re having an Endless OS Foundation meetup), but if you wanted to do the same journey 1 or 2 days later then it shouldn’t differ significantly. In any case, I can put you in touch with others making this journey if you want.

As with many things in GLib, the allocations in this code use refcounting, rather than new/free, to manage object lifecycles. This makes it quite hard to debug lifecycle issues, because the location of a bug (a ref or unref call which isn’t correct) can be quite far removed (in time and code) from where the effects of that bug become visible. This is because the effects of refcounting problems only become visible when an object’s refcount reaches zero, or when the program ends and its refcount still hasn’t reached zero.

While debugging this code, I tried an approach I haven’t before: changing some of the ref calls on the buggy object to be copy calls instead. (Specifically, changing g_object_ref() to g_dbus_message_copy().) That split up the lifecycle of the object into smaller pieces, narrowing down the sets of ref/unref calls which could be buggy. Ultimately, this allowed me to find some bugs in the code, and hopefully those are the bugs causing the refcounting issue. Since the issue is intermittent, it’s a bit hard to be sure.

This debugging approach was possible in this case because the object I was debugging is immutable, so passing around copies of it doesn’t affect the behaviour of other bits of code vs passing around the original. Hence this approach is only applicable in some situations. But it’s another good reason why using immutable objects is quite helpful when writing code, and it’s certainly an approach I’m going to be using again when I can.