I am a big believer in the four opens:

The Four Opens are a set of principles guidelines that were created by the OpenStack community as a way to guarantee that the users get all the benefits associated with open source software, including the ability to engage with the community and influence future evolution of the software.

• Open Source
• Open Design
• Open Development
• Open Community

There is an implied “to the public” in each of the four opens, at least how I have understood it over the past many (many) years. I have repeatedly advocated for open (to the public) discourse and transparency when working with companies like CloudBees and Databricks as they have engaged with open source projects.

The mounting negative pressures and in some cases outright hostility towards free and open source projects has me reconsidering the implied “to the public” and how these communities may need to evolve in the future.

While I have never been a fan of invite-only Discord or Slack servers, both of which are used by the Apache Datafusion project for some odd reason. There are good reasons to put the project’s shared spaces in slightly more private and slightly less AI-accessible systems. A little bit of privacy can lead to more candid conversations and potentially a stronger feeling of community and safety.

My first line of thinking led me to the idea of “vouching” which I recall mitchellh posting about in the fediverse, but I couldn’t find a good linkable reference.

Vouching is what we did as kids when a new friend was suggested to join the mischief, somebody would vouch for the new kid and say “hey, they’re my neighbor, they’re cool” and then we would go start new trouble together. In the context of an open source community vouching can:

• Help build a web of trust without every person necessarily knowing each new person
• But also vouching means there is a higher tendency for a community to be homogeneous, since it will be less welcoming to random new-comers.

I think vouching could also exacerbate the likelihood of a Jia Tan where the web of trust within the community is compromised by a malicious actor. Getting one member to vouch for you may lower the guard of all of the other members of the community making these style of attacks easier to pull off.

Since I started writing this post a whole week has passed by, without any new ideas or patterns popping into mind. I’m curious how others are thinking about it, so please let me know on Mastodon or via email rtyler@~

systemd as an init system is pretty good. Once upon a time I worked on porting launchd to FreeBSD and so I have some familiarity with the silliness of most init systems.

systemd as a katamari at the root level of most Linux systems is not “pretty good.” There have been numerous tendrils of what is understood to be “systemd” which are of lesser quality and have resulted in security issues.

Anyways, I hope you get the point. systemd as an init system: good. systemd as a operating system: bad.

The drama du jour is about the latter.

One should not obey in advance. Especially in the domain free and open source software which is literally a political project.

I stumbled into this blog post through Planet Debian by a debian maintainer which is patently absurd.

Recently, the free software Nazi bar crowd styling themselves as “concerned citizens” has tried to start a moral panic by saying that systemd is implementing age verification checks or that somehow it will require providing personally identifiable information.

The author is correct insofar that systemd did not add age verification. However most of the folks upset with the change are upset that their Linux systems are obeying in advance.

systemd did make changes in order to obey. To take part in anti-free restrictions under the guise of “age verification” From the pull request

Stores the user’s birth date for age verification, as required by recent laws in California (AB-1043), Colorado (SB26-051), Brazil (Lei 15.211/2025), etc.

The whole motivation of the change was to obey in advance to these unjust laws.

The author then goes on to make some equally absurd claims about how this functionality is important for porents to implement controls on computers, for the children! Clearly this person must not know any actual children, or even parents for that matter. Children are excellent at finding ways to circumvent restrictions. The idea that a user-modifiable piece of data on a local machine should be trusted for “parental controls” is so detached from reality that I originally thought they were making a sarcastic joke.

I think this tongue-in-cheek systemd-censord post does better than anybody to exclaim how absolutely ludicrous this obeying in advance is:

Systemd units will be created for every desired censorship function, and will be started based on the user’s location. For example, a unit for Kazakhstan will implement the government-required backdoor, a unit for China will implement keyword scans and web access blocks (more on this later), a unit for Florida will ban all packages with “trans” in the name (201 packages in current stable distribution), a unit for Oklahoma will ensure all educational software is compliant with the Christian Holy Bible, a unit for the entire United States will prevent installation of any program capable of decoding DVD or BluRay media, and a unit for California will provide the user’s age to all applications and all web sites from which applications may be downloaded. As can be seen, multiple units may be started for a given location.

Do not obey in advance.

At my height any seat
can feel like misery.

I wouldn’t be here today,
if not for last night’s delay.

35E, 35E
trapped in this humid
sneeze of
humanity.

Everything is expensive, and still it sucks.
The cheapest coffee was four lousy bucks.

The grumpiness was extreme at the TSA
acting in their theatre for deferred pay

35E, 35E
I’m not sure in which
timezone
I should be.

United customer support has been totally outsourced,
hour seventeen on the phone; just the worst.

For the shareholders the texts all lie,
about being powered by GenAI.

35E, 35E
all of the staff
on this flight and the last
have been really kind and patient which is a testament to their professionalism and hospitality despite the overtly customer-hostile environment of modern American commercial aviation.

Compressed Apache Arrow tables over HTTP

When discussing transport protocols for sending data between services at work recently, a colleague asked “why can’t we just yeet Arrow over HTTP?” It turns out, you absolutely can and Arrow IPC streams even have a registered MIME type:

Content-Type: application/vnd.apache.arrow.stream

Understanding Parquet format for beginners

A great introduction to the Apache Parquet format and why it makes so many things better with large data storage systems like Delta Lake. I have written on this topic before and encourage you to take another read through this blog post by some maintainers of the parquet crate.

Every layer of review makes you 10x slower

Every layer of approval makes a process 10x slower [..]

Just to be clear, we’re counting “wall clock time” here rather than effort. Almost all the extra time is spent sitting and waiting.

• Code a simple bug fix: 30 minutes
• Get it code reviewed by the peer next to you: 300 minutes → 5 hours → half a day
• Get a design doc approved by your architects team first: 50 hours → about a week
• Get it on some other team’s calendar to do all that (for example, if a customer requests a feature): 500 hours → 12 weeks → one fiscal quarter

This inspired these thoughts which I shared with the delta-rs community:

“what if we didn’t require code review for merging into main”

I’m exploring the thought more about what we might need to make that happen. “Why would you do such a thing, code review is so valuable!”  I do find code reviews valuable but we do seem to lose a lot of flow time due to timezones, differing work schedules, and a number of other things. For something without a lot of changes, especially bug fixes that come with tests I would be much more comfortable with maintainers merging once CI goes green.

Some pieces of the puzzle that I think would be needed:

• Soft caps on pull requests. I saw this mentioned somewhere else, but implementing a soft cap of <500 lines per pull request can help people avoid massive unreviewable changes that are simpler to integrate.
• Incorporating some of the benchmarking work into CI that has already been explored. If performance of key operations is not affected and the build is green, go for it.
• Stronger semantic version checks: if our APIs have not changed and all tests pass, I’m generally comfortable with landing stuff by maintainers.
• Implementing Apache Software Foundation style release candidates and voting: this is where we would put a mandatory bottleneck, rather than some jokey slack emojis like I tend to do, implementing a true release candidate process that requires review and vote before we push something to users.

All of this is to say that reviews can still be requested, but I would love to see us land more improvements faster and I think we have a bunch of different schedules that can make pushing each change through a review queue a lot slower than necessary.

Conditional Impls in Rust

It’s possible in Rust to conditionally implement methods and traits based on the traits implemented by a type’s own type parameters. While this is used extensively in Rust’s standard library, it’s not necessarily obvious that this is possible.

I have been vaguely aware of this functionality but haven’t really taken the time to consider it, so I really appreciated this post walking through the conditional impl functionality in Rust.

This video struck a chord for me. Contrasting the Bay Area hustle culture to the things that make life worth living:

San Francisco has always a destination for those seeking their fortunes. The frenetic enthusiasm radiates through seemingly everything there.

I also really enjoyed the energy of San Francisco when I first moved there. I had nothing else but work.

The trade-off for my relentless focus on my career was a tremendous level up in a short amount of time. I wouldn’t be where I am today without a few years of judicious networking and workaholism.

San Francisco was “Lord of the Flies” when I described it to friends from elsewhere. Awash in adult boys, untethered from the real world. I would hang out with men 10 years older then me who were doing the same dumb shit I was, except I was in my early twenties, a commonly accepted time to be foolish.

I did not want to end up like them and increasingly put both physical and mental distance between the them and myself.

There is more to life than panning for gold.

Today I was talking with an elder almost twice as old as me, who casually offered:

I still get up at 5am; at this age there are only so many sunrises left to see.

I’m going to try to not stay up too late dwelling on the comment, lest I miss tomorrow’s sunrise.

For years the conventional wisdom around Delta Lake has been to not connect user-facing/online systems to Delta tables. Basically, don’t point your Django app at your Delta tables. This continues to be a decent guideline but definitely not a rule and I have the performance data to back that up.

My talk abstract:

Scribd hosts hundreds of millions of documents and has hundreds of billions of objects across our buckets. Combining large-language models with a massive amounts of text has required investment in our new Content Library architecture.  We selected Delta Lake as the underlying storage technology but have pushed it to an extreme. Using the same Delta Lake architecture we offer both direct data access for data scientists in Databricks Notebooks and online data retrieval in milliseconds for user-facing web services.

In this talk we will review principles of performance for each layer of the stack: web APIs, the Delta Lake tables, Apache Parquet, and AWS S3.

The work done by myself and my colleague Eugene in this area has been heavily related to my previous research around Low latency Parquet reads which informed work named Content Crush, which I have explored more on the Scribd tech blog and on the Screaming in the Cloud podcast.

I really hope that I am able to share results at Data and AI Summit from this incredibly challenging work that I am undertaking. But even if I don’t, blog posts like my musings on Multimodal with Delta Lake, scaling streaming Delta Lake applications, and a myriad of other articles I have published can be pieced together to form the larger mosaic of insane large-scale data work I have been hammering on!

For the longest time I used the built-in VNC support to connect to machines because the vm console command would use /usr/bin/cu which would inevitably trap my console and no amount of ~>~>~><~D~>D<S~>D<~><D~<L would help me exit.

Somewhere along the line tmux support was added to vm-bhyve and now vm console <name> simply opens up a new tmux window!

I host everything under /vm on the machine, so in /vm/.config/system.conf:

console="tmux"

This seems like a simple thing to be excited about, and it is, but it makes VMs wildly more accessible and useful for me.

The developer was trying to write some brand Terraform for the project they were working on, not their area of expertise but it needed to be done. They had an experience which I recognized from my explorations with earlier models where I also just wanted the model to write some miserable Terraform resources because I didn’t want to myself. Except when prompted, it was as if I asked “write me some Terraform to provision this resource, wrong answers only.”

I also thought I must be insane and just using the magic numbers box incorrectly. Seeing the exact same situation play out for another developer, with a different model, years later, I feel like i understand the problem!

There is not enough open source use of Terraform to copy.

Plenty of great open source terraform modules exist (Putin khuylo!) but perilously few open source examples exist of using Terraform. I believe this to be because the vast majority of Terraform is closed source and therefore not scraped and ingested into these models.

If code-generating AI tools don’t want to suffer from the dead internet theory, the data has to come from somewhere.

The machine relies on code being open sourced.

LTJG Kaffee: Colonel Jessep! Did you author that code?

Judge Randolph: You don’t have to answer that question!

Col Jessup: I’ll answer the question. You want answers?

LTJG Kaffee: I think I’m entitled to them.

Col Jessup: You want answers?!

LTJG Kaffee: I want the truth!

Col Jessup: You can’t handle the truth!

Son, we live in a world that has models, and those models have to be seeded by humans who code.

Who’s gonna do it? You? You, Lieutenant Weinberg?

I have a greater responsibility than you can possibly fathom.

You weep for the roadmap, and you curse the upstream. You have that luxury. You have the luxury of not knowing what I know – that the launch delay, while tragic, probably saved time; and my existence, while grotesque and incomprehensible to you, saves time.

You don’t want the truth because deep down in places you don’t talk about at parties, you want me feeding that model – you need me feeding that model.

We use words like “fork,” “code,” “libre.” We use these words as the backbone of a life spent supporting something. You use them as a punch line.

I have neither the time nor the inclination to explain myself to a man who rises and sleeps under the blanket of the productivity that I provide and then questions the manner in which I provide it.

I would rather that you just said “thank you” and went on your way. Otherwise, I suggest you pick up an editor and submit a PR.

Either way, I don’t give a DAMN what you think you’re entitled to!

LTJG Kaffee: Did you author that code?

Col Jessup: I did the job –

LTJG Kaffee: – Did you author the code?

Col Jessup: YOU’RE GOD DAMN RIGHT I DID!

One of the big arguments against generative AI-based coding tools is that they were trained on billions of lines of copyrighted and licensed works in the open source ecosystem, and they strip those works of all attribution and violate the terms of the licenses.

Yesterday there was some fervor about the Supreme Court allowing a lower court decision to stand in my timeline. I have been following this topic for a few weeks after reading this Congressional Research Service report: Generative Artificial Intelligence and Copyright Law (PDF) and considering how some of the guidance might affect the use of generative AI in open source projects.

kat nailed it with their toot

So uh

does this mean that there is now precedent that at least “agentic” dev systems, potentially any genAI dev system, now leaves companies open to their code no longer being considered copyrightable if they use these systems?

kat is right that this is huge.

In the Delta Lake project we rely on Developer Certificate of Origin (DCO) with guidance from the Linux Foundation. Yes, that Linux Foundation.

From the DCO:

The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

From the Apache License:

“Licensor” shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.

[..]

“Contributor” shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.

I am no lawyer but I do have at least a 12th grade reading comprehension level.

If AI-generated works are not copyrightable, then it is not possible for somebody to license under any open source license, much less assert via DCO that they are able to do so.

2026-03-04 update: Software Freedom Conservancy has a blog post on the topic which is worth a read. I understand the narrowness of the scope of judgement which is being referred to, but my opinion on the situation is taking into consideration the other guidance from the Congressional Research report and the US Copyright Office oplicies as they stand today.

Again, this is a big deal.

From the Congressional Research report

The AI Guidance states that authors may claim copyright protection only “for their own contributions” to such works, and they must identify and disclaim AI-generated parts of the works when applying to register their copyright.

The only viable solution I can imagine is that all AI-generated code contributions in open source projects is considered public domain and commented appropriately. Otherwise I don’t see a sensible path forward for AI-generated code in open source projects.

Today Hetzner, a major cloud services provider in Europe announced

There have been drastic price increases in various areas in the IT sector recently. That is why, unfortunately, we must also increase the prices of our products.

The costs to operate our infrastructure and to buy new hardware have both increased dramatically. Therefore, our price changes will affect both existing products and new orders and will take effect starting on 1 April 2026.

Last year for Earth Day I wrote on the Buoyant Data blog

Time is money. In the cloud time is measured and billed by the vCPU/hour and the most efficient software is always the cheapest.

Nothing makes the case for more efficient software like more expensive hardware!

In the past five years I have repeatedly seen success in taking a system written in a less-efficient platform, redesigning and rebuilding in Rust, and reaping the rewards in lower operational costs.

For a simple exercise, imagine a service which costs $100,000/year to operate, that’s roughly $1,900 a week. Assuming a developer’s time costs roughly $6,000 a week, taking a month to rebuild the service might cost $25,000. The efficiency needed is then only about 25% to pay off that rewrite in a year, but what I have consistently seen is an order of magnitude change in efficiency.

Instead of costing $100k, these newly deployed services tend to cost less than 10-20% of their predecessors. Recouping the cost of conversion in a couple of months, freeing up money to go towards different investments.

The biggest cost to contend with is opportunity cost and that one is much harder to model, and also much less subject to changing prices by your vendors.