Holden's Blog

Yet Another AI Security OSS Externality

2026-09-29T11:20:59.709-07:00

Yet Another Rant About AI Security OSS Externality

IMPORTANT DISCLOSURE: This blog post is written in my personal capacity reflecting on the recent Apache Spark patch releases (3.5.9, 4.0.4, and 4.1.3). These are not the views of the Apache Software Foundation (ASF), the Apache Spark project, or any of my employers past or present. Both the ASF and the AI research lab have been provided an opportunity to comment on an earlier draft of this blog post and correct any information they see as incorrect and while many individuals have provided feedback in their personal capacity, there is not (as of yet) any official comment from either.

AI has fundamentally changed the volume of security reports in all projects, especially OSS projects where the barrier to analysis is even lower. We need to revisit how we handle security disclosures in the age of AI-driven pull requests and security auditing. Open Source Software (OSS) has long been for the community, and while some of us have jobs which give us time to work on open source, it's most often largely a volunteer effort for the public good, with the ASF focused on Software For the Public Good since it was incorporated in 1999. Security is of the utmost importance and requires greater care in collaboration to avoid accidental disclosure. For security teams, when working with OSS projects, please be flexible on release timelines when issues are not actively being exploited. If we have to move our release schedule to meet your planned disclosure date, it's more likely to cause us to ship more security issues rather than fewer.

While some labs have early access programs, in my experience they're not working. Despite there being an agreement for early access to one of the labs' "frontier security models", we only got access to a scan in August. While a team at that same lab, that we'll call Team X, is, seemingly, using their frontier models from at least February to report security bugs and issue deadlines for resolution and disclosure.

The old model of "here's a bug, you have ~90 days to fix your issues before we publish how to exploit your code" came about because software vendors, mostly closed source but some open too, were in the habit of ignoring security reports (or being openly hostile to them). Previously security reports took a human to make, and we'd get a few every month, but now with AI our security report inbox is looking more like the promotions tab in my Gmail. Additionally, more bug fixes are landing later in the release candidate cycle from AI driven PRs, so these previously reasonable hard security disclosure deadlines are making it more likely we ship worse security bugs while we try and play catch up. To that end, I want to call on all frontier AI labs to put more funding into mitigating the impact of their tools on the OSS projects which are vital to their success.

To put it in “corporate” the key asks from this entire post are fairly simple:

Frontier AI labs should provide real usable access to their frontier models for OSS
AI labs should be careful in teams using their models in house for security in advance of their own programs for “trusted” access.
More flexibility on disclosure dates is needed given the impacts of AI
A pony with attached coffee cup holder^ (and unlimited coffee) (jk)

The core of the issue remains that reviewer bandwidth is the scarcest thing in large open source projects. While tools like CodeRabbit are amazing (big shout-out to them for their OSS plans), they still don't capture everything. Thanks to the increasing frequency of attempted (and sometimes successful) malicious PRs and takeovers of OSS projects, we can't just leave it to the robots to argue amongst themselves. Maintainer burnout is a real security risk that we as an industry are not prepared to handle.

While these suggestions are only a starting point, the big AI labs can do a number of things to mitigate some of the downsides. One is providing usable non-time locked free accounts to the people whose software they used. For some of the bigger projects, the AI labs have staff who are able to do things like manage releases, but they're often too busy working on internal issues to have the time to work upstream. I think giving engineers something akin to Google's "20% time" (which some jokingly called 120% time) to give back to the projects they're using could help.

I want to double down on the "usable" part. While Anthropic has done better than any of the other AI labs at trying to provide free tooling for OSS developers, their effort still doesn’t do enough. In trying to use my Anthropic provided OSS maintainer account on a recent Spark security issue I tripped its safety guards and eventually ran out of tokens*. In practice, I've found paid inference on open weight models** to be more effective at addressing security issues. Most OSS maintainers, including myself, don't yet have usable direct free access to the frontier security models, meaning we're pushed into a reactive mode rather than pro-active.

For me these issues came into focus during the recent ASF Spark 3 and 4 patch releases, available everywhere fine bits can be downloaded. Most impactful was a security issue reported by Team X where because of a combination of a lack of communication in the initial disclosure and lack of flexibility on the surprise deadline once the release managers became aware, we had to scramble for extra reviewers on complex separate security issues.

Team X reported security issues in the Apache Spark project around February 25th of this year. This report was (thankfully) accompanied by a human-reviewed proposed patch and no specific deadline in the initial report, which to be fair is better than we would get with a frontier model alone. So far, so good.

However, out of band, Team X then gave a vendor who distributes and contributes to Apache Spark (not my current employer) 90 days to fix this, or Team X would disclose the issue publicly. While the vendor requested an initial extension to line up with the planned OSS release schedule, which Team X granted, the deadline was not communicated to the broader OSS group, including myself, until much later.

While working on the most recent releases, we got roughly 30 post-first-line filter security reports in the week before the first patch release candidate, a few of which turned out to be actionable. Part way through the release cycle, I became aware of the deadline from Team X and reached out to the head of security at the lab to further extend the deadline so we could address the outstanding security issues. Some of these issues were blocked on knowledge of specialized libraries I personally don't have. Initially Team X was unwilling to extend the deadline any further, but when I explained how our 72-hour release voting normally works, they were willing to grant one extra week if the vote failed. During this negotiation, the amazing ASF security folks found a reviewer for the complex issue I was wrangling. We were able to get known issues fixed and the release candidate vote thankfully passed, but that outcome was far from certain.

If we had not been able to find reviewers for the encryption fix, we would have had to ship a partial release with a known broken encryption issue. At the end of the day, Team X has still not published the disclosures as of the end of September, strongly suggesting that they could have given us more time. As a release manager, this is of course frustrating and had personal impacts on my vacation coffee consumption being more necessary rather than enjoyed, but the impacts could have been much worse (not just missing out on limoncello).

I am not particularly anti-AI as far as things go. Having worked on making AI tools to help people appeal health insurance denials, I think there are potential positive cases where AI can help people.

I'd also love to give a shout-out to my point of contact at the frontier AI lab for helping unravel the game of telephone and getting me that extra week.

I don't think this problem is unique to Team X, the process is broken and compounding the existing reviewer constraint. Reviewer bandwidth has almost always been the biggest bottleneck in OSS projects, and the decreasing cost to generate code will continue to push against this bottleneck. When issues are discovered in OSS projects that are not actively being exploited, requiring releases from the OSS project with a fixed timeline is counterproductive. Release cycles are likely going to keep getting longer (see how big the latest Linux RCs are), and we should allow time for this. For the humans driving the robots, when there is a public interest-reason like an active exploit, not just a conference or PR opportunity, for providing a disclosure deadline, make sure to include that in your initial report whenever possible.

I'd like to end by reiterating my request to all of the frontier AI labs: please offer truly usable free plans for OSS developers, be flexible with your disclosure deadlines when no active exploits exist. In addition, there are some amazing engineers at these labs, if you give your staff time to participate in OSS maintenance tasks it could make a big difference. There's a strong history of industry and OSS working together in both ASF and other projects from Spark to PyTorch to the Linux kernel. That and a pony with a coffee cup holder and a coffee, but I'll take more tokens in place of a pony (I still want the coffee though :p).

If you're coming to Glasgow for Community over Code, I'll be around and looking to catch up with other OSS folks for their views on how we can try and make the externalities from AI less terrible, including chatting with folks at the responsible AI round table, and talking about how to make PySpark even faster. For SF based folks I'll be at QConSF in November talking on using AI powered transpilation to make UDFs faster. And for folks regardless of location I'm going to be doing a virtual talk end of October on dealing with AI security reports (I'll post on LI and blog when the registration link is available).

P.S.

I am well aware this is a bit of a “do as I say not as I did” situation as I did report some Yahoo! Mail security vulnerabilities purely through side channels back in University almost two decades ago. But I was young and did not have the resources of a multi-billion dollar company behind me, and I would argue the security implications of unencrypted e-mail passwords are a bit different than data processing software with schema inference driven class instantiation vulnerabilities.

* I try to use the account for other OSS work, not just security. It is more successful for non-security work.
** I don't currently have enough vRam to run these on my own hardware, but if anyone has a few hundred spare RTX5090s or DGXs and a few 20a circuits give me a call :p
^ Did you know that's a real thing? I mean not the unlimited coffee, but they make coffee cup holders for saddles. In practice since I live in a city and do not have space for a pony let alone the skills, I ride a Tiger 900 (without coffee cup holder) and an LX150 (with coffee cup holder). If the AI labs are reading this: coffee cup holder for motorcycle with unlimited coffee is an acceptable substitute 😛

New year new job, same projects

2026-02-12T07:51:00.000-08:00

I’m stoked to announce that I’ve joined Snowflake to continue working on OSS Apache Spark :) I’ve got a post on the Snowflake blog talking about the work we’re doing https://careers.snowflake.com/us/en/blogarticle/building-apache-spark-in-the-open-at-snowflake —

Making Hibernate work on Ubuntu 22.04 (jammy) on the Framework Laptop w/full disk encryption

2022-05-30T17:44:00.002-07:00

Making Hibernate work on Ubuntu 22.04 (jammy) on the Framework Laptop w/full disk encryption

The 0th step is disabling secure boot in the bios (reboot, press F2 to get into the BIOS, disable secure boot).

The first step is to make a swapfile large enough for your device to suspend to.

sudo swapoff /swapfile

sudo dd if=/dev/zero of=/swapfile bs=$(cat /proc/meminfo | awk '/MemTotal/ {print $2}') count=1024 conv=notrunc

sudo mkswap /swapfile

sudo swapon /swapfile

Then make sure it's enabled on reboot, e.g: echo '/swapfile swap swap defaults 0 0' | sudo tee -a /etc/fstab

If there is another swap partition or file in /etc/fstab you may need to take it out.

Find the device your swapfile is located on (e.g. findmnt -no source -T /swapfile), note here some guides recommend using the UUID, in my experience, this resulted in some errors.

To find the resume offset of your file run filefrag -v /swapfile and then look at the first value under physical_offset.

Set `GRUB_CMDLINE_LINUX_DEFAULT="splash resume=${RESUME_DEVICE} resume_offset=${RESUME_OFFSET} nvme.noacpi=1"` in `/etc/default/grub` and then update grub with sudo update-grub

If you want to enable hibernation on lid close set `HandleLidSwitch=hibernate` in `/etc/systemd/logind.conf`.

Taking a Break (surgery)

2020-10-31T10:18:00.001-07:00

Once again I am having surgery. As with all surgery, there is a chance for complications, so while I hope to be back up and at most of my usual things by the end of November, there is a non-zero chance that plan won't pan out. Unlike last time (fixing bones from a Vespa/motorcycle crash), this is one I've had in the back of my mind for a bit (facial feminization surgery) and I'm really excited about it.

I am going to try and do some blogging during my recovery period -- if I feel up to doing so (predicting how recovery is going to be is HARD). I do know that when I feel like doing streams again, that'll be without my face visible before I feel comfortable with myself on camera again -- thanks for understanding. I will continue to do code reviews and fix any bugs I've introduced whenever possible, but I will be slower to respond.

If you are an American citizen, please vote! And regardless of where you are when you can, vote for love over hate. Local elections can have huge impacts too, and even in liberal cities, I've seen people running on platforms that would severely hurt myself, my friends, and our community. I love San Francisco, and I don't want to have to leave (especially after finally getting my green-card this year).

Socially Distant Air Hugs & Kisses,

Holden

Scaling Python for ML (new blog)

2020-10-26T16:46:00.002-07:00

I've started a new blog over at http://scalingpythonml.com/ exploring the different tools to scale Python ML code. I'm just getting started but you can also follow along with my YouTube channel.

The Importance of Reviewing PR comments @ Scala Days EU 2019

2019-06-12T02:42:00.001-07:00

Come join me on Wednesday 12 June @ 17:45 for The Importance of Reviewing PR comments.I'll update this post with the slides soon.Come see to the talk or comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Validating Spark ML Jobs-Stopping Failures Before Production on Apache Spark @ @SparkAISummit SF 2019

2019-04-24T11:10:00.001-07:00

Thanks for joining me on 2019-04-24 at @SparkAISummit SF 2019 San Francisco, CA, USA for Validating Spark ML Jobs-Stopping Failures Before Production on Apache Spark.

The slides are at http://bit.ly/2KYpqig.

Validating spark ml jobs stopping failures before production on Apache Spark @ Spark Summit 2019 from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Powering Tensorflow with Big Data @ CERN Computing Seminar

2019-04-03T23:26:00.001-07:00

Thanks for joining me on 2019-04-03 for Powering Tensorflow with Big Data.

The slides are at http://bit.ly/2MWFpi3.

Powering tensor flow with big data using apache beam, flink, and spark cern 2019 (3) from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

PySpark on Kuberntes @ Python Barcelona March Meetup

2019-03-21T10:25:00.001-07:00

Thanks for joining me on 2019-03-21 at Python Barcelona March Meetup 2019 Barcelona, Spain for PySpark on Kuberntes.

The slides are at http://bit.ly/2Fv7Uwj.

PySpark on Kubernetes @ Python Barcelona March Meetup from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Contributing to Spark 3 @ Spark BCN Meetup

2019-03-19T10:17:00.001-07:00

Thanks for joining me on 2019-03-19 at Spark BCN Meetup 2019 Barcelona, Spain for Contributing to Spark 3.

The slides are at http://bit.ly/2HwlXFf.

Contributing to Apache Spark 3 from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

PyData Hong Kong - Making the Big Data ecosystem work together with Python: Apache Arrow, Spark, Flink, Beam, and Dask @ PyData Hong Kong

2019-02-18T18:02:00.001-08:00

Thanks for joining me on 2019-02-19 at PyData Hong Kong 2019 Hong Kong for PyData Hong Kong - Making the Big Data ecosystem work together with Python: Apache Arrow, Spark, Flink, Beam, and Dask.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Validating Big Data Jobs An exploration with @ApacheSpark & @ApacheAirflow (+ friends) @ FOSDEM

2019-02-05T06:50:00.001-08:00

Thanks for joining me on 2019-02-03 at FOSDEM 2019 Brussels, Belgium for Validating Big Data Jobs An exploration with @ApacheSpark & @ApacheAirflow (+ friends).

The slides are at http://bit.ly/2wMwRiF.

Validating big data pipelines - FOSDEM 2019 from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Introducing @Kubeflow (w. Special Guests Tensorflow and @ApacheSpark) @ FOSDEM

2019-02-02T01:30:00.001-08:00

Thanks for joining us (@holdenkarau,@rawkintrevo) on 2019-02-03 at FOSDEM 2019 Brussels, Belgium for Introducing @Kubeflow (w. Special Guests Tensorflow and @ApacheSpark).I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Apache Spark on Kubernetes -- Avoiding the pain of YARN @ Pre-FOSDEM Belgium Kubernetes Meetup

2019-02-01T02:34:00.001-08:00

Thanks for joining me on 2019-02-02 for Apache Spark on Kubernetes -- Avoiding the pain of YARN.The talk covered:

Apache Spark is one of the most popular big data tools, and starting last year has had integrated support for running on Kubernetes. This talk will introduce some of the use cases of Apache Spark quickly (machine learning, ETL, etc.) and then look at the current cluster managers Spark runs on and their limitations. Most of the focus will be around running non-Java code, and the challenges associated with dependencies along with general challenges like scale-up & down. Its not all sunshine and roses though, I will talk about some of the limitations of our current approach and the work being done to improve this.

.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Predicting areas for PR Comments based on Code Vectors & Mailing List Data @ FOSDEM

2019-01-02T00:07:00.001-08:00

Thanks for joining us (@holdenkarau,@krisnova) on 2019-02-03 at FOSDEM 2019 Brussels, Belgium for Predicting areas for PR Comments based on Code Vectors & Mailing List Data.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Understanding Spark Tuning with Auto Tuning (or how to stop your pager going off @ Data Day Texas

2018-12-31T15:44:00.001-08:00

Thanks for joining me on 2019-01-26 at Data Day Texas 2019 Austin, TX, USA for Understanding Spark Tuning with Auto Tuning (or how to stop your pager going off.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!) -- now with Scooters @ Scala eXchange 2018

2018-12-14T06:18:00.001-08:00

Thanks for joining me on 2018-12-14 at Scala eXchange 2018 London, UK for Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!) -- now with Scooters.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Holden @ Kiwi Code Mania: talk title TBD @ Code Mania

2018-12-14T03:17:00.001-08:00

Thanks for joining me on 2019-05-15 at Code Mania 2019 Auckland, New Zealand for Holden @ Kiwi Code Mania: talk title TBD.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!) @ Big Data Spain

2018-11-14T02:56:00.001-08:00

Thanks for joining me on 2018-11-14 at Big Data Spain 2018 Madrid, Spain for Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!).

The slides are at http://bit.ly/2S1y3HJ.The video of the talk is up at http://bit.ly/2Gxk23h.

Validating Big Data Pipelines - Big Data Spain 2018 from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Big Data w/Python on Kubernetes (PySpark on K8s) @ Big Data Spain

2018-11-14T02:53:00.001-08:00

Thanks for joining me on 2018-11-15 for Big Data w/Python on Kubernetes (PySpark on K8s).

The slides are at http://bit.ly/2RWsxpA.The video of the talk is up at http://bit.ly/2R9x4bE.

Big data with Python on kubernetes (pyspark on k8s) - Big Data Spain 2018 from Holden Karau

And some related links: recorded demos, setup livestreamComment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

A Basic Introduction to PySpark Dataframes by exploring ASF Gender Diversity Data - workshop @ @pyconca

2018-11-09T17:49:00.001-08:00

Thanks for joining us (@holdenkarau,@math_foo) on 2018-11-10 at @pyconca 2018 Toronto, ON, Canada for A Basic Introduction to PySpark Dataframes by exploring ASF Gender Diversity Data - workshop.

You can find the code for this talk at https://github.com/holdenk/diversity-analytics/.The slides are at http://bit.ly/2RNhG1g.There is a related video you might want to check out.

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

PyConCanada After Lunch Keynote Sunday @ @pyconca

2018-11-08T21:02:00.001-08:00

Thanks for joining me on 2018-11-08 for PyConCanada After Lunch Keynote Sunday.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Dealing With Contributor Overload @ Festival de Software Libre

2018-11-02T13:58:00.001-07:00

Thanks for joining us (@holdenkarau,@griscz) on 2018-11-02 at Festival de Software Libre 2018 Puerto Vallarta, Jalisco, Mexico for Dealing With Contributor Overload.I'll update this post with the slides soon.Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Diversity in OSS with Holden & Gris @ Festival de Software Libre

2018-11-02T10:27:00.001-07:00

Thanks for joining us (@holdenkarau,@griscz) on 2018-11-02 at Festival de Software Libre 2018 Puerto Vallarta, Jalisco, Mexico for Diversity in OSS with Holden & Gris.

The slides are at http://bit.ly/2JBYgcw.

Keynote Open Source Diversity - Festival del Software Libre from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

End to End ML with Kubeflow: Scaling with Big & Tiny Data (+ deep learning of course) @ signal

2018-10-13T16:55:00.001-07:00

Thanks for joining me on 2018-10-17 at signal 2018 San Francisco, CA, USA for End to End ML with Kubeflow: Scaling with Big & Tiny Data (+ deep learning of course).

The slides are at http://bit.ly/2pWFUKj.The video of the talk is up at http://bit.ly/2GBMp0B.And if you want there is a related codelab you can try out.

Intro - End to end ML with Kubeflow @ SignalConf 2018 from Holden Karau

Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback