NSHipster

Replay

2025-12-31T00:00:00-08:00

The year is 2025. You’re writing tests for networking code in your Swift app.

You could hit the live API, but then your tests are slow, flaky, and fail whenever that third-party service has a bad day. You could stub URLSession, but then you’re maintaining two implementations of your networking layer. You could maintain JSON fixtures by hand, but there’s no way to capture real responses automatically, so your fixtures go stale without anyone noticing.

There’s a better approach: Record real HTTP traffic once, then replay it instantly for every subsequent test run. This pattern has been battle-tested for over fifteen years in other languages.

Replay brings it to Swift.

In February 2010, Myron Marston created VCR for Ruby. Just like a videocassette recorder could capture television for later playback, VCR captured HTTP interactions so tests could replay them without hitting the network. Record once, play back forever.

The idea spread. Python got VCR.py and pytest-recording. Java got Betamax (continuing the home video theme). Go got go-vcr.

I’ve used VCR and pytest-recording for years and always missed having something comparable in Swift. DVR from Venmo came closest, using the same URLProtocol injection point that Replay uses. But it was built for a different era of Swift — before we had the tools to make something that felt really nice.

For Auld Lang Syne

Being in conversation with that prior art means asking what’s different now. Two things stand out:

First, HAR became a de facto standard. When Myron built VCR, there was no widely-adopted format for HTTP archives, so he invented one using YAML. That’s where the “cassette” terminology comes from. But around the same time, Jan Odvarko on the Firefox developer tools team was creating HAR (HTTP Archive). Today, every major browser exports HAR. As do Charles Proxy, Proxyman, mitmproxy, and Postman.

Replay uses HAR instead of inventing a new format. This means you can capture traffic from Safari’s Network tab and drop it directly into your test fixtures. You can inspect fixtures with any text editor.

Second, Swift finally has the extension points we need. Swift Testing traits — especially the TestScoping protocol from Swift 6.1 — enable the kind of declarative, per-test configuration that pytest fixtures provide. Package plugins let us build integrated tooling that feels native.

These capabilities simply didn’t exist before — not in any way that felt intuitive or convenient.

How Replay Works

Add .replay to a test and Replay intercepts HTTP requests, serving responses from a HAR file instead of hitting the network:

import Testing
        import Replay
        struct User: Codable {
        let id: Int
        let name: String
        let email: String
        }
        @Test(.replay("fetchUser"))
        func fetchUser() async throws {
        let (data, _) = try await URLSession.shared.data(
        from: URL(string: "https://api.example.com/users/42")!
        )
        let user = try JSONDecoder().decode(User.self, from: data)
        #expect(user.id == 42)
        }

The .replay("fetchUser") trait loads responses from Replays/fetchUser.har. Your production code uses URLSession normally. No protocols to define. No mocks to inject. Replay’s interception uses built-in affordances in the Foundation URL Loading System, so it works with URLSession.shared, custom sessions, and libraries like Alamofire.

The Recording Workflow

The first time you run a test with .replay, it fails intentionally:

❌  Test fetchUser() recorded an issue at ExampleTests.swift
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
        ⚠️  No Matching Entry in Archive
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
        Request: GET https://api.example.com/users/42
        Archive: /path/to/.../Replays/fetchUser.har
        This request was not found in the replay archive.
        Options:
        1. Run against the live network:
        REPLAY_PLAYBACK_MODE=live swift test --filter <test-name>
        2. Record the archive:
        REPLAY_RECORD_MODE=once swift test --filter <test-name>

This is deliberate. Accidental recording could capture credentials, PII, or session tokens. By requiring you to opt into recording explicitly, Replay ensures you’re always aware when network traffic is being captured.

So let’s do that now:

REPLAY_RECORD_MODE=once swift test --filter fetchUser

This hits the real API, captures the response, and saves it to Replays/fetchUser.har. From then on, the test runs instantly against the recorded fixture. There’s something deeply satisfying about watching a test suite that used to take minutes finish in seconds.

What’s in a HAR File

A HAR file is just JSON:

{
        "log": {
        "version": "1.2",
        "creator": { "name": "Replay", "version": "1.0" },
        "entries": [
        {
        "request": {
        "method": "GET",
        "url": "https://api.example.com/users/42",
        "headers": [{ "name": "Accept", "value": "application/json" }]
        },
        "response": {
        "status": 200,
        "content": {
        "mimeType": "application/json",
        "text": "{\"id\":42,\"name\":\"Alice\"}"
        }
        }
        }
        ]
        }
        }

Human-readable. Editable. Compatible with a vast ecosystem of tools. 🧑‍🍳💋

Handling Sensitive Data

HAR files have a way of accumulating secrets. Session cookies, authorization headers, API keys — all captured faithfully alongside the responses you actually care about.

Replay addresses this with filters that strip sensitive data during recording:

@Test(
        .replay(
        "fetchUser",
        filters: [
        .headers(removing: ["Authorization", "Cookie"]),
        .queryParameters(removing: ["token", "api_key"])
        ]
        )
        )
        func fetchUser() async throws { /* ... */ }

The general principle: configure filters before recording, not after.

Flexible Matching

By default, Replay matches requests by HTTP method and full URL. For APIs with volatile query parameters (pagination cursors, timestamps, cache-busters), you can configure looser matching:

@Test(.replay("fetchUser", matching: [.method, .path]))
        func fetchUser() async throws { /* ... */ }

Available matchers include .method, .url, .host, .path, .query, .headers([...]), .body, and .custom(...) for arbitrary logic.

Inline Stubs

Replay also supports inline stubs:

@Test(
        .replay(
        stubs: [
        .get("https://api.example.com/health", 200, ["Content-Type": "application/json"], {
        #"{"status": "ok"}"#
        })
        ]
        )
        )
        func testHealthCheck() async throws {
        let (data, _) = try await URLSession.shared.data(
        from: URL(string: "https://api.example.com/health")!
        )
        #expect(String(data: data, encoding: .utf8) == #"{"status": "ok"}"#)
        }

This is useful for testing error handling, edge cases, or scenarios where the response content matters less than the status code.

Fifteen years of refinement across Ruby, Python, JavaScript, and other ecosystems have established clear best practices for HTTP recording: capture real traffic, strip sensitive data, fail fast when fixtures are missing, provide good error messages when things go wrong. Replay brings all of that to Swift.

If you’ve been struggling with flaky API tests, or putting off testing your network code because the overhead felt too high, resolve to give Replay a try.

Replay is open source on GitHub. Let me know what you think!

Manim

2025-10-01T00:00:00-07:00

A few years ago, Twitter user @38mo1, a chemistry teacher from Japan, posted a Halloween × Boolean logic meme that I liked enough to save to ~/Downloads. And ever since, I’ve had a yearly reminder every October 1st to open the image and get a seasonal chuckle 🎃

More recently, I wanted to teach my kids some advanced snake maths (long story 🐍) and remembered that the software 3blue1brown uses for their videos was open-source.

Which is all to say that this year for Halloween, NSHipster is dressing up as a beloved YouTuber. How’d I do?

Neat, right? Everything used to create that video available on GitHub. Read on to learn more about how to get started on your next great explainer.

Back in 2015, Grant Sanderson found himself at a hackathon with a desire to practice his Python skills. What emerged was “very scrappy code for visualizing functions as transformations” — a playful experiment that would eventually become the foundation of both the 3Blue1Brown YouTube channel (now 6.5+ million subscribers strong) and Manim itself.

0 to 1

Manim has a reputation for being difficult to install. Some of this is a natural consequence for a tool whose audience extends to self-described non-programmers. Some of this is intrinsic to working with motion graphics. And being written in Python doesn’t do it any favors (though uv pretty much solves all the complaints anyone could have).

Docker was more or less invented to solve the problem of packaging Python applications with system dependencies, so it’s entirely appropriate to wash your hands of everything and run Manim with Docker:

$ docker run --rm -it -v "/full/path/to/your/directory:/manim" manimcommunity/manim manim -qm scene.py MySceneName

…but that feels like giving up.

And besides, you’ll appreciate having a proper Python development environment once you’re up-and-running. Stick with me, and I promise everything will be A-OK (pinky swear 🤙).

An opinionated setup guide for Manim on macOS in 2025

First, let’s do some mise en place:

# Install mise
        $ brew install mise
        # Install Python and uv
        $ mise use -g python@3.13 uv@latest

Homebrew is a package manager for macOS. It’s the best way to install the system dependencies we need to run Manim.
Mise is a polyglot tool manager. While you could use Homebrew to install Python, you often need to keep a few versions around to work across different projects. So that’s what we’ll recommend here.

Mise is slick. I like it way more than anything else I’ve tried, and I think you’ll like it, too.
uv is a Python package manager. It’s wicked fast and just works™.

It’s hard to say too many nice things about uv. It really is that good.

Now, let’s create our project and make ourselves at home:

# Create a new project
        $ uv init my-explainer
        $ cd my-explainer
        # Now open with your preferred $EDITOR

Next, go ahead and install Manim’s system dependencies:

# Install Manim dependencies
        $ brew install pkg-config cairo # for graphics
        $ brew install --cask mactex-no-gui # for ƒοrmυℓαѕ
        $ brew install sox # for voiceovers

Cairo is a 2D graphics library. You need pkg-config to get the Pycairo bindings working properly.
MacTex is a LaTeX distribution for macOS. You’ll need this to make a proper maths explainer with impressive-looking formulas. We recommend downloading mactex-no-gui, which foregoes the GUI apps that ship in the full version.
SoX is a utility for converting audio files to different formats. Manim has some nice-_ish_ affordances for putting voice-overs into your scenes, and you’ll need SoX to take advantage of them.

Finally, let’s run a health check to see if we’re still on the happy path:

# Is everything working? (🫣)
        $ uv run manim checkhealth

Huzzah!

Setting the Scene

The Manim API is rich and well-documented, with an extensive collection of examples. Rather than attempt to explain everything here, let’s look at one concrete example to get a sense of how it all works.

Here’s a standalone “Trick XOR Treat” scene:

from manim import *
        class TrickXORTreat(Scene):
        def construct(self):
        # Panel background
                panel_bg = RoundedRectangle(width=7, height=5, corner_radius=0.2).set_fill(
        GREY, opacity=1
        )
        # Title
                title = Text("Trick XOR Treat").scale(0.8)
        title.move_to(panel_bg.get_top() + DOWN * 0.4)
        # Two circles for the Venn diagram
                c1 = Circle(radius=1.4, color=BLACK).move_to(LEFT * 0.8)
        c2 = Circle(radius=1.4, color=BLACK).move_to(RIGHT * 0.8)
        # Create individual fill shapes for XOR (exclusive OR)
                left_only = Difference(c1, c2).set_fill(ORANGE, opacity=0)
        right_only = Difference(c2, c1).set_fill(ORANGE, opacity=0)
        center = Intersection(c1, c2).set_fill(GREY, opacity=0)  # XOR excludes center
        
        # Create faces on left and right sections
                face = VGroup()
        # Left face
                left_eye = Circle(radius=0.2, color=BLACK, fill_opacity=1)
        left_eye.move_to(left_only.get_center() + UL * 0.3)
        left_mouth = ParametricFunction(
        lambda t: [t, 0.15 * np.sin(t * PI * 5), 0], t_range=[-0.3, 0.3]
        ).set_stroke(color=BLACK, width=8)
        left_mouth.move_to(left_only.get_center() + DOWN * 0.2)
        # Right face
                right_eye = Circle(radius=0.2, color=BLACK, fill_opacity=1)
        right_eye.move_to(right_only.get_center() + UR * 0.3)
        right_mouth = ParametricFunction(
        lambda t: [t, 0.15 * np.sin(t * PI * 5), 0], t_range=[-0.3, 0.3]
        ).set_stroke(color=BLACK, width=8)
        right_mouth.move_to(right_only.get_center() + DOWN * 0.2)
        face.add(left_eye, left_mouth, right_eye, right_mouth)
        # Combine everything
                panel = VGroup(
        panel_bg,
        left_only,
        right_only,
        center,
        VGroup(c1, c2),  # Circle outlines
                    title,
        face,
        )
        panel.move_to(ORIGIN)
        # Animate the panel appearing with smooth scaling
                self.play(FadeIn(panel_bg, scale=1.1), Write(title, run_time=1.2), run_time=1.0)
        # Draw circles sequentially
                self.play(Create(c1, run_time=1.0))
        self.wait(0.5)
        self.play(Create(c2, run_time=1.0))
        self.wait(0.5)
        # Dramatic filling animations with smooth rate functions
                self.play(
        left_only.animate.set_fill(ORANGE, opacity=1),
        run_time=1.2,
        rate_func=smooth,
        )
        self.wait(0.5)
        self.play(
        right_only.animate.set_fill(ORANGE, opacity=1),
        run_time=1.2,
        rate_func=smooth,
        )
        self.wait(0.5)
        # Animate faces appearing on top of fills with bounce
                self.play(FadeIn(face, shift=UP * 0.3, scale=0.8), run_time=0.8)
        # Final pause
                self.wait(1.5)

The Manim API is refreshingly procedural— a nice break from the declarative nature of SwiftUI, if you ask me. You create objects, position them, and then explicitly tell them what to do. The .animate syntax provides some syntactic sugar for transformations, but at its core, this is imperative programming at its finest.

Enter the Development Loop

Our example project includes a Mise task for quickly previewing our scene. It runs Manim at a low resolution (480p) and frame rate (15 fps), and then opens the generated file in QuickTime Player.

$ mise run preview

The project also has a render task for when you’re ready to share it around:

$ mise run render

If everything works as intended, you should see something like this:

(Voiceover)

No 3blue1brown explainer video would be complete without the dulcet baritone of its creator. Alas, Mr. Sanderson had previous obligations that took him away from our costume party, so we’ll have to improvise. Let’s crank up the spook factor and have an AI… ghost? I guess? narrate our flick.

ElevenLabs has some scary-good text-to-speech models. While manim-voiceover provides integration, it’s currently a bit temperamental with Python 3.13 and doesn’t support the latest v3 models for hyper-realistic voiceovers.

The official manim-voiceover package doesn’t play nicely with Python 3.13, and doesn’t support ElevenLabs’ SOTA v3 models. I found a fork that got me what I needed, but YMMV.

Here’s the error message I wrestled with for the better part of an hour, for posterity:

uv add manim-voiceover[elevenlabs,transcribe] openai-whisper
        × Failed to build `openai-whisper==20230314`
        ├─▶ The build backend returned an error
        ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

To get started, you import from manim_voiceover (_fixed) and update the scene to inherit from VoiceoverScene.

from manim_voiceover_fixed import VoiceoverScene
        from manim_voiceover_fixed.services.elevenlabs import ElevenLabsService
        class TrickXORTreat(VoiceoverScene):
        def construct(self):
        self.set_speech_service(
        ElevenLabsService(
        voice_name="Liam",
        model="eleven_v3",
        transcription_model=None,  # <-- workaround for https://github.com/ManimCommunity/manim-voiceover/issues/114
                    )
        )
        with self.voiceover(
        text="""[annoyingly high-pitch nasal pedantic] Well, actually,
        it's an exclusive or, also known as XOR..."""
        ):
        # This wait() automatically waits until the voiceover finishes!
                    self.wait()

The wait() inside the context manager waits until the voiceover finishes — no manual timing required. For recorded voiceovers especially, this dramatically cuts down on how much time you spend in post getting the timing right. In fact, the finished video at the top was entirely rendered by Manim— without any touch-ups in Final Cut or Premiere!

The first time you run this code, Manim will helpfully (albeit unexpectedly) prompt you to enter an API token, which you can get in your account dashboard. That’ll get saved to an .env file and that should be the end of it.

The best explainer videos don’t just teach, they spark curiosity, they delight, they make complex ideas feel approachable. That’s what drew me to 3blue1brown in the first place, and why I’ve kept that silly meme in my Downloads folder all these years.

So if you’ve been sitting on a meme in your ~/Downloads folder, or just want an excuse to dust off those Python skills, maybe this Halloween you dress up as a math YouTuber, too. The costume’s open source, and honestly? It’s a pretty good look.

@isolated(any)

2025-08-04T00:00:00-07:00

Ahh, @isolated(any). It’s an attribute of contradictions. You might see it a lot, but it’s ok to ignore it. You don’t need to use it, but I think it should be used more. It must always take an argument, but that argument cannot vary.

Confusing? Definitely. But we’ll get to it all.

To understand why @isolated(any) was introduced, we need to take a look at async functions.

let respondToEmergency: () async -> Void

This is about as simple a function type as we can get. But, things start to get a little more interesting when we look at how a function like this is used. A variable with this type must always be invoked with await.

await respondToEmergency()

This, of course, makes sense. All async functions must be called with await. But! Consider this:

let sendAmbulance: @MainActor () -> Void = {
        print("🚑 WEE-OOO WEE-OOO!")
        }
        let respondToEmergency: () async -> Void = sendAmbulance
        await respondToEmergency()

The explicit types are there to help make what’s going on clear. We first define a synchronous function that must run on the MainActor. And then we assign that to a plain old, non-MainActor async function. We’ve changed so much that you might find it surprising this even compiles.

Remember what await actually does. It allows the current task to suspend. That doesn’t just let the task wait for future work to complete. It also is an opportunity to change isolation. This makes async functions very flexible!

Just like a dispatcher doesn’t sit there doing nothing while waiting for the ambulance to arrive, a suspended task doesn’t block its thread. When the dispatcher puts you on hold to coordinate with the ambulance team, that’s the isolation switch - they’re transferring your request to a different department that specializes in that type of work.

But change to where, exactly?

Ok, so we know that async functions, because they must always be awaited, gain a lot of flexibility. We are close, but have to go just a little further to find the motivation for this attribute.

func dispatchResponder(_ responder: () async -> Void) async {
        await responder()
        }
        await dispatchResponder {
        // no explicit isolation => nonisolated
        print("🚒 HONK HOOOOONK!")
        await airSupport()
        print("🚁 SOI SOI SOI SOI SOI!")
        }
        await dispatchResponder { @MainActor in
        print("🚑 WEE-OOO WEE-OOO!")
        }

We now have a function that accepts other functions as arguments. It’s possible to pass in lots of different kinds of functions to dispatchResponder. They could be async functions themselves, or even be synchronous. And they can be isolated to any actor. All thanks to the power of await.

Except there’s a little problem now. Have a look at dispatchResponder on its own:

func dispatchResponder(_ responder: () async -> Void) async {
        await responder()
        }

The type of responder fully describes everything about this function, except for one thing. We have no way to know its isolation. That information is only available at callsites. The isolation is still present, so the right thing happens at runtime. It’s just not possible to inspect it statically or even programmatically. If you’ve encountered type erasure before, this should seem familiar. The flexibility of async has come with a price - a loss of information.

This is where @isolated(any) comes in.

Using `@isolated(any)`

We can change the definition of dispatchResponder to fix this.

func dispatchResponder(_ responder: @isolated(any) () async -> Void) async {
        print("responder isolation:", responder.isolation)
        await responder()
        }

When you apply @isolated(any) to a function type, it does two things. Most importantly, it gives you access to a special isolation property. You can use this property to inspect the isolation of the function. The isolation could be an actor. Or it could be non-isolated. This is expressible in the type system with (any Actor)?.

Functions with properties felt really strange to me at first. But, after thinking for a minute, it became quite natural. Why not? It’s just a type like any other. In fact, we can simulate how this all works with another feature: callAsFunction.

struct IsolatedAnyFunction<T> {
        let isolation: (any Actor)?
        let body: () async -> T
        func callAsFunction() async -> T {
        await body()
        }
        }
        let value = IsolatedAnyFunction(isolation: MainActor.shared, body: {
        // isolated work goes here
        })
        await value()

This analogy is certainly not perfect, but it’s close enough that it might help.

There is one other subtle change that @isolated(any) makes to a function that you should be aware of. Its whole purpose is to capture the isolation of a function. Since that could be anything, callsites need an opportunity to switch. And that means an @isolated(any) function must be called with an await — even if it isn’t itself explicitly async.

func dispatchResponder(_ responder: @isolated(any) () -> Void) async {
        await responder() // note the function is synchronous
        }

This makes synchronous functions marked with @isolated(any) a little strange. They still must be called with await, yet they aren’t allowed to suspend internally?

As it turns out, there are some valid (if rare) situations where such an arrangement can make sense. But adding this kind of constraint to your API should at least merit some extra documentation.

How @isolated(any) Affects Callers

All of the task creation APIs — Task initializers and TaskGroup — make use of @isolated(any). These are used a lot and are usually encountered very early on when learning about concurrency. So, it’s completely natural to run into this attribute and think:

“Ugh another thing to understand!”

It’s reasonable because the components of a function type dictate how it can be used. They are all essential qualities for API consumers. They are the interface.

Parameters
Return value
Does it throw?
Is it async?

This is not an exhaustive list, but what’s important is all of these are things callers must care about. Except for @isolated(any), which is the opposite. It doesn’t affect callers at all.

This, I think, is the root of a lot of confusion around @isolated(any). Unlike other qualities of a function, this attribute is used to capture information for the API producer.

I’m so close to saying “you can and should just ignore @isolated(any)“. But I just cannot quite go that far, because there is one situation you should be aware of.

Scheduling

To help understand when you should be thinking about using @isolated(any), I’m going to quote the proposal:

This allows the API to make more intelligent scheduling decisions about the function.

I’ve highlighted “intelligent scheduling”, because this is the key component of @isolated(any). The attribute gives you access to the isolation of a function argument. But what would you use that for?

Did you know that, before Swift 6.0, the ordering of the following code was undefined?

@MainActor
        func threeAlarmFire() {
        Task { print("🚒 Truck A reporting!") }
        Task { print("🚒 Truck B checking in!") }
        Task { print("🚒 Truck C on the case!") }
        }

Ordering turns out to be a very tricky topic when working with unstructured tasks. And while it will always require care, Swift 6.0 did improve the situation. We now have some stronger guarantees about scheduling work on the MainActor, and @isolated(any) was needed to make that possible.

Take a look at this:

@MainActor
        func sendAmbulance() {
        print("🚑 WEE-OOO WEE-OOO!")
        }
        nonisolated func dispatchResponders() {
        // synchronously enqueued
        Task { @MainActor in
        sendAmbulance()
        }
        // synchronously enqueued
        Task(operation: sendAmbulance)
        // not synchronously enqueued!
        Task {
        await sendAmbulance()
        }
        }

These are three ways to achieve the same goal. But, there is a subtle difference in how the last form is scheduled. Task takes an @isolated(any) function so it can look at its isolation and synchronously submit it to an actor. This is how ordering can be preserved! But, it cannot do that in the last case. That closure passed into Task isn’t actually itself MainActor — it has inherited nonisolated from the enclosing function.

I think it might help to translate this into GCD.

func dispatchResponders() {
        // synchronously enqueued
        DispatchQueue.main.async {
        sendAmbulance()
        }
        // synchronously enqueued
        DispatchQueue.main.async(execute: sendAmbulance)
        // not synchronously enqueued!
        DispatchQueue.global().async {
        DispatchQueue.main.async {
        sendAmbulance()
        }
        }
        }

Look really closely at that last one! What we are doing there is introducing a new async closure that then calls our MainActor function. There are two steps. This doesn’t always matter, but it certainly could. And if you need to precisely schedule asynchronous work, @isolated(any) can help.

isolated(all)

All this talk about @isolated(any) got me thinking…

It’s kinda strange that only some functions get to have this isolation property. It would certainly feel more consistent to me if all functions had it. In fact, I think we can go further. I can imagine a future where an explicit @isolated(any) isn’t even necessary for async functions. As far as I can tell, there is no downside.

And a little less syntactic noise would be nice. Perhaps one day!

isolated(some)

We do have to talk about that any. It’s surprising that this attribute requires an argument, yet permits only one possible value. The reason here comes down to future considerations.

The concrete actor type that this isolation property returns is always (any Actor)?. This is the most generic type for isolation and matches the #isolation macro. Today, there is no way to constrain a function to only specific actor types, such as @isolated(MyActor). The any keyword here was chosen to mirror how protocols handle this. But accepting an argument leaves the door open to more sophisticated features in the future.

And that really fits the spirit of @isolated(any). Doing a little work now in exchange for flexibility down the road.

Because you’ll see it in many foundational concurrency APIs, it’s very natural to feel like you must understand @isolated(any). I’m 100% behind technical curiosity! In this case, however, it is not required. For the most part, you can just ignore this attribute. You will rarely, if ever, need to use it yourself.

But if you ever find yourself capturing isolated functions and passing them along to other APIs that use @isolated(any), you should consider adopting it. It could prove useful. It’s even a source-compatible change to add or remove this attribute from an async function.

So there you have it.

As with many parts of the concurrency system, there’s a surprising depth to @isolated(any). Thankfully, from a practical perspective, we can enjoy the ordering guarantees of task creation that it enables without needing to master it. And one less thing on this journey is most welcome.

Isolated maybe, but never alone.

Uncertain⟨T⟩

2025-07-25T00:00:00-07:00

You know what’s wrong with people? They’re too sure of themselves.

Better to be wrong and own it than be right with caveats. Hard to build a personal brand out of nuance these days. People are attracted to confidence — however misplaced.

But can you blame them? (People, that is) Working in software, the most annoying part of reaching Senior level is having to say “it depends” all the time. Much more fun getting to say “let’s ship it and iterate” as Staff or “that won’t scale” as a Principal.

Yet, for all of our intellectual humility, why do we ~~write~~ vibe code like this?

if currentLocation.distance(to: target) < 100 {
        print("You've arrived!") // But have you, really? 🤨
        }

GPS coordinates aren’t exact. They’re noisy. They’re approximate. They’re probabilistic. That horizontalAccuracy property tucked away in your CLLocation object is trying to tell you something important: you’re probably within that radius. Probably.

A Bool, meanwhile, can be only true or false. That if statement needs to make a choice one way or another, but code like this doesn’t capture the uncertainty of the situation. If truth is light, then current programming models collapse the wavefunction too early.

Picking the Right Abstraction

In 2014, researchers at the University of Washington and Microsoft Research proposed a radical idea: What if uncertainty were encoded directly into the type system? Their paper, Uncertain<T>: A First-Order Type for Uncertain Data introduced a probabilistic programming approach that’s both mathematically rigorous and surprisingly practical.

As you’d expect for something from Microsoft in the 2010s, the paper is implemented in C#. But the concepts translate beautifully to Swift.

You can find my port on GitHub:

import Uncertain
        import CoreLocation
        let uncertainLocation = Uncertain<CLLocation>.from(currentLocation)
        let nearbyEvidence = uncertainLocation.distance(to: target) < 100
        if nearbyEvidence.probability(exceeds: 0.95) {
        print("You've arrived!") // With 2σ confidence 🤓
        }

When you compare two Uncertain values, you don’t get a definitive true or false. You get an Uncertain<Bool> that represents the probability of the comparison being true.

The same is true for other operators, too:

// How fast did we run around the track?
        let distance: Double = 400 // meters
        let time: Uncertain<Double> = .normal(mean: 60, standardDeviation: 5.0) // seconds
        let runningSpeed = distance / time // Uncertain<Double>
        // How much air resistance?
        let airDensity: Uncertain<Double> = .normal(mean: 1.225, standardDeviation: 0.1) // kg/m³
        let dragCoefficient: Uncertain<Double> = .kumaraswamy(alpha: 9, beta: 3) // slightly right-skewed distribution
        let frontalArea: Uncertain<Double> = .normal(mean: 0.45, standardDeviation: 0.05) // m²
        let airResistance = 0.5 * airDensity * frontalArea * dragCoefficient * (runningSpeed * runningSpeed)

This code builds a computation graph, sampling only when you ask for concrete results. The library uses Sequential Probability Ratio Testing (SPRT) to efficiently determine how many samples are needed — maybe a few dozen times for simple comparisons, scaling up automatically for complex calculations.

// Sampling happens only when we need to evaluate
        if ~(runningSpeed > 6.0) {
        print("Great pace for a 400m sprint!")
        }
        // SPRT might only need a dozen samples for this simple comparison
        let sustainableFor5K = (runningSpeed < 6.0) && (airResistance < 50.0)
        print("Can sustain for 5K: \(sustainableFor5K.probability(exceeds: 0.9))")
        // Might use 100+ samples for this compound condition

Using an abstraction like Uncertain<T> forces you to deal with uncertainty as a first-class concept rather than pretending it doesn’t exist. And in doing so, you end up with much smarter code.

To quote Alan Kay:

Point of view is worth 80 IQ points

Before we dive deeper into probability distributions, let’s take a detour to Monaco and talk about Monte Carlo sampling.

The Monte Carlo Method

Behold, a classic slot machine (or “fruit machine” for our UK readers 🇬🇧):

enum SlotMachine {
        static func spin() -> Int {
        let symbols = [
        "◻️", "◻️", "◻️",  // blanks
        "🍒", "🍋", "🍊", "🍇", "💎"
        ]
        // Spin three reels independently
        let reel1 = symbols.randomElement()!
        let reel2 = symbols.randomElement()!
        let reel3 = symbols.randomElement()!
        switch (reel1, reel2, reel3) {
        case ("💎", "💎", "💎"): return 100  // Jackpot!
        case ("🍒", "🍒", "🍒"): return 10
        case ("🍇", "🍇", "🍇"): return 5
        case ("🍊", "🍊", "🍊"): return 3
        case ("🍋", "🍋", "🍋"): return 2
        case ("🍒", _, _), // Any cherry
        (_, "🍒", _),
        (_, _, "🍒"):
        return 1
        default:
        return 0  // Better luck next time
        }
        }
        }

Should we play it?

Now, we could work out these probabilities analytically — counting combinations, calculating conditional probabilities, maybe even busting out some combinatorics.

Or we could just let the computer pull the lever a bunch and see what happens.

let expectedPayout = Uncertain<Int> {
        SlotMachine.spin()
        }.expectedValue(sampleCount: 10_000)
        print("Expected value per spin: $\(expectedPayout)")
        // Expected value per spin: ≈ $0.56

At least we know one thing for certain: The house always wins.

Beyond Simple Distributions

While one-armed bandits demonstrate pure randomness, real-world applications often deal with more predictable uncertainty.

Uncertain<T> provides a rich set of probability distributions:

// Modeling sensor noise
        let rawGyroData = 0.85  // rad/s
        let gyroReading = Uncertain.normal(
        mean: rawGyroData,
        standardDeviation: 0.05  // Typical gyroscope noise in rad/s
        )
        // User behavior modeling
        let userWillTapButton = Uncertain.bernoulli(probability: 0.3)
        // Network latency with long tail
        let apiResponseTime = Uncertain.exponential(rate: 0.1)
        // Coffee shop visit times (bimodal: morning rush + afternoon break)
        let morningRush = Uncertain.normal(mean: 8.5, standardDeviation: 0.5)  // 8:30 AM
        let afternoonBreak = Uncertain.normal(mean: 15.0, standardDeviation: 0.8)  // 3:00 PM
        let visitTime = Uncertain.mixture(
        of: [morningRush, afternoonBreak],
        weights: [0.6, 0.4]  // Slightly prefer morning coffee
        )

Uncertain<T> also provides comprehensive statistical operations:

// Basic statistics
        let temperature = Uncertain.normal(mean: 23.0, standardDeviation: 1.0)
        let avgTemp = temperature.expectedValue() // about 23°C
        let tempSpread = temperature.standardDeviation() // about 1°C
        // Confidence intervals
        let (lower, upper) = temperature.confidenceInterval(0.95)
        print("95% of temperatures between \(lower)°C and \(upper)°C")
        // Distribution shape analysis
        let networkDelay = Uncertain.exponential(rate: 0.1)
        let skew = networkDelay.skewness() // right skew
        let kurt = networkDelay.kurtosis() // heavy tail
        // Working with discrete distributions
        let diceRoll = Uncertain.categorical([1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1])!
        diceRoll.entropy()  // Randomness measure (~2.57)
        (diceRoll + diceRoll).mode() // Most frequent outcome (7, perhaps?)
        // Cumulative probability
        if temperature.cdf(at: 25.0) < 0.2 {  // P(temp ≤ 25°C) < 20%
        print("Unlikely to be 25°C or cooler")
        }

The statistics are computed through sampling. The number of samples is configurable, letting you trade computation time for accuracy.

Putting Theory to Practice

Users don’t notice when things work correctly, but they definitely notice impossible behavior. When your running app claims they just sprinted at 45 mph, or your IRL meetup app shows someone 500 feet away when GPS accuracy is ±1000 meters, that’s a bad look 🤡

So where do we go from here? Let’s channel our Senior+ memes from before for guidance.

That Staff engineer saying “let’s ship it and iterate” is right about the incremental approach. You can migrate uncertain calculations piecemeal rather than rewriting everything at once:

extension CLLocation {
        var uncertain: Uncertain<CLLocation> {
        Uncertain<CLLocation>.from(self)
        }
        }
        // Gradually migrate critical paths
        let isNearby = (
        currentLocation.uncertain.distance(to: destination) < threshold
        ).probability(exceeds: 0.68)

And we should consider the Principal engineer’s warning of “that won’t scale”. Sampling has a cost, and you should understand the computational overhead for probabilistic accuracy:

// Fast approximation for UI updates
        let quickEstimate = speed.probability(
        exceeds: walkingSpeed,
        maxSamples: 100
        )
        // High precision for critical decisions
        let preciseResult = speed.probability(
        exceeds: walkingSpeed,
        confidenceLevel: 0.99,
        maxSamples: 10_000
        )

Start small. Pick one feature where GPS glitches cause user complaints. Replace your distance calculations with uncertain versions. Measure the impact.

Remember: the goal isn’t to eliminate uncertainty — it’s to acknowledge that it exists and handle it gracefully. Because in the real world, nothing is certain except uncertainty itself.

And perhaps, with better tools, we can finally stop pretending otherwise.

Model Context Protocol (MCP)

2025-03-07T00:00:00-08:00

Language Server Protocol (LSP) revolutionized how programming languages integrate with developer tools. Model Context Protocol (MCP) aims to do the same for a new generation of AI tools.

But before we lay our scene of two protocols (both alike in dignity), let’s take a moment to motivate the problem a bit more.

Tool Use & Agents

Today’s frontier models are extraordinary. But they’re limited in two key ways:

They don’t know anything outside their training data
They can’t do anything outside of predicting the next token

To their credit, LLMs (kinda, sorta) understand these limitations, and have been trained to ask for what they need. We (humans) have settled on a practice called tool use that lets AI do just that.

Here’s an example:

User: Write a limerick about today's weather in Portland, Oregon.
Client, to Assistant →: Also, there's this "Get Weather" tool available if you need it. It takes a set of coordinates and returns the current temperature and conditions.
Assistant: <thinking> To write an accurate poem, I should get the weather. From my training, I know Portland's coordinates to be 45.5155° N, 122.6789° W </thinking>; Tool call: {tool: "Get Weather", latitude: 45.5155, longitude: -122.6789}
← Client, to User: Is it alright to make that tool call?
User: 👍
Client, to Assistant →: Tool result: {temperature: 12, conditions: 'cloudy', humidity: 85%}
Assistant: Here's your limerick about Portland's weather:; There once was a city called Portland
Where clouds made the skies quite importland
At twelve degrees C
It's as gray as can be
With a dampness that's truly absorbland!

Pretty clever, right?
But then again, so what?

On its face, this is a very slow, expensive way to do something better-served by a quick Google search — or sticking your head out a window, for that matter.

But here’s the thing:
Tool use gives rise to agency (✨𝓮𝓶𝓮𝓻𝓰𝓮𝓷𝓬𝓮✨)

As the saying goes, Quantity has a quality all its own. Give a language model a dozen… a hundred… a thousand tools — hell, give it a tool for making more tools. How confident are you that you could find a problem that couldn’t be solved by such a system?

We’re only just starting to see what’s possible.

But back to more practical matters:
Let’s talk about where we are today with Model Context Protocol.

The New M × N Problem

We’ve written previously about Language Server Protocol, and the M × N problem. LSP’s challenge was connecting M editors with N programming languages MCP faces a similar challenge, of connecting M clients with N resources. Without MCP, each AI application must implement custom integrations for every data source it wants to access.

This creates the same kind of fragmented ecosystem that plagued development tools before LSP:

Some AI applications offer deep integration with specific data sources but limited support for others
Different applications implement similar integrations in incompatible ways
Improvements to one integration rarely benefit the broader ecosystem

Like LSP, MCP transforms this M × N problem into an M + N problem through standardization. Rather than each AI application implementing custom integrations, it only needs to support the MCP standard. In doing so, it gains access to all MCP-compatible data sources and tools 🌈

How Model Context Protocol Works

MCP follows a client-server architecture similar to LSP:

The client is typically an AI application or development environment
For example, Claude Desktop, Zed, and Cursor.
The server is a program that provides access to data and/or tools

Requests and responses are encoded according to the JSON-RPC 2.0 specification. Communication between client and server happens over Stdio (stdin/stdout) or HTTP with Server-Sent Events transports.

Like LSP, MCP has clients and servers negotiate a set of capabilities. When a client connects to a server, it sends an initialize message, with information about what protocol version it supports. The server responds in kind.

From there, the client can ask the server about what features it has. MCP describes three different kinds of features that a server can provide:

Prompts: Templates that shape how language models respond. They’re the difference between getting generic text and precise, useful results. A good prompt is like a well-designed API - it defines the contract between you and the model.
Resources: Reference materials that ground models in reality. By providing structured data alongside your query, you transform a model from a creative writer into an informed assistant with domain-specific knowledge. (Think: databases, file systems, documents)
Tools: Functions that extend what models can do. They allow AI to calculate, retrieve information, or interact with external systems when simple text generation isn’t enough. Tools bridge the gap between language understanding and practical capability.

Our previous example handwaved the existence of a “Get Weather” tool. MCP gives our client a standard way to consult various connected services.

To get a list of available tools on an MCP, the client would send a tools/list request to the server:

{
        "jsonrpc": "2.0", 
        "id": 1, 
        "method": "tools/list", 
        "params": {}
        }

In our example, the server would respond:

{
        "jsonrpc":"2.0",
        "id": 1,
        "result": {
        "tools": [
        {
        "name": "get_weather",
        "description": "Returns current weather conditions for the specified coordinates.",
        "inputSchema": {
        "type": "object", 
        "properties": {
        "latitude": { "type": "number" },
        "longitude": { "type": "number" }
        },
        "required": ["latitude", "longitude"]
        }
        }
        ]
        }
        }

The client can share this list of tools with the language model in a system prompt or a user message. When the model responds wanting to invoke the get_weather tool, the client asks the user to confirm tool use. If the human-in-the-loop says 🆗, the client sends a tools/call request:

{
        "jsonrpc": "2.0", 
        "id": 2,
        "method": "tools/call", 
        "params": { 
        "name": "get_weather",
        "arguments": {
        "latitude": 45.5155,
        "longitude": -122.6789
        }
        }
        }

In response, the server sends:

{
        "jsonrpc":"2.0",
        "id": 2,
        "content": [
        {
        "type": "text",
        "text": "{\"temperature\": 12, \"conditions\": \"cloudy\", \"humidity\": 85}"
        "annotations": {
        "audience": ["assistant"]
        }
        }
        ]
        }

The client then passes that result to the AI assistant, the assistant generates a response with this information, and the client passes that along to the user.

That’s pretty much all there is to it. There are plenty of details to get bogged down with. But that’s what LLMs are for. Now is the time for vibes coding.
MCP is punk rock.

How do I start?

MCP is an emerging standard from Anthropic. So it’s no surprise that Claude Desktop is most capable of showing off what it can do.

Once you have Claude Desktop installed, you can peruse the myriad example servers available.

Or, if you want to skip straight to la crème de la crème, then have a taste of what we’ve been cooking up with MCP lately:

iMCP

Fun fact! The word “paradise” derives from an old word for “walled garden”.

Ironic how Apple has a way of making your digital life a living hell sometimes.

For many of us who exist in Apple’s walled garden, we’re often frustrated by the product design and software quality that gets between us and our data. Spotlight search is stuck in the ‘00s. Apple Intelligence didn’t live up to the hype. Siri seems doomed to suck forever.

That was our motivation for building iMCP.

iMCP is a macOS app for connecting your digital life with AI. It works with Claude Desktop and a growing list of clients that support MCP. It gives MCP access to your calendars, contacts, even messages — no small feat!

Download it today and get a taste of some real Apple intelligence.

mcp-swift-sdk

In the process of building iMCP, we built a Swift SDK for Model Context Protocol servers and clients.

If you’re inspired to build your own MCP app and like working in Swift more than Python or TypeScript, definitely give this a try!

hype

If, however, you have accepted Python into your heart as I have, then I’d recommend checking out another project I’ve been working on: hype.

My goal with hype is to eliminate every barrier between writing Python code and calling it in a way that’s useful. Add the @hype.up decorator to a function to instantly generate an HTTP API, a CLI, a GUI, or an MCP.

# example.py
        import hype
        from pydantic import Field
        @hype.up
        def divide(
        x: int,
        y: int = Field(gt=0),
        ) -> int:
        """
        Divides one number by another.
        :param x: The numerator
        :param y: The denominator
        :return: The quotient
        """
        return x // y

Start up an MCP server with the hype command:

$ hype mcp example.py

emcee

But really, the best code is code you don’t have to write. If you already have a web application with an OpenAPI specification, you can use another tool we built — emcee — to instantly spin up an MCP server to it.

We think emcee is a convenient way to connect to services that don’t have an existing MCP server implementation — especially for services you’re building yourself. Got a web app with an OpenAPI spec? You might be surprised how far you can get without a dashboard or client library.

In case it’s not abundantly clear, we here at NSHipster dot com are pretty bought into the promise of Model Context Protocol. And we’re excited to see where everything goes in the coming months.

If you’re building in this space, I’d love to hear from you ✌️

Ollama

2025-02-14T00:00:00-08:00

“Only Apple can do this” Variously attributed to Tim Cook

Apple introduced Apple Intelligence at WWDC 2024. After waiting almost a year for Apple to, in Craig Federighi’s words, “get it right”, its promise of “AI for the rest of us” feels just as distant as ever.

While we wait for Apple Intelligence to arrive on our devices, something remarkable is already running on our Macs. Think of it as a locavore approach to artificial intelligence: homegrown, sustainable, and available year-round.

This week on NSHipster, we’ll look at how you can use Ollama to run LLMs locally on your Mac — both as an end-user and as a developer.

What is Ollama?

Ollama is the easiest way to run large language models on your Mac. You can think of it as “Docker for LLMs” - a way to pull, run, and manage AI models as easily as containers.

Download Ollama with Homebrew or directly from their website. Then pull and run llama3.2 (2GB).

$ brew install --cask ollama
        $ ollama run llama3.2
        >>> Tell me a joke about Swift programming.
        What's a Apple developer's favorite drink? 
        The Kool-Aid.

Under the hood, Ollama is powered by llama.cpp. But where llama.cpp provides the engine, Ollama gives you a vehicle you’d actually want to drive — handling all the complexity of model management, optimization, and inference.

Similar to how Dockerfiles define container images, Ollama uses Modelfiles to configure model behavior:


        FROM mistral:latest
        PARAMETER temperature 0.7
        TEMPLATE """
        {{- if .First }}
        You are a helpful assistant.
        {{- end }}
        User: {{.Prompt}}
        Assistant: """

Ollama uses the Open Container Initiative (OCI) standard to distribute models. Each model is split into layers and described by a manifest, the same approach used by Docker containers:

{
        "mediaType": "application/vnd.oci.image.manifest.v1+json",
        "config": {
        "mediaType": "application/vnd.ollama.image.config.v1+json",
        "digest": "sha256:..."
        },
        "layers": [
        {
        "mediaType": "application/vnd.ollama.image.layer.v1+json",
        "digest": "sha256:...",
        "size": 4019248935
        }
        ]
        }

Overall, Ollama’s approach is thoughtful and well-engineered. And best of all, it just works.

What’s the big deal about running models locally?

Jevons paradox states that, as something becomes more efficient, we tend to use more of it, not less.

Having AI on your own device changes everything. When computation becomes essentially free, you start to see intelligence differently.

While frontier models like GPT-4 and Claude are undeniably miraculous, there’s something to be said for the small miracle of running open models locally.

Privacy: Your data never leaves your device. Essential for working with sensitive information.
Cost: Run 24/7 without usage meters ticking. No more rationing prompts like ’90s cell phone minutes. Just a fixed, up-front cost for unlimited inference.
Latency: No network round-trips means faster responses. Your /M\d Mac((Book( Pro| Air)?)|Mini|Studio)/ can easily generate dozens of tokens per second. (Try to keep up!)
Control: No black-box RLHF or censorship. The AI works for you, not the other way around.
Reliability: No outages or API quota limits. 100% uptime for your exocortex. Like having Wikipedia on a thumb drive.

Building macOS Apps with Ollama

Ollama also exposes an HTTP API on port 11434 (leetspeak for llama 🦙). This makes it easy to integrate with any programming language or tool.

To that end, we’ve created the Ollama Swift package to help developers integrate Ollama into their apps.

Text Completions

The simplest way to use a language model is to generate text from a prompt:

import Ollama
        let client = Client.default
        let response = try await client.generate(
        model: "llama3.2",
        prompt: "Tell me a joke about Swift programming.",
        options: ["temperature": 0.7]
        )
        print(response.response)
        // How many Apple engineers does it take to document an API? 
        // None - that's what WWDC videos are for.

Chat Completions

For more structured interactions, you can use the chat API to maintain a conversation with multiple messages and different roles:

let initialResponse = try await client.chat(
        model: "llama3.2",
        messages: [
        .system("You are a helpful assistant."),
        .user("What city is Apple located in?")
        ]
        )
        print(initialResponse.message.content)
        // Apple's headquarters, known as the Apple Park campus, is located in Cupertino, California.
        // The company was originally founded in Los Altos, California, and later moved to Cupertino in 1997.
        let followUp = try await client.chat(
        model: "llama3.2",
        messages: [
        .system("You are a helpful assistant."),
        .user("What city is Apple located in?"),
        .assistant(initialResponse.message.content),
        .user("Please summarize in a single word")
        ]
        )
        print(followUp.message.content)
        // Cupertino

Generating text embeddings

Embeddings convert text into high-dimensional vectors that capture semantic meaning. These vectors can be used to find similar content or perform semantic search.

For example, if you wanted to find documents similar to a user’s query:

let documents: [String] = …
        // Convert text into vectors we can compare for similarity
        let embeddings = try await client.embeddings(
        model: "nomic-embed-text", 
        texts: documents
        )
        /// Finds relevant documents
        func findRelevantDocuments(
        for query: String, 
        threshold: Float = 0.7, // cutoff for matching, tunable
        limit: Int = 5
        ) async throws -> [String] {
        // Get embedding for the query
        let [queryEmbedding] = try await client.embeddings(
        model: "llama3.2",
        texts: [query]
        )
        // See: https://en.wikipedia.org/wiki/Cosine_similarity
        func cosineSimilarity(_ a: [Float], _ b: [Float]) -> Float {
        let dotProduct = zip(a, b).map(*).reduce(0, +)
        let magnitude = { sqrt($0.map { $0 * $0 }.reduce(0, +)) }
        return dotProduct / (magnitude(a) * magnitude(b))
        }
        // Find documents above similarity threshold
        let rankedDocuments = zip(embeddings, documents)
        .map { embedding, document in
        (similarity: cosineSimilarity(embedding, queryEmbedding),
        document: document)
        }
        .filter { $0.similarity >= threshold }
        .sorted { $0.similarity > $1.similarity }
        .prefix(limit)
        return rankedDocuments.map(\.document)
        }

For simple use cases, you can also use Apple’s Natural Language framework for text embeddings. They’re fast and don’t require additional dependencies.

import NaturalLanguage
        let embedding = NLEmbedding.wordEmbedding(for: .english)
        let vector = embedding?.vector(for: "swift")

Building a RAG System

Embeddings really shine when combined with text generation in a RAG (Retrieval Augmented Generation) workflow. Instead of asking the model to generate information from its training data, we can ground its responses in our own documents by:

Converting documents into embeddings
Finding relevant documents based on the query
Using those documents as context for generation

Here’s a simple example:

let query = "What were AAPL's earnings in Q3 2024?"
        let relevantDocs = try await findRelevantDocuments(query: query)
        let context = """
        Use the following documents to answer the question. 
        If the answer isn't contained in the documents, say so.
        Documents:
        \(relevantDocs.joined(separator: "\n---\n"))
        Question: \(query)
        """
        let response = try await client.generate(
        model: "llama3.2",
        prompt: context
        )

To summarize: Different models have different capabilities.

Models like llama3.2 and deepseek-r1 generate text.
- Some text models have “base” or “instruct” variants, suitable for fine-tuning or chat completion, respectively.
- Some text models are tuned to support tool use, which let them perform more complex tasks and interact with the outside world.
Models like llama3.2-vision can take images along with text as inputs.
Models like nomic-embed-text create numerical vectors that capture semantic meaning.

With Ollama, you get unlimited access to a wealth of these and many more open-source language models.

So, what can you build with all of this?
Here’s just one example:

Nominate.app

Nominate is a macOS app that uses Ollama to intelligently rename PDF files based on their contents.

Like many of us striving for a paperless lifestyle, you might find yourself scanning documents only to end up with cryptically-named PDFs like Scan2025-02-03_123456.pdf. Nominate solves this by combining AI with traditional NLP techniques to automatically generate descriptive filenames based on document contents.

The app leverages several technologies we’ve discussed:

Ollama’s API for content analysis via the ollama-swift package
Apple’s PDFKit for OCR
The Natural Language framework for text processing
Foundation’s DateFormatter for parsing dates

Looking Ahead

“The future is already here – it’s just not evenly distributed yet.” William Gibson

Think about the timelines:

Apple Intelligence was announced last year.
Swift came out 10 years ago.
SwiftUI 6 years ago.

If you wait for Apple to deliver on its promises, you’re going to miss out on the most important technological shift in a generation.

The future is here today. You don’t have to wait. With Ollama, you can start building the next generation of AI-powered apps right now.

op run

2025-01-01T00:00:00-08:00

.env files. If you’ve worked on a web application, you’ve probably seen one.

While they certainly get the job done, .env files have shortcomings that can create friction in development workflows.

We’ve touched on .env files in past articles about xcconfig files and secret management on iOS. But this week on NSHipster we’re taking a deeper look, exploring how the lesser-known 1Password CLI (op) can solve some problems many of us face managing secrets day-to-day.

The Problem of Configuration

Around 2011, Adam Wiggins published “The Twelve-Factor App”, a methodology for building modern web applications that has since become canon in our industry.

The third of those twelve factors, “Config”, prescribes storing configuration in environment variables:

“Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.”

This core insight — that configuration should be separate from code — led to the widespread adoption of .env files.

A typical .env file looks something like this:

DATABASE_URL=postgres://localhost:5432/myapp_development
        REDIS_URL=redis://localhost:6379/0
        AWS_ACCESS_KEY_ID=AKIA...
        AWS_SECRET_ACCESS_KEY=wJa...
        STRIPE_SECRET_KEY=sk_test_...

You add this file to .gitignore to keep it out of version control, and load these variables into your environment at runtime with a tool or library.

Simple enough. So what’s the problem?

.env Files in Practice

Despite their apparent simplicity, .env files introduce several points of friction in development workflows:

First, there’s the perennial issue of onboarding: How does a new team member get what they need to run the app locally? The common solution is to have a .env.sample / .env.example file in version control, but this creates a maintenance burden to keep it in sync with the actual requirements. And in any case, developers still need to go on a scavenger hunt to fill it out before they can be productive.

Then there’s the multi-environment problem: As soon as you need different configurations for development, staging, and production, you end up with a proliferation of files: .env.development, .env.test, .env.staging… Each requiring its own .sample / .example counterpart.

But perhaps most pernicious is the challenge of managing changes to configuration over time. Because .env files aren’t in version control, changes aren’t, you know… tracked anywhere 🥲

Enter the 1Password CLI (`op`)

You may already use 1Password to manage your passwords and other secrets. But what you might not know is that 1Password also has a CLI that can integrate directly with your development workflow.

op lets you manage 1Password from the command-line. You can do all the CRUD operations you’d expect for items in your vault. But its killer features is the op run subcommand, which can dynamically inject secrets from your 1Password vault into your application’s environment.

Instead of storing sensitive values directly in your .env file, you reference them using special op:// URLs:

# .env
        IRC_USERNAME=op://development/chatroom/username
        IRC_PASSWORD=op://development/chatroom/password

import Foundation
        guard let username = ProcessInfo.processInfo.environment["IRC_USERNAME"],
        let password = ProcessInfo.processInfo.environment["IRC_PASSWORD"] else {
        fatalError("Missing required environment variables")
        }
        // For testing only - never print credentials in production code
        print(password)

Run this on its own, and you’ll fail in proper 12 Factor fashion:

$ swift run
        ❗️ "Missing required environment variables"

But by prepending op run we read in that .env file, resolve each vault item reference, and injects those values into the evironment:

$ op run -- swift run
        hunter2

You’re even prompted to authorize with Touch ID the first time you invoke op run.

Ready to give this a test drive? Here’s how to get started:

A Step-By-Step Guide to Using the 1Password CLI in .env Files

Step 1: Install and Configure the 1Password CLI

On macOS, you can install the CLI with homebrew:

$ brew install 1password-cli

Then, in the 1Password app, open Settings (⌘,), go to the Developer section, and check the box labeled “Integrate with 1Password CLI”.

Running any op subcommand should prompt you to connect to the app.

If you get off the happy path, consult the official docs to get back on track.

Step 2: Create a Shared Vault

Create a new vault in 1Password specifically for development secrets. Give it a clear name like “Development” and a useful description.

Step 3: Migrate Existing Secrets

For each entry in your .env file, create a corresponding item in 1Password. Choose the appropriate item type:

API Credential

For third-party service API keys

Fields: username, credential

Password

For first-party secrets, like encryption keys

Fields: username, password

Database

For hosted PostgreSQL databases and the like

Fields: type, server, port, database, username, password

Step 4: Update Your .env File

Replace raw values in your .env file with op:// references using the following format:

op://vault/item/field

Each reference consists of three components:

The vault name (e.g., “development”)
The item name or UUID
The field name from the item

For example, here’s how you might reference credentials for various services:

# Reference by item name (case-insensitive)
        AWS_ACCESS_KEY_ID=op://development/AWS/username
        AWS_SECRET_ACCESS_KEY=op://development/WorkOS/credential
        # Reference by item UUID
        STRIPE_SECRET_KEY=op://development/abc123xyz789defghijklmnop/password
        # Using different field names based on item type
        DATABASE_HOST=op://development/db/server
        DATABASE_USER=op://development/db/username
        DATABASE_PASSWORD=op://development/db/password
        DATABASE_NAME=op://development/db/database

You can locate the UUID for any item in 1Password by clicking the "More actions" button (⋮, whatever you want to call that) and selecting "Copy item UUID".

Both item name and UUID references work, but using UUIDs can be more reliable in automation contexts since they're guaranteed to be unique and won't change if you rename the item.

Once you’ve replaced all sensitive values with op:// references, you can safely commit your .env file to version control. The references themselves don’t contain any sensitive information – they’re just pointers to your 1Password vault.

You might find that .env files are excluded by your global Git configuration. To override this, add the following to your repository’s .gitignore:

# Override global .gitignore to allow .env containing op:// references
        !.env

The exclamation point (!) negates a previous pattern, allowing you to explicitly include a file that would otherwise be ignored.

Step 5. Update Your Development Script

Whatever command you normally run to kick off your development server, you’ll need to prepend op run -- to that.

For example, if you follow the “Scripts to Rule Them All” pattern, you’d update script/start like so:

#!/bin/sh
        - swift run
        + op run -- swift run

op run does a neat trick by creating a pseudoterminal (PTY) pair to redact secrets if printed out directly to stdout:

$ op run -- env
        LANG=en_US.UTF-8
        IRC_USERNAME=<concealed by 1Password>
        IRC_PASSWORD=<concealed by 1Password>

This behavior can be turned off with the --no-masking option.

Advantages Over Traditional .env Files

op run solves many of the problems inherent to .env files:

No More Cold Start Woes: New team members get access to all required configuration simply by joining the appropriate 1Password vault.
Automatic Updates: When credentials change, they’re automatically updated for everyone on the team. No more out-of-sync configuration.
Proper Secret Management: 1Password provides features like access controls, versioning, and integration with Have I Been Pwned.

Potential Gotchas

Like any technical solution, there are some trade-offs to consider:

Performance: op run adds a small overhead to command startup time (typically less than a second). ¹
stdout/stderr Handling: As mentioned above, op run modifies stdout/stderr to implement secret masking, which can interfere with some terminal applications. ²
Dev Container Support: If you use VSCode Dev Containers, you may encounter some friction with the 1Password CLI. ³

Driving Technical Change

The implementation is often the easy part. The real challenge can be getting your team on board with the change.

First, state the problem you’re trying to solve. Change for change’s sake is rarely helpful.

Next, figure out who you need to get buy-in from. Talk to them. Articulate specific pain point that everyone recognizes, like the frustration of onboarding new team members or the time wasted debugging configuration-related issues.

Once you’ve gotten the green light, move slowly but deliberately. Start small by migrating a single credential, or maybe all of the credentials in a smaller project. Build up confidence that this approach is a good fit — both technically and socially.

Managing development secrets is one of those problems that seems trivial at first but can become a significant source of friction as your team and application grow.

The 1Password CLI offers a more sophisticated approach that integrates with tools developers already use and trust.

While it may not be the right solution for every team, it’s worth considering if you’re feeling the pain of traditional .env files.

As We May Code

2020-07-07T00:00:00-07:00

Chris Lattner often describes LLVM as a process of lowering.

You start at the highest level of abstraction, source code written in a programming language like Swift or Objective-C. That code is parsed into an abstract syntax tree, (AST), which is progressively transformed into lower-level, intermediate representations until it finally becomes executable binary.

What if, instead of lowering source code down for the purpose of execution, we raised source code for the purpose of understanding?

You could say that we already do this to some degree with syntax highlighting
(func f() → func f()), structured editing, and documentation generation. But how far could we take it?

In this article, I’d like to share an idea that I’ve been kicking around for a while. It’s something that’s come into greater focus with my recent work on swift-doc, but first started to form during tenure in Apple Developer Publications, back in 2015.

The idea is this:
What if we took the lessons of the semantic web and applied them to source code?

Specifically:

Representation: Software components should be represented by a common, language-agnostic data format.
Addressability: Packages, modules, and their constituent APIs should each have a unique URL identifier.
Decentralization: Information should be distributed across a federated network of data sources, which can cross-reference one another by URL.

I grew up with the Internet, and got to see it, first-hand, go from an obscure technology to the dominant cultural force. So much of what I see in software development today reminds me of what I remember about the web from 20 years ago. And if you’ll forgive the extended wind-up, I think there’s a lot we can learn by looking at that evolution.

Web 1.0 The Web of Documents

Tim Berners-Lee launched the World Wide Web from a NeXT workstation 27 years ago. His vision for a globally-distributed, decentralized network of inter-connected documents gave rise to the Internet as we know it today. But it was also part of an intellectual tradition dating back to the 1940s, which includes Vannevar Bush’s Memex, Ted Nelson’s Xanadu, and Doug Engelbart’s Mother of All Demos.

In those early days, the knowledge being shared was primarily academic. As the userbase grew over time, so too did the breadth and diversity of the information available. And, for a time, that’s what the Internet was: fan sites for Sunbeam toasters, recipes for Neapolitan-style pizza, and the official website for the 1996 film Space Jam.

But the web of documents had limits.

If you wanted to shop for appliances, see the menu of a pizza shop, or get local showtimes for a movie, you might be able to do that on the early Internet. But you really had to work at it.

Back then, you’d start by going to a directory like Yahoo! or DMOZ, navigate to the relevant topic, and click around until you found a promising lead. Most of the time, you wouldn’t find what you were looking for; instead, you’d disconnect your modem to free up your landline and consult the yellow pages.

This started to change in the early ’00s.

Web 2.0 The Social Web

With Perl CGI and PHP, you could now easily generate web pages on-the-fly. This enabled eCommerce and the first commercial uses of the Internet.

After the ensuing dot-com bubble, you had technologies like Java applets and Flash bring a new level of interactivity to web sites. Eventually, folks figured out how to use an obscure API from Internet Explorer 5 to replicate this interactivity on normal webpages — a technique dubbed AJAX. Interacting with a page and seeing results live, without reloading a page? This was huge. Without that, social media might not have taken off as it did.

Anyway, the server-side APIs powering those AJAX interactions on the client, they were the secret sauce that let the Internet evolve into what it is today.

Remember “mashups”?

Thanks to all of these (often unsecured) AJAX endpoints, developers could synthesize information across multiple sources in ways that nobody had ever thought to do. You could get someone’s location from Fire Eagle, search for photos taken nearby on Flickr, and use MOO to print and deliver prints of them on-demand.

By the end of the decade, the rise of social networks and the early promise of mashups started to coalesce into the modern Internet.

Web 3.0 The Web of Data

The term “Web 3.0” didn’t catch on like its predecessor, but there’s a clear delineation between the technologies and culture of the web between the early and late ’00s.

It’s hard to overstate how much the iPhone’s launch in 2007 totally changed the trajectory of the Internet. But many other events played an important role in shaping the web as we know it today:

Google acquiring the company behind Freebase, giving it a knowledge graph to augment its website index.
Facebook launching Open Graph, which meant everything could now be “Liked” (and everyone could be targeted for advertisements).
Yahoo releasing SearchMonkey and BOSS, two ambitious (albeit flawed) attempts to carve out a niche from Google’s monopoly on search.
Wolfram launching Wolfram|Alpha, which far exceeded what many of us thought was possible for a question answering system.

The Internet always had a lot of information on it; the difference now is that the information is accessible to machines as well as humans.

Today, you can ask Google “Who was the first person to land on the moon?” and get an info box saying, “Commander Neil Armstrong”. You can post a link in Messages and see it represented by a rich visual summary instead of a plain text URL. You can ask Siri, “What is the airspeed velocity of an unladen swallow?” and hear back ~~“I can’t get the answer to that on HomePod”~~ About 25 miles per hour.

Think about what we take for granted about the Internet now, and try to imagine doing that on the web when it looked like this. It’s hard to think that any of this would be possible without the semantic web.

GitHub.com, Present Day The Spider and The Octocat

READMEs on GitHub.com today remind me of personal home pages on Geocities back in the Web 1.0 days.

Even with the standard coat of paint, you see an enormous degree of variance across projects and communities. Some are sparse; others are replete with adornment.

And yet, no matter what a project’s README looks like, onboarding onto a new tool or library entails, well reading.

GitHub offers some structured informational cues: language breakdown, license, some metadata about commit activity. You can search within the repo using text terms. And thanks to semantic / tree-sitter, you can even click through to find declarations in some languages.

But where’s a list of methods? Where are the platform requirements?
You have to read the README to find out! (Better hope it’s up-to-date 😭)

The modest capabilities of browsing and searching code today more closely resemble AltaVista circa 2000 than Google circa 2020. Theres so much more that we could be doing.

RDF Vocabularies The Owl and The Turtle

At the center of the semantic web is something called RDF, the Resource Description Framework. It’s a collection of standards for representing and exchanging data. The atomic data entity in RDF is called a triple, which comprises:

a subject (“the sky”)
a predicate (“has the color”)
an object (“blue”)

You can organize triples according to a vocabulary, or ontology, which defines rules about how things are described. RDF vocabularies are represented by the Web Ontology Language (OWL).

The ideas behind RDF are simple enough. Often, the hardest part is navigating its confusing, acronym-laden technology stack. The important thing to keep in mind is that information can be represented in several different ways without changing the meaning of that information.

Here’s a quick run-down:

RDF/XML: An XML representation format for RDF graphs.
JSON-LD: A JSON representation format for RDF graphs.
N-Triples: A plain text representation format for RDF graphs where each line encodes a subject–predicate–object triple.
Turtle: A human-friendly, plain text representation format for RDF graphs. A superset of N-Triples, and the syntax used in SPARQL queries.
SPARQL: A query language for RDF graphs.

Defining a Vocabulary

Let’s start to define a vocabulary for the Swift programming language. To start, we’ll define the concept of a Symbol along with two subclasses, Structure and Function. We’ll also define a name property that holds a token (a string) that applies to any Symbol. Finally, we’ll define a returns property that applies to a Function and holds a reference to another Symbol.

@prefix : <http://www.swift.org/#> .
        @prefix owl: <http://www.w3.org/2002/07/owl#> .
        @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
        @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
        @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
        :Symbol rdf:type owl:Class .
        :name rdf:type owl:FunctionalProperty ;
        rdfs:domain :Symbol ;
        rdfs:range xsd:token .
        :Structure rdfs:subClassOf :Symbol .
        :Function rdfs:subClassOf :Symbol .
        :returns rdf:type owl:FunctionalProperty ;
        rdfs:domain :Function ;
        rdfs:range :Symbol .

<?xml version="1.0" encoding="utf-8" ?>
        <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
        xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
        xmlns:owl="http://www.w3.org/2002/07/owl#">
        <owl:Class rdf:about="http://www.swift.org/#Symbol"></owl:Class>
        <owl:FunctionalProperty rdf:about="http://www.swift.org/#name">
        <rdfs:domain rdf:resource="http://www.swift.org/#Symbol"/>
        <rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#token"/>
        </owl:FunctionalProperty>
        <rdf:Description rdf:about="http://www.swift.org/#Structure">
        <rdfs:subClassOf rdf:resource="http://www.swift.org/#Symbol"/>
        </rdf:Description>
        <rdf:Description rdf:about="http://www.swift.org/#Function">
        <rdfs:subClassOf rdf:resource="http://www.swift.org/#Symbol"/>
        </rdf:Description>
        <owl:FunctionalProperty rdf:about="http://www.swift.org/#returns">
        <rdfs:domain rdf:resource="http://www.swift.org/#Function"/>
        <rdfs:range rdf:resource="http://www.swift.org/#Symbol"/>
        </owl:FunctionalProperty>
        </rdf:RDF>

Parsing Code Declarations

Now consider the following Swift code:

struct Widget { … }
        func foo() -> Widget {…}
        func bar() -> Widget {…}

We can use SwiftSyntax to parse the code into an AST and SwiftSemantics to convert those AST nodes into a more convenient representation.

import SwiftSyntax
        import SwiftSemantics
        var collector = DeclarationCollector()
        let tree = try SyntaxParser.parse(source: source)
        collector.walk(tree)
        collector.functions.first?.name // "foo()"
        collector.functions.first?.returns // "Widget"

Combining this syntactic reading with information from compiler, we can express facts about the code in the form of RDF triples.

{
        "@context": {
        "name": {
        "@id": "http://www.swift.org/#name",
        "@type": "http://www.w3.org/2001/XMLSchema#token"
        },
        "returns": "http://www.swift.org/#returns"
        },
        "symbols": [
        {
        "@id": "E83C6A28-1E68-406E-8162-D389A04DFB27",
        "@type": "http://www.swift.org/#Structure",
        "name": "Widget"
        },
        {
        "@id": "4EAE3E8C-FD96-4664-B7F7-D64D8B75ECEB",
        "@type": "http://www.swift.org/#Function",
        "name": "foo()"
        },
        {
        "@id": "2D1F49FE-86DE-4715-BD59-FA70392E41BE",
        "@type": "http://www.swift.org/#Function",
        "name": "bar()"
        }
        ]
        }

_:E83C6A28-1E68-406E-8162-D389A04DFB27 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.swift.org/#Structure> .
        _:E83C6A28-1E68-406E-8162-D389A04DFB27 <http://www.swift.org/#name> "Widget"^^<http://www.w3.org/2001/XMLSchema#token> .
        _:4EAE3E8C-FD96-4664-B7F7-D64D8B75ECEB <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.swift.org/#Function> .
        _:4EAE3E8C-FD96-4664-B7F7-D64D8B75ECEB <http://www.swift.org/#name> "foo()"^^<http://www.w3.org/2001/XMLSchema#token> .
        _:4EAE3E8C-FD96-4664-B7F7-D64D8B75ECEB <http://www.swift.org/#returns> _:E83C6A28-1E68-406E-8162-D389A04DFB27 .
        _:2D1F49FE-86DE-4715-BD59-FA70392E41BE <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.swift.org/#Function> .
        _:2D1F49FE-86DE-4715-BD59-FA70392E41BE <http://www.swift.org/#name> "bar()"^^<http://www.w3.org/2001/XMLSchema#token> .
        _:2D1F49FE-86DE-4715-BD59-FA70392E41BE <http://www.swift.org/#returns> _:E83C6A28-1E68-406E-8162-D389A04DFB27 .

@prefix swift: <http://www.swift.org/#> .
        @prefix owl: <http://www.w3.org/2002/07/owl#> .
        @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
        @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
        @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
        _:Widget rdf:type :Structure ;
        swift:name "Widget"^^xsd:token .
        _:foo rdf:type :Function ;
        swift:name "foo()"^^xsd:token ;
        swift:returns _:Widget .
        _:bar rdf:type :Function ;
        swift:name "bar()"^^xsd:token ;
        swift:returns _:Widget .

Encoding our knowledge into a standard format lets anyone access that information — however they like. And because these facts are encoded within an ontology, they can be validated for coherence and consistency. It’s totally language agnostic.

Querying the Results

With an RDF graph of facts, we can query it using SPARQL. Or, we could load the information into a graph database like Neo4j or a relational database like PostgreSQL and perform the query in Cypher or SQL, respectively.

PREFIX
        swift: <http://www.swift.org/#>
        SELECT ?function ?name
        WHERE {
        ?function a swift:Function ;
        swift:returns ?type ;
        swift:name ?name .
        ?type swift:name "Widget" .
        }
        ORDER BY ?function

MATCH (function:Function)-[:RETURNS]->(symbol:Symbol {name: 'Widget'})
        RETURN function

CREATE TABLE symbols (
        id UUID PRIMARY KEY,
        name TEXT,
        );
        CREATE TABLE functions (
        returns_id UUID REFERENCES symbols(id),
        ) INHERITS (symbols);
        --
        SELECT f.id, f.name
        FROM functions f
        INNER JOIN symbols s USING (returns_id);
        WHERE s.name = 'Widget'
        ORDER BY name

Whichever route we take, we get the same results:

id	name
4EAE3E8C-FD96-4664-B7F7-D64D8B75ECEB	foo()
2D1F49FE-86DE-4715-BD59-FA70392E41BE	bar()

Answering Questions About Your Code

“What can you do with a knowledge graph?” That’s kind of like asking, “What can you do with Swift?” The answer — “Pretty much anything” — is as true as it is unhelpful.

Perhaps a better framing would be to consider the kinds of questions that a knowledge graph of code symbols can help answer:

Which methods in Foundation produce a Date value?
Which public types in my project don’t conform to Codable?
Which methods does Array inherit default implementations from RandomAccessCollection?
Which APIs have documentation that includes example code?
What are the most important APIs in MapKit?
Are there any unused APIs in my project?
What’s the oldest version of iOS that my app could target based on my current API usage?
What APIs were added to Alamofire between versions 4.0 and 4.2?
What APIs in our app are affected by a CVE issued for a 3rd-party dependency?

The possibilities get even more interesting as you layer additional contexts by linking Swift APIs to different domains and other programming languages:

How is this Swift API exposed in Objective-C?
Who are the developers maintaining the packages that are pulled in as external dependencies for this project?
What’s the closest functional equivalent to this Swift package that’s written in Rust?

Future Applications The Promise of What Lies Ahead

Any fact becomes important when it’s connected to another.

Umberto Eco, Foucault’s Pendulum

Operating on code symbolically is more powerful than treating it as text. Once you’ve experienced proper refactoring tools, you’ll never want to go back to global find-and-replace.

The leap from symbolic to semantic understanding of code promises to be just as powerful. What follows are a few examples of potential applications of the knowledge graph we’ve described.

Flexible Search Queries

GitHub’s advanced search provides an interface to filter results on various facets, but they’re limited to metadata about the projects. You can search for Swift code written by @kateinoigakukun in 2020, but you can’t, for example, filter for code compatible with Swift 5.1. You can search code for the string “record”, but you can’t disambiguate between type and function definitions (class Record vs. func record()).

As we showed earlier, the kinds of queries we can perform across a knowledge graph are fundamentally different from what’s possible with a conventional faceted, full-text search index.

For example, here’s a SPARQL query to find the urls of repositories created by @kateinoigakukun and updated this year that contain Swift functions named record:

PREFIX
        swift: <http://www.swift.org/#>
        skos: <http://www.w3.org/2004/02/skos/core/#>
        sdo: <http://schema.org/#>
        SELECT ?url
        WHERE {
        ?function a swift:Function ;
        swift:name "record" ;
        skos:member ?repository .
        ?repository a sdo:SoftwareSourceCode ;
        sdo:contributor ?contributor;
        sdo:url ?url ;
        sdo:dateModified ?date .
        ?contributor a sdo:Person ;
        sdo:username "kateinoigakukun" .
        FILTER (?date >= "2020-01-01")
        }
        ORDER BY ?url

Linked Documentation

When faced with missing or incomplete documentation, developers are left to search Google for blog posts, tutorials, conference videos, and sample code to fill in the gaps. Often, this means sifting through pages of irrelevant results — to say nothing of outdated and incorrect information.

A knowledge graph can improve search for documentation much the same as it can for code, but we can go even further. Similar to how academic papers contain citations, example code can be annotated to include references to the canonical APIs it interacts with. Strong connections between references and its source material make for easy retrieval later on.

Imagine if, when you option-click on an API in Xcode to get its documentation, you also saw a list of sample code and WWDC session videos? Or what if we could generate sample code automatically from test cases? Wouldn’t that be nice?

All of that information is out there, just waiting for us to connect the dots.

Automatic µDependencies

John D. Cook once observed, code reuse is more like an organ transplant than snapping LEGO blocks together. Fred Brooks similarly analogized software developers to surgeons in The Mythical Man-Month.

But that’s not to say that things can’t get better — it’d be hard to argue that they haven’t.

Web applications were once described in similar, organic terms, but that came to an end with the advent of containerization. Now you can orchestrate entire multi-cloud deployments automatically via declarative configuration files.

Before CPAN, the state of the art for dependency management was copy-pasting chunks of code you found on a web page. But today, package managers are essential infrastructure for projects.

What if, instead of organizing code into self-contained, modular chunks ourselves, we let software do it for us? Call it FaaD (Functions as a Dependency).

Say you want an implementation of k-means clustering. You might search around for “k-means” or “clustering” on GitHub and find a package named “SwiftyClusterAlgorithms” (😒), only to discover that it includes a bunch of functionality that you don’t need — and to add insult to injury, some of those extra bits happen to generate compiler warnings. Super annoying.

Today, there’s no automatic way to pick and choose what you need. (Swift import syntax (import func kMeans) is a lie) But there’s no inherent reason why the compiler couldn’t do this for you.

Or to go even further: If everything compiles down to web assembly, there’s no inherent requirement for that implementation of k-means — it could be written in Rust or JavaScript, and you’d be none the wiser.

At a certain point, you start to question the inherent necessity of software packaging as we know it today. Take it far enough, and you may wonder how much code we’ll write ourselves in the future.

Code Generation

A few months ago, Microsoft hosted its Build conference. And among the videos presented was an interview with Sam Altman, CEO of OpenAI. A few minutes in, the interview cut to a video of Sam using a fine-tuned version of GPT-2 to write Python code from docstrings.

def is_palindrome(s):
        """Check whether a string is a palindrome"""
        return s == s[::-1] # ← Generated by AI model from docstring!

And that’s using a model that treats code as text. Imagine how far you could go with a priori knowledge of programming languages! Unlike English, the rules of code are, well, codified. You can check to see if code compiles — and if it does compile, you can run it to see the results.

At this point, you should feel either very worried or very excited.
If you don’t, then you’re not paying attention.

Taking Ideas Seriously The Shoemaker’s Children

The use of FORTRAN, like the earlier symbolic programming, was very slow to be taken up by the professionals. And this is typical of almost all professional groups. Doctors clearly do not follow the advice they give to others, and they also have a high proportion of drug addicts. Lawyers often do not leave decent wills when they die. Almost all professionals are slow to use their own expertise for their own work. The situation is nicely summarized by the old saying, “The shoe maker’s children go without shoes”. Consider how in the future, when you are a great expert, you will avoid this typical error!

Richard W. Hamming, “The Art of Doing Science and Engineering”

Today, lawyers delegate many paralegal tasks like document discovery to computers and doctors routinely use machine learning models to help diagnose patients.

So why aren’t we — ostensibly the people writing software — doing more with AI in our day-to-day? Why are things like TabNine and Kite so often seen as curiosities instead of game-changers?

If you take seriously the idea that AI will fundamentally change the nature of many occupations in the coming decade, what reason do you have to believe that you’ll be immune from that because you work in software? Looking at the code you’ve been paid to write over the past few years, how much of that can you honestly say is truly novel?

We’re really not as clever as we think we are.

Postscript Reflection and Metaprogramming

Today marks 8 years since I started NSHipster.

You might’ve noticed that I don’t write here as much as I once did. And on the occasions that I do publish an article, it’s more likely to include obscure historical facts and cultural references than to the obscure APIs promised by this blog’s tagline.

A few weeks out now from WWDC, I should be writing about DCAppAttestService, SKTestSession, SwiftUI Namespace and UTType. But here we are, at the end of an article about the semantic web, of all things…

The truth is, I’ve come around to thinking that programming isn’t the most important thing for programmers to pay attention to right now.

Anyway, I’d like to take this opportunity to extend my sincere gratitude to everyone who reads the words I write. Thank you. It may be a while before I get back into a regular cadence, so apologies in advance.

Until next time, May your code continue to compile and inspire.

WWDC 2020

2020-06-26T00:00:00-07:00

Like everything else in 2020, this year’s WWDC had to be a little different if it was going to happen at all.

When Apple first announced that the conference would be fully remote, nobody knew what that would look like, exactly. What parts of the dubdub experience would be kept in this new format? What details would be lost in translation? Could they actually pull it off?

As it turns out, going fully remote wasn’t merely good enough — it was, in many ways, superior to the original thing. There’s a lot to like about the new format.

The videos are well-produced, and let each presenter’s personality really shine. Everybody looks and sounds great.

Sessions are tight and well-paced. Rather than stretching or cramming content into a fixed time slot, they’re as long as they need to be. And thanks to this more digestible format, we’re starting to see WWDC clips being shared around, which is new and refreshing.

To be honest, it’s hard to imagine ever going back to a physical conference.

However, as someone who had the privilege of attending WWDC in years past, there are things I’m going to miss (and some that I decidedly won’t).

🥰	😫
Refrigerators stocked with Odwalla smoothies	Trying to download the latest Xcode beta over hotel internet
Lunchtime sessions	Eating lunch at or around Moscone
WWDC track jackets saving the lives of first-time attendees from the cold of San Francisco summer	Being in San Francisco, generally
Eating burritos on the terrace of McEnery Convention Center during WWDC check-in	Being in San Jose, generally
Guessing who would be playing at Bash this year	Hearing the same handful of songs on repeat before and after every session (this song in particular)
Watching Apple executives dance at Bash
Talking to people as you wait in line for the keynote on Monday morning	Waking up late and having to settle for watching from the overflow room
Leaving at the end of the week with a mix of hope, fear, and inspiration ^*	Being sick for the next week with dubdub flu

^* I’d like to hold on this last point for a moment.

In the Before Times, many of us traveled far from home to attend WWDC. There was a physical and temporal delineation between life before, during, and after the conference. Flying out of SFO, SJC, or OAK, you escaped Apple’s “reality distortion field” and its surrounding echo chamber. You returned to your normal life.

This year? Not so much.

WWDC 2020 was just another week in this bizarre existence amidst this pandemic. Not only is there no escape from the “reality distortion field”, there isn’t even a “normal life” for us to leave or return to.

So here we are, filled with anxious excitement and fear; our corporeal forms replaced by Memoji floating in a black, digital void lit only by the screens of our soon-to-be-obsolete MacBooks Pro.

Excitement

I don’t have a real sense of how everything went over this year. There wasn’t any applause (or heckling) at this year’s keynote to gauge the temperature in the room. There were no parties to overhear shop talk and hot takes. There was no line for lunch to make small talk with a fellow attendee.

But if Twitter is anything to go on, my overall impression is that everyone is really excited.

Which is fine. I get it.

But it should come as no surprise that things announced at WWDC are exciting — that’s the whole point of having a developer conference in the first place. Apple has the best marketing in the world, and WWDC is Apple’s way of marketing to us.

Here’s the thing about excitement: It’s kryptonite to developers.

Excitement messes with our ability to focus on one thing, which is already a big struggle for a lot of us (myself included). When you’re excited, it’s almost impossible to get anything done.

There are plenty of voices in the community who are echoing this excitement. I can’t add anything to that discussion. And besides, that’s not really where my head’s at right now.

Trivial Pursuit

I briefly considered reviving the NSHipster Quiz for WWDC 2020, but it didn’t feel right. With everything going on in the world, Apple trivia just isn’t where my head or heart are right now.

To give you a sense of what I mean, here’s what I had for Round 3, whose theme was inspired by Richard Hamming:

Question 1.: What are the important problems of your field?
Question 2.: What important problems are you working on?
Question 3.: If what you are doing is not important, why are working on it?

Temperance

If you’re serious about solving a problem, you owe it to yourself to temper any excitement that distracts you from making real progress.

In last year’s write-up for WWDC 2019, I concluded with the following, as a counterpoint to the conference’s theme of #mindblown 🤯:

Taking care of yourself — sleeping enough, eating right, exercising regularly — will do more to improve your productivity than any language or framework out there. Your ability to communicate and collaborate with others will always be a better predictor of success than your choice of technology stack. Your relationships with others are the most significant factors of success and happiness in life.

I stand by this advice, boring as it may be.

It’s been an exciting week, so take a moment to collect yourself. Go on a walk. Take a hike. (Be safe about it.) Do whatever you need to break free of the “reality distortion field”. Once you do, you’ll have the necessary distance to determine what new technologies you should pay attention to and what you can ignore for now.

We have a lot of work ahead of us.

Language Server Protocol

2018-11-05T00:00:00-08:00

In October 2018, Apple announced on the Swift.org forums that it was starting work to adopt the Language Server Protocol (LSP) for Swift and C languages.

At Apple we are making it a priority to support high-quality tooling for all Swift developers, including those working on non-Apple platforms. We want to collaborate with the open-source community and focus our efforts on building common infrastructure that can be shared by Xcode and other editors and platforms. To that end, [ … ] we’ve chosen to adopt LSP.

Argyrios Kyrtzidis, October 15^th, 2018

This is arguably the most important decision Apple has made for Swift since releasing the language as open source in 2014. It’s a big deal for app developers, and it’s an even bigger deal for Swift developers on other platforms.

To understand why, this week’s article will take a look at what problem the Language Server Protocol solves, how it works, and what its long-term impacts may be.

Imagine a grid with each row representing a different programming language (Swift, JavaScript, Ruby, Python, etc.) and each column representing a different code editor (Xcode, Visual Studio, Vim, Atom, etc.), such that each cell represents the level of support that a particular editor has for a language.

Up until recently, what you’d find was a patchwork of compatibility across the various combinations. Some editors offered deep integration with a few languages and little to no support for anything else, whereas other editors aimed to be general-purpose with at least a modicum of support for many languages. (The term IDE is often used to describe the former.)

Case in point: You’d be stubborn not to use Xcode for app development and foolish to use it for anything else.

For an editor to have better support for a particular language, it needs to write integration code — either directly in the code base or via a plugin system. Due to implementation differences across languages and editors, improvements to, say, Ruby support in Vim wouldn’t translate into better support for Python, nor could they be applied to make Ruby work better in Atom. The end result: inconsistent support across technologies and a lot of wasted effort.

The situation we described is often referred to as an M × N problem, where the number of integrations is the product of M editors and N languages. What the Language Server Protocol does is change this M × N problem into a M + N problem.

Rather than an editor having to implement support for each language, it only needs to support the LSP. And in doing so, it gets the same level of functionality for all languages that support the LSP.

Language Server Protocol provides a common set of functionality for supported languages, including:

Syntax Highlighting
Automatic Formatting
Autocomplete
Syntax
Tooltips
Inline Diagnostics
Jump to Definition
Find References in Project
Advanced Text and Symbol Search

Rather than reinventing the wheel for each new technology, tools and editors can invest in better usability and more advanced functionality.

How Language Server Protocol Works

If you’re an iOS developer, you may be most familiar with the terms server and protocol in the sense of communicating with web applications in JSON format via HTTP. This actually isn’t too far off from how the Language Server Protocol works.

In the case of LSP, the client refers to the editor — or more generally, the tool — and the server refers to an external program run locally in a separate process.

As for the protocol itself, LSP resembles a simplified version of HTTP:

Each message consists of a header part and a content part.
The header part has a required Content-Length field containing the size of the content part in bytes, and an optional Content-Type field (application/vscode-jsonrpc; charset=utf-8 by default)
The content part uses JSON-RPC to describe the structure of requests, responses, and notifications.

Whenever something happens in the tool, such as the user jumping to the definition of a symbol, the tool sends a request to the server. The server receives that request and then returns an appropriate response.

For example, imagine that a user opens the following Swift code in an Xcode-like editor that supported the Language Server Protocol:

class Parent {}
        class Child: Parent {}

When the user ⌘-clicks the symbol Parent in the inheritance clause on line 2, the editor jumps to the definition of the Parent class on line 1.

Here’s how LSP enables this interaction behind the scenes:

First, when the user opens the Swift code, the editor launches its Swift language server in a separate process, if it isn’t running already, and performs any additional setup.

When the user executes the “jump to definition” command, the editor sends the following request to its Swift language server:

{
        "jsonrpc": "2.0",
        "id": 1,
        "method": "textDocument/definition",
        "params": {
        "textDocument": {
        "uri": "file:///Users/NSHipster/Example.swift"
        },
        "position": {
        "line": 1,
        "character": 13
        }
        }
        }

Upon receiving this request, the Swift language server uses a compiler tool like SourceKit to identify the corresponding code entity and find the location of its declaration on the preceding line. The language server then responds with the following message:

{
        "jsonrpc": "2.0",
        "id": 1,
        "result": {
        "uri": "file:///Users/NSHipster/Example.swift",
        "range": {
        "start": {
        "line": 0,
        "character": 6
        },
        "end": {
        "line": 0,
        "character": 12
        }
        }
        }
        }

Finally, the editor navigates to the file (which, in this case, is already open), moves the cursor to that range, and highlights the token.

The beauty of this approach is that the editor did all of this without knowing anything about the Swift programming language other than that .swift files are associated with Swift code. All the editor needs to do is talk to the language server and update the UI. And knowing how to do that, the editor can follow the same procedure to facilitate this interaction for code written in any language with a language server implementation.

Language Server Protocol Support in Clang / LLVM

If the M + N diagram from before looks familiar, it might be because it’s the same approach taken by LLVM.

At the core of LLVM is an intermediate representation (IR). Supported languages generate IR using a compiler frontend, and that IR can generate machine code for any platform supported by a compiler backend.

The LLVM compiler frontend for C languages is called Clang. It’s also used by Swift for inter-operability with Objective-C. In its recent 5.0.0 release, Clang added a new tool called Clangd, LLVM’s implementation for the Language Server Protocol.

In April 2018, Apple announced to the LLVM mailing list that it was switching the focus of its development efforts from libclang to Clangd as the primary way to create interactive tooling.

Now you might think, “So what?” Apple is among the most prominent supporters of the LLVM project — having, among other things, employed the project’s founder, Chris Lattner, for over a decade. Apple’s decision to switch from one obscure Clang tool to another would seem to be an implementation detail (so to speak).

What makes this announcement quite interesting is that Clangd appears to have been created entirely outside of Apple, with significant contributions from Google and other companies. This announcement signals a significant shift in the direction of tooling development going forward — something that would be confirmed 6 months later on the Swift.org forums.

Getting Started with Language Server Protocol

Xcode 11.4 includes sourcekit-lsp in its default toolchain. You can use the xcrun command to get the path to the language server executable:

$ xcrun -f sourcekit-lsp
        /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/sourcekit-lsp

Check out our article about Visual Studio Code to get started with our go-to editors. Beyond that, the sourcekit-lsp project on GitHub has instructions for integrating with Sublime Text, Vim, Emacs, and others..

Potential Consequences of Apple’s Support of Language Server Protocol

It’ll take some time to feel the full impact of these developments, but believe me: your patience will be rewarded. Here are just a few of what I believe will happen as a result of LSP in the coming months and years.

Swift Becomes More Appealing as a General-Purpose Programming Language

Although Swift is used primarily for app development, it was designed from the start to be a capable general-purpose programming language. Between Swift for TensorFlow, SwiftNIO, and other projects, we’re just starting to see the promise of what Swift can do beyond the App Store.

Among the biggest factors holding Swift back from mainstream adoption up to this point has been its reliance on Xcode.

It’s a lot to ask, say, a web developer or machine learning engineer to download Xcode just to try Swift when there are so many great alternatives with a much lower barrier to entry. Support for the Language Server Protocol should make it significantly easier for folks outside the Apple ecosystem to evaluate Swift with the same, familiar tools they use for everything else.

Xcode Gets Better

Adopting LSP isn’t just about making Swift work better in other editors; Xcode stands to benefit immensely, as well.

Consider this forum post from Project Lead for Swift at Apple, Ted Kremenek:

The LSP service [Argyrios] is describing will be functionally more powerful than SourceKit is today.

LSP is an opportunity for the Xcode team to take a fresh approach to Swift integration, and to capitalize on all of the improvements to the language and tooling in the four years since its 1.0 release.

Our first glimpse into this overhauled infrastructure comes by way of IndexStoreDB: a powerful new API for querying code symbols in Swift projects from a Clang index.

Xcode (Eventually) Becomes More Capable

The benefit of LSP isn’t limited to Swift and Objective-C; from another post by Argyrios in that thread:

Getting Xcode to use our new LSP service should make it viable to use other LSP services as well, and it’s something that we are interested in, but we don’t have specific plans to announce at this moment.

The main focus for the current efforts are to improve the story for Swift. But once that’s done, it should be relatively straightforward to have those benefits cascade down to other languages with LSP support.

The architecture of software reflects the structure and values of the organizations that create it. The converse is true as well, to some extent.

By adopting the open Language Server Protocol standard for Xcode, Apple is making good on its commitment to the success of Swift on platforms outside the Apple ecosystem. And I think it’ll work: tooling (or lack thereof) is often the key decider in which technologies gain mindshare. But perhaps more importantly, I believe this decision demonstrates an increased willingness within (at least some small part of) the company for collaboration and transparency.