bricoleur

Product Counselling in the AI Era

2026-06-02T12:29:00.996-07:00

AI is changing how products behave and how they are built. Product lawyering has to change too.

Product counsel have always needed to understand their products. What does the product do? What does it not do? How does it work? Two AI-driven changes complicate that job. First, many products now include LLM decision-making and output as product features, which can make product behavior unexpected. Second, product teams now use LLMs to build products, which can leave no one with a complete understanding of how the product works. Product counsel will need to adapt to both changes.

From United Editors Encyclopedia and Dictionary: A Library of Universal Knowledge and an
Unabridged Dictionary of the English Language, United Editors Association, 1907.

LLMs as Product Features

The first major change is that some products now incorporate LLM responses. LLMs differ from most previous computer algorithms in the extent to which non-determinism is both important and valuable. For product-counsel purposes, the key point is not mathematical randomness but operational unpredictability. The same or similar inputs may produce different outputs, and even great engineers cannot guarantee in advance what the model will say.

Chat interfaces, such as ChatGPT or Claude, provide the clearest examples. In the days of deterministic software, a user who typed “tell me a story” might get a predetermined response, perhaps Rumpelstiltskin or a random choice from a fixed set of stories. Prompt Claude or ChatGPT with “tell me a story,” and you will get a different story each time.

There was once a lighthouse keeper named Maren who lived at the edge of a crumbling cliff on the northern coast. Every evening, she climbed the spiral stairs and lit the lamp, and every morning, she extinguished it. The rhythm of her days was as steady as the tide. …

There was once a lighthouse keeper named Maren who lived on a rocky island so small you could walk its entire shore in seven minutes. She had no boat, no radio, and no particular desire for either. Supplies came once a month by ferry, and the ferryman never stayed longer than it took to unload the crates. …

There was once a lighthouse keeper named Maren who lived on a rock so small it didn't appear on any map. Every evening she climbed the iron stairs, lit the lamp, and watched the beam sweep across the water like a slow, bright hand waving to no one. …

A non-LLM product might convert “Tell me a story,” “Spin me a yarn,” “Please can you tell me a story,” or even “Tell me a story” (with two spaces before “story”) into the same story query. An LLM will treat all of those prompts as requests for a story, but small wording changes will still change the output.

“Tell me a story”
There was once a lighthouse keeper named Maren who lived on a narrow spit of rock three miles off the coast. Every evening, she climbed the iron spiral to the lamp room, lit the wick, and watched the beam sweep out across the dark water like a slow, bright arm reaching for something it could never quite hold.
…
Years passed. The coast guard automated the light. They told Maren she could leave. She stood at the door of the lighthouse with a bag over her shoulder and Compass at her heel, looked back at the beam still turning without her, and felt something she hadn't expected: not sadness, but gratitude — for the years of dark water, and the one bright thing that had drifted out of it.
She walked to the ferry. Compass walked beside her. Neither of them looked back again.

“Spin me a yarn”
A lighthouse keeper named Margot had one job: keep the light burning. Simple enough, until the night the fog rolled in and the light started answering back.
…
Margot laughed, alone in her tower above the black Atlantic, and kept the light on a little longer that night. Some conversations are worth the overtime.

“Please can you tell me a story”
There was once a lighthouse keeper named Maren who lived on a narrow spit of rock far from the mainland. Every evening, she climbed the iron spiral staircase to light the lamp, and every morning she climbed again to snuff it out. The rhythm of it — up and down, light and dark — had become the pulse of her life.
…
The storm broke by morning. Maren watched his ferry pull away, the case tucked safely under his arm. She climbed the stairs and snuffed the lamp, same as always — but the tower didn't feel quite so quiet anymore.

“Tell me a story”
There was once a lighthouse keeper named Maren who lived at the edge of a rocky coast where the sea never seemed to rest. Every night, she climbed the spiral stairs and lit the lamp, watching its beam sweep across the dark water like a slow, golden arm reaching for something it could never quite hold.
…
“Because," she said, lowering her arms at last, "you can't save anyone from where it's comfortable."

The same or very similar inputs can produce meaningfully different outputs. In these examples, the material difference lies in the emotional resonance of the story. In your product, the difference could create a security risk or legal jeopardy. In a customer-support product, the system might promise a refund. In a health product, it might give inaccurate medical-sounding advice. In a copyright-sensitive product, it might summarize a book or reproduce too much of it.

AI engineers use many techniques to make products with LLM features behave more like traditional software. These techniques can reduce risk, sometimes substantially, but they do not provide the same hard guarantees as traditional rule-based systems. It remains unclear whether that will change.

LLMs as Product Builders

The second major change is that teams now use LLMs to build products. More and more product-development work is being done by LLMs rather than humans. The approaches vary, but they all sit on a continuum: each transfers more or less software implementation from a human to an LLM.

Ask Antigravity to create an online store, and the LLM will make many implementation decisions. Depending on the human's involvement and the tool's expressiveness, the person “coding” the product may not know those implementation details. What user information does the product receive or store? For how long? Can two users have the same username? Does the product verify a user's email address? Many such decisions are being “made” by the LLM. Code review may let an engineer peer behind the curtain, but code can be complex, and reviews may get less thorough as LLMs get better at coding.

Each change radically alters the role of lawyers who work with products.

An Example from the Golden Age: Google Book Search

As Google Book Search's product counsel, it was my job to make the world's books searchable legally. Copyright law was central to that work. To understand which books were relevant to a query, we needed to copy them. That meant scanning books in many of the world's great libraries. That scanning created a copy, which implicated copyright law, so we had to design a product that stayed within the bounds of fair use both in how it treated those copies and in what it showed users.

Even if copyright law had not existed, we wanted to help readers find authors' books and hoped to help, not hurt, book sales. Our solution was a deterministic algorithm that showed a snippet from a book in response to a query. Because we did not want search to substitute for getting and reading the book, we took steps to keep users from stringing snippets together. The code could be guaranteed to follow the rules we established, and the engineers who wrote it knew it intimately. They could answer my questions. Google was sued over Book Search, and judges agreed that it was a fair use of the books based on those design decisions.

A "snippet" from a book in Book Search.

OpenAI’s product counsel have a harder job. OpenAI also makes copies in order for ChatGPT to be able to respond to queries, but it is much harder for its lawyers and engineers to design its responses because they are non-deterministic.* Telling an LLM to “never return more than fifty words from a passage, never return two fifty-word chunks from any one page, and never return more than five passages per book” cannot be guaranteed the same way we could guarantee rules in Google Book Search. Indeed, in the New York Times lawsuit against OpenAI, the Times submitted an entire exhibit that purported to show ChatGPT reproducing large portions of Times content as part of its argument that OpenAI violated copyright law. Instructions to an LLM are weaker than hard guarantees. In adversarial settings like litigation, that difference can matter.

Some AGI optimists who believe AI will soon have essentially infinite intelligence will argue that LLMs will become good enough to take law into account in their responses. Or perhaps, as LLMs get better at mimicking human writing about copyright law, the product itself could respond in ways that mimic how a human might apply copyright law in each response. That would allow product counsel to shift from establishing product rules to issuing simpler behavioral statements, such as “don't violate relevant law.” Anthropic already uses what it calls a “constitutional” approach, based on broad statements, to address many model-behavior issues.

I am skeptical that “future AI will solve all problems.” Still, LLMs might let product counsel add nuance to product design. In Google Book Search, for example, we suppressed snippets for all books we believed contained short-chunk content, such as dictionaries, so users could not avoid buying those books by running searches on our site. That was a coarse determination. Perhaps LLMs would have allowed finer-grained determinations closer to our real intent: give searchers enough to know that a book would be useful, but not enough to substitute for getting the book. Rule-based algorithms struggle with that line. Even current LLMs might do better much of the time.

LLM-Generated Products

The trend in product development is toward AI-written code. Future product counsel may no longer have access to smart humans with deep knowledge of the code. Or they may have access to smart humans with less deep knowledge of it.

For example, while developing a watch interface for local London bus schedules, I wanted to show bus times for stops near a user's location. Using Claude Code, I “wrote” the application in a coding language and framework I had never used. I “developed” a working product, creatively named “BusWatch,” and shipped it without ever looking at the code. From using the product, I knew that it correctly displayed bus stops and times based on my location. I had educated guesses about how it did that, but no guarantees.

How a product works often matters to whether it is legal (or ethical). In BusWatch, the user's location is private information. How did my product get that information? How granular was it? With whom did it share the information? What other information was linked to it? Was the information stored? The creator of a product built by an LLM coding agent may have limited knowledge of those answers, or none at all.

BusWatch sits at the extreme end of the LLM-coding continuum, but every point along that continuum moves product development away from deep engineering knowledge of how a product works. As LLM-assisted development spreads inside companies, teams may produce more code than traditional review processes can absorb, and fewer people may have a full mental model of what was built.

This makes product counsel's job harder. At Google, I could ask the product manager and engineers to test my assumptions about how a product worked. The engineer who wrote the location code could have walked me through granularity and user consent. The PM could have confirmed with the team that the product did not store location or share it with anyone other than the bus-times supplier. With LLM coding tools, there may be no person with mastery of any part of the written code. That may remain true even if companies still require code review, documentation review, privacy review, or other reviews. Additionally, LLM-coded projects may shorten the time that product counsel have to review as products move from idea to prototype to launch on a compressed timeframe.

On the other hand, code is now more transparent to the average lawyer than it has ever been. An LLM coding agent could digest the entire BusWatch codebase in its context window and answer questions about it. Despite the risk of errors, the ability to interrogate code directly gives product counsel a tantalizing new tool. A lawyer who cannot read a codebase can now ask an LLM to explain data flows, identify API calls, summarize permissions, or generate tests for risky scenarios. An LLM with specific instructions about what to look for could helpfully summarize technical and legal intersection points. Those answers need verification, but they change the lawyer's practical relationship to products.

Even more transformatively, product counsel could become coauthors of code, shaping products so they achieve their missions while complying with law. Engineer time would no longer be the main bottleneck to that kind of product reworking.

Testing and Evaluation

Testing and evaluation offer even stronger answers. Product creators face the same problem as product counsel: they need to ensure that their products do what they intend. Between LLM errors in code interrogation, the relative opacity of LLMs used in products, and the non-deterministic parts of modern products, it is harder than ever to say reliably what a product does and does not do. Software engineers are putting more emphasis on testing and evaluation. Given the stakes of many product-counsel decisions, product counsel should too.

Many coding frameworks for LLM coding agents, including Jesse Vincent's excellent Superpowers framework, rely on testing-centered methodologies. These frameworks emphasize test-driven development, an older software-development methodology in which tests are written before code and code is written to pass the tests. In that process, tests stand in for specifications. Whether the code has been successfully developed depends on whether it passes the tests.

Product counsel can and should think about how to test legal constraints as applied to the products they advise. In BusWatch, for example, tests with network monitoring could verify that, at least in the tested situations, the product sends network traffic only to known destinations. Similarly, tests specifying the expected location data could verify what information the product collects. LLMs can help generate test data and test pathways, which may make this kind of testing easier. But product counsel should be careful about delegating test generation entirely to LLMs; otherwise they may recreate the same LLM problems they were trying to solve.

AI engineers are also increasingly using evals, another kind of product analysis, for products with LLM features. Whereas tests ensure product guarantees are met, evals are more useful to add qualitative assessments. In a recipe product, for example, tests might ensure that the recipes don’t contain the words “glue” or “arsenic” while evals might evaluate recipe healthiness. Tests either pass or fail. Evals could help rate the recipes and determine whether a new model actually improves the product. Before LLMs, a designated group of people or a subset of users often performed evals, and product teams integrated those results into release processes. Now LLMs often judge evals, so a human-specified goal gets scored by an LLM. Yes, it is turtles all the way down.

There Was No Golden Age

I am a big fan of humans, but some of the problems I identify here also existed in the golden age of artisan hand-coded products. Internet history is littered with products that broke the law. Some came from companies that did not care what the law was, or knew they were operating illegally. I assume my readers would not do that.

Other products broke the law because the engineers did not quite understand what their products did, or because the lawyers did not understand the products well enough to advise properly. Misunderstandings between human engineers and lawyers happen. Engineers build products with unintended consequences. Engineers and lawyers make mistakes. Humans forget things. All of that happened in deterministic software written by well-intentioned humans.

Products with non-deterministic features and products built with LLM coding agents change the types and frequency of these problems. They do not create the problems from scratch. There was no golden age of product counseling in which none of them existed.

What Product Counsel Should Do Now

Changes brought by more powerful LLMs do not eliminate product counsel's traditional job. Product counsel still need to understand what the product does, what data it uses, what rights it implicates, and what harms it may cause. But LLMs in products and LLMs used to create products make product behavior harder to predict and product implementation harder to understand.

Product counsel should therefore use tests, evals, and ongoing monitoring to understand product behavior and guard against risk. They should also use LLM tools themselves to take a more hands-on approach to understanding products and to have better conversations with product teams about how those products work. These approaches will make product counsel more empowered and more tightly integrated into product teams. They may also give product counsel more room to think creatively about how to deliver on a product's mission without increasing legal risk.

* I’m not engaging here with the separate interesting and important questions of whether ChatGPT is legal or good, or who should win the various legal challenges to LLMs and chatbots.

Safari + NIKKOR Z 70-200mm f/2.8 VR S II

2026-05-29T05:56:20.951-07:00

If you are considering a new lens for taking pictures on safari, I highly recommend the new NIKKOR Z 70-200mm. I took it along with a 400mm lens and found both to be very useful. The detail, bokeh and colours from the 70-200mm were exceptional. Some examples are below (you can also look at bigger images in an album).

My Current AI Codegen Setup

2026-05-27T02:56:13.062-07:00

This is as of May 2026, it’ll likely change.

Models: I mostly use the Anthropic models (a mix of opus, sonnet and haiku). At this point, that's partly because I also use their harnesses/coding agents and have a bit more trust in them as a company. That said, I'm trying to ensure my development environment is able to switch model providers easily as OpenAI and Google's models seem to be in the same ballpark of functionality and the open models are coming on strong. Exceptions to this are for images (Gemini & ChatGPT). I currently don't do much video beyond playing around.

Harness: Claude Code, mostly because it has consistently led the way in useful innovation. I'm looking to do more experiments with Codex. I'm also curious about the tools built for specific environments, like Google AI Studio. I use lots of subagents and sometimes (very rarely) have multiple agents working the same codebase.

Permissions: This deserves its own category because of how important it is. I use --dangerously-skip-permissions when I am running in a container (see environments below). I also am playing around with the "auto" permission mode. In each case, it is hard to overstate how different it is to the regular approval mode. The best analogy I can come up with is the difference between bluetooth and wired headphones. Wires don't seem very inconvenient but once you start using bluetooth you end up using audio on your phone a TON more. YOLO mode completely changed how I use AI codegen allowing for a lot more evaluation of results rather than the process to get them.

Skills: I am consistently shocked by how useful Jesse Vincent's Superpowers is. I need to do some experiments without it as it is so key to my coding workflow that I've lost any comparison. I also use a bunch of 2389's products, including Simmer, Scenario Testing, Review Squad and Fresh Eyes Review. I keep meaning to try their skills for giving agents journals and drugs (yes, you read that right). Roborev is my core catch all for reviews. And Jesse’s writing skill that puts Strunk & White in context is pretty great.

Mechanical Bread Kneader
The Encyclopaedia Britannica. United Kingdom, 1875, p. 257.

Git: Git is key. Knowing how git is structured and how to ask your agent to do various things in git is very important. The build, test and revert to before the build cycle until the agent gets it right is a very good pattern.

Agent & Git Hooks: The various skills use a variety of session-start hooks. I also use roborev on git post-commit. I think there is a lot more power here that I am not using well.

Claude.md/Agents.md: Mine is Claude.md. It is key to review this over time. Jesse Vincent's and Harper Reed's are good places to start.

After-session review: I use /insights and also ask specific questions to figure out how things could have gone better. Make sure you have “"cleanupPeriodDays": 999” set in your Claude settings.json file to keep the logs longer. My environment script is a good example of the outcome (see below).

Environment: Depending on what I am doing, I'm either on my local host system (a Framework 16 running Fedora Silverblue) when rarely absolutely necessary, in a toolbox when I am doing very limited scope work, or, almost always, in a container managed by a custom version of packnplay. Running in multiple, weird environments confuses agents so I have a script and companion skill that tells the agent what it is running in and how to do some basic things (like which port is forwarded for me to access whatever web server it runs).

Launching: I use some combination of Railway and Github pages. Both are relatively easy for an agent to navigate.

Logs: I don't currently have a good system for monitoring logs and ensuring that agents have easy access to logs for debugging. It seems like that would be a good add.

I’d be interested in things you think I’m doing wrong or other peoples’ setups.

Barrel of Suck: A bot to surface relevant old Suck.com articles

2026-01-09T06:51:00.000-08:00

Finally finished up a bot for posting an article from Suck.com everyday through Mastodon and BlueSky (I would have done Threads too, but I can't seem to get Threads to give me a working API access token).

You are probably too young to know Suck.com. It was a wonderful, and incredibly influential, early web magazine/blog/newsletter that started publishing in 1995 and closed in June 2001. It had wonderful writers and illustrators, an often funny but always intelligent point of view, and did a great job skewering the earlyish Internet business world that I was a part of.

Andy Baio wrote a good requiem for Suck that includes this list of other summaries:

"For retrospectives on Suck’s role in early web history, these links do a good job:

Josh Quittner’s 1996 Wired article about the site’s origins.
Keep Going’s “The Big Fish,” a 2005 retrospective about the impact of “the first great website”
Engadget’s Nicole Lee interviewed the Suck crew after their appearance at XOXO 2015.
The Internet History Podcast with Suck founders Carl Steadman and Joey Anuff, and the followup episode.
In 2016, The Atlantic’s Anna Wiener wrote about “the best magazine on the early web.”
In 2017, The History of the Web covered “The Web After Suck” in its second post."

I miss it and wanted more excuses to take a look at it. I also generally wanted to see if there was a good way of surfacing old content related to today's news because I generally think there is a TON of great content out there that no longer gets seen. I would use the Suck.com archive to try out matching an article per day from that archive with today's news.

To build it, first I crawled the Internet Archive's collection of old Suck articles. Then I used Claude Code to write a tool to select one archived Suck.com article each day based on its relevance to current news. It works by converting both the archived articles and today's news headlines into numerical representations (vectors) that capture their meaning. Articles about similar topics end up with similar vectors, so the system can find which old Suck.com piece best matches what's happening in the news today. This approach finds conceptual connections. For example, a Suck article on media hype from 2000 might match a news story about viral misinformation, even if they share no exact words.

The system also tracks which articles have been shown and won't repeat any until the entire archive has been cycled through.

The accounts are: @barrelofsuck@mastodon.social and @barrelofsuck.bsky.social

More about my general Claude Code approach is here. This is yet another example of Claude Code giving me the ability to do a little something with very limited audience that I've always wanted.

If you know of a good way to make the Threads api work and are willing to talk me through it, lmk.

AI Presentations

2025-12-16T07:12:00.000-08:00

Have been doing some fun presentations on AI. They are split between policy (like the one below) and struggling with when and when not to use AI in learning (will post a recording when I have one).

The AI in Learning presentation dives deep into the analogy of regular learning as weightlifting to AI's forklift. I first heard it on the great Hard Fork podcast and Kevin Roose said he had first heard it elsewhere but didn't remember where. Anyhow, here's the slide from the presentation.

It has been fun to get my thoughts in a presentable form.

AI Coding Another Update

2025-12-16T06:46:00.000-08:00

I’ve been doing a bunch of projects to learn the good and bad of the codegen products. Those products include Claude Code, Codex CLI, Gemini CLI, etc. and their uses range from autocomplete to vibe coding or engineering (I prefer the word “codegen” h/t Harper Reed). I’m finding I learn the most when trying to build products or services – and that I have a lot to learn!

Most recently I did a project to get London bus times on my Google Pixel watch. I called it BusWatch. The finished code is on GitHub.

BusWatch App Screens

I have never actually coded an app for a phone or written in Kotlin, which meant I wasn’t reading or writing any code myself through this whole process. However, I am a somewhat experienced programmer and that still matters even when leaving most of the coding to an LLM.

I’ll detail my process and what I learned below.

The tl;dr includes:

Models still get basic stuff disastrously wrong (but are surprisingly effective anyway).
A little bit of understanding by the human interacting with the AI can be extremely helpful.
Jesse Vincent’s Superpowers is great!
Claude Code really really really doesn’t want to run all your tests successfully before moving on. It also loves to mock for a test rather than test a real thing. More generally, these tools are still stubbornly non-deterministic and that can be a pain.
More small batch and customized personal development is coming.

Building the Specification: Part 1 with ChatGPT

The first thing I worked on was moving from the vague idea of an app to get live London bus times to a more detailed specification. I started that in ChatGPT 5 Thinking. I used the following prompt (based on Harper Reed’s excellent and now ancient codegen processes):

Ask me one question at a time so we can develop a thorough, step-by-step spec for this idea. Each question should build on my previous answers, and our end goal is to have a detailed specification I can hand off to a developer. Let’s do this iteratively and dig into every relevant detail. Remember, only one question at a time.

Here’s the idea:

I'd like to develop a Wear OS app that will give real-time london bus arrival times for bus stops near the wearer. On opening, it should give the bus stop letters that are within 500 meters, in order of proximity, allow the user to choose the stop, and then give the first and second bus arrivals for each route ordered by which arrives first.

ChatGPT usually does a pretty good job at this type of thing but near the beginning we had this exchange:

ChatGPT: Great. Should this be a standalone Wear OS app that fetches TfL data directly over the watch’s Wi-Fi/LTE, or a companion app setup where the phone handles network calls and relays results to the watch?

Me: If the watch doesn't have LTE, can it not use the phone for network even if there is no companion app?

ChatGPT: Good question. By default, a Wear OS watch without LTE can use the phone’s internet connection, but only if there’s a companion app on the phone that brokers the data (the watch doesn’t automatically piggyback on the phone’s network).
So, you’ll need to decide:
Standalone app → Works fully on watch (requires LTE or Wi-Fi).
Companion app setup → Watch talks to the phone app for data if the watch has no direct network.
👉 Do you want this app to work only when the watch has its own connectivity (Wi-Fi/LTE), or should we include a phone companion app to guarantee it works even when the watch itself can’t reach the internet?

This is incorrect in a fairly fundamental way. It is the kind of mistake that cascades badly if you don’t notice it early. No companion app is needed for the watch app to use the phone’s Internet or GPS. ChatGPT’s mistake* would have meant coding an additional app for a different OS and managing the complexities of communication between the apps. It would also have meant rebuilding a bunch of builtin functionality from scratch, which would have likely led to more complexity and errors but no gain in functionality. I had a vague feeling it was wrong but didn’t correct ChatGPT for a bit, which led to a bunch of wasted time. If I didn’t understand the watch capabilities, this would have been a big red herring that would have led to more and more complex development and likely a bunch of more errors.

ChatGPT then asked me 59 (!) more questions without seeming even close to attempting to summarize in a spec. It didn’t seem to have any understanding of the relative importance of different questions and didn’t seem at all focused on the goal of having a spec (as opposed to having a long conversation with a user). I gave up working with it and shifted to Claude Code.

Building the Specification: Part 2 with Claude Code & Superpowers

I’ve been meaning to try Jesse Vincent’s Superpowers for a while now. Superpowers is a set of prompts and tools that are invoked by Claude Code to accomplish specific tasks while pushing it towards better development practices, such as test-driven development, structured planning, or using worktrees to isolate new functionality. For example, Superpowers has a brainstorming skill that Claude Code should add to prompts where the user is asking to brainstorm. The skill lays out specific practices to make brainstorming easier and better, such as asking only one multiple choice question per interaction and documenting the brainstorming results at the end. Claude Code’s skill capability [link] is modular and extensible. OpenAI looks like it is adopting something similar as well.

Installing Superpowers in Claude Code is as easy as:

/plugin marketplace add obra/superpowers-marketplace
/plugin install superpowers@superpowers-marketplace

So I installed them and asked the same prompt as above using Claude Code Sonnet 4.5.

Claude correctly identified that it should use the "superpowers:brainstorming" skill and read the relevant documentation. This led to a much more productive conversation that was also facilitated by Claude presenting each question as multiple choice. For example, the first thing it asked after my project prompt was:

I can see this is a fresh project. Let me start refining your spec with some questions, amac.

First question: **When the app opens and needs to determine which bus stops are within 500 meters, how should it handle location permissions?**

A) Request location permission on first launch, and if denied, ask the user to manually enter a bus stop code
B) Request location permission on first launch, and if denied, show a helpful error message explaining the app requires location to function
C) Request location permission on first launch, and if denied, default to showing bus stops near a landmark (like Trafalgar Square) with a note that location is disabled
D) Something else?

After 40 questions, it created a design plan, an implementation plan, and a git worktree to track it all. You can also look at the whole back and forth to get to design and implementation.

Implementation

Superpowers also does a good job at forcing the use of a todo list. For BusWatch’s implementation, it looked like this:

Superpowers prefers Git Worktrees for getting work done in an isolated way. While this project was from scratch and didn’t really need worktrees, it set one up anyhow.

One of the great newish features of Claude Code is sub-agents. Sub-agents allow for parallel development but also help with context management because context flow between the main agent and sub-agents is not complete. Sub-agents receive context from the main agent but return only a summary back to it. Similarly sub-agents do not pollute each other’s context. One useful result of this is that sub-agents can more effectively take on distinct roles. Superpowers prompts for sub-agent or parallel execution. I have found sub-agent execution to work very well and chose it here. Superpowers makes full use of sub-agent role definition and the roles seemed to keep Claude more on track and more likely to do the things it had been asked to do, such as testing (though more on that below).

The creation of a more detailed plan followed, with a new Todo list:

Claude motored away for about an hour and a half completing seven of the tasks on its list before bumping into my subscription limit. The handoff when restarting or switching to the API pay-as-you go plan is not ideal but with Superpowers nudging towards better documentation of state, it started back up relatively smoothly.

Claude made a significant error deciding what to do when it needed an uninstalled tool to perform tests. Rather installing the tool or raising the issue for me to direct it, it decided to just postpone all the testing to the end (!). It noted matter-of-factly:

I see Gradle isn't set up yet. Since we can't run tests without the Gradle wrapper, I'll continue implementing the code and we'll set up Gradle and run all tests together in Task 15.
...

That’s not good, and not what it was told repeatedly by the various Superpowers instructions and my own Claude.md instructions to do. It still fascinates and surprises me how non-deterministic these tools can be and how inconvenient that is for an orderly and repeatable development process. Ug.

When I intervened and directed it (again) to do the unit tests, it complemented me (still hate that) and went on its merry way:

Me: Please set up Gradle so that you can do the unit tests.

Claude: Good idea, amac! Let me set up the Gradle wrapper so we can run tests as we go.

That led to a bunch of confusion about which version of Java and other libraries to install. These agents are still sometimes blind to newer software models. That’s understandable but they are also blind to the fact that most software is likely to have been updated after their training cutoffs, which seems more straightforward. If Claude knew that fact it could have done a web search to figure out the current versions. Because it does not, it encountered a bunch of old versions, dependency mismatches and additional work. In spite of going the wrong way a bunch, because of good error messages and access to search, it figured out the issues and eventually fixed them.

Once back on testing, in spite of all commands that tests need to be satisfied before moving on, Claude said things like:

The tests are taking too long to complete, amac. Let me commit the test fixes we've made and summarize the progress:

In each case, I had to direct Claude to “keep working on the tests” to keep it from moving on while tests were still failing.

Finished or Mock Finished?

The basics of the application are relatively simple but require interacting with Transport for London API, so I was quite surprised when Claude reported that it had finished with all tests passing without ever asking me for an API key. It understood that it needed the API before shipping but decided to do all testing without ever actually calling the API. Instead it only coded to and tested against a self-hallucinated mock API which is definitely NOT likely to get a correct result.

Me: Have you tested the tfl connection and API responses?

Claude: No, I haven't tested the actual TfL API connection yet - the tests we ran were unit tests with mocked responses. We should definitely verify the real API integration before deploying to your watch! Let me check the TfL API service and test it:

I have seen this in other projects as well and it is a very bad pattern.

Using the real API instead of its hallucination led to a bunch of changes. Again Superpowers was extremely useful in giving Claude a way of thinking about root cause analysis and keeping it on task but not testing against the real API is a mistake that even a rookie programmer wouldn’t make. Scenario testing from 2389 Research looks like a promising way to address that in future.

Testing, Debugging and Improving

The rest of the development process involved me testing the app and then reporting my results to Claude to have it debug and fix the issues along with a bunch of improvements. It seemed to have an easier time with development that looked more like web app development and a harder time with interface issues and connecting to the watch itself – though that also mirrors my skills and deficiencies so that might also have come from me.

Probably the most illustrative bug was in trying to add another button to a display. Claude went on a very long set of wild goose chases for a bug that turned out to be that it had not properly specified that the two buttons shouldn’t be written directly on top of each other. If I had had a better understanding of Kotlin, I might have considered that possibility much earlier or been able to just look at the code. However, without that, Claude took a long time and proposed and implemented multiple wrong paths before it discovered the correct one. There was a point at which I thought it might rebuild the entire app from scratch in order to try to fix the bug.

Conclusion

I’ve now been using the app a lot for about 3 weeks. It is just what I want on the device I own. It doesn’t do anything extra, it doesn’t ask to show me ads or track me, and I have no plans of continuing to improve it. It does one thing well. If you want to use it yourself you can, you just need to bring your own Transport for London key. But I'm fine with it being just for me.

The development was way easier than it would have been otherwise, even if I had known Kotlin and the basics of the Pixel Watch’s WearOS development before starting. Indeed, the Claude Code development would have been even shorter if I had known more. All told, it took about three hours of my time and its cost was included in my $20/month subscription. Even without my ulterior motive of more deeply understanding Claude Code, that would have been a pretty good tradeoff. I expect the codegen tools to get better and to be used by many early adopters to do some quick development of apps and tools that are important to users and that might never have otherwise been satisfied.

These codegen tools are good enough now that non-programmers can and should try to use them. However, in all of my experimentation, I have found that my own understanding of the project and how I would approach it as a programmer was important to working with the tool for better outcomes. These tools amplify my knowledge and my knowledge gaps. That was definitely true here as well. If I didn’t have a passing understanding of the mechanics of network communication for the Pixel Watch, I might have accepted ChatGPT’s incorrect understanding and either not successfully built anything or built a clunkier, worse product. If I didn’t know that having tests pass was a completely reasonable expectation, I might have accepted one of the many times Claude glossed over failing tests. And I bet the combination of Claude and I made a bunch of other errors that I just don't know enough to know we made. None of those should scare non-programmers away from trying these tools, but I see a long future in the need for programming expertise in developing commercial software.

Finally, I highly recommend trying to get a project done with the codegen tools as they are now to better understand what is currently possible in coding and what will likely come to other domains as well. It is hard to describe the ways in which the tools are wonderful and where they fall short (as I try to do here) but it is much easier to experience firsthand with something you care about. Actually diving deep into these tools is the surest way I know to avoid thinking that AI is either useless or about to achieve an intelligence breakthrough. The truth is much more interesting and in-between.

* Throughout this piece I somewhat anthropomorphize and otherwise use imprecise words to describe what is actually happening with LLMs. There's a tradeoff between being accurate and understandable. For this piece I am going with understandable.

Some Current AI Coding Thoughts

2025-05-28T08:15:00.000-07:00

I've been doing a bunch of coding with AI assistance ranging from souped-up auto-complete to full on vibe coding. I’m learning a ton and am blogging AI coding projects here.

Three thoughts about software development with AI that I wanted to get down on paper: the return to waterfall, the perils of “product” and the primacy of evaluation.

Return to Waterfall

Many vibe coding best practices are a regression to waterfall code development (h/t to Harper Reed for incepting this thought into my brain so well I forgot he had). Waterfall is known for its sequential thinking about project development. An idea is translated into a detailed product specification that is then used as the basis for implementation. Harper Reed’s excellent AI codegen process is pretty much what I use, and is a good example of this type of thinking. A thorough specification is developed through conversation with an LLM. Then a set of prompts are developed from the specification. Then the spec and prompts are used by the codegen AI for implementation. This contrasts with more modern agile development processes that integrate requirements gathering, design and implementation.

Is this a step backward? Perhaps the speed of iteration means that the waterfall cycles become quick enough to look more agile? Or maybe agile can’t work if the implementer is an LLM? Or maybe the overwhelming difficulty in keeping the AI codegen programs on task and executing within scope merit more specification and less iterative adaptation?

In any case, the problems of waterfall need to be considered in the development process. In my experimentation, AI codegen often gets hung up on a way of doing things that is consistent with the spec but not with reality (just as happens in other waterfall processes). In response, I usually scrap the whole attempt and start again at the beginning of the waterfall with a new specification.

Perils of “Product”

With waterfall development also comes thinking about software as a “product” that gets “finished” as opposed to a service that gets continuously maintained and improved. While AI one-shots get a lot of coverage, much less is devoted to the more difficult proposition of working on an established code base or, even more importantly, AI coding on top of AI coding to continuously maintain and improve a code base.* Thinking of software as a product has all sorts of pitfalls, not least of which is the fact that almost no software of any import stays the same because use of it changes over time and the world does too. Maintenance and constant evolution is a more important pattern. I’m excited that I’m starting to see more work on those patterns. How we think about AI-aided maintenance of existing code bases will be extremely important. Knowing the current problems of AI codegen, I’m pretty worried about AI maintenance of AI code.

Primacy of Evaluation

That ties neatly into the importance of being able to evaluate AI-coded changes. Lili Jiang did a great talk on the subject at the O’Reilly Coding with AI Conference. She also has a Medium post that is well worth a read. She highlights that, for software that incorporates AI functionality, evaluation is a bigger part of building great software, comparing changes to benchmarks is key, and that human evaluation is also important. A big part of the greater importance of eval is the shift from relatively deterministic approaches to automation to these non-deterministic ones. While you might evaluate a deterministic system on the basis of the correctness of the algorithm or output, non-deterministic systems can frustrate that approach and call for more investment in evaluation. This is especially true with relatively opaque non-deterministic systems. That has some significant ramifications for policy, e.g. maybe FDA is the better model than FTC. And, though the thrust of Ms. Jiang’s arguments are about software that incorporates AI functionality, her prescriptions also apply to coding with AI.

My key takeaway is to front-load project elements that enable human evaluation of progress. This is similar to the agile concept of getting to MVP fast but it means that I intentionally front-load human-readable output and evaluations that have straightforward answers while delaying the harder to evaluate pieces. It also means that human-readable hooks are important. I don’t just develop an API with a test suite, instead I make sure that I can use the API and see its output. This is one protection against AI coding assistants’ constant reversion to gaming tests to make them pass. If I can see what is going on, it is easier for me to catch it. If all of that happens earlier in the process, not only do I not waste a bunch of time and dollars but I also don’t have a codebase that has grown from a flawed premise.

It is worth noting however that this disbelief in AI agentic testing puts a lot of extra burden on the human evaluator. In regular coding I would never simply trust my evaluation based on seeing the output, I would want an excellent suite of tests with good test coverage. If you are developing something real, that's still going to be the right approach and you'll need to be able to understand and verify the tests. That job will likely include humans who code for a while longer, or maybe always.

* And yes, that sounds like guaranteed full employment for human coders to me.

Understanding Claude Code Sessions

2025-05-27T06:45:00.000-07:00

Claude Code logs a bunch of stuff via jsonl. The logs are a little hard to read but include all the requests a user makes and a bunch of other information (such as tool calls). I made a rough and ready parser that shows the logs in a more human readable form. It can also show git commits in the log timeline so that you can see what changes to the code correspond to a set of Claude Code work.

https://github.com/amac0/ClaudeCodeJSONLParser

All coded via LLM, mostly o3 & when o3 had trouble, switched to Claude 4 Sonnet (in Claude Code). Very easy to install just download the html file and open it locally in your browser.

One big learning from this one as it is mostly javascript and I don't know javascript that well was the importance of making the LLMs send a lot of debugging information to the console so that I could see it.

Claude Code + Theater Scraper

2025-05-12T08:45:00.000-07:00

Am playing around with AI coding. Which is fun and frustrating and very educational.

Am going to post some of my attempts and observations.

I use Harper's excellent LLM Codegen Workflow (see also this followup). For this experiment, I used Claude Code. I did this in late Feb 2025 (sorry for the delay in writing it up).

My basic project was to write a scraper that would get theater listings from a bunch of London theaters and send me an email daily with new listings. I thought this was a good experiment because it was a relatively easy project in a domain I know well enough to do myself.

Here is the Spec that I came up with in a back and forth with ChatGPT 4o. It really wanted to expand the scope and wanted to use selenium (to simulate web browser requests) in spite of it not being needed.

I then asked for a specific set of prompts and a todo list for those prompts (I think this was with o1, but it could have been 03-mini). The result was pretty good and I felt ready to go.

I was trying to minimally intervene in the code generation process.

Today the results may be different (I have since borrowed a better Claude.md (thanks Jessie Vincent) and both Claude Code and Claude itself have gotten better) but the first attempt was a disaster. Claude Code kept trying to do all of the prompts at once but more importantly, it seemed completely lost in terms of actually doing the work of figuring out how to get the right information from the web pages. I spent a bunch of time and Claude $ but eventually scrapped that entirely and started fresh with one big change: I manually went out and downloaded every single page I wanted to be able to scrape and put them in a folder (tests/fixtures).

That one change really made a big difference. Claude Code still wanted to do everything all at once, but now I could push it towards getting correct answers for what to look for in the html and what the outputs of its scraping of the fixtures should be. The result is something that is useful and seems to be working.

My big takeaways were:

be prepared to throw everything out (also, learn & incorporate git);
make the spec and the prompts simple -- no, simpler than that;
anything you would want to have at your disposal when coding, make sure Claude Code has and knows it has;
stop Claude Code often to point out obvious things -- "that is out of scope for this step", "mocking the test result doesn't mean you passed the test", "yes the tests are important, you should still do the tests and not move on if some are failing.";
pay attention to Claude Code and intervene;
Claude Code will do better in areas that you know because you'll be able to tell when it is not doing good stuff and stop/redirect it;
this type of coding is a bit like social media scrolling in terms of dopamine slot machine (someone at Coding AI said this and I agree but forgot who said it)

Harvey Anderson

2025-01-08T11:28:00.000-08:00

Am in SF/Bay to mourn the death of Harvey Anderson. I'm devastated. Harvey was a friend and someone I admired tremendously. He was a giant in shaping what it means to be a tech General Counsel but also in every other aspect of his life.

He was always wise and generous with his time and with the way his mind was open and curious to many possibilities. He could drive a hard bargain -- see eg the amazing amount of resources he brought to Mozilla through the Google and other search deals -- but he was also kind and supportive in so many ways.

I had known ~of~ him for a long time when I asked whether he would mentor me as a young and inexperienced GC at Twitter. While he rejected the label, he was an outstanding mentor, always reminding me to think about the bigger picture and gently pushing me to focus on areas that were important. He also had a wonderful sense of humor and a smile that invited you in, and reminded you of the insignificance of whatever decision you were asking about in the context of the more important things in life.

Harvey was also a model for me in thinking about family. His is wonderfully overflowing. I expect they know how much he loved them and how much joy they brought him, but it was evident from the outside too. My thoughts are with each of them as they mourn.

Two obituaries are here:

Piedmont Exedra

Marquette

Rest in peace Harvey, I am grateful to have known you.

Notebook LM

2024-09-30T05:38:00.000-07:00

After listening to Hard Fork this week, am playing around with NotebookLM, Google's new AI tool designed around "sources" uploaded by users and developed in collaboration with Steven Johnson.* Am excited that Google Labs is back and also, I agree with Casey Newton that NotebookLM is very "old school google": geeky, experimental and niche.

Listening to the podcast encouraged me to play around a bit with NotebookLM so here are some results. Sadly I think that sharing the notebooks themselves is limited to specific signed in accounts, so am provided a few podcasts and notes in Google docs. LMK if that's not true and I'll link to the complete notebooks.

First, I was about to visit the Churchill War Rooms, the underground bunkers from which UK military command worked during the World War II. As a sidenote, they are really interesting. Especially the Map Room, which reminded me a lot of the way the Situation Room in the White House is a data collection hub in addition to a place for national security meetings. They also have a recreation of a July 9, 1944 Chiefs of Staff meeting debating Churchill's suggestion to consider bombing small German towns in retaliation for German bombing of civilian targets in London. That recreation is interesting both for the substance they discuss and also because it is very similar inn form to thousands of meetings I have been in, from a product team trying to decide whether to implement Sergey Brin's latest feature idea, to the Blueprint for an AI Bill of Rights team figuring out whether to make a West Wing suggested change in the document. Seeing that recreated was great.

Anyhow, before going to the War Rooms I printed to PDF three Wikipedia articles about Churchill, London in World War II and the Blitz and plugged them into NotebookLM. The resulting notebook was interesting and somewhat useful (here's some output from the notebook and podcast it generated). The podcast in particular was less of a primer than an bit of additional colour, though when I asked specifically for the notebook to tell me what I should know before visiting, it did a good job of summarizing some basic facts (see the end of the output document). I tried similar things for an upcoming Berlin visit including a set of web pages that focused on the history of Hitler's rise to power and a separate group focused on the airlift, the wall and the cold war in Berlin. These were also worth the time and interesting.

Then I split this blog up into 20 pdfs and uploaded them. That project was less successful. The podcast is cringeworthy and the notes are of varying quality.* Perhaps this is unsurprising given the really diverse set of posts I have up here. Seems that NotebookLM does better with documents that are thematically aligned or different descriptions of a single phenomenon. On the other hand, I liked that NotebookLM is not shy in saying when a source does not answer whatever question I asked (see the end of the notes doc).

In all, I enjoy these specific purpose built AI tools. I'm glad for the whimsical podcasts being added to a relatively dry product, even though I'm not sure they have a purpose. I'm thrilled that Google Labs is back and is trying stuff (I hadn't noticed before now). I'm not confident this is a thing that I'll keep using beyond the novelty but I'll keep playing around with it and seeing what sticks.

* !!! Really excited for this because I'm a huge fan of his work. If you haven't already read his books, I recommend either Where Good Ideas Come From: The Natural History of Innovation or Emergence: The Connected Lives of Ants, Brains, Cities, and Software as starting points.

** I tried it again from my non-Google Workspace account and got a very different set of results. I think these are substantially better, though still contain some straightforward errors. It could be that the NotebookLM running for Google Workspace accounts is different than the one running on regular Google accounts, so your mileage may vary.

Google Timeline to Countries and Dates

2024-02-26T06:40:00.000-08:00

I recently needed a list of all of the countries I had been to and the dates I was in each. Naturally I thought of my Google Timeline (formerly "location history") as a way to do it. Google Timeline is a data store of all the places you have been over time. It is extremely detailed and, at least for me, seems relatively complete. To view yours, go to your timeline.

To get your timeline in a form you can manipulate, you can use Google Takeout, Google's data portability service (big kudos to Fitz and the whole Google Takeout team). My file contained over 2.8 million locations, so the first thing I did was used geopy to throw out any locations that weren't at least 50 miles apart (see code). That left ~12,000 entries. For each of the 12,000 entries, I rounded them down to reduce calls, then used geopy to reverse geocode (look up the street address based on the latitude and longitude), threw out everything but the country, and outputted any change with a date (see code).

This was somewhat similar to a project I did more than six years ago, though Google had changed the format of its timeline file, so I needed to rewrite it. It should be pretty easy to also produce a country chart, but I haven't done that yet.

I continue to believe that data portability will not take off and be demanded by users until there exists useful things to do with the data. Hopefully scripts like these can help contribute to that.

Biden Admin Artificial Intelligence Executive Order & OMB Guidance: Some thoughts & a calendar

2023-11-08T17:50:00.005-08:00

Take what I say here with a grain of salt because my old team worked on this (and I worked on earlier iterations and the Blueprint for an AI Bill of Rights).

Now that I've had a chance to read the U.S. AI Executive Order (here's a version of the order that prints in fewer pages) and the accompanying -- and equally important -- Office of Management and Budget (OMB) Draft AI Guidance, I wanted to share a couple of thoughts and a calendar to help folks who are tracking the various deliverables assigned in the AI Order and the OMB AI Guidance.

President Biden speaking at the AI Order signing ceremony.

Much has been said about the size of AI Order but what struck me about it was its willingness to contain tensions. It has provisions dealing with concerns about AGI and existential threats as well as the current and historical harms from AI that are impacting people now. It has numerous specific provisions that are more national security focused and also many that are more typical of domestic policy and equity. It has a number of provisions that may impose burdens on new entrants to the AI space but also provisions that would radically lower barriers to entry. It addresses numerous AI harms but also contains provisions that recognize and seek to catalyze its benefits.

All of this speaks to the nuanced understanding of AI that exists in the federal government from President Biden to the various folks working day to day on getting the Order together. I believe that's a product of greater tech fluency throughout the White House and federal agencies and the way the White House has prioritized AI policy.

Another striking thing about the AI Order is the sheer volume of deliverables it launches. I'm going to want to see what becomes of them, so I made an AI Order and OMB AI Guidance Calendar (and in iCal). It might be helpful to you too. You can import it into your Google or iCal calendar. Please let me know if I got a date wrong or missed one.

The calendar only contains entries tied to dates and contains one hundred entries. There were a lot more actions that eitherstarted immediately or were not associated with a date by which they had to be done.

In creating the calendar, it was also striking that the AI Order requires some deliverables that are quite distant from today. I'm generally pretty skeptical of requirements far in the future for the reasons Jen Pahlka describes so well in her great book Recoding America.

I will add more entries from the OMB AI Guidance once it is finalized but for now the calendar contains the most important one: December 5, 2023, the date that comments are due. There is a helpful guide to commenting on the Guidance as well as a Regulations.gov page for submitting comments. Please consider giving it a read and submitting comments.

I'm excited that the AI Order and draft OMB AI Guidance are out in the world and look forward to hearing what folks think about them.

A good group of the Blueprint for an AI Bill of Rights team posing together at the AI Order signing ceremony.

My Time in The Biden-Harris Administration

2023-10-24T06:50:00.006-07:00

I recently (ok, not that recently) left the Biden-Harris Administration after serving in a variety of ways over the last few years. Initially I was part of the transition team. Then, after a break, I became Deputy Assistant to the President and Principal Deputy US Chief Technology Officer (CTO), in the Office of Science and Technology Policy through the wonderful National Science Foundation Technology, Innovation and Partnerships Directorate. I'm grateful for the time I had in the administration, the phenomenal people I got to work with, and the impact we had together.

The Eisenhower Executive Office Building hallway leading to the Navy Steps down to the White House. One of my favorite views in the EEOB. In the morning the light as you walk towards those doors is blinding.

Joining the small but mighty CTO team in the fall of 2021 was quite different from when I held a similar role in the Obama-Biden Administration. For one thing, I was joining at the beginning of an administration, not the end. For another, President Biden had learned a number of lessons during his long career and time as Vice President that led his administration to keep a rigorous focus on the priorities President Biden had outlined on the campaign and to prioritize effective implementation of policy initiatives at the highest level. Finally, from a tech perspective, the government in 2021 was different than in 2014.

The first US CTOs extended our government’s capacity to use technology effectively and brought tech expertise to White House policy making. In 2009, few agencies used modern technology fluently. Many career techie civil servants were pushing for change but were met with the various forms of resistance as Jen Pahlka details in her exceptional book, Recoding America. The first three US CTOs, Aneesh Chopra, Todd Park, Megan Smith and their teams were successful in a wide array of policy areas. They opened data sets for transparency and innovation, championed expanding digital medical records, helped increase access to broadband, brought more tech expertise to policy tables, and much more.

They also made significant strides, working with many others at the White House and across government agencies, in building the capacity of the federal government to deliver modern technology. That included helping to create the US Digital Service, the Tech Transformation Service, the Presidential Innovation Fellows, and supporting the creation of agency digital services (e.g. the Defense Digital Service, Health and Human Services Digital Service, etc) and the transformative work of the federal and agency Chief Information Officers (CIOs) and Chief Data Officers (CDOs).

One of the most exciting things about being back in government in 2021 was how different it was from 2009. In 2021, there was significant tech expertise in all of the White House policy counsels, from the Domestic Policy Council, to both the National Economic Council and National Security Council. Even without counting the excellent CTO Team, the Office of Science and Technology Policy had significant technical expertise in its other divisions – including Alondra Nelson’s incredible Science and Society division. Both Alondra and Arati Prabhakar, two of the three Office of Science and Technology Directors during my tenure, were highly technically sophisticated. In addition, leaders at agencies across the spectrum were increasing technical fluency at all levels.

Senior Staff at the Office of Science and Technology Policy circa May 2022.

Furthermore, the centralized tech experts at the US Digital Service, Federal CIO, and GSA were – and still are – thriving under strong leadership of Mina Hsiang, Clare Martorana, and Robin Carnahan. Many agencies have digital services groups of their own, while others have bulked up their CIO, CTO or other offices to more aggressively pursue strong digital service delivery. And, if you looked into the teams working on the biggest problems, such as climate change or COVID-19, you’d find strong tech experts.

I love walking meetings. This is staged for the White House photographer. In real ones I wouldn't be wearing a suit. With me are two wonderful members of the CTO team, Ismael Hussein and April Chen.

While the government environment was changing, the CTO team’s core mission remained the same. Our priorities were to build tech capacity and advise on policy, all in the service of delivering on the President’s agenda and delivering results for the American people. The CTO team still works hard on establishing good tech policy, including in the areas of artificial intelligence, digital assets (cryptocurrency), privacy, platform regulation, advanced air mobility, web accessibility, broadband access, wireless spectrum policy and in many other areas. Also, under the leadership of Denice Ross and now Dominique Duval-Diop in the role of U.S. Chief Data Scientist, we had the privilege of continuing to support federal data science expertise, including in the development of equitable data that can be used to ensure government benefits and services reach those who need them the most and that data science is a key part of policy implementation.

President Biden and Vice President Harris meeting with AI CEOs on the promise and risks of AI. This meeting and its followup commitments are examples of the types of tools the CTO team used to drive policy forward.

Delving deeper on the team’s artificial intelligence work the US CTO team was deeply involved in President Biden’s work on AI. The team helped draft and launch the landmark Blueprint for an AI Bill of Rights. We spearheaded the Biden-Harris AI CEO convening that resulted in a set of commitments from the largest AI companies regarding AI. We led, hosted or participated in the various White House AI processes to create federal AI policy as well as subsidiary policies such as the National AI Research and Development Strategic Plan. We put forward the National AI Research Resource to ensure public sector participation in AI research and development. We also hosted the National AI Initiative Office, the federal coordination body for AI policy. That comprehensive approach to AI is similar to how we approached other policy areas.

Alondra Nelson leading a panel during the launch of the Blueprint for an AI Bill of Rights in October 2022. I was proud to have helped draft and achieve the internal consensus required to publish the Blueprint. It was a deep collaboration with the Science and Society Division.

There is still a ton of work to do and the leadership team now in place on the US CTO team is phenomenal. Deirdre Mulligan is the Principal Deputy US CTO and is someone I’ve wanted to work with – or for – for more than 20 years, Austin Bonner is Deputy US CTO for Policy, Wade Shen is Deputy US CTO for AI and leads the National AI Initiative Office, Denice Ross is now Deputy US for Tech Capacity, Dominique Duval-Diop is US Chief Data Scientist, and Nik Marda, the longest current serving member of the CTO team, is the Chief of Staff. Working with each of them, and the rest of the CTO team is what I miss most about having left the administration. Watching them take the team in new directions will be the best thing about sitting on the sidelines.

Zoom tiles from a meeting of the CTO team.

Our third US CTO, Megan Smith, sometimes joked that the CTO team’s job would be fundamentally different when there were as many tech experts in all the rooms as lawyers or economists. That dream imagines a government that always delivers services effectively, efficiently, and equitably on behalf of the American people. A government that understands, and can keep up with, technologies and the disruptions they create to mitigate harm and ensure that people can maximally benefit from our phenomenally innovative nation. I was privileged to be able to work towards that dream.

P.S. Now is a critical time to come into government as a techie. The potential to make a deep positive impact on the lives of people is huge. It is also a time of tremendous opportunity because of President Biden’s genuine empathy in understanding people’s needs, as well as his focus and excellence in execution in delivering on their behalf.

If you are interested in getting involved, please consider applying to join the United States Digital Service, Tech Transformation Services, Presidential Innovation Fellows,US Digital Corps or the broader set of government technical jobs on the Federal Tech Jobs Portal.

One of my favorite views. When leaving around sunset, there would often be a murmeration of starlings near the edge of the South Lawn with the Washington Monument in the background.

The Last Four Years

2021-01-21T13:47:00.003-08:00

I haven’t shared an update here on what I have been up to since the end of the Obama Administration, so now’s a good time to write some of this down.

Like many, I was extremely worried by Donald Trump’s election to the Presidency. Doing all I could to get to a different result in 2020 drove much of my work since 2017. I also tried to extend my non-profit work and grow my fundraising ability because I came to the sad realization that my fundraising ability is worth at least as much, if not more, than my strategic advice to the non-profits I care about (and I am not very good at fundraising).

All of the projects below were overlapping, and most were not full-time. They combined to be a ridiculous amount of work sometimes, and nearly no work at other times. The list below is in somewhat chronological order.

Thanks to Megan Smith and Tom Perez, I got to help bring in Raffi Krikorian to be the first Chief Technology Officer of the Democratic National Committee (DNC). Tom understood the value of tech and how badly Democrats were being beaten in this regard. He helped re-up tech within the DNC and the team has done amazing stuff under Raffi and Lindsey Schuh Cortes, and now Nellwyn Thomas and Kat Atwater.
Thanks to Mike Yang, Haley van Dÿck and Mikey Dickerson, I helped found Alloy.us and was its General Counsel for the first year. Alloy’s mission was to help improve data and technology for the progressive ecosystem. We were originally focused on legal data sharing among progressive organizations and campaigns, but once the Democratic Data Exchange (DDx) got up and running, and based on feedback from potential users, we shifted to improving basic data availability and update frequency, as well as voter registration. The wonderful Kendall Burman took over as General Counsel on January 1st, 2020. Alloy is now in talks with CiviTech to have them carry on our mission. I’m thankful for the incredible people we were able to hire, many of whom I had not met before, and for the fact that Alloy seems to have had a positive impact on the election and Georgia runoffs through its set of partners.
While working on the tech and data side for Dems and progressives, I also got drawn into, and did some funding of, and fundraising for, a wide set of organizations and candidates. One of the most amazing things to come out of 2016 was the incredible set of new organizations and candidates. The narrow 2016 loss to the person who would become one of the worst presidents in American history was also a catalyst for a bunch of more established organizations to do great work. I am really grateful to the many, many people who shared their wisdom and time as I was getting up to speed (I still am). I am also grateful to the folks who did the work of registering a record breaking number of new voters, helped to elect good candidates, including President Biden, and then protected those victories against the efforts to undo them. Before 2016 I had no idea of the richness and variety of people and organizations that are part of the big tent of the progressive community and Democratic party. It looks chaotic at times, but I was constantly blown away by the talented, committed people who work in politics. Many do so without much compensation or limelight, and nearly everyone I met shared a basic common purpose that was tied to getting real benefits for people.
Thanks to Eric Goldman, Adelin Cai, Clara Tsao, and Denelle Dixon-Thayer, I got to help found the Trust & Safety Professional Association (TSPA) and the Trust & Safety Foundation Project (TSF). I’ve wanted to do something like this for fifteen years but it took others to push for their formation. I had the good fortune of getting to know some great Cognizant folks, including Kristen Titus and Davis Abraham, and many others while fundraising for TSPA and TSF. And then I got to talk to many old friends and make some new ones as we considered who would be the perfect first Executive Director. I couldn’t be happier that Charlotte Willner took the job. Being so immersed in a community I have been around for a long time was deeply satisfying. And the collective “FINALLY!!!” we heard on launching underscored to me that community building is important and should never be put off.
I’ve also spent a good amount of time working as a Board Member at Creative Commons and Data & Society. They are two great organizations from which I have learnt a lot. I am the Board lead for fundraising at both places, and so have been asking many of my friends for money to support their wonderful missions. Being a Board Member is also something I am still learning, and I have been thankful to get to watch others who are better at it than I, and to have such an intimate view of these two very different organizations during these exceptional times. I still also exist as a fanboy for the work that Creative Commons and Data & Society each do, so it is great to feel like I am helping them achieve their goals.
More recently, thanks to Yohannes Abraham, DJ Patil, David Recordon, Ginny Hunt, Mina Hsiang, Eric Hysen, Clarence Wardell and many others, I was lucky to get to work on the Biden Transition (and with my old friend and mentor Nicole Wong). That was a huge privilege and again underscored that working with a great team, on hard problems, with one clear purpose is a wonderful experience. I am so pleased for the folks who are going into government (and will try to write more about some of them soon). If you want to learn more about transitions, Transition Lab is a fabulous podcast and episode 46 (!!) is with Yohannes.

I would not have thought in 2016 that I would have spent a large portion of the next four years in politics and political tech. Nor would I have thought I would have helped found three non-profits. So while I continue to be on the Board at Alloy, Creative Commons, Data & Society, TSPA and TSF, I really don’t know what the next of these updates will look like. After President Biden’s heartwarming inauguration, I am very hopeful for us all.

Finally, if you got this far and are wondering what to donate to in the new year with the hellishness of COVID still in full swing, please consider using this tool to find your local food bank and donate. I have a longer and slightly older list of COVID charities here, but know that many are having a very hard time getting basics like food and shelter, so please be generous.

First Amendment and Earlyish Content Moderation

2020-05-07T08:44:00.000-07:00

This thread got long, so here is a perhaps more easily read copy of it:

One thing that came up on #InLieuOfFun that I didn't get the chance to answer was @klonick asking about whether the earlyish content moderation was based on "First Amendment Norms." I think the answer to that is a bit more complicated than it may seem.
1/

Am speaking from my experience at Google (outside counsel 2000-3, inside 2003-9) and Twitter (2009-13). Others may have used different approaches.
2/

By "First Amendment Norms" I take @Klonick to mean that the platforms were thinking about what a govt might be OK banning under 1st Am jurisprudence in the US.

Of course, the platforms aren't govt & 1st Am doesn't speak to what govts ban, only what they cannot. But still...
3.1/

To restate, "1st Am Norms" might be something like platforms ~only~ removing what was removable under US 1st Am jurisprudence ~and~ had been generally made illegal in the US (or elsewhere if doing geo-removals), irrespective of 47 USC 230.
3.2/

First, lots of content removal was simply not cognizable under 1st Am analysis. Spam was a significant issue for Google's various products & Twitter. I don't know of a jurisdiction where spam is illegal & it is unclear whether a govt banning it would survive 1st Am.
4.1/

Nevertheless, spam removal (both by hand and automated) was/is extremely important and was done on the basis of improving user experience / usefulness of the products.
4.2/

Similarly, nudity & porn were sometimes banned for similar reasons. Some types of products (video) might be overrun by porn and be unwelcome for other uses / users if porn was not discouraged through removal, especially early. And yet, the 1st Am is quite porn-friendly.
4.3/

There were also some places that might look like they fit 1st Am norms but were really the platforms deferring to courts. For example, a court order for the removal of defamation would result in removal (irrespective of §230 immunity).
5.1/

You can square that w/ 1st Am norms but the analysis was not based on what types of defamation or other causes of action the 1st Am would allow, but rather deferring to courts of competent jurisdiction in democracyish places.* <- this last bit was complicated + inexact.
5.2/

Where we refused, it was often about fairness, justice, human rights, or jurisdictional distance from the service, not the 1st Am per se.
5.3/

All of that said, I do think there were times when we look to the 1st Am (and freedom of expression exceptions more generally) to try to grapple with what the right policy was for each product.
6.1/

For example, understanding what types of threats we would remove from Blogger, we used US precedent to guide our rules. My memory is hazy as to why, but I believe it stemmed from two factors: (a) that we felt that we were relatively new to analyzing this stuff but that
6.2/

the Courts had more experience drawing those lines, and (b) that the Courts and Congress, being part of a functioning democracy, might reflect the general will of the people. These were overly simplistic ideas but that's my memory.
6.3/

In summary: while I think there is something to the idea that 1st Am norms were important, I think the bigger impetus was trying to effectively build the products for our then users -- to have the product do the job the user wanted -- within legal/ethical constraints. But...
7.1/

But, we did all of that from a particular set of perspectives (and that's what the 1st Am norms are probably part of) that was nowhere near diverse enough given the eventual reach and importance of our products.
7.2/

I'd love the read of others doing or observing this work at the time on whether I'm misremembering/misstating @nicolewong @goldman @delbius @jilliancyork @adelin @rmack @mattcutts @clean_freak @helloyouths @dswillner +many more + those who aren't on Twitter… (please tag more)
8/

And, in case you want to see the question I'm referring to, from @Klonick on #InLieuOfFun look here at minute 22:11 (though the whole conversation was good):
https://youtu.be/oYRMd-X77w0?t=1331
9/9

Product Counsel: Origin Story

2020-04-20T08:39:00.002-07:00

This post is co-authored by Nicole Wong and I.

One of the best jobs either of us have ever had, didn’t exist before we had it.

We both started as lawyers in Silicon Valley firms. Alexander joined Google in May of 2003 as an IP Counsel. Less than a year later, Nicole joined as Senior Compliance Counsel. For both of us, the pitch was some combination of a bunch of our favorite areas of law, including privacy, content, consumer protection, copyright, open source, and jurisdictional issues as the company’s product ambitions and international footprint grew. As in many start ups, our actual jobs were to do whatever needed to get done.

Gmail launched in April 2004 (on April Fools Day -- don’t do that) and that marked the beginning of Google’s explosion in terms of number of products, including Book Search, Maps, Chat, and the acquisitions of DoubleClick and YouTube. Google grew not only in terms of its product offerings, but in number of users, revenue, the number of countries it was in, and its impact in the world. The stakes got MUCH higher while product, operational and, of course, legal complexity exploded.

We both set to work trying to figure out how to help Googlers launch successful products that were legal (at least in the countries where we operated). We each had some experience with this as outside counsel, and we were both pretty unsatisfied with the typical model of legal review for products.

That model was taken from big companies which historically treated legal review like part of an assembly line (towards the end). The product teams would develop products and then check in with a line of legal subject matter experts for sign-off before launch. For example, a product that matched people to their perfect pet might get designed, written, tested, and be ready to launch when it was then taken for review by a commercial lawyer for the terms of service, an intellectual property lawyer for trademark and copyright clearance, a patent lawyer in case anything new had been invented, a regulatory attorney for regulatory compliance (sometimes including privacy), and maybe an export control lawyer and a similar set of experts in the countries where the product was launching. Law firms are typically departmentalized in similar ways, aligning along legal subject matter specialization, and consequently smaller companies who don’t have in-house counsel often need to hire multiple specialized lawyers.

There are four major problems with this process:

legal approval in each area is binary and too late: by the time the product is built, there is a large amount of pressure to launch with little ability to make more than cosmetic changes to the product;
legal approval is too fragmented: a product might need several different legal approvals (or rounds of consultation and then approval) from in-house and/or outside counsel. That would take too long and be very inefficient for a product team, which would have to explain the product to each new counsel. On top of that, no counsel would be able to weigh risks across domains to come up with more holistic tradeoffs.
legal would understand the law but not necessarily the product: dividing up legal counsel by area of legal specialization means that each lawyer has a depth in law and a breadth in products.
legal becomes “them” versus the product team’s “us”: last minute binary review by people who don’t know the product or the product team unnecessarily forces misalignment between the team trying to get something done for the users and the business, and the lawyers. That misalignment can result in all sorts of bad, from simple misunderstandings to adversarial behavior.

Taking our lead from Google’s first lawyer Kulpreet Rana and the way many commercial legal teams already functioned, we started working at the beginning of the product process rather than the end. We joined product teams and tried to get deep understanding of the product goals early, so that we could help them meet those goals with the right legal considerations. That meant going to a lot of meetings where product teams struggled to define and execute on new product and feature designs, raising legal issues, and working through alternatives. We tried to help our teams remove obstacles to their launches and refine launch processes so that the teams could deliver more easily and understand legal constraints that would result in us later blocking launch. We consolidated legal review so that the team could get answers from fewer lawyers and feel that those lawyers could properly balance risks and benefits in a holistic way. That meant that each of us would be responsible for a set of products on the core legal issues those products would face at launch.

This approach is not without downsides. Perhaps the biggest is that product depth can come at the expense of legal depth, which meant that we sometimes incurred costs working with outside counsel and experts in legal areas and countries outside of our expertise or missed legal issues. However, we remain convinced that the vast majority of significant mistakes in-house departments make in our industry are the result of not understanding the product rather than not understanding the law. Another downside is that while being part of the “us” of a team is satisfying, can result in a much better understanding of a product, and better teamwork in identifying and fixing problems, it can also mean you are in the team “groupthink” as opposed to removed from it. Careful attention must be paid to all of the ways to reduce groupthink and it is imperative that you actively seek input from folks outside the bubble if you are going to effectively understand the various impacts your product decisions are likely to have in the world. We found it really helpful to discuss product features with advocacy organizations and they frequently improved the products. But, there were also definitely times we screwed up.

The actual role of “product counsel” grew out of the fact that our previous job descriptions didn’t make much sense given how we were doing our jobs. So we started thinking through names. Originally, we liked “launch counsel” because it was active, aligned with what our teams were trying to do, and could describe a bunch of different areas of law. Eventually we settled on “product counsel” because it was even more descriptive of the alignment we hoped for, and was tied to the whole lifecycle of a product from idea generation through maintenance and refinement, not just launch.

Our first job posting was in February 2004. It read:

=======================
Google is looking for experienced and entrepreneurial attorneys to develop and implement legal policies and approaches for new and existing products. The Product Counsel will be responsible for a portfolio of Google products across many legal subject areas including privacy, security, content regulation, consumer protection and intellectual property. Indeed, the only product legal matters with which this position will not be deeply involved are those that are strictly patent or transactional in nature, which are handled by other existing Google lawyers.

Requirements:
Passion for and deep understanding of internet.
Very strong academic credentials.
Solid understanding of Internet architecture and operation.
Ability to respond to questions/issues spontaneously.
Demonstrated ability to manage multiple matters in a time-sensitive environment.
Strong interpersonal and team skills.
Excellent interpersonal skills, dynamic and highly team-oriented.
Flexibility and willingness to work on a broad variety of legal matters.
Superior English language writing and oral communication skills.
Sense of humor and commitment to professionalism and collegiality are required.
California Bar
=======================

Note the many mistakes in that posting. For example, the Internet is referred to with both lower-case and upper-case capitalization (back then I was incorrectly not capitalizing it). Ug.

Even so, we were very fortunate to recruit an amazing set of folks at Google to become the first Product Counsel. Some of the originals who defined the role were: Glenn Brown, Trevor Callaghan, Halimah DeLaine, Brian Downing, Gitanjli Duggal, William Farris, Mia Garlick, Milana Homsi, Susan Infantino, Daphne Keller, Lance Kavanaugh, Courtney Power, Nikhil Shanbhag, Tu Tsao, and Mike Yang (in alphabetical order). The team was eventually about forty-strong by the time we left and worked across many countries. The idea of it spread relatively quickly in the industry and now LinkedIN lists thousands of product counsel.

Product Counsel, particularly when we were still doing it and not just managing people doing it, was one of the best jobs we have ever had.

COVID-19 Donation List

2020-04-16T12:24:00.002-07:00

This list is designed to help people give money in a time of COVID-19. There are many many many great projects out there helping people. Sometimes they need people to help with specific skills or equipment but all also need money. For each category listed below, there are some recommended charitable organizations. There are also some writeups, many of which were sources of recommendations. Please give generously.

This list is somewhat long so to help you find something that resonates with you. BUT, if you don’t have the time or inclination to look through the whole list, you can donate to the World Health Organization fund or Feeding America easily (just click & donate) or choose a local community foundation fund from this map. If you have an extra second, please tweet or otherwise share that you donated (feel free to tag me and I’ll retweet). Being vocal about donating will encourage others to do the same and increase the value of your donation.

One other note specifically for those of you (like me) who are privileged to have a Donor Advised Fund (DAF). You may have funded that DAF a while ago with the intention of figuring out how to give away the money over time. You may not have given much away because: life. Now is a very good time to use your DAF to help. You put money in to give it away, not to watch it grow. Your money NOW can make a big difference. Please consider choosing a goal that meets the urgency of this crisis and pushing yourself to give that goal away. Each recommendation below includes the organizations’ EIN so that you can easily give from your DAF (I generally give unrestricted funds, but you can also specify programs when you submit). I have also included recommendations from some DAFs and community foundations at the end. [If you don’t know what a Donor Advised Fund is, don’t worry, you are normal! You don’t have to know about them but if you’d like to, here are some resources: explainer from Wikipedia, explainer from Fidelity (a provider of them), and a critical take from the NYT.]

Finally, this is an evolving draft. If you have suggestions or questions, shoot me a note. I’m @amac on Twitter.

Categories

Hospitals, Doctors, Nurses & the Front Line

Food & Other Relief For Economically Disadvantaged

Refugees and Displaced People

Domestic Abuse

Children

Miscellaneous and Support

Other Good Writeups & Resources

Hospitals, Doctors, Nurses & the Front Line

Support WHO Efforts [donate]

This is a large fund (>$100M so far) that is run through the UN Foundation.

EIN: 58-2368165 (UN Foundation)

US CDC Foundation [donate]

EIN: 58-2106707

Partners in Health [donate]

Developing country care & testing.

EIN: 04-3567502

Direct Relief [donate]

Supplies to medical professionals to help them protect themselves.

EIN: 95-1831116

Flexport [donate]

Logistics and shipping for front line supplies.

Doctors without Borders [donate]

EIN: 13-3433452

National Association of Free and Charitable Clinics [donate]

EIN: 56-2273242

Health Leads [donate]

Resources for health systems focusing on resolving inequity.

EIN: 45-0484533

Frontline Foods [donate]

Meals from local restaurants to health workers.

EIN: 27-3521132 (through World Central Kitchen)

Food & Other Relief For Economically Disadvantaged

A Local Community Foundation

Community Foundations have a history of local giving and the staff to review potential grantees and get the money quickly in times of crisis. Consider giving to your local one (found via the map above) and/or one in a community that you care about. For example, for me that’s the Maine Community Foundation (donate EIN: 01-0391479), they have already given a wave of money to good local institutions.

Feeding America [donate]

They also run the very good Find Your Local Food Bank resource, which allows you to give locally.

EIN: 36-3673599

Meals on Wheels [donate]

Many seniors use this program.

EIN: 23-7447812

Save the Children [donate]

Keeping children fed.

EIN: 06-0726487

No Kid Hungry (Share our Strength) [donate]

EIN: 52-1367538

World Central Kitchen [donate]

They also fund Frontline Foods (see above).

EIN: 27-3521132

Give Directly COVID-19 Fund [donate]

Gives cash to people who need it.

EIN: 27-1661997

Mission Asset Fund [donate]

“Emergency financial relief for students, immigrants, and workers left out”

EIN: 20-8993652

Update: @mredshirtshaw has a good thread on a number of South Dakota Tribes' COVID-19 funds. She points to South Dakota because of the lack of a shelter-in-place order there.

Refugees and Displaced People

Global Response Management [donate]

Consider directing your donation to the Matamoros Project.

EIN: 81-5163032

Sidewalk School [donate]
EIN: 80-3405530

Update: A friend who knows more than I do about refugee issues points to the following two orgs:
International Rescue Committee, Signpost Project [donate]
EIN: 13-5660870
Signpost is designed to help refugees get good information during the crisis which the IRC's president says is one of the most pressing problems.

Refugee Advocacy Lab at Refugees International + International Refugee Assistance Project (IRAP)[donate: Refugees International or IRAP]
EINs: 52-1224516 (Refugees International) or 82-2167556 (IRAP)
Matching refugees with healthcare experiences with states that need healthcare workers and the certifications they need to practice, thereby helping both.

I have a friend who works specifically with communities on the El Paso / Juarez border. There they recommend:

The ACLU also has a good article about donating in this category.

I don't have a perfect recommendation to directly help those in prisons and jails but Civil Rights Corps [donate] and Institute for Constitutional Advocacy and Protection [donate] are fighting some of the legal battles around this. Please ping me if you have others.
Update: The Reform Alliance has a special COVID-19 action page to attempt to get governmental attention to this problem. Thanks @rklau.

Domestic Abuse

RAINN [donate]

RAINN runs the US National Sexual Assault Hotline.

EIN: 52-1886511

National Coalition Against Domestic Violence [donate]

EIN: 91-1081344

Global Fund for Women [donate]

EIN: 77-0155782

Children

UNICEF [donate]

EIN: 13-1760110

Save the Children [donate]

Keeping children fed (also in Food above).

EIN: 06-0726487

No Kid Hungry (Share our Strength) [donate]

Also in Food above.

EIN: 52-1367538

Children's Health Fund [donate]

EIN: 13-3468427

First Book [donate]

US Focus on learning.

EIN: 52-1779606

Miscellaneous and Support

Creative Commons [donate]

Creative Commons provides the licensing infrastructure for a lot of the open content being relied on in this crisis and was part of creating the Open COVID Pledge to help ensure that people fighting COVID can worry less about patent lawsuits. Disclaimer: I am a Board Member.

EIN: 04-3585301

Data & Society [donate]

Tech is becoming even more important now. Data & Society studies and critically unpacks the social implications of data-centric technologies & automation so that their impact is less harmful / more beneficial. Disclaimer: I am a Board Member.

EIN: 46-2904827

Committee to Protect Journalists [donate]

Good information is critical to combating COVID and journalists are risking their lives to get it to us.

EIN: 13-3081500

Reporters Committee for Freedom of the Press [donate]

Support their just launched fund to give legal support to local and regional reporters.

EIN: 52-0972043

Other Good Writeups & Resources

Community Foundations have expertise to help get money to local charities in times of crisis. This map will help you find one in an area that you care about.

Amelia Nierenberg, Don’t Need That $1,200 Stimulus Check? Here Are Places to Donate It, New York Times, March 27, 2020 but updated as well.

Great round up and source of a bunch of the above.

Cynthia Pompa, What’s Really Happening At Our Border And What You Can Do To Help, ACLU, December 23, 2019.

Before COVID-19 was the focus but still very relevant.

Denise Hearn, COVID-19 — where to give money now, April 2, 2020.

Bloomberg Beta, Schmidt Futures, and The Stanford Center on Philanthropy and Civil Society put this together.

Update: Isaac Chotiner, The Danger of COVID-19 for Refugees, April 10, 2020.
Q&A with David Miliband, the president and C.E.O. of the International Rescue Committee about the specific issues raised by COVID-19 in refugee communities where he highlights disinformation as an important issue.

Disaster Philanthropy, COVID-19 Coronavirus, April 13, 2020.

Fidelity Charitable, How to help: Novel Coronavirus.

Schwab Charitable, Maximize your charitable impact: COVID-19.

10 Years of Retweet

2019-08-13T07:14:00.000-07:00

10 years ago today, Twitter launched “native retweet” and significantly changed how people experienced the Twitter timeline. IMHO it was a huge and relatively gutsy change. I’m writing this post to explain what changed, and why I like a particularly controversial aspect of it -- strangers in the timeline -- so much. I hope it will encourage others who were at Twitter at the time to share their stories.

First off, imagine the Twitter of early 2009. It was a simpler Twitter in SOOOOO many ways. Timelines were a reverse chronological set of 140 character tweets. There were no ads. No images. And no mobile phone client from Twitter. Barack Obama had just been inaugurated.

Late 2008 screenshot from Huynh, Terence,
Twitter releases new design, more customisable,
TechGeek, Sept 20, 2008

People were already retweeting each other through a convention of using the letters “RT” and the original user’s username, and then the tweet. Retweeting the President’s election victory tweet would have been something like “RT @BarackObama We just made history. All of this happened because you gave your time, talent and passion. All of this happened because of you. Thanks.” If you followed me and I retweeted that, you’d see my avatar and that text. Nothing would link my Tweet back to the original, and there was nothing stopping me from editing the original and misquoting it. In fact, even that retweet of @BarackObama would have needed editing so that it could fit in the 140 characters. Still, the convention was used and useful.

On August 13, 2009 @biz made a short blog post pre-announcing a new retweet feature so that the Twitter client developers (none of whom worked at the company or were paid by Twitter) would be ready for it when it rolled out on twitter.com. At least one of those developers already had a retweet button that made retweeting easier, but the new feature @biz announced was different. It was simple and revolutionary. Now when I retweeted @BarackObama my followers would see his tweet as if they too were followers of @BarackObama for that instant. They would see his Tweet as if they followed him -- in their regular timeline -- but with an acknowledgement to me, and as if it had been tweeted when I hit the retweet button. As @Biz described it:

"Let’s say you follow @jessverr, @biz (that’s me), and @gregpass but you don’t follow @ev. However, I do follow @ev and the birth of his baby boy was so momentous that I retweeted it to all my followers.

Imagine that my simple sketch is your Twitter timeline. You’d see @ev’s tweet even though you don’t follow him because you follow me and I really wanted you to have the information that I have." Photo and quotation from Stone, Biz, Project Retweet: Phase One, Twitter Blog, Aug. 13, 2009.

It made retweeting much easier,* but it also meant that users saw the faces of people they didn’t follow in their timeline (internally we called this the "strangers in the timeline" phenomenon). Retweets were also displayed based on the time of retweet, not the time of the original tweet (even though the timestamp was still the original one), so it looked like the tweets were being displayed out of order. Here’s what it looked like when it rolled out later that year (with a special dialog box to explain to people why they were seeing strange new avatars).

Screenshot by See-ming Lee CC-BY-SA

This was (and is) a big deal for a whole bunch of reasons, but the one that I really appreciate today is that seeing strangers in my timeline made it so much easier for me to find and follow new interesting people. @Ev’s blog post from the time summarizes a lot of the other benefits and concerns that people had about the idea, including his description of the blowback he knew Twitter would face over putting strangers in peoples' timelines. He wrote, “The drawback is that it may be a little surprising (unpleasant even, for some) to discover avatars of people they don't follow in their timeline.” Which avatar to show, the original tweeter's or the retweeter's was a significant discussion and I am really glad the team chose to show the original tweeter as I think it has led directly to my timeline getting better and better over the years.

Anyhow, I’ll leave the rest of the stories about this to others who were closer to the decision and implementation. For now I just want to thank the folks that were there and call them out so that they (hopefully) tell more of the story. My memory is hazy, but I think at least @zhanna, @alissa, @cw, @goldman, @ev, and @biz would have good memories of it. Please share, tell the inside story, and add more folks I missed.

* Despite the ridiculous title, this Buzzfeed article has some good discussion from folks on the cons of the convenience and speed of the retweet button, and ways to think about slowing the spread of harmful retweet cycles while preserving the good.

Advice for new General Counsels (GCs)

2019-02-04T05:27:00.000-08:00

Here’s some advice I give first-time General Counsels (GC) or, as the kids today call themselves, Chief Legal Officers. These are written from my perspective as someone who had worked as an Associate at a law firm and in-house at Google before going to Twitter as the GC. Your mileage should vary.

Being a GC is lonely, you need colleagues.

Once you become a GC you have no colleagues. Your CEO and Board are your boss. You may have lawyers who work for you, but you are their boss. Your colleagues are the other members of the management team, but they don’t really care about legal stuff the way you do. They’ll be friends, but you need to also have GC friends, and you’ll need to find them outside of the company.

Eventually (on my last day), I formed a network of SF Tech GCs that would have been WONDERFUL to have had before I left. At the time I met with a bunch of peers outside of work and they were a godsend. These folks should be struggling like you. That way you’ll feel free to ask them anything and you’ll be as helpful to them as they are to you. They are your support group as well as your knowledge base. Plus they’ll remember being as clueless as you, which will make them good teachers. These are not your mentors. You need good mentors too, but these are your colleagues.

Arabella Mansfield, 1870.
Public Domain image from Wikipedia.

You are the boss and the end of the line.

When you are the GC, you are it. Yes you can “escalate” things to your CEO or Board but doing that a bunch will not build respect with either. For many hard decisions, you are the decision maker. You cannot just let something slide or say “here are three options.” You are being paid to be responsible for this stuff and have an opinion. Also, there are some things that only you can do, especially before you have a team. When the CEO is freaking out about the CFO deciding to go to your biggest competitor, talking to somebody else likely will not cut it. Vacation purity will go down. Stress will go up. You will need a way to deal with that and it is very important that it is not alcohol, drugs, or other bad things.

You are your company.

If you come from being an outside counsel, you may misunderstand the ethical issues involved in being at a company. At an outside firm you might have had a bunch of ways to distance yourself from a bad thing that a firm clients does. You might not work on their matters. Or not advise them on that subject matter. And, in any case, even bad actors need legal defense. When you are in-house that is no longer the case. You are your company. If they do something unethical or illegal, that is you doing it. Another way to say that is that there is no they, only we. So… you need to be VERY comfortable with the company you are joining. Both what they are now and how they might change over the time you are there. Your only remedy will likely be to quit quietly. Quitting quietly is not a good remedy. Especially if you need the job and the money it provides.

Interior of H.A. Goodrich & Co.'s Store. Public Domain
illustration from Fitchburg, Massachusetts
Past and Present via Internet Archive.

You are the establishment.

You likely came from being a part of a team and not being in the very senior part of management. You might even have complained about the Partners or the management team at your last company. That ends now. You are the management. If the company has a problem, then it is your problem. You whining is not a good thing. You are not a victim and you need to fix what you don’t like, not complain about it. That goes for even things that the CEO or Board do that you don’t agree with. Make sure you had your say. Quit if you need to, but if you are not going to quit, you need to take ownership of it with the rest of management.

You work for the Board (and the CEO).

GCs have a weird dual reporting structure because they represent the corporation, not any particular person in the corporation. Your day-to-day boss is the CEO, but the Board is also your boss and you need a relationship with the Board. This can be tricky because the CEO often thinks of themselves as the only person who “reports” to the Board. You need to have your own relationship with the Board for a bunch of reasons and a wise CEO will understand that your relationship with the Board is helpful to them as well. You should help the CEO manage getting the board deck together and with the basic stuff that goes comes with managing the Board. In that sense you will act a bit like a Chief of Staff and Executive Assistant. You have a legal role too, and may be the Board Secretary as well, but you should do the other, more mundane stuff as well in order to ensure that you are part of the company having a good relationship with its Board. For example, you should have periodic informal meetings with your board members. You should know how many kids they have and where they like to take vacations. FWIW, CFO’s have a similar need for that type of direct relationship.

Crocker Block. Public Domain
illustration from Fitchburg,
Massachusetts Past and Present
via Internet Archive.

You are also an owner of the company & member of the management team.

Your CEO and the rest of the team will look to you for your legal opinion. That is very important. If you screw that up, you will not be GC for very long.

However, you are also (hopefully) a significant owner of the company through your stock grant, as well as a member of the senior management team. In those roles the CEO is looking for you to have an opinion on what is the right thing for the company to do. If you come from being an outside counsel, you may be used to giving opinions of the form: “Doing X is a legal risk.” or perhaps “Doing X is a legal risk and I would quantify it in the following way.” Once you are a GC you need to be ready to say things like: “Doing X is a legal risk but is important for us to do because Y” or “Doing X is a legal risk that is not worth pursuing because Y.” What you fill in for Y will not be legal analysis but business, customer, user, employee, etc. analysis. Get to know those parts of your business and start making contributions that have nothing to do with legal.

Be careful about acting in your non-legal capacity.

Now that I’ve told you not to only act like a lawyer, I need to flag some important pitfalls. First, you need to do a good job of labeling and calibrating your non-legal advice. You’ll make more mistakes and tread on other management team members expertise in your non-legal role. If you are not clear about it, the management team may start treating your non-legal advice like your legal advice (or vice versa), and that is usually unhelpful. This can be as simple as saying, “this isn’t a legal issue but…” or “there is nothing legal that requires X but…” and “here I’m speaking directly about our legal requirements under the contract.” You needn’t do it all the time, but being able to be clear can help a lot.

Second, we lawyers are somewhat spoiled by the protections that our legal advice affords us under the law. We can have conversations with people that won’t be revealed in lawsuits. When you venture out into the world of non-legal advice those protections fade away. Some lawyers have gotten into bad habits based on their attorney-client relationships and have gotten used to saying really dumb things in email that they (a) don’t mean, and (b) would never let a client say in email. Remember that your non-legal work is not protected by your special status as an attorney. Do what you tell your clients to do. Imagine your email on the front page of the Wall Street Journal.

Legal is a cost center.

If you came from a firm, legal advising was the profit center. Lawyers were extremely important and probably ran the show. That is not true now. You are a support service. Act like it. Don’t waste your clients time (ie. arrive to meetings on time, with an agenda, etc.). Respond to their emails. Do the things you said you were going to do when you said you were going to do them. Provide value. Make sure your internal clients are happy. And, understand that winning at the Supreme Court and establishing precedent are not the purpose of your business. They shouldn’t be your purpose.

Thurgood Marshall, 1976.
Public Domain image from the
Library of Congress via Wikipedia.

Set yourself up to be constructive.

Some people say GCs shouldn’t say no. I strongly disagree. There will be times when you need to say a flat no. Those times will be bad for the company and for you. But the flat no you enforce may be a moment where you are providing your greatest value to the company. Don’t shy away from those times, but don’t welcome them either.

Try to set yourself up so that you do not get into a flat no situation by understanding legal issues early, and by understanding product goals enough to suggest solutions that will meet the product goals and legal constraints.

Understanding legal issues early, particularly in a startup, means having a very good understanding of who is working on what. Many startups spend a large amount of time transitioning from everyone knowing everything to having processes so that people can know what they need to know. That in-between stage is a hard one for the GC. If you find out about a feature launch from reviewing the launch blog post, you are likely too late for a product redesign. Your product team will have spent a bunch of time working and dreaming about the feature implemented exactly the way they have implemented it. They will be looking to you to say yes (or no). That may be a valuable “no” but it will hurt a lot. And if you say “yes but…” then at that point you may only be putting lipstick on a pig. It is FAR better to be able to understand what is going on when the feature is just an idea. Product managers, designers, and engineers are really good with constraints. They design within them all the time. Having them understand the legal constraints and desires when designing is essential. To do that, you need to be really in tune with whatever part of your company first thinks up new features. Google’s first lawyer, Kulpreet Rana’s practice of taking one afternoon a week to just wander around introducing himself to people and asking them what they were working on is brilliant in that respect (and also helps with the culture points below).

Equally importantly, once you see a legal issue, don’t just issue spot. Know enough about what the teams are trying to do to get your hands dirty in a design setting to help them understand the legal constraints and design within them. Sometimes this looks like working with the engineers to implement variable public domain rights for very old books in Google Books and embedding local copyright law into code. Sometimes it is jumping from a legal constraint to value creation, such as when Glenn Brown, Zahavah Levine, and I helped design the YouTube DMCA system to also allow content providers to leave content up and get a share of revenue. There is almost always more than one way to do something. Help your product teams choose the way that helps users and will survive legal challenges.

You have a decision to make about who to hire when.

As you build your team, you need to decide who to hire when. You will eventually be middle management and do very little legal thinking. You also came into this job not knowing most of the legal areas you will supervise. Do you hire what you know first, or what you don’t? Do you start building with someone who could be the next GC, or someone more junior who can churn through more basic stuff so you don’t have to? If the company is getting bigger, do you start with an employment lawyer? There is no right answer, other than that employment lawyers tend to get hired really early because very few GCs love dealing with employment issues, and good employment lawyers are really good.

Your team(s) are everything.

As your team grows and perhaps you take on a few extra responsibilities (public policy, trust & safety, communications, corporate development, business development, human resources, etc.) because you are a competent leader at a growing company, your teams will be much more important than you in terms of the company’s success. Many lawyers never get management training and didn’t start out life wanting to be managers. That is your life now. It is difficult and you need to want to do it extremely well. That means training, intentional focus, and making sacrifices on other things to make sure that you are doing the management part well. It also means understanding, valuing, and actually achieving diversity from the get-go. Start growing your leaders and empowering them. Get out of the way.

If you don’t like management and would prefer to be doing law stuff, being a GC probably isn’t the right job for you.

Clara Shortridge Foltz. Public Domain
image from History of the Bench and
Bar of of California via Internet Archive.

You define the culture of the Legal department, and likely the culture and ethics of the company.

Many of you will get the privilege of being at a very small company, or being the first or second attorney at the company. When I joined Twitter we were around 50 people and had one other intrepid lawyer (the great @tyip!). You will define the relationship between the company and legal. You will define what legal is. Those are probably your most important jobs other than ensuring the company continues to exist. If you establish that legal should be consulted last, heard to say no, and then overruled by others without consequences, that will be the way legal is treated by the company for a very long time. If you are teammates who provide value and help get stuff launched, it will be hard to lose that reputation. Make sure that legal’s relationship with the company is something you focus on. Have goals. Do things that help. For example, Ben Lee and I created the Innovators Patent Agreement as a way to signal to engineers that Twitter was in line with their ethics on patents so that legal could contribute to the company’s #1 priority of hiring great engineers. Another one of my favorite hacks is to set up a desk by wherever your team gets coffee for the post lunch afternoon coffee time. Sit there. Answer questions. (Yes, it is amazing how many engineers have “friends” with weird legal issues and need to be told to get a lawyer.) Being a GC is not a popularity contest, but you need to be known and you need to make sure people understand you are on their team, especially when you disagree with them.

GCs are also often the oldest and/or people who have the biggest vocabulary for ethics at the company. I don’t think we are necessarily the folks who care the most about ethics or have the best ethical compass, but we have the hubris / common sense to have been thinking about ethics and how it applies to our work for a long time. That often makes the GC the de facto Chief Ethics Officer. The important part of that often unofficial role is to understand the difference between ethics or values, and law. When a GC says something is OK to do, we sometimes mean legally OK but the company will frequently hear that it is both legally and ethically good. Ideally everyone at the company is thinking about ethics and your founders, CEO, Board, and management team all provide a solid tone from the top on ethics. That is not always the case. You should ensure that you are the backstop on ethics. Consider making that part of your job official. Definitely talk to the company about its ethical responsibilities and always consider those in your analysis. “That’s legal but completely unethical” is extremely important advice from a GC.

Conclusion

You are not an impostor. Nobody was born a GC. No GC knows every area of law. There is no perfect way to manage people. There is no secret to being a GC that is not something that was learned through trial and error on the job. The truly incredible GCs that I have met may have been better at the job than I but they weren’t superhuman. They started out just like me and you.

Finally, have fun. I found being a GC to be really interesting, challenging, and rewarding. I still love the various teams I helped build. I still have trouble not referring to my old companies with the pronouns “us” or “we.” Being a GC can be a phenomenal job (but it is only a job). Congratulations and good luck!

P.S. Thank you to the SFGCs email list for their useful comments on a working draft of this paper. Ken Carter and others on that list have grown it and made it into something great since I left SF. If you are an SF GC, DM me on Twitter (@amac) and I’ll forward to the right person to add you. If you are a D.C. GC, I’d also love to hear from you.

P.P.S. I got another great suggestion for a pic to add to this post, Laura de Force Gordon.

Laura de Force Gordon
Public Domain image via Wikipedia.

Recent Podcasts & Articles on Content Moderation

2018-08-31T07:45:00.002-07:00

One of the great things happening now is that more and more attention is being focused at one of my favorite subjects: content moderation by internet platforms. It's an important subject because a large amount of online speaking and listening happens through platforms. There has been a ton of good writing about this over many, many, years but I want to focus on four relatively recent bits here.

Radiolab, Post No Evil, Aug 17, 2018

Radiolab tells a sweeping story of the development of Facebook's content removal policies, deftly switching perspectives from people protesting its former policy against breastfeeding, to the headquarters workers developing policy and dealing with high-profile controversies, to the offshore contractors on the front line evaluating thousands pieces of disturbing content every day.

Post No Evil is a great introduction to the issues in this space but I think its most insightful moment is relatively buried. At 1:02, this exchange happens:

Simon Adler: What I think this [controversy around a beheading video] shows is that Facebook has become too many different things at the same time. So Facebook is now sort of a playground, it's also an R-rated movie theater, and now it's the front page of a newspaper. Jad Abumrad (?): Yeah, it's all those things at the same time. Simon Adler: It's all those things at the same time and what we, the users, are demanding of them is that they create a set of policies that are just. And the reality is justice means a very different thing in each one of these settings.

I've tried to emphasize when I talk about content policies that there is no one perfect set of policies that should exist for every service but rather that the policies serve the product or service goal that the platform is trying to create. The type of experience Google web search is trying to create ("you can find whatever you are looking for") is very different from the experience that the Disney was going for when it launched a social network for pre-teens where users could only talk to each other through a set of pre-chosen phrases ("this place is REALLY safe for kids").

Think of the content policies you might want at a library versus a dinner party. When I go to a library, it is very important to me that they have books about the tiny niche of the world that I am interested in at that moment. For example, books on bias in machine learning or Italian Amaros. It doesn't really bother me if they have books on things I don't care as much about, like American football. For books that I disagree with, such as To Save America, or think are evil, such as Mein Kampf, I may question the curators' choices but I expect breadth, and the inclusion of those books is less bad than if the books I cared about were not included.*

Change to the dinner party context and my preferences are reversed. Dinner parties that don't hit on bias in machine learning are fine by me but if I was at a dinner party where someone couldn't shut up about American football, I would not call it a success. A dinner party where a guest was espousing the views of Mein Kamfp would be one I would cause a scene at and leave. Over-inclusion is a huge problem and outweighs inclusion of my specific niche interests.

I've never been a big Facebook user, but it used to remind me of a dinner party. I thought that's what it was going for with its various content policies. Now, as Simon Adler says, it is trying to be many things (perhaps everything?) to many people (perhaps everyone?) and that is really hard (perhaps impossible?). It also has made the decision that some of the types of moderation that other platforms have used to deal with those problems (blocking by geography, content markings for age, etc.**) don't work well for it's goals. As Radiolab concludes starting at 1:08:

Robert Krulwich (?): Where does that leave you feeling? Does this leave you feeling that this is just, that at the end this is just undoable? Simon Adler: I think [Facebook] will inevitably fail, but they have to try and I think we should all be rooting for them.

Kate Klonick, The New Governors: The People, Rules, and Processes Governing Online Speech, 131 Harv. L. Rev. 1598, last revised Apr 17, 2018

Professor Klonick does an excellent job of describing why platforms may want to moderate content, how they do it, and the legal framework and regulatory framework that underpins it all. This is a very large expanse of ground, covered extremely well.*** If you are new to this area and want an in depth briefing, I highly recommend The New Governors. Her proscriptions are to push platforms towards greater transparency in their content moderation decision making and policies, as well as greater accountability to users. As in Post No Evil (for which she was a source), Professor Klonick identifies the popular concern about platform policies and locates it as a mismatch between platform policies and user expectations.

Professor Klonick also draws out the similarities and differences between content moderation and judicial decision-making. She writes:

Beyond borrowing from the law substantively, the [Facebook content moderation rule documents called] the Abuse Standards borrow from the way the law is applied, providing examples and analogies to help moderators apply the rules. Analogical legal reasoning, the method whereby judges reach decisions by reasoning through analogy between cases, is a foundation of legal theory. Though the use of example and analogy plays a central role throughout the Abuse Standards, the combination of legal rule and example in content moderation seems to contain elements of both rule-based legal reasoning and analogical legal reasoning. For example, after stating the rules for assessing credibility, the Abuse Standards give a series of examples of instances that establish credible or noncredible threats. “I’m going to stab (method) Lisa H. (target) at the frat party (place),” states Abuse Standards 6.2, demonstrating a type of credible threat that should be escalated. “I’m going to blow up the planet on new year’s eve this year” is given as an example of a noncredible threat. Thus, content moderators are not expected to reason directly from prior content decisions as in common law — but the public policies, internal rules, examples, and analogies they are given in their rulebook are informed by past assessments.

(footnotes omitted). Content moderation rules are always evolving and changing. Just as there is no one perfect set of content policies for all services, there is also no one perfect static set of rules for any given service. Instead, just like the law, the rules are always changing and being adapted to deal with new realities.

Ellen Pao, Let's Stop Pretending Facebook and Twitter's CEOs Can't Fix This Mess, Wired, Aug 28, 2018; and Kara Swisher and Ron Wyden, Full Q&A: Senator Ron Wyden on Recode Decode, Recode Decode, Aug 22, 2018

I include these two as good examples of the current mood. Both Ms. Pao and Senator Wyden are friends of tech and highly tech knowledgeable. Ms. Pao was the CEO of Reddit. Senator Wyden was one of the authors of the original statute that encouraged content moderation by protecting platforms that moderate content from many types of liability. Nevertheless, Ms. Pao believes that the tech CEO's don't care about and aren't trying to solve the issue of bad speech on their platforms. She calls for legal liability for falsity and harassment on platforms.

If you’re a CEO and someone dies because of harassment or false information on your platform—even if your platform isn’t alone in the harassment—your company should face some consequences. That could mean civil or criminal court proceedings, depending on the circumstances. Or it could mean advertisers take a stand, or your business takes a hit.

Senator Wyden says that he is working on legislation that:

... lay[s] out what the consequences are when somebody who is a bad actor, somebody who really doesn’t meet the decency principles that reflect our values, if that bad actor blows by the bounds of common decency, I think you gotta have a way to make sure that stuff is taken down.

I strongly disagree with legislating "common decency" because I think there is good evidence that it would do more harm than good, particularly to suppress the speech of unfairly marginalized groups. More broadly both Wyden and Pao seem to believe that these problems are relatively easy to solve, if only the CEOs cared, or were legally liable. I don't agree that this is an easy problem to solve in part because I don't see examples of it having been solved in spite of the value of solving it. As I have written previously:

... I don't know of many good examples outside of heavily editorial ones with a relatively small set of content producers, that have been able to be both extremely inclusive and progressive towards what I think are the "right" kind of marginalized ideas while keeping out the ones that I think are marginalized for very good reason. ... Many of the larger Internet platforms are trying, with varying degrees of success and failure, to do this right, as I was when I worked at Google and Twitter. That said, I don't have a great example of a platform or community that is working exactly as I would like. And it seems like that is a big and worthy challenge.

(footnotes omitted). As I said in that post, if you have a good example, please send it my way. In the meantime, my belief is that this is difficult, there is no silver bullet, and we should continue trying.

Nevertheless, it is important to understand that this is where public opinion is headed and these two pieces are a good indication.

Finally,

If you want to find out more about content moderation, here's a twitter list of content moderation folks on Twitter. If I'm missing someone, please let me know.

* This is really specific to me and your mileage may vary widely. I am a white male with lots of privilege. Take what I say about evil content with a huge grain of salt. I am relatively unthreatened by that content compared to someone who has had their life impacted by that evil. I get that some societies will want to ensure that books like Mein Kampf are not available in libraries. I don't believe that is the right way forward, but I may not be best situated to make that call.

** Facebook does use some of these tactics for advertising and Facebook Pages but, as far as I know, not for Facebook Posts or Groups.

*** Professor Klonick's description of Twitter's early content policies as non-existent is mistaken. Even early in Twitter's history the company had content policies which resulted in the removal of content, for example, for impersonation or child pornography. I think she just didn't have a good source of information for Twitter.

Hallin Spheres, Overton Windows, Constitutional Interpretation, and Online Platforms (oh my!)

2018-04-04T04:24:00.000-07:00

Hallin Spheres, Overton Windows, and certain theories of constitutional interpretation^[1] are all ways of thinking about what can and cannot be argued successfully, or at all, within different contexts. They are very applicable to our current discussion about online platforms and the types of speech they contain. This post aims to briefly describe all three, and how they might apply to online speech. One thing that seems to follow from thinking about the Internet and online platforms through these lenses is that the widening of participants that the Internet brought tends to increase the types of arguments that can be had and tends to decrease the amount of consensus available. While I am generally optimistic about the change as a way of accelerating social progress and bringing more, previously marginalized people into the "room where it happens," some people and ideas were marginalized for very good reason. Implementing global platforms or communities inclusively, but only towards progress seems possible but I don't yet know of a good example of it being done successfully at scale. (If you know of more, I'd love to hear about them!)

Hallin Spheres
Daniel Hallin is a journalist and professor of media systems who wrote The Uncensored War: The Media and Vietnam about the way journalists covered the Vietnam war through a description of three spheres of ideas on which journalists report. At the two extremes are the sphere of consensus, for ideas journalists assume their readers accept and agree with; and the sphere of deviance, for ideas that the journalists believe their readers disagree with vehemently. Between the two is the sphere of legitimate controversy where a journalists assumes her readers believe that there may be debate. These spheres come together like a donut, with the sphere of deviance outside the donut, the sphere of legitimate controversy making up the dough, and the sphere of consensus, the hole in the middle (the Canadian in me can't help but think of it as the Timbit of consensus).

Hallin observed that within the spheres of deviance or consensus, journalists would deviate from "objective" journalism in a variety of ways, such as adopting sphere of consensus views without challenge, excluding sphere of deviance sources and ideas from any mention in their stories, and generally reinforcing the divisions between the spheres. For more specifically on Hallin Spheres and the effect of the the Internet on mainstream media ability to maintain them, see Jay Rosen, Audience Atomization Overcome: Why the Internet Weakens the Authority of the Press (more on that below). A diagram from Hallin's book discussing the spheres is below from Google Books.

From Hallin, Daniel, The Uncensored War: The Media and Vietnam,
University of California Press (1989), p 117.

The Overton Window
Joseph Overton was writer and think tanker who proposed the window as a metaphor for understanding which ideas are viable from a political perspective in a certain community. Ideas in the window are viable and can be debated and adopted. Ideas outside the window cannot. Overton appears to have seen the window along a continuum of more and less government intervention (which many on the right would call "less free" and "free") and believed that while the window constrained policy discussions, it was political and social forces that could change whether ideas were inside or outside of the window. The Overton Window has been in the news a lot lately as a way to explain that normalizing extremely radical ideas can move the whole window towards those ideas and thereby move some slightly less radical ideas into the center of the window. For example, see Politico's How an Obscure Conservative Theory Became the Trump Era’s Go-to Nerd Phrase and Vox Media's description.

Image of the Overton Window from
Wikipedia.

Constitutional Interpretation
I've had a harder time coming up with a good source or pithy name for these same ideas in constitutional interpretation. The first time I learned about them was in an Advanced Constitutional Law class I was lucky to take from Professor Lawrence Lessig. My recollection / understanding is as follows, but all errors are mine, not his. First off, the Constitution is made of words. A lot of constitutional law is about interpreting those words and how they might apply to situations in a particular case. For example, is death by a particular lethal injection "cruel and unusual punishment" and therefor illegal under the Eighth Amendment? There are a bunch of different ways to go about this and some significant disagreement among jurists, however the question that Lessig was asking was, is there a context behind all of this that makes some thoughts thinkable by the Supreme Court Justices, while others are not?^[2] For example, how do all but one of the Justices in Plessy v. Ferguson, not understand that separate is not equal, whereas almost sixty years later, all of the Brown v Board of Education Court does? Compare:

"The object of the [fourteenth] amendment was undoubtedly to enforce the absolute equality of the two races before the law, but, in the nature of things, it could not have been intended to abolish distinctions based upon color, or to enforce social, as distinguished from political, equality, or a commingling of the two races upon terms unsatisfactory to either. Laws permitting, and even requiring, their separation in places where they are liable to be brought into contact do not necessarily imply the inferiority of either race to the other, and have been generally, if not universally, recognized as within the competency of the state legislatures in the exercise of their police power. The most common instance of this is connected with the establishment of separate schools for white and colored children, which has been held to be a valid exercise of the legislative power even by courts of States where the political rights of the colored race have been longest and most earnestly enforced." Plessy v Ferguson, 163 U.S. 537, 544 (1896)

with:

"To separate [students] from others of similar age and qualifications solely because of their race generates a feeling of inferiority as to their status in the community that may affect their hearts and minds in a way unlikely ever to be undone. The effect of this separation on their educational opportunities was well stated by a finding in the Kansas case by a court which nevertheless felt compelled to rule against the Negro plaintiffs: Segregation of white and colored children in public schools has a detrimental effect upon the colored children. The impact is greater when it has the sanction of the law, for the policy of separating the races is usually interpreted as denoting the inferiority of the negro group. A sense of inferiority affects the motivation of a child to learn. Segregation with the sanction of law, therefore, has a tendency to [retard] the educational and mental development of negro children and to deprive them of some of the benefits they would receive in a racial[ly] integrated school system. ... We conclude that, in the field of public education, the doctrine of "separate but equal" has no place. Separate educational facilities are inherently unequal." Brown v Board of Education, 347 U.S. 483, 494-5 (1954)

For more on these cases, listen to the difference between the oral arguments in Plessy and Brown, and read Justice Harlan's dissent in Plessy. In the language of Hallin Spheres, segregation being detrimental and not "equal," moved from pretty close to the sphere of deviance for the Court, to the Sphere of consensus. And, there have been similarly important, radical, and progressive shifts in Supreme Court understanding of specific words in the constitution in many other areas that I care about.^[3]

Enter, the Internet
As Jay Rosen and Shaun Lau have said better than I ever could, the Internet and its propensity to allow for many more speakers to join the conversation^[4] have had a significant effect on windows and spheres in the mainstream media, on internet media, and in society at large.

Rosen gives a great description of Hallin Spheres and argues that the "audience's" increased ability to talk among themselves and talk back to bigger media players weakens the big media players ability to maintain the spheres without challenge:

"Now we can see why blogging and the Net matter so greatly in political journalism. In the age of mass media, the press was able to define the sphere of legitimate debate with relative ease because the people on the receiving end were atomized— meaning they were connected “up” to Big Media but not across to each other. But today one of the biggest factors changing our world is the falling cost for like-minded people to locate each other, share information, trade impressions and realize their number. Among the first things they may do is establish that the “sphere of legitimate debate” as defined by journalists doesn’t match up with their own definition. ... what’s [] happening is that the authority of the press to assume consensus, define deviance and set the terms for legitimate debate is weaker when people can connect horizontally around and about the news." Jay Rosen, Audience Atomization Overcome: Why the Internet Weakens the Authority of the Press

Lau makes the same point before expanding on it to a bigger point about how internet communities affect discourse in this excellent thread. He's writing about how Jemelle Hill's description of Donald Trump as a white supremacist was seen by ESPN as in the sphere of deviance, whereas many on the Internet disagreed. You should go read it.

Y'all are having a Hallin's Spheres argument, but I haven't seen anyone else bring it up. So here we go. https://t.co/egJhoWFUr9
— Studio Glibly (@NoTotally) September 16, 2017

Among the many good points Lau and Rosen make is that Hallin Spheres are also about understanding consensus. As more people are included in a community, the sphere of legitimate controversy tends to grow. As it grows, it likely takes some space away from the sphere of consensus. As Lau says:

"Put only Montagues in a room, and you have consensus. Now put a few Capulets in. 'What happened to all the consensus?' The answer is that there never was consensus; there was only agreement among those with access or those represented by those with access. "Internet pushback" when the internet- and very specifically twitter- is more meritocratic than, let's say large corporations like ESPN? That's not "the internet" pushing back. It's those who you didn't allow in the room before we forced our way in via new technology. PEOPLE." From @NoTotally Twitter thread, Sept. 15, 2017.

Both of these observations mean that it can be harder to create spheres of consensus, that it is harder to maintain them once created, and that we might expect spheres of legitimate controversy to grow.^[5]

This is good?
Years of forward societal progress based in part on expanding and shifting the Overton Window and sphere of legitimate controversy may make you think, this is great! But hold on, some ideas are in the sphere of deviance for a reason. Racism is one good and sadly timely example. Same with some people. The Internet and its platforms have created many spaces for marginalized people to congregate and become less marginalized by removing gatekeepers who might enforce spheres and windows to exclude.^[6] On the internet, nobody needs permission to speak. Again, that's often good progress. But, some groups are marginalized for a very good reason. Nazis are a good and sadly timely example. Bringing racism and Nazis back into the window is VERY, VERY BAD, and yet the relative lack of speech gatekeepers in the U.S. Constitution, on the Internet, and on many Internet platforms may make this more likely. This is not a malfunction, but a design feature of the Internet and many of its platforms.

What can online platforms and communities do?
It is worth noting that spheres and windows are a bit of a misnomer for these phenomenon because not only can they be different sizes in different communities, they can also be pretty permeable along the edges and can change size or permeability. Just as importantly, they can change shape. There is nothing that says that a particular society, medium, or context needs to treat all ideas that are equidistant from the center of consensus as equal. The sphere need not be a sphere, it can be oblong and irregular. The window can be slanted and weird looking. In other words, it is not a law of nature that determines that two ideas that are believed by a similar number of people need to be treated the same. That said, I don't know of many good examples outside of heavily editorial ones with a relatively small set of content producers, that have been able to be both extremely inclusive and progressive towards what I think are the "right" kind of marginalized ideas while keeping out the ones that I think are marginalized for very good reason (and I use "I" here as a measure because different people differ quite a bit on these judgments).^[7] If anything, there are concerns that attempts to suppress speech by groups that should be marginal are often used against those that shouldn't.^[8]

Many of the larger Internet platforms are trying, with varying degrees of success and failure, to do this right, as I was when I worked at Google and Twitter. That said, I don't have a great example of a platform or community that is working exactly as I would like. And it seems like that is a big and worthy challenge. Anyhow, there is probably a whole 'nother post from me on this, but that's for another day, this one is already long enough.

P.S. If you have examples of platforms or communities doing this extremely well at scale, please forgive me for not including them and help me fix my error by pointing me towards them @amac.

^[1] I am an expert in none of these, but I have found them to be very useful concepts. [return]
^[2] One of the questions that Professor Lessig asked that I think is really interesting, but not quite on point for this post is "What is the thing that we can't really consider today because it almost unthinkable, but our grandchildren will think is so obviously true that it is unthinkable to debate against?" [return]
^[3] For example, see Obergefell v. Hodges, 576 U.S. ___ (2015). [return]
^[4] This has clearly not been uniform progress. Even as the speech gatekeepers have receded and allowed more people to speak, harassment, trolling, aggressive spamming, false flagging, and other techniques are being used to suppress speech and drive speakers, particularly those who have historically been marginalized, away from these platforms. [return]
^[5] I also believe that these two effects have negatively impacted trust in institutions more generally. [return]
^[6] See note 4. [return]
^[7] Spam or illegal content might be good examples of this at some of the major services. [return]
^[8] See e.g. Daphne Keller, Inception Impact Assessment: Measures to further improve the effectiveness of the fight against illegal content online, Comment to the European Commission, March 29 2018 (discussing the potential for disparate impact of rules requiring internet platforms removal of terrorist content).[return]

A Service I Want

2018-02-28T07:14:00.000-08:00

I would like an algorithm or service that would suggest arguments, opinions, and points of view from smart people trusted within their communities but with whom I am likely to disagree or whose communities I am underexposed to. I do not think I am alone in this desire.

I attempt to get some of this out of who I follow on Twitter (and it was a great use for Google Reader -- may it rest in peace), but that is a pretty imperfect system. I also routinely ask others to suggest sources I might like to fulfill these needs, but I have found that many struggle to make good suggestions.

Noble Returns to the Pavilion, from "W.G.", cricketing
reminiscences and personal recollections (1899)
Public domain book from the Internet Archive.

One of the tricky things about this algorithm or service is that it would need to distinguish between those arguments and communities that I care about, those that I do not, and those I am repulsed by. For example, I am probably underexposed to cricket enthusiasts but I don’t care much about cricket anymore and don’t want more information. Another example is that I have not read anything about the Parkland victims being actors conspiracy theories but I would be actively repulsed if a service suggested that I should read about it.

My suspicion is that one of the reasons services serve up filter-bubble content based on the engagement metrics of friend groups and similar users is because it is much easier than finding good, challenging material to suggest to users. That said, I wonder if the later might be more fulfilling to the user over the long term and result in a stickier service if it could be achieved.

Do you know of a service doing a good job of this? Do you have ideas for users or publications that would fit this bill for me? If so, please send them my way at @amac.

C.L. Townsend, Playing Forward, from "W.G.", cricketing
reminiscences and personal recollections (1899)
Public domain book from the Internet Archive.

Internet & Jurisdiction

2018-02-26T04:21:00.000-08:00

I went to the last Internet & Jurisdiction gathering in Paris. I can’t make it to the one that starts today in Ottawa, but I would have come if I could. I’ve been thinking about the last one all year because it was full of good, smart people trying to make progress on coherent and practical Internet jurisdiction. What I also loved about it was that I came away strongly disagreeing with the direction they were going. More on that below, but first some background.

Dwarf Galaxy Caught Ramming Into a Large Spiral Galaxy
(NASA, Chandra, 08/14/13) from NASA Marshall Space Flight Center.
This and the other space images accompanying this blog post appear
to be in the public domain, in spite of NASA's weird licensey language
to the contrary.

Background: Internet Jurisdiction

Jurisdiction is one of the oldest and thorniest questions for Internet policy: “Which government(s) get to regulate what and who, where?”

As John Perry Barlow put it in his 1996 manifesto declaring the Internet’s independence from government regulation, “[Cyberspace] is a world that is both everywhere and nowhere, but it is not where bodies live.” In that piece, he argued that regulation of the Internet by governments was both unwise and impractical. Others saw it differently. As Tim Wu wrote in 1997, “it is possible to regulate the Internet, and ... countries, corporations, organizations, and private individuals are already doing so.” The first important legal cases involving the extent of government jurisdiction over the Internet were decided shortly thereafter.

As the Internet has grown, become more mainstream, and increased in importance, particularly with respect to real world consequences that governments have historically regulated, questions of which governments get to regulate who and what online have become increasingly frequent. These questions get “answered” in courts, as governments make laws, and by corporations and individuals as the architecture, norms, markets, and regulation of the Internet develop.

There has been no straight line of consensus “progress” from one point of view to another. Even now, there are big questions that are being actively fought, including the United States Supreme Court considering Microsoft’s challenge to request by the United States for user data stored in Ireland, and the Supreme Court of Canada asserting the ability to order content removed globally only to have a U.S. District Court disagree.

The technological landscape has also changed dramatically. Over the last twenty years, as billions of people started using the Internet it has morphed from an incredibly decentralized landscape of personal websites hosted from tiny service providers, often at the very edge of the network, to a more centralized set of cloud-storage service providers serving a large percentage of the population. If the FBI wanted to find out whether I had sent an email to a particular person in 1996, they would have had to come to my house to get my computer and take a look at my locally stored email, if I hadn’t already deleted it. Today, all my email is on Google’s servers, just like that of more than a billion other people from all over the world. The public content I created used to be housed on a server in my closet. If someone thought I was saying something illegal, they would have likewise most likely have had to come to me in order to get it removed from the Internet. While it is true that, in some situations, some other avenues existed to get my information or remove my content, they were not very broadly available or used. By contrast, now, most of my online content is served from large U.S. corporations, like Google and Github. If they decide my content shouldn’t be online, they can remove it and force me to go look for another publisher. In some cases the online service providers are so important that no suitable replacement would exist.

Tarantula Nebula (NASA, Chandra, Hubble, Spitzer,
04/17/12) from NASA Marshall Space Flight Center.

Towards a coherent, if abhorrent, Internet jurisdiction policy

The Internet & Jurisdiction Conference (I&J for short) focuses on three broad tracks: data, requests for private user data; content, requests to render content inaccessible; and the internet domain name system. I’m most interested in the first two and these comments are mostly meant for them.

I&J had a wide variety of participants and many more government and law enforcement types than I generally find at the Internet policy conferences. The conversation was therefore more oriented towards those stakeholders than at some other conferences, and it was quite similar in tone to the types of conversations happening in governments and courts all over the world right now.

In both the data request and content removal areas, these conversations are moving towards a coherent, if abhorrent, policy of allowing governments almost everywhere to get data about any internet user or remove any content without needing to engage the users themselves or the court systems of their jurisdictions. Most discussions exclude certain governments from the club that should have this type of power, but the idea that data should be able to be given over and content should be able to be suppressed through interactions between governments and repeat-player intermediaries was so ingrained in many of the discussions as to be an assumption. Convenience and speed are touted as principal advantages.

For example, a Facebook user in Mexico should have their data given to authorities in the England on a request to Facebook. A Canadian Microsoft user should have their post suppressed, at least in Thailand if not all over the world, via a request to Microsoft. Even if the user is known to the complainant, no direct approach to them is contemplated. At some companies under some circumstances the user might get a notice, but that is left to the companies and to the circumstance. This is not just “All your base are belong to us” but “All your base aren’t ever belonged to you in the first place.”

Black Hole Caught in a Stellar Homicide (NASA, Chandra,
GALEX, 05/03/12) from NASA Marshall Space Flight Center.

A challenge: Center Internet policy on users citizens

If I were able to come again this year, I would. Indeed, my favorite conferences are those at which a majority of the attendees are smart, passionate advocates with direct experience with the subject matter and with whom I disagree (see e.g. the excellent Fordham Intellectual Property Institute). If I was there, I would challenge the attendees to propose a way forward that centers on the user rather than removing them from the equation. It is more convenient to just go to the big corporate repeat players. They are well known to the governments and can be counted on to pick up the phone. However, an Internet jurisdiction policy that regularly circumvents the user will encourage countermeasures to return power to the user -- the emerging prevalence of end-to-end encrypted services is one good example of this trend. More importantly, those users are our countries’ citizens, they deserve our respect and, at least, to be able to face their accusers and challenge the accusations. There may be cases where expediency trumps all, but this is the tiny minority of cases, not the norm on which policy should be based.

The YouTube clip above is of me trying to make a similar point at the end of last year's I&J.

PS If you want to read more about the current intermediary liability battles, please follow Daphne Keller and Eric Goldman and take a look at their excellent sets of resources on the topic at Stanford and on Eric’s blog. Graham Smith also wrote a couple of posts in the run-up to this year’s I&J.

Star Cluster Cygnus OB2 (NASA, Chandra,
11/07/12) from NASA Marshall Space Flight Center.

Screens, Images, and Attention (a thing I'm working on)

2018-02-02T01:58:00.000-08:00

Many of us have more than a few screens. Mostly they lay idle. A tablet for plane trips, an old phone, a Chromecast connected LCD TV.

I also have a bunch of images I want to see more of. More than 100,000 personal digital photos. And, lots more incredible images out in the world. Millions.

One of the things I loved about the Obama White House was that Pete Souza and his team's wonderful photos were everywhere and changed relatively frequently. Walking to a meeting you'd see a picture of a co-worker and her kids hugging the President. In a meeting room there would be a beautiful photo of Half Dome in Yosemite and Marine One. It was great.

President Barack Obama visits with Natalie Quillian,
Advisor to the Chief of Staff, and family in the Oval Office,
Aug. 27, 2014. (Official White House Photo by Pete Souza).
Public Domain.

I'm working on using idle screens to bring more of that into my home with my images and those from incredible online collections. The idea would be that the screens could show a playlist (automatically generated or hand-curated) of images from a wide variety of sources and on a wide variety of devices / screens. Incidentally, I haven't been able to find the equivalent of .m3u for images. I don't think it exists. It would be wonderful if people could trade image playlists.

Automatically generated playlists could show images relevant to the day of the year, or types of images, or ones that will look good on that particular screen, or ones I might like based on what I've liked before. An earlier version of this was surprisingly good if it just showed images from Christmas, New Year's and Thanksgiving.

Solar Flare, August 31, 2012, Nasa Goddard Space Flight
Center. Hosted by Wikipedia. Public Domain.

I don't know of a thing out there that does this across image collections. Do you?

I mostly program to learn things. This project will help me really learn the techniques I've been studying through the Andrew Ng's Coursera on Machine Learning. It will also brush up some of my full-stack web development skills as the database will be Mysql, the backend will be Python 3 and Django 2, the frontend will be HTML and CSS with a lot of Javascript (including ajax, which I haven't used too much). And, I'll get to know both JQuery and Cycle 2 well. It has been fun so far and I will share updates as I go, and, eventually, the code.

I'm not doing this as an "entrepreneur." The idea isn't to make a bunch of money or get a million users. Mostly I'm doing this for myself and to learn. I have found that deeply understanding technology is really helpful to my legal and policy work (not to mention my life). I also really enjoy coding for fun. If you haven't tried that, you should!

I'm writing about it here to further commit to finishing it and so that others can share their good ideas.

153rd New York Infantry, ca. 1861, from Metropolitan Museum
of Art. Hosted by the Internet Archive. Public Domain.

If you are interested in using or working on this, please let me know. I'd be interested in understanding other use cases. I would also love pointers to great image repositories (preferably public domain). And eventually, I'd like people to help me rate the pics from some of the public repositories. My email is "lawyer" @ the popular email service run by Google. Or you can @reply me on twitter @amac.