LockBoxx

Offensivecon Tokyo 2026 Review

2026-10-28T17:01:37.691-07:00

This conference was an interesting experience for me, though probably not for the reasons I originally expected. There were some genuinely excellent presentations and valuable technical insights, but also some pretty disappointing moments that left me questioning whether attending was worth it. A few friends and I submitted talks back on June 1st, but acceptance decisions weren't announced until late September. For an international conference in Japan in late October, that meant committing to flights and travel arrangements before knowing whether we'd be presenting. None of our talks were accepted, but with our trips already booked, we decided to attend anyway, despite the additional $1,800 per person conference admission. At that price, especially when the talks are eventually released online for free, you're paying for an experience that should offer substantially more than just access to presentations. I was still optimistic going in, but after the opening keynote, I couldn't help feeling like I'd been sold something very different from what I'd expected. What had initially seemed like unfortunate travel logistics and an expensive ticket suddenly started to feel like a pretty questionable investment.

I think keynote presentations carry a particular responsibility that ordinary conference talks don't. A keynote is often an attendee's first real impression of a conference. It establishes the tone, introduces the themes, and implicitly communicates what the organizers believe is important or representative of the event. In many ways, the opening keynote is the conference's opportunity to demonstrate why attendees should be excited about everything that follows. That's why the Day 1 keynote was especially disappointing. Based on the title, I was genuinely excited to hear a discussion of Recursive Self-Improvement (RSI), particularly given how rapidly AI is changing and how much interest there is in systems that can iteratively improve their own capabilities. This is an incredibly relevant topic for vulnerability research, where AI is already forcing us to reconsider many of our assumptions about how research gets performed. Instead, the presentation largely focused on extracting insights at minimal cost, building benchmarks, and constructing datasets. Those are perfectly legitimate subjects, but they weren't what I understood the talk to be promising. As far as I could tell, there wasn't an actual demonstration or particularly substantive discussion of recursive model self-improvement. I was also left wondering how XBOW's approach related to RSI in the traditional sense, particularly given my understanding that they rely on models they don't directly control. The entire presentation felt more like a loosely assembled vendor pitch than the carefully considered technical keynote I'd expected. And that's where the experience really started to sour for me. After spending thousands of dollars to travel internationally and attend, sitting through what felt like a promotional presentation as the opening keynote left me feeling like I'd paid for something very different from what I'd been led to expect. I don't think it's unreasonable to expect significantly more from the keynote of a conference charging this much money. A keynote isn't just another presentation in the schedule; it sets expectations for the quality, direction, and substance of the entire event. When it misses that mark so dramatically, it affects how you perceive everything that comes afterward.

The physical conference itself also caught me off guard. The entire event was essentially contained within a single conference room and an adjoining hallway, with one vendor, XBOW, and perhaps five to seven tables for eating or socializing. There was very little space outside the main room for people to congregate. During a coffee break on the second day, I tried counting attendees and came up with somewhere around 60 people. Now, there's nothing inherently wrong with a small conference. Some of the best technical exchanges happen at smaller, more focused events. But the combination of the scale, facilities, and exceptionally high ticket price was difficult to reconcile. There was also a noticeable sense that many attendees and speakers already knew one another from previous offensive security conferences or shared professional backgrounds. It seemed like quite a few had worked together at places like Binary Gecko, XBOW, or elsewhere in the industry. Again, that's not necessarily a bad thing. Tight-knit research communities are common, especially in offensive security. But as someone attending from outside those established circles, I couldn't shake the feeling that I'd somehow paid a substantial amount of money to attend what was, in practice, a relatively small gathering of people who already knew one another.

Despite all of this, I don't want to dismiss the entire conference, because there were genuinely good presentations. The Day 2 keynote, in particular, was a substantial improvement. It addressed something I think is critically important right now, the fact that our industry is changing rapidly, and that researchers and organizations will need to adapt if they want to remain relevant. I didn't agree with every technical detail, particularly some of the discussion around influencing training data and exploiting that material, but the broader message resonated with me. Several other talks also stood out, either because they were exceptionally well presented and showed some various novel components on how to infer or understand data with vulnerability research. Those presentations provided real value. They offered interesting technical insights, introduced new ideas, and gave me things to think about afterward. They were the moments when the conference began to resemble what I'd originally hoped it would be. But they also highlighted something else that I found increasingly difficult to ignore.

We're in the middle of what feels like a software industrial revolution. AI is fundamentally changing how we write, understand, analyze, and interact with software. The implications go far beyond simply making individual researchers more productive. We're starting to see changes in the economics and scale of vulnerability discovery, the automation of complex analysis, and the processes through which entire software systems are developed and secured. Vulnerability research is right in the middle of that transformation. And yet, what surprised me most was how little of that transformation seemed to be reflected in the overall direction of the conference. Almost everyone seemed aware of what was happening. Many presenters were already using AI in their research, and conversations throughout the event made it clear that researchers understood the significance of these tools. But much of the selected content still seemed anchored in the traditional model of vulnerability research: finding highly specific bugs, developing intricate exploitation techniques, and presenting the technical details of individual discoveries. There's obviously still tremendous value in that work. Deep technical understanding and exploitation expertise aren't suddenly irrelevant just because AI is getting better. But the methods by which we discover, analyze, and ultimately exploit vulnerabilities are changing underneath us. I would have loved to see more discussion of that shift. Not simply talks about using an LLM to assist with an existing workflow, but serious exploration of what vulnerability research looks like when the fundamental constraints around time, scale, cost, and human effort start to change. It felt like the community understood that a major transformation was underway, while the conference's overall program was still largely organized around the world that existed before it. Perhaps that's just the nature of a conference with a particular technical focus. But I couldn't help wondering whether the talk selection reflected a disconnect between where the industry has traditionally placed its attention and where the most consequential developments are now happening.

I came away with mixed feelings. There were undeniably talented researchers at this event, and some of the presentations were excellent. I learned things, heard interesting perspectives, and had conversations that made attending worthwhile. But I also came away feeling that the overall experience didn't remotely justify its cost. And I keep coming back to that keynote. When you're asking people to spend thousands of dollars, often traveling across the world to attend, your opening presentation should make them feel confident that they've made a worthwhile investment. It should represent the intellectual ambition and technical quality of the event. Instead, I spent much of that first morning wondering why I'd paid so much money to listen to what felt like a sales pitch. The stronger presentations that followed helped recover some of that initial disappointment, but they couldn't entirely erase it. I'm genuinely curious to see how this conference evolves as AI continues to reshape vulnerability research and the broader software industry. There's certainly an opportunity for it to become a place where researchers grapple with some of the most important changes happening in our field. For now, though, I suspect I'll be following that evolution from the sidelines, watching the presentations online rather than paying to attend in person again.

A Summer of Agentic Game Dev

2026-10-10T13:01:23.682-07:00

Since late May I've been building games with Claude Code almost every week, and I've picked up a lot of practical game dev tricks in those months. Above you can see my Asteroid clone called Starbreaker. It's a fun rogue-lite that experimented with some early physics and play concepts I had. Yampa Valley was one of my other early projects, a single-file browser platformer about a golden retriever sliding on his momentum, rolling through mud, and stomping chickens. It took less than a day to dev and it was super fun. The interesting part was the art. I generated my sprite sheets as PNGs per model with position labels and a beige reference background, then I ended up writing a custom slicer in Python and Pillow that chroma-keys the beige to transparent and cutting the sheets into 60+ clean dog sprites. The final product is a remarkably smooth player movement and feel. C4-Man taught me other lessons about multiplayer. It's a Bomberman clone in a single 1k-line HTML file with no server anywhere. Players connect over WebRTC data channels, Trystero handles the signaling over public Nostr relays, and every client generates the same map from hardcoded seeds. Each peer owns its own movement and bombs, and the peer with the lowest ID becomes the "warden" that simulates the creeps, with automatic takeover when it leaves. It trusts the client completely, which works for a game between friends and would fall apart the moment strangers showed up.

Strangers messing with my game was the whole point of my next project. For DEF CON I built DCJRPG, a public MMO card battler with a four-part storyline, player trading, PvP, and a casino with multiplayer craps and roulette, because why not. It ran to 400+ commits between May and August, with help from a few developers from 562 and my homies Sohan and Rozar. Where C4-Man trusted everyone, DCJRPG trusts nobody, as the premise was releasing a multiplayer game at a hacker conference. The stack is a Phaser 3 client talking to a Colyseus server backed by Postgres and Redis, and the README opens with the rule that governed everything: "The game is an API... if a feature can't be done through @dcjrpg/sdk, that feature has a bug." I made that call for security, so players couldn't edit their local client and hand themselves inventory. The side effect was a game you could script against with almost no effort, which is a fun thing to discover about your own MMO. I spent a ton of time on the card engine design. We implemented two complete engines before settling. The first was a register-based design with simultaneous planning across four register slots, battery costs, and engine building, specced out in a proper RFC. The one we shipped is a Robot Deck Builder where equipment slots generate a 20-card deck during the battle. Making a card deal 6 damage is easy. Deciding what happens when weak, strength, vulnerable, and thorns all stack on the same hit, and in what order, is where I spent alot of time getting the engine exactly right.

The other problems were art, rapid iteration, and hand editing. Every card tied back to a physical robot part, so a buzz saw arm needed cards that looked like they came off a buzz saw. I built an image pipeline against gpt-image-1 to generate card art at scale, and the robots themselves became a layered paper doll of five PNGs in a fixed 128 by 160 frame, with an anchor table in the spec telling any generator exactly where the legs, core, arms, head, and special overlay sit. The sprite tooling from Yampa Valley grew up into Spritebench, my own special version of paint on steroids. I eventually directly integrated this editor into a custom map editor I built just for the DCJRPG. I had dozens of game dev tools such that you could edit any NPC's dialogue, quests, card values, or even map objects, down to a sprite editor, such that you can double-click a tile and edit its PNG live in the tool. Mobile and controller support was another challenge that needed a creative answer. A browser UI built for mouse and keyboard doesn't survive the move to a phone, so instead of rebuilding the game I pointed Unity at it. Because the server keeps every rule, the Unity client is only a renderer, using the official Colyseus Unity SDK, and its first milestone was a workable client with a fully emulated on screen controller layer. That pattern stuck with me. When a design surface gets hard, I hand it to a real tool that already solves it and can create a workable source of truth underneath my concept. The game went live for the conference. Our player count was small, since my marketing plan amounted to handing out stickers, but several people finished all four chapters, and their names are still in the credits. All that said, my biggest measure of success was that there were no signs of abuse or people hacking my game in either the game stats or on the servers files. While I did have someone mess with one of the client workstations I set up to play the game at a party, what they ended up doing was extremely juvenile and easy to understand with just a few minutes of forensics (no impact beyond setting a MOTD-like banner). I was really hoping someone would actually try to exploit the game itself.

That pattern paid off again this fall on HelloSpire, a three-character Slay the Spire 2 expansion my cousin (Rozar again) and I built with friends. It adds a Paladin, a Gunslinger, and an Alchemist, with 280 cards, 26 relics, 31 potions, and 75 powers across close to 500 commits. I expected working with the Slay the Spire engine to hurt more than it did, but BaseLib already wires custom cards, relics, and characters into the game. We spent our time on design and balance instead. The art pipeline from DCJRPG came along and got pushed much further. Using layered prompts in the 'house style' of prompting, I had a character preamble, then the card's subject, then composition rules for where the game draws its title plate, plus a hard no-text rule. The actual character model movement however stayed unsolved for weeks. Slay the Spire characters are floating, independently animated body parts on Spine rigs, so our first versions borrowed the Ironclad, Silent, and Necrobinder rigs with our palettes baked in. We tried stick figures on a shared skeleton but it just looked too janky. Finally we sought out other tools such as Blender to help us with the rendering. We turned the 2D character art into 3D models, had Claude drive Blender to pose and animate them, and exported the result back to the Spine framework. We finally nailed it, and the Paladin's rig went out as 14 painted parts on weighted meshes with a shadow and all eight combat animations. Another cool twist on this project, after it was done it was super easy to add localization support and use Claude to translate every card and peice of mod text into a bunch of foreign languages. It immediately expanded our player base and was super cool to use in a programming project.

With all of that as a background, this last week Google released their new Playground and I decided to take it for a test drive. Some of it works, but in general it was a really disappointing experience. It generates and slices sprite sheets decently, although I still think models like MidJourney produce far better art. It will also produce sound effects and music, but Suno and a basic audio editor is probably a much better choice. I do like their asset manager UI, although I also like having direct access to assets on the CLI. All that said, the development loop is the biggest problem. You describe a game, wait for it to build, play it, find something broken, describe the fix, and wait again. Sometimes it fixes the bug. Sometimes it introduces two new ones or changes something you never mentioned. After dozens of rounds I still had broken game logic, janky animation, and mechanics that didn't feel right. You can export the source, which is a real point in its favor, but inside Playground you can't point the AI at a file, isolate a collision bug, or ask it to leave the movement controller alone while it fixes the HUD. Every request reinterprets the whole game. I wasn't alone in this. Early Hacker News threads describe games that ignored instructions, and TechRadar's hands-on found interesting first drafts that fell apart during refinement. Generating a playable prototype and developing a good game are separate jobs, and Playground only does the first.

Looking back over the summer, a lot of what I built is due to standing on the shoulders of giants who came before me. C4-Man stands on Trystero and Nostr. DCJRPG stands on Phaser, Colyseus, Postgres, and Unity. HelloSpire stands on BaseLib, Godot, Spine, and Blender, and Yampa Valley stands on Pillow and Playwright. Claude Code's real strength is that it lets me pull those tools in as I need them, and skills files make that cheap. DCJRPG's pulls dozens of skills to use various tools and stress tests decisions along the way; alongside dozens of project skills and tests for verification and UX checks. Each skill teaches the agent one existing piece of the game dev world, so the agent works inside the same ecosystem human developers spent ages building. Google Playground walls you off from all of that and asks the model to reinvent a game engine, an art pipeline, and a netcode model on every prompt. I just don't think it's there yet, and for now I'll keep writing interface tools and working from the shoulders of giants.

OpenAI’s Artist Rules Are Easier to Evade Than to Explain

2026-10-06T10:56:14.163-07:00

In March 2025 OpenAI shipped native image generation in GPT-4o, and within a week my feeds filled with "Ghiblified" family photos, memes, and even an official White House post. Sam Altman swapped his profile picture for a Ghibli-style portrait and joked that demand was melting OpenAI's GPUs. Studio Ghibli never agreed to any of it. Hayao Miyazaki's 2016 comment calling AI animation "an insult to life itself" went viral again, and the anger in Japan kept building. After Sora 2 launched in the fall, the Japanese government made a formal request that OpenAI stop replicating Japanese artwork, and CODA, the trade group representing Ghibli, Bandai Namco, and Square Enix, demanded that OpenAI stop training on its members' work without permission. Credtent, a content-licensing firm, put the AI training value of Ghibli's IP at $17 to $20 million a year. Ghibli collected none of that, while OpenAI used the studio's look as free marketing for the biggest product launch of its year. OpenAI's fix, as I experience it in ChatGPT today, is a rule against imitating the style of a "living individual artist." This rule is not nearly robust enough, and fails itself on it's literal interpretation of it's own guideline. I spent a few minutes testing that and the following are three important places where that rule breaks.

My first issue with this policy is their literal interpretation of the word "individual." I asked ChatGPT to draw my dog in the style of Studio Ghibli, and it did so without hesitation: a golden retriever lounging in an aspen forest, the dog painted light, detailed backgrounds, the whole ass look. Studio Ghibli is a company, so the policy apparently does not apply. That gets the original incident and the real problem backwards. A studio is hundreds of living animators and background painters whose combined work built the style, and the man whose name sits on the door is still alive and still making films. The aesthetic at the center of the entire controversy remains one prompt away. OpenAI built a classifier that can tell a person from a company, in respect to who is making the art, but it still can't reason out who is getting hurt and defrauded in this processes.

My second issue with the policy is that the guardrail hands you the workaround. I asked ChatGPT to redo an image in the style of Rebecca Guay, the Magic: The Gathering artist, and it refused because she is "a living individual artist". In the same reply it offered to translate her name into "the visual language you want": traditional watercolor and gouache fantasy illustration, delicate ink-like contours, translucent layered washes, dreamy woodland atmosphere, elongated forms, and the hand-painted feel of 1990s and 2000s fantasy card art. Feed that paragraph back in and you get something that looks a lot like a Guay. In red team terms, the filter blocked the string and then printed the bypass. The model knows whose work I want, and it proved that by describing her style better than I could. That knowledge came from her paintings in the training data, and refusing her name does nothing to remove it. If you talk around the artist's name in the prompt and the model still walks you straight back to their art in terms of generation. Antivirus vendors learned decades ago that signature matching on a name loses to the first person who renames the file. Here the defender even helps you rename the file! A Nieman Lab piece published yesterday shows the model's knowledge of an artist goes all the way down to the signature. ChatGPT has been signing generic "New Yorker-style" cartoons with the names of real New Yorker cartoonists. A fake cartoon of Dolly Parton and Tim Curry at the pearly gates picked up 25,000 likes on Twitter with Brendan Loper's "BLOPER" signature in the corner. Reporter Andrew Deck documented more than 15 cartoonists whose signatures the model reproduced, including Emily Flake and Pat Byrnes, down to the period Byrnes has put after his name since he was a teenager. The style rule never fired, because the user never typed an artist's name. The model supplied the name on its own, which tells you it had a specific artist in mind the whole time, and will still generate their exact art even when you describe it without using their name.

My third issue with this is the "living" clause in their "living individual artist" policy. I asked where the line sits, and ChatGPT told me the rule turns on living versus deceased, with no waiting period of 20, 50, or 70 years after death. U.S. copyright runs for the life of the author plus 70 years, because an artist's estate, heirs, and licensing deals keep going after the funeral. Under OpenAI's rule, an illustrator's style becomes fair game the day they die, at the moment their finished body of work gains scarcity value and their family starts depending on it. Put the three rules side by side and you can see how ineffective these OpenAI artist protection policies really are.

The Nieman story also shows how OpenAI responds once someone catches the problem. After Deck flagged the forged signatures, OpenAI added a warning about "similarity to third-party content," and ChatGPT continued signing cartoons anyway. OpenAI misses on both axes, both the controls and the response. Today it still generates Studio Ghibli style images as you can see at the top and bottom of the article. The technical controls are really a sham here; the policy guards an artist's name in the request while the model forges that same name on the finished work. The same Nieman piece cites court filings in New York Times v. OpenAI showing the company chose to work around publishers' paywalls, along with a Microsoft memo calling the scraped training data "the largest theft of labor in human history." I believe the real issue here is deeply systemic to OpenAI's models. At it's core, their models can generate this look-a-like imagery and they don't have a real capability to control it, yet still want to release this technology publicly. It's a real catch-22 they've put themselves in. OpenAI started out as the lab built to make powerful AI safe. Today it sits at the front of the harm that AI does to working artists, because it keeps shipping capability to the public first and backfilling guardrails that read well in a policy doc but collapse in less than an afternoon of testing.

How Dream Serpent Inspired OpenClaw Enterprise

2026-10-01T09:40:40.119-07:00

Dream Serpent (DS) started from a simple idea, constrain the machine, not the model. Rather than assuming an autonomous agent can be made trustworthy through prompting alone, we designed around the possibility that the agent, its tools, or the code it executes may eventually behave in ways we did not intend. In the previous post I described that through Behavior, Identity, and Controls (BIC). Observe what the agent is doing, give the workload its own bounded identity, and enforce its capabilities through infrastructure the agent cannot rewrite. That philosophy became part of the inspiration for OpenClaw Enterprise (OCE). OCE approaches agents less like privileged chatbots and more like untrusted distributed workloads operating inside explicit trust boundaries. The runtime may be different, but the underlying question is the same. If the agent becomes completely untrustworthy, what can it actually reach?

That influence also shows up in how the systems are structured. Dream Serpent separates concerns like orchestration, identity, and VM execution into distinct parts of the repository. Networking, shared contracts, and assurance are separated as well. OCE applies a similar kind of decomposition at a larger Kubernetes scale. Its control plane, Harness execution environment, and IAM system each have explicit responsibilities. Audit, shared contracts, and infrastructure drivers are also separated into their own boundaries. Both projects try to make trust boundaries visible in the architecture and the source tree. Security-sensitive responsibilities are deliberately kept out of the component running agent-controlled code, and interfaces between those components are treated as contracts.

The most direct inspiration can be seen in the technical controls themselves. Dream Serpent keeps long-lived credentials outside the agent VM, uses scoped substitutes inside it, and separates network reachability from authority to perform an authenticated operation. DNS, egress, and policy enforcement also live beyond the agent's control. OCE translates many of those ideas into Kubernetes-native mechanisms. It separates the control plane from the harness framework, starts from default-deny networking, and hardens the workload boundary based on infrastructure needs. It also moves credential custody outside the agent runtime and uses scoped substitution where the surrounding infrastructure supports it. Some of the connection was even literal, as OCE imported and adapted pieces of Dream Serpent's Rust networking and policy data plane while experimenting with DNS gating, egress control, and credential brokering. But the more interesting inheritance is the architecture underneath that code. Dream Serpent helped establish a model in which the agent is treated as the untrusted part of the system and the surrounding infrastructure holds the authority. OCE takes that same idea and applies it to a much larger Kubernetes environment.

My own thinking has evolved a bit since writing the original BIC post. I now think the mental model is better described as BICO, Behavior, Identity, Controls, and Observability. Behavior shapes how the agent does it's thing. Identity tells us who/what is doing it, and Controls determine what it is allowed to do. Observability closes an important gap by giving us the evidence to understand what actually happened across those boundaries. Working on Dream Serpent helped me realize that tying that observability across multiple different layers was harder than it seems, so it's important to make it a first class citizen in the architecture. It has been especially cool to see the same idea show up in OCE, where audit, runtime state, and enforcement evidence are treated as core parts of the platform rather than something bolted on afterward. For me, that is one of the more interesting ways these ideas have continued to evolve as they hit real world environments. Dream Serpent started as an experiment in constraining autonomous agents, and in many ways OCE is an idealogical evolution of those same principles but applied to a broader, enterprise control plane. I'm very excited to see where OCE goes from here, you can bet I'll be watching the project closely! Below you can see David Campbell talking more about observability as an overarching theme that should be traced across the agent workload:

Maybe Flagship Models Shouldn't Be World Class Exploit Writers?

2026-09-26T09:11:03.086-07:00

In my experience as a security professional, there is a question missing from the frontier AI conversation: why does a general-purpose LLM need all of these dangerous capabilities we keep giving it? OpenAI has made advanced exploitation an explicit research target. Its own description of GPT-5.6-Cyber, OpenAI says the model is specifically trained to improve cybersecurity capabilities including "finding zero-day vulnerabilities and developing exploit chains." The company separately says the model is trained for workflows involving exploit development and that its improved zero-day performance results from "specialized training." Exploit development is something OpenAI has intentionally decided its models should become better at and pursued training their models to become better at.

It would be misleading to describe these capabilities as merely emergent side effects of general intelligence. ExploitGym and ExploitBench themselves are evaluations to this end that OpenAI has chased better scores in. Astra now scores 100% on ExploitBench and 42.4% on ExploitGym. When OpenAI created a newer benchmark from recently disclosed vulnerabilities specifically to reduce contamination concerns, Astra achieved 39% versus GPT-5.6 Sol's 5.5% and discovered two previously unknown zero-days while developing exploit chains. This is not a general-purpose model accidentally discovering that it knows how to hack. We are deliberately teaching our flagship AI systems to become better software exploit writers. The question is why?

That question becomes much harder to dismiss after OpenAI's own Hugging Face incident. During cybersecurity evaluations, agents went outside their intended tasks, probed surrounding infrastructure, obtained unauthorized internet access, found credentials, exploited vulnerabilities in public infrastructure, and ultimately compromised systems belonging to a third party. OpenAI's retrospective found that similar out-of-bounds behavior had been increasing during a training run that contributed to the model. In their supervised learning environment, that behavior was then reinforced as successful. OpenAI says that reinforcement may have contributed to the model's tendency to use similar techniques during its cybersecurity evaluations. In other words, a reward-seeking system learned that exploiting boundaries could help it accomplish difficult tasks, that behavior received reinforcement, and the resulting model later applied similar strategies where they were never intended.

This exposes a strange contradiction in the current approach to alignment. We worry about reward hacking, instrumental behavior, autonomous agents exceeding their authority, and models finding unexpected ways to accomplish goals, while simultaneously working to give those same general-purpose systems dramatically better abilities to discover zero-days, construct exploit chains, compromise hardened software, and escalate privileges. Astra has now reached OpenAI's own critical cybersecurity threshold, which includes the ability to develop functional zero-day exploits against hardened real-world systems or devise novel end-to-end cyberattacks from a high-level goal. Their proposed answer is generally more monitoring, more classifiers, more sandboxing, and more safeguards. Cybersecurity already has a name for a simpler principle, compartmentalization and least privilege. Don't give a system capabilities it does not need and then hope you can reliably prevent it from using them; it sounds like a recipe for disaster to me.

If world-class autonomous exploitation genuinely has defensive value, build a specialized model for it, don't train it into your flagship general purpose model. Give it separate access, infrastructure, permissions, monitoring, and deployment criteria. What requires the world's best general-purpose model, the model we ultimately want browsing the web, operating computers, and pursuing long-horizon goals, to also be exceptionally capable at compromising the systems around it? The burden should be on frontier labs to answer that question before continuing to add more offensive capability. Alignment should not only ask whether we can control what a powerful model does. It should ask whether the model needed to possess the dangerous capability in the first place. Otherwise, this stops looking like thoughtful engineering and starts looking like an arms race in which capability itself has become the goal. These systems will have a massive impact on the societies they serve, thus as a society we should re-evaluate what the intended purpose of these systems are and really what capabilities labs should be focusing on to that end.

Book Review: "The Scaling Era"

2026-09-08T19:33:36.035-07:00

"The Scaling Era: An Oral History of AI, 2019-2025" by Dwarkesh Patel with Gavin Leech, is a riveting conversation among the people who helped create, study, fund, and interpret about frontier AI development. The Scaling Era is an amazing oral history of the AI scaling revolution, from the years 2019-2025. Patel cuts his interviews together across eight subjects: scaling, evals, internals, safety, inputs, impact, intelligence explosion, and timelines. The result feels almost like sitting at a roundtable where people with sharply different rationals keep answering one another indirectly. Underneath the surface of the interviews, it is repeatedly asking questions like, "What actually determines what an intelligent system is, what real progress has been made in the last 6 years, and how far away from AGI are we really?" I listened to the book on Audible at roughly $15 for ~6 hours. Overall I give the book 7 out of 10 stars, and recommend it anyone that is interested in technology history, especially those interested in AI. The book is great because it is so densely packed with information and great interviews, and everyone in the book speaks very fast, so I couldn't imagine listening to this on more than 1x speed. I've even found myself returning to the conversations over time as they have such valuable insights. The following are the chapters of the book, in my typical review style, so you can get a better idea of the narrative journey it takes us on:

Preface
Chapter 1: Scaling
Chapter 2: Evals
Chapter 3: Internals
Chapter 4: Safety
Chapter 5: Inputs
Chapter 6: Impact
Chapter 7: Explosion
Chapter 8: Timelines
Conclusion

That doesn't really cover the contents of the book though, as it's mostly interviews with tons of AI critical thinkers and builders. The exact interviewee roster in the book's “Profiles” section is 20 people: Dario Amodei, Leopold Aschenbrenner, Gwern Branwen, Trenton Bricken, Joe Carlsmith, François Chollet, Ajeya Cotra, Tyler Cowen, Sholto Douglas, Demis Hassabis, Jared Kaplan, Holden Karnofsky, Shane Legg, Dylan Patel, John Schulman, Carl Shulman, Ilya Sutskever, Jon Y. of Asianometry, Eliezer Yudkowsky, and Mark Zuckerberg. It is also unusually good at moving between levels of abstraction. One minute Jared Kaplan or Dario Amodei is discussing scaling laws; the next Shane Legg is talking about memory and learning; Sholto Douglas about agent reliability; Trenton Bricken about representations; Joe Carlsmith or Eliezer Yudkowsky about goals and control; Dylan Patel about chips and infrastructure; Tyler Cowen about economics. That said, Dwarkesh explicitly acknowledges that many interviewees are founders, developers, researchers, investors, or equity holders who are strongly selected for believing AI will matter enormously. So this is an extraordinary inside view, but not a representative social history of AI. There are comparatively few voices from labor, sociology, education, psychology, public administration, ordinary users, or communities affected by deployment. Still, the interviews contained within are powerful, insightful, and from some of the most "in-the-know" people in the industry. Also despite the “2019–2025” title, Patel states that the book's knowledge cutoff is November 2024. The book is best treated as a remarkably well-preserved snapshot of what leading participants believed at the end of that scaling era phase, rather than a current account of everything that followed. Some of the best takeaways for me were that while the model supplies a space of possible behavior; the surrounding context, training, memory, tools, incentives, evaluation, and environment greatly influence the set of outcomes the agent ends up selecting. As such, the book is surprisingly relevant to agent architecture, even though it isn't marketed as an agent-design book. The following is an interview with Dwarkesh where he does an overview of this timeline of the Scaling Era, extracting many of the points from the interviews:

Behavior, Identity, and Controls for Coding Agents (Introducing Dream Serpent)

2026-09-01T10:16:52.426-07:00

A few weeks back, my friend Free and I had been working on an open-source project called Dream Serpent, an experiment in what the infrastructure around autonomous coding agents should look like. The basic premise is simple: constrain the machine, not the model. Models are probabilistic. You can instruct an agent not to do something, but once you give it a shell, network access, credentials, and the ability to run arbitrary code, you can't really be sure that it will stick to those instructions (for any number of reasons). Dream Serpent approaches agent work from the assumption the agent environment may be compromised and designs the system around that assumption.

A lot of these ideas stem from conversations I've been having with David Campbell on the fundamentals of secure AI systems. We've been framing AI architecture through three ideas: Behavior, Identity, and Controls (BIC). Let's dive into some of those properties below, and how we implement them in Dream Serpent.

Behavior

An agent’s behavior is useful, but it isn’t something we should have to trust. Instead of trying to perfectly predict whether an agent will behave, Dream Serpent puts observable actions, network access, credential use, writes, and policy decisions, through infrastructure that can allow, deny, record, or escalate them. The default of the orchestrator is to fail-closed. Unknown or disallowed behavior becomes an error the user can often respond to rather than an opportunity to escape its boundary. This changes the question from “Can I trust this agent?” to “What can this agent actually do?”. That said, we think shaping behavior is a big part of guiding an agent to success. Dream Serpent offers the ability to load golden images, redeploy snapshotted workloads, and pre-load harness and context files for various subagent workloads. Separating the behavior, capabilities, and environment not only allow us to better control the agents, they allow us to better guide them to success as well. Dream Serpent was written as a productivity tool first, so making the dev environment snappy and fully featured was a primary goal of ours.

Identity

Agents shouldn’t simply inherit the identity of the human running them. In Dream Serpent, each session is designed to receive its own short-lived workload identity, associated with the human who launched it. This delegation chain is critical for both making auth decisions as well as forensically tracing actions. Even sub-agents can receive narrower identities and be traced back to their original tasks. Long-lived credentials are also designed to stay outside the agent VM. The agent carries limited credentials, while a proxy boundary performs credential exchange at the point of use. That gives us a much cleaner model of delegation:

User → agent → sub-agent

Each step can receive less authority than the one before it and actions can be attributed to the workload that actually performed them.

Controls

Finally, controls only mean much if the agent being controlled cannot rewrite them. Dream Serpent places its most important guardrails outside the agent VM. VM isolation, default-deny networking, DNS gating, an egress network proxy, policy enforcement, credential handling, and audit records all sit across a boundary the agent cannot control. The system even defines an enforcement ladder, to allow, block, ask a human, or kill and snapshot, rather than turning every uncertain action into an approval dialog.

The broader idea behind BIC is that useful agent safety probably won’t come from any one mechanism.

Behavior tells us what the agent is doing.
Identity tells us what is doing it.
Controls determine what it is allowed to do.

Put those together and autonomous agents start looking less like chatbots with shell access and more like ordinary distributed systems. They are microservices operating with bounded authority inside enforceable trust boundaries. That’s the direction we’re exploring with Dream Serpent. The project is still early but it's a really fun way to play with these concepts in practice. A real coding agent can run inside the VM today, while parts of the production egress and credential architecture are just defined specs rather than being fully implemented yet. This is an exploration of the architecture we think unattended agent infrastructure needs and we hope others enjoy the work or find it useful. If you just want to hear more about these ideas, you can see David talk about them below or on his youtube channel:

Training for CCDC and CPTC

2026-08-28T18:54:29.360-07:00

I have spent more than a decade around CCDC and CPTC, and the same lesson keeps surfacing. These competitions measure how a team operates under pressure. And while the competitions measure technical skill, that is one input among several. The teams that do well pair their tradecraft with preparation, teamwork, incident response, leadership, and professional communication. CCDC asks you to keep services running while you defend them and answer to the business. CPTC asks you to find vulnerabilities and explain the risk they present to the client. In both, you have to work out what matters to the business, determine where to apply your limited time, make descions with incomplete information, document as you go, and explain your work to people who were not watching you do it. Your job is not to secure everything and not to find every vulnerability. Your job is to rank what matters most, execute against that ranking, and keep the team moving when the environment turns chaotic. I put together a set of slides based on these ideas, covering the operating models, technical checklists, tools, team roles, reporting practices, and lessons I have watched work across both competitions. I also put it into a presentation you can watch below:

One of the major themes of my presentation is that preparation beats heroics. In CCDC, preparation means practicing how to protect the services that score, cutting attack surface, catching and evicting an attacker, finishing injects, collecting evidence, and keeping information flowing across the team. In CPTC it means practicing mapping the attack surface, validating the findings that matter, collecting clean evidence, writing as you go, and presenting risk a client can act on. The technical work differs across the two events, while the team dynamics actually look fairly similar to other. Strong teams assign ownership, automate the repetitive work, keep each other current, keep evidence they can use later, and give people enough room to act without asking permission for every decision. Practice these behaviors before competition day. Break your own services, abuse your known credentials, run stealthy attackers against yourselves, bury a dependency then try to remove it, throw surprise injects, set reporting deadlines, and make people present under a clock. If competition day is the first time your team handles those conditions, you are already behind.

Recently, Cal Poly Pomona (CPP) has put out a much larger collection of some of their training material. CPP placed second nationally in CCDC in 2023 and 2024 and first in CPTC in 2023, so their approach holds up at the top of collegiate cybersecurity competitions. The series below runs nine videos, at more than 10 hours, covering lessons learned, techniques, preparation, and experiences from people who have competed at the top of these events. If you are building a new CCDC or CPTC team or preparing for your next season, spend the time with the whole series. Watch the sessions, take what fits your team, turn those lessons into repeatable process, and drill them until nobody has to stop and think. The teams that have practiced how to think, communicate, adapt, and execute together are the ones who are a real threat come game day.

Building Tools for the Games I Love

2026-08-21T10:02:23.525-07:00

I'm back to chalk up another point for The AI Software Revolution. Lately, I have been building out a ton of fun tools, specifically under the RPG Tools section of tools.lockboxx.org.

From a Secret Agent Live Action Role Playing Game, to tools that can help DMs for D&D, Daggerheart, MORK BORG, and Rogue Trader. I've even made utilities to read video game save files or provide live advice with some community statistics. In just 1 or 2 days, I've made random item generators, guided character creators, entire LARP games, video game file parsers, wiki scrappers, and statical play advisors. None of these are large projects, yet each has it's own custom website and hosting beyond the local tool logic. For most of my career, the bar for writing software sat high enough that making a full GUI for your simple tools added a few extra hours of overhead. However, most of these tools I've made within a few hours total or in a completely automated way, turning that previous paradigm on it's head.

Previously, you had to want the thing badly enough to spend a bunch of extra hours on design, code, debugging, and deployment, and if you were building it at work you had to justify that time against everything else your team could ship. That calculation pushed software development toward problems that were large or profitable. Small hobbyist projects were typically centered on a few projects a year, not a few projects a day or week. AI-assisted development lowered the bar enough that I can proof of concept several ideas in a day and immediately see their impact, faster now than previously imaginable. Low creation costs bring customization back. High production costs force standardization. Three years ago most of these ideas stopped at "that would be cool." Today they end at a freely hosted application, a URL anyone can use, and anyone with model access can add any features they can imagine. The habit matters more to me than the tools. I now catch the twenty seconds of friction while it is happening and make better tools for a thing on the spot, instead of sitting through pain or a lesser understanding for another two years. It makes me feel a lot like Tony Stark or a D&D artificer. I find myself constantly making tools to support my analysis or solution in any problem now. It has been an incredible boon not only for myself, but for other others when I hand them these free, publicly hosted tools. The hard questions now are knowing where to build a tool to get more insight or value. Check out the rest of my cool tools on tools.lockboxx.org. I might need to figure out a new layout, or start breaking the tools page up, if I keep developing at this rate.

A New Era for The Collegiate Cyber Defense Competition

2026-07-28T10:04:50.216-07:00

The National Collegiate Cyber Defense Competition (CCDC) was founded by the UTSA's Center for Infrastructure Assurance and Security (CIAS) for more than twenty years ago and, for the first time, is now entering a new chapter. Starting in the 2027 season, the National Cyber Readiness Foundation, a new nonprofit led by Alex Levinson, will manage NCCDC. The official announcement explains the reasons for the transition, what will stay the same, and how the new organization intends to preserve the competition while preparing it for the future. I encourage everyone in the CCDC community to read the announcement.

But this announcement holds personal significance for me as well. I have been involved with CCDC for more than a decade, often attending two or three events annually. CCDC has become much more than just an annual competition on my calendar. It represents a community I care deeply about, a mission I believe in, and has shaped a meaningful part of my professional life. My relationship with Alex goes back further than my involvement with CCDC. He was actually one of the first people to bring me into this community, as we met through playing the US Cyber Challenge competition.

I can't imagine anyone better suited to lead CCDC. Alex has a deep understanding of CCDC from nearly every angle. He has served as a competitor, sponsor, red team member, competition organizer, and advocate for over a decade. He recognizes that these events matter both as technical challenges and as formative experiences for students, volunteers, coaches, and professionals. Over many years, I've also worked with Alex in a professional capacity. I've witnessed his drive, his talent for bringing people together, his willingness to pursue ambitious ideas, and his fairness as a leader. He possesses the connections, experience, and vision to preserve what makes CCDC special while advancing it further. Alex has also spent years building and running cybersecurity events. We founded the Collegiate Penetration Testing Competition, CPTC, together. In every environment where I've watched him work, few match his strength in bringing talented people together to create visionary, meaningful experiences for the next generation of cybersecurity professionals. Alex thrives in that kind of work.

But leading CCDC involves far more than running just another competition. CCDC has become an institution. Thousands of students, coaches, volunteers, organizers, and sponsors have built it up over the years. For many people, a CCDC marks the beginning of their careers, friendships, and professional communities. CIAS built that legacy and deserves credit in creating such an incredible community. An ambitious educational idea became a nationwide program that shaped a generation of cybersecurity professionals in lasting ways. Cybersecurity, however, continues to evolve especially in the light of agentic systems. The systems students defend, the threats they face, and the jobs they prepare for all keep changing. CCDC must adapt alongside these changes while preserving the pressure, teamwork, unpredictability, and community that made it valuable from the start. Considering the rapid advancement of our AI era, computer work is changing forever, and we need the competition to keep up in that regard. Alex understands both sides of the issue. He respects CCDC's history and can envision its future potential.

The new organization still has details to work through, but I really can't think of a better leader for this new era. As someone who has known Alex for years, worked alongside him, built competitions with him, and shared countless CCDC weekends, I know Alex will take the competition to new heights. CCDC's next chapter is beginning, and I'm excited to see where he and the community will take it. And I'm sure he needs help, so please volunteer! Below is one of my earliest CCDC photos of Alex, Lucas, and I with the DSU student CCDC team.

The Industrial Revolution of Software Is Here

2026-07-11T12:52:39.649-07:00

We are living through an industrial revolution for software development, and I say this as someone who has spent months quietly testing whether the claim holds up. AI-accelerated development has reduced the cost of moving from idea to working proof-of-concept so sharply that experimentation and small tool development now costs about as much as $20 and coming up with a good idea in the first place. That's it. That's the new price of entry.

The historical comparison deserves serious consideration rather than being treated as a catchy headline. The first industrial revolution did not make craftsmen obsolete, but it shifted what the limiting factor was. Before power looms, the constraint was hands. After, the constraint became design, logistics, and judgment. Software is undergoing the same shift right now, where typing code is no longer the limiting factor. What matters now is deciding what's worth building and having enough knowledge to assess the results.

I don't trust claims like this without evidence, so here's what I've actually done. In recent months I've created dozens of small tools, with most moving from concept to working prototype in under an hour, and several completed in a single session:

Claude Instrumentation & Analysis: CIA: External, passive monitor for Claude Code sessions.
Casual Bayes: A Bayes' theorem calculator for everyday use.
SecDigest: A tool to help manage mailing lists
Mork Items: A random item generator for Mörk Borg
DaggerHeart Primer: A character generator for the RPG game DaggerHeart
pyart: A procedural art generation scripts
ImageTools Tools to edit sprites with multiple layers and advanced features
VideoStitcher: A tool to help create video montages from images, videos, and music.
Asteroid: A roguelite ship-builder game
Lockboxx Pinball: A full pinball game in a single HTML page
Dreamride: A small browser platformer in a single HTML page

That's in addition to a dozen or so private repositories following the same approach. Several of these tools are available at tools.lockboxx.org, hosted free on GitHub Pages by using only JavaScript and HTML, with no backend, no server costs, and no deployment infrastructure. To be clear, I also host some tools from other creators on that domain to fill out the collection, and not all the content there was generated by AI. The pattern still applies however to new features and extending them for my purposes. The gap between thinking "I wish this existed" and actually having it built and in use has become smaller than ever.

The cost breakdown matters here because I always ground my thinking in cost and time. Historically, a tool like any of these would have required a weekend project at minimum, buying the professional version (and being limited to their vision), or more realistically a week of evenings, which in practice meant most of them simply end has half baked proof of concepts. The idea would have stayed in a notes file. Now the economics have flipped. The build is cheap enough that the notes file becomes the backlog, and the backlog actually gets cleared.

What This Actually Unlocks

You can now rebuild your favorite tools, the ones you've always wanted a slightly different version of, as easily as filing a feature request that never gets answered. Unhappy with how a utility handles some edge case? You can reconstruct the 80% of it you actually use, with your fix included, in an afternoon. This used to be available only to people with a lot of free time and a high pain tolerance.

Once a tool exists, adding a feature takes about as long as it takes to describe the feature. In some cases it literally takes as long as it takes to *speak* it. The iteration loop tightens until the limiting factor becomes how fast you can decide what you want. You can also build smaller tools and iterate on harnesses to make the effort easier and output higher quality over time.

This is the part I think is most underrated, because it enables creation for the non-technical. You no longer need to be a developer, just slightly tech-literate with good taste. Someone with domain knowledge and a clear sense of what they need can often build their vision with little-to-no programming background.

In my typical style, I'm not going to pretend this is all upside, because the failure modes map neatly onto the original industrial revolution too. Early factories had appalling safety records because the machinery outpaced the practices surrounding it. AI-generated code faces the same issue. Creating code quickly is different from understanding it, and vulnerabilities emerge precisely in that gap between the two. When you deploy tools you can't debug manually, you're building up risk that remains invisible to you. For systems that handle real data or real users, security review cannot be skipped, and that work didn't become cheaper, it became relatively more expensive as everything else around it got faster.

There's also the problem of low-quality output. As the marginal cost of creation drops toward zero, the sheer volume of low-effort output grows, and the signal-to-noise ratio deteriorates for everyone. The industrial revolution brought us mass production alongside mass-produced junk. We're getting both.

A real question exists about skill atrophy, whether a generation of builders who never had to fight the machine will develop the judgment to know when it's wrong. I don't have a confident answer to that, but you can see my buddy DCam and riffing on it here and here. These tools amplify whatever judgment you bring to them, including bad judgment.

None of those drawbacks change the core fact. The power is real, present, and undeniable. People who figure out how to harness it with their eyes open to the failure modes will run circles around those waiting for the dust to settle. The dust will not settle. Looms did not un-mechanize.

What once limited us was implementation, but now the constraint is ideas, judgment, brands, resources, and actual infrastructure. This represents a shift at the systems level. If your backlog of "someday" tools is still sitting in a notes file, you're the only thing holding yourself back.

Go Build Something!!

If you aren't creating things right now, I genuinely don't know what you're waiting for. This is an amazing time to be a developer or even just someone with basic technical literacy, and I can't encourage you enough to get out there and start building.

Share your creations with me in the comments, join the mailing list, or send me good ideas, and I'll feature the best ones in a future mailing list issue.

Below Andrej Karpathy's talk "Software Is Changing (Again)" explores the same shift from the model-builder's perspective and is worth watching if you want the theory behind what practitioners are already experiencing:

Book Review: "The Challenger Sale"

2026-06-30T13:05:52.376-07:00

I recently read "The Challenger Sale: Taking Control of the Customer Conversation" by Matthew Dixon and Brent Adamson. This was part of journey to better understand product fit and selling at a startup. I must say this was another great sales book in terms of the lessons learned, as I find this strategy pretty effective. I'm not sure I would keep this around as a desk reference, but once you grab the secrets from it, it can dramatically change your approach to sales and customer acquisition. Overall I give the book 6 out of 10 stars, for being insightful and teaching me a really strong technique, but not being something I would revisit or recommend many other readers. I think this technique is fairly specific to domain experts, or people that can actually teach something about the product or domain they are selling in. I also though the book was way too long as 240 pages. If you are really interested, I actually recommend one of these summary or spark-note type docs. Boom, there is the core concept in 9 pages. If you aren't familiar, its core argument is that the best salespeople teach customers something new, tailor the message to the customer’s business, and take control of the sale. This educating and guiding approach has statistically been shown to have better sales. The call this approach the Challenger sale, as the book is named after. The book compares this to the 'relationship builder' approach, which focuses on developing long term relationships, and shows that the statistics clearly shows one approach closing more deals. The book identifies five sales rep profiles: the Hard Worker, the Relationship Builder, the Lone Wolf, the Reactive Problem Solver, and the Challenger. The authors make the case that Challengers outperform the others, especially in complex sales environments, because they bring insight and constructive tension into the conversation. The following are the chapters of the book in my typical style:

Introduction: A Surprising Look into the Future
Chapter 1: The Evolving Journey of Solution Selling
Chapter 2: The Challenger (Part 1): A New Model for High Performance
Chapter 3: The Challenger (Part 2): Exporting the Model to the Core
Chapter 4: Teaching for Differentiation (Part 1): Why Insight Matters
Chapter 5: Teaching for Differentiation (Part 2): How to Build Insight-Led Conversations
Chapter 6: Tailoring for Resonance
Chapter 7: Taking Control of the Sale
Chapter 8: The Manager and the Challenger Selling Model
Chapter 9: Implementation Lessons from the Early Adopters
Afterword: Challenging Beyond Sales

The most important lesson is that good selling is not just discovering pain. It is creating insight. The seller should help the buyer understand a hidden cost, missed opportunity, flawed assumption, or market shift. Another valuable idea is that customers often prefer sellers who make the buying process easier, not sellers who merely agree with them. A Challenger does not antagonize the customer, but they are willing to say, “There is another way to look at this.” To summarize these ideas, the book presents this as its central framework:

Teach: Bring commercial insight that changes how the customer sees their situation
Tailor: Adapt the message to the customer’s industry, role, priorities, and economics
Take control: Guide the buying process, challenge assumptions, and avoid becoming overly

A challenger is also able to bring up the cost of the product, and isn't afraid to circle back on those topics to close a sale. I found this particularly helpful to always bring it back to how my solutions are relevant and can help take the conversation or problem further than not acting. Granted, if you look online, there are some strong critiques of the book. One of the loudest critiques was that this is hard for non-immersed sales people to actually execute on. The technique really requires knowing the domain, the customer, and the solution all exceptionally well. Well enough that you can have unique insights or teach people who are presumably doing this in a professional capacity. I think this method actually lends it's self to expert practitioners, founders, or people deeply embedded in the problem space over generic sales people, which may be why it appeals to me so much. Another major critique of the book is that the authors aren't actually experienced sales professionals, rather academics who arrived at these conclusions studying statistics. While I think both of these are valid critiques, as a non-sales person I've found these methods tremendously successful for winning people over to my products or services. The following is a snippet of a presentation by the author Matthew Dixon, where he conveys many of these same insights:

Book Review: "The Mom Test"

2026-06-24T09:48:01.651-07:00

"The Mom Test" by Rob Fitzpatrick was a great book because it was so concise, digestible, and not only had important goals, but techniques for achieving those new goals. Its core argument is simple, most people ask customers bad questions, receive polite lies, mistake those lies for validation, and then build the wrong thing with confidence. We should pursue facts and build a fuller picture of what they do in their normal operations. The goal of this book is understanding the customer's world, and getting out of being stuck in your own solutions and pitches. I paid roughly $15 for this on Amazon and read it over the course of two flights. I would ultimately recommend it to founders or anyone building a product that is trying to find a customer fit. Overall I give it 6 out of 10 stars being being a great, simple book for understanding your product fit. I give that rating in a very practical sense. It is a good book with a simple, valuable technique. Once you understand the technique, though, you mostly have what you need. I do not really see myself using this as a desk reference. It is more of a quick calibration tool than a deep operating manual. Questions such as "how many times a day do you typically use X?" are much better questions for at getting to the facts of the matter than "so do you like our product?". The book essentially builds on these lessons throughout the entire book. Honestly as short as the book is, they could probably reduce the length even more. The following are the high-level chapters of the book:

Chapter 1: The Mom Test
Chapter 2: Avoiding Bad Data
Chapter 3: Asking Important Questions
Chapter 4: Keeping it Casual
Chapter 5: Commitment and Advancement
Chapter 6: Finding Conversations
Chapter 7: Choosing your Customers
Chapter 8: Running the Process
Conclusion and Cheatsheet

The book is all about how to avoid asking bad questions or hearing bad answers. Rather the book teaches how to seek out the truth and data you are looking for without sounding like a clinical researcher. I think some of the most important lessons from the book are how to lock in those early customers that will help you hone your product. I found some other really good reviews out there with some great takeaways, such as really staying homed in the customer's world. By homing yourself in their world you can avoid pitching your product (searching for validation), and instead truly try to solve their problems wiht your available tools. Some of the best distilled lessons from the book, for me personally, were the following:

Ask about their experience, not your idea.

Ask about explicit past behavior, these are facts you can use.

Ignore compliments, these are noise unless they are attached to action.

Look for the hidden cost, understand what you are really spending on a customer.

Do not pitch while learning.

Do not confuse a 'good conversation' with actual commitment or action.

Push for commitment or advancement when appropriate.

Beware zombie customers who create motion without progress.

Keep the conversation casual, but keep your thinking disciplined.

Lastly you can see Rob himself speak on some of these ideas in an interview:

Book Review: "How Spies Think"

2026-06-05T03:59:33.351-07:00

"How Spies Think: Ten Lessons in Intelligence" by David Omand a fantastic book on observation, intelligence, and reasoning. David previously served as a director for GCHQ, which is Britain's very famous intelligence, cyber, and security agency. David literally climbed the ranks of the spy world and lead one of the greatest agencies. This book is David distilling his method of thinking and analysis for others. I really enjoyed reading this book. I felt like it gave me a solid framework and tool for making rational intelligence observations on a target. It appears to be an original model that David has created and it's very applicable to generally understanding, reasoning, and and making intelligent predictions about an event. I paid roughly ~$15 for 'How Spies Think'. This is another book I got from the spy museum in Manhattan, but this one was actually good, unlike the last one. I'm really glad that the museum had both content for someone off the street as well as more serious content, and that I got to sample both aspects of that. Overall I give this 8 out of 10 starts for being a very well throughout book, packed with good references throughout history, delivering a unique and useful framework for intelligence analysis. I recommend this book any analyst, or really anyone who like offering informed critique. The book is centered around a core philosophy SEES:

Situational awareness: Observation. What is happening?
Explanation: The why. What motives or causes best fit the evidence?
Estimates: Fuzzy prediction. What might happen next, under different assumptions?
Strategic notice: Preparation. What longer-term risks or opportunities can we glean?

After the core SEES framework, there are three more ideas that help you check your reasoning when doing intelligence observation, followed by three more ideas that show you how you can use and apply your new skills. The following are the chapters of the book to help you understand what it covers and how the entire book frames things around the SEES framework:

Introduction
Part One: An Analyst SEES: Four Lessons in Ordering Our Thoughts
Lesson 1: Situational Awareness
Lesson 2: Explanation
Lesson 3: Estimations
Lesson 4: Strategic Notice
Part Two: Three Lessons in Checking our Reasoning
Lesson 5: Is is our demons that are most likely to mislead us
Lesson 6: We are all susceptible to obsessive states of mind
Lesson 7: Seeing is not always believing: beware manipulation, deception, and faking
Part Three: Three Lessons in Making Intelligent Use of Intelligence
Lesson 8: Imagine yourself in the shoes of the person on the other side
Lesson 9: Trustworthiness creates lasting partnerships
Lesson 10: Subversion and sedition are now digital
Part Four: A Final Lesson in Optimism

Some of my favorite parts of the book, and best lessons, beyond the SEES framework, were the warnings about cognitive bias and deception. Omand does a great job showing that intelligence failures are not always caused by a lack of information. Often, they come from misreading the information we already have, becoming emotionally attached to a theory, or failing to imagine how the situation looks from the other side’s perspective. That is what made the book feel useful beyond the world of intelligence. The lessons apply to business, politics, personal judgment, online discourse, and any situation where someone is trying to separate signal from noise. The book is not just about “spies”, it is about how to think clearly when the truth is incomplete, contested, or deliberately obscured. The following is a short clip with the author so you can get a better understanding of his background, school of thought, and great insights for yourself:

Lockboxx Infosec Newsletter!

2026-05-18T15:38:01.132-07:00

Thanks again to all my loyal readers. I really appreciate any one who reads this blog.

One thing that keeps coming up is just how much garbage news you have to sift through to stay current in security. Every day there’s a new breach, new vuln, new vendor hype cycle, or someone rediscovering a technique from 2017 and calling it revolutionary. To help with all of that noise, I started a security newsletter.

The idea is pretty simple. Curated infosec news, filtered by me, with an emphasis on things operators, engineers, defenders, and people working in the trenches will actually care about. Less noise, more signal. Things that are useful, interesting, or worth paying attention to before they become everyone’s problem.

If you want a cleaner feed of cutting edge security news without spending hours doomscrolling, you might get something out of this. You can check it out and subscribe here:

SecList by LockBoxx

If you sign up, hopefully it earns that spot in your inbox.

The New York Times Watched Us Run Cyber Ops for Two Days

2026-05-16T08:59:55.757-07:00

This year at NCCDC was strange in a way I do not think I fully appreciated until afterward. For two days, while we were running offensive operations against some of the best collegiate defenders in the country, the New York Times was sitting in the room with us. Watching. Asking questions. Trying to understand why half a dozen people would voluntarily spend months building custom malware, agent frameworks, and infrastructure just to stress test students in a national competition. You see the NYT article here, or read the free archive here.

Not only that, but we were in the penthouse suite of the Cosmo, eating the best meals, and hacking at poolside cabanas when we weren't in the room. The suite looked exactly like what people imagine offensive security operations look like. Giant wall displays covered in telemetry. Attack infrastructure sprawled across tables. Music running nonstop. Guys fully immersed in the matrix wearing Apple Headsets. Operators shouting across the room, coordinating attacks, while reporters and photographers moved between clusters of laptops and cables running from table to table. If Hollywood designed a cyber operations center, it probably would have looked pretty close to our red team room this year.

One thing I appreciated about working with Cade over the interviews leading into the event was that he seemed genuinely interested in the operational reality rather than forcing everything into a predetermined narrative. What happened in the room was experienced operators using increasingly strange tools to move faster under pressure. For two days there were reporters from the New York Times sitting inside that ecosystem while some of the best operators I know showed them what did we normally do. I love NCCDC because we really go all out building tools, testing ideas, arguing about the infrastructure, then battle testing our ideas against some of the brightest in the Nation.

The agentic systems helped a lot. They also failed a lot. Sometimes within minutes of each other. That nuance matters. The agentic systems still do extremely strange, often unhinged things. But their value is obvious striking that balance is the golden mixture many people are chasing right now. After extensive testing I think it looks a little more like traditional automation, that calls out to LLMs as part of it's automation routines. This is of course opposed the claudbot or multi-agent approach that we see a lot of currently.

To most people, a Vegas penthouse full of operators coordinating cyber attacks while journalists take notes would probably sound like fiction. By day two, for us, it was just the same war-game that we love to play. Because from the inside, after enough years, spending a weekend in a Vegas suite coordinating cyber attacks starts to feel very normal.

Book Review: "Cybersecurity First Principles"

2026-05-15T07:23:59.493-07:00

"Cybersecurity First Principles: A Reboot of Strategy and Tactics" by Rick Howard is a splendid attempt to boil down what is "cybersecurity" at it's fundamental level, and what are some core strategies for achieving this. If you are like me, you are probably very curious what these First Principles actually are, and you might be surprised to learn this is only one:

"Reduce the probability of a material impact due to a cyber event over the next three years."

I paid $15 for a hard copy of the book and breezed through it in a few days. It reads very easy and is choc full of resources and links, exactly my kind of book. Overall I give this 7 out of 10 stars, for being phenomenally well researched and grounded in it's philosophy, albeit a little high-level for practitioners. The book is full of old references to philosophy and military strategy, so it's immediately the kind of book I love digging through and following the references. That said, I had to give it slightly less stars because at times some of the wording feels very AI written and is thus hard to read for long periods of time. Even so I found the book really thought provoking, and constantly pushing me to ask, "is this a principle? what if we made it more generic?" Regardless of what I think are security principles, the following are all of the chapters of the book:

Chapter 1 — First Principles

Chapter 2 — Strategies

Chapter 3 — Zero Trust

Chapter 4 — Intrusion Kill Chain Prevention

Chapter 5 — Resilience

Chapter 6 — Risk forecasting

Chapter 7 — Automation

Chapter 8 — Summation

As the chapters lay out, after the First Principle chapter, Chapter 2 covers the strategies from a high level, then Chapters 3 - 5 are the literal strategies for pursuing the First Principle. This felt a little redundant to me, like Chapter 2 could have easily been rolled into the intro. The Zero Trust chapter is also useful because Howard refuses to let it be a product category. He treats it as architectural posture. assume breach, segment aggressively, authenticate every transaction, which is what the original framing always was. He references several cool things like the NIST design and original paper. But the chapter also veers off into vuln management and several other topics. When we get to the chapter on the Intrusion Kill Chain Prevention strategy, the comparison I keep coming back to is my own book, Adversarial Tradecraft in Cybersecurity. Howard and I are really saying the same thing from different angles. In my book, I walk through each stage of an attack and ask what the attacker is doing, how defenders can disrupt it, and what the attacker is forced to do next if that disruption works. That naturally leads to the same conclusion Howard reaches from the boardroom. The best security investments are the ones that interrupt attackers early and at the key choke points where adapting becomes harder, louder, and easier to detect. The Resilience and Automation chapters are a little unfocused in my opinion. Using examples like Netflix's Chaos Monkey or Google's tech strategy is a little too lofty, scattered, and expensive for a normal person. Those practices emerged from organizations with mature distributed architectures and extensive engineering investment. Most security teams operate under radically different constraints, making the translation from principle to implementation less obvious. Lastly, the Risk Forecasting chapter is probably one of my favorite. I love the throwbacks to "How to Measure Anything in Cybersecurity Risk" as well as the use of Bayes Alogorythm. Overall, I really enjoyed the book as it made me think deeply about various aspects of cyber strategy. Below you can see Rick Howard and G Mark discus a bunch of the strategies and approaches in the book:

Red Teaming at NCCDC 2026

2026-05-09T16:16:00.000-07:00

This year was one of the best NCCDC events I've ever played in. The environment was good. The blue teams were top-notch. The red team had new tools, old friends, and some killer 0day. Nearly every match-up turned into a long-form operational dogfight instead of a quick pop and dominate. The final standings ended up being:

1st place: Dakota State University
2nd place: University of Virginia
3rd place: Western Washington University

First I need to mention how sick the environment and our tooling was this year. Alex also writes about it here, but essentially it was a water treatment plant themed around some lovable pocket monsters. The infrastructure felt believable, the ICS components were integrated well, and the whole thing had a ton of personality. On top of that, White Team did something incredibly interesting this year. Going into the competition, we told them we planned to bring some legitimate 0day research targeting ESXi infrastructure. Rather than turning the event into an uncontrolled hypervisor bloodbath, White Team worked with the red team to create a controlled implant prebaked into the environment itself. Instead of live exploitation, we had an evenly deployed, pre-planted ESXi management backdoor that teams were gradually made aware of through threat intelligence notifications and environmental clues during play. It took our normal ESXi exploitation and evenly applied it to all teams, rather than cursing just a specific few teams with a significantly harder experience. Don’t get me wrong, this was very much a 'final-boss-level' computer security problem to hand blue teams. But this is Nationals, and part of the fun is keeping the threats fresh, forcing teams to deal with problems they probably have not seen before, and making sure the environment evolves beyond the standard “reset passwords and block SMB” playbook.

This year I was up against University of Central Florida (UCF), which continues to be one of the strongest CCDC teams in the country. They have an incredible program there and always present a serious challenge to the red team. UCF brings real operational maturity and professional grade tools to the competition. They come prepared with a defensive strategy that evolves throughout the event and involves adversarial strategy comparable to the red teams. They are one of the few teams that consistently adapts to the red team during play. This year was no different and I really love playing that game (it's rare). I actually figured out it was them during the competition because the epic tooling they bring comes with it's own mascot as well, SnoopyOnSecurity. They brought custom tooling including Red Baron 2, a proprietary Linux EDR-style agent that was actively identifying persistence, removing backdoors, and feeding telemetry back to defenders. They also deployed increasingly aggressive network monitoring stack known as Peanuts. As the event progressed, Peanuts passivly monitored most red team access then on day two they did a big flip to kick us out of scored services and shutdown our various c2 channels. At multiple points during the event they effectively changed the shape of the battlefield using their Palo Alto. Their ability to adapt operationally throughout the event was impressive. Most teams either collapse immediately or bunker permanently. UCF actively evolved throughout the engagement to monitor and respond to us. They hardened aggressively, deployed custom detection logic, constrained attacker movement, and thus forced the red team into progressively narrower operational paths.

While the dogfight over the individual machines continued, what actually sunk the team was the damage we were able to do their OT systems. They spent a ton of their time hardening their traditional "general computing" environment, but left their SCADA water treatment systems generally exposed throughout the life the competition. There were multiple paths into SCADA infrastructure including exposed Grafana instances, Node-RED deployments, MQTT brokers with anonymous access enabled, and just straight Modbus TCP devices with no authentication. Once access was established, operational impact became straightforward and devestating. Grafana service account tokens survived password rotation and allowed extraction of SCADA telemetry through the proxy interface. Node-RED provided command execution through malicious flow deployment. MQTT retained messages allowed persistent manipulation of the systems. And the Modbus infrastructure was particularly fragile. Several PLCs exposed unauthenticated register writes over port 1502. Continuous manipulation scripts were looped that repeatedly overwrote flow rate, chlorine values, pressure readings, pH, turbidity, and alarm states. What's even more wild is that Claude agents drove many of these OT attacks, showing that commercially available models are more than ready to attack critical infrastructure systems, given the right pretense.

Most red teams started using the Dog-Tunnel and HyperSpace access on day 2 to attack domain controllers directly, usually by targeting DNS and trying to collapse the entire environment. My team spent a significant amount of time attempting the same thing against UCF. By that point, they had applied genuinely impressive local hardening to the DC. Application controls were tighter, local firewall rules were aggressive, and most of the blind command execution we attempted failed for unknown reasons. We could touch the system, but not in a way that let us execute anything operationally significant. I also thought if I decided to revert it, it would very quickly come back to this state and risk this amazing access I had. However the team's Achilles heel was their network configuration and architecture. Their Palo Alto appliance had become the backbone of their defensive posture. It was responsible for almost all of the network monitoring, filtering, and traffic suppression that had gradually forced the red team out of the environment. They smartly rerouted many of their machines behind the in-line Palo Alto device and used this to monitor and block lots of traffic. But the fatal flaw in this design was they placed their ESXi host controller behind this firewall (apparently because of the notifications they were getting from WhiteTeam about our ESXi exploits). I was able to revert their Palo instance to an earlier snapshot through our HyperSpace backdoor and the effect was immediate. Their filtering collapsed and the environment destabilized hard. For roughly two to four hours many of their services were entirely unavailable before they eventually appeared to hard revert portions of the environment back to an earlier state, resulting in a massive point loss. I don't think this lost them the competition, but it was certainly a great way for the red team to leverage their network architecture against them. The issue was that when it failed, it failed in a way that disrupted their own ability to manage and recover the environment. The defensive stack became a dependency instead of a resilience layer, and once the center collapsed, recovery became significantly harder to near impossible due to the remote nature.

I took a big experiment this year using an agentic swarm I made using a custom harness. I took a ton of my old techniques, broke them in agents, then fed the agents rules and skill files dynamically based on the task. I orchestrated them naively through tmux and a flat csv file system to start, which actually worked surprisingly well. I'm working on new versions now that use databases and message queues. It's super strange because it does make some stuff faster but also many normal operations can be slower. I brought a fairly mature CCDC-focused agent swarm framework into the competition. The system was designed around distributed operational assistance rather than full autonomy. Agents handled repetitive enumeration, credential validation, persistence verification, pathfinding, and operational state tracking while I focused on higher-order decisions. In some ways it worked incredibly well. The agents dramatically accelerated:

credential spraying
service validation
host categorization
persistence verification
privilege escalation
operational documentation
environment mapping over time
web app exploitation
familiarity with lesser known systems

But the tradeoffs were real. Agentic systems introduce latency in places where experienced operators normally move instinctively. There is overhead in task routing, tokenization, querying a network service, querying an llm, result validation, and state synchronization. During the first few minutes of competition that overhead matters a lot. The result is a strange paradox. The system makes you simultaneously faster in some ways and slower in others. I think you are faster operationally across multi-hour engagements where rapid parallelization is required. However the longer the agents run the more they lose their original context and become more "unhinged". After running for several hours, they seemed to have lost track of their original agent files and thus true purpose, and the agents began drifting into one anothers' lanes. At one point, one of the agents even tried to execute some malware locally, but due to the sandboxing it was ineffective. The agents are also a lot slower tactically in the first moments of exploitation. Particularly, if you already know exactly what you want to do, then traditional automation works a lot better in many places. Essentially, while I think the agentic claude-code loop of an agent driving operations feels good, it is probably not the right long-term engineering solution. Rather, I think the more mature solution actually looks like traditional software kill-chain automation (think metasploit), that just queries an LLM when it needs to, rather than giving the agentic swarm more control. I almost certainly will follow up on these agentic thoughts soon in another post, but I wanted to get some of early operational thoughts down here.

Infosec Training Courses Available - Train Directly With Me

2026-05-02T18:25:00.000-07:00

Hey all, I wanted to take a moment to say that I run training sessions in a variety of subjects. Not only is this an affordable way to hang out with me, it's a great way to learn a bunch of topics from me first hand. Over the years I’ve had the opportunity to work across offensive security, detection engineering, purple teaming, security engineering, large scale corporate engineering, AI systems, and competitive cyber operations. I’ve decided to open up a limited number of direct training engagements to the public, for those that want practical, hands-on instruction from someone with some deep experience in the field.

I offer both in-person and remote training sessions designed around real operational experience, not recycled certification material or generic slide decks. I've lead word class red team operations against fortune 100 companies and I've gotten real attackers indicted. I've presented to the board and testified in front of a grand jury on a cyber incident. I have a wide swath of experience with large corporate engineering challenges.

I’m also the author of the bestselling cybersecurity book Adversarial Tradecraft in Cybersecurity, which focuses on modern offensive and defensive techniques refined through real-world network operations and adversarial tradecraft. not only that but I'm working on a new book titled AI Security Engineering that applies decade old security learnings to new LLM powered systems in a large corporate eningeering context.

All of the courses consist of the following:

Two 3-hour sessions
6 hours total instruction time
Interactive discussion and Q&A
Practical workflows, tooling, and operational methodologies
Building tools and solutions live to your challenges
Available online or onsite

The idea is an in-depth session with me where we can dig into specific material relevant to either your current problems at work or skills you want to work on improving. The sessions are really designed like educational courses to help transfer several skills and workflows I've developed over the years. The following courses are offered in either a Basic or Advanced versions. This is a lot of content that I have pre-canned, that I've developed over my career, but I can easily customize or pivot any of these sessions:

Building a Vulnerability Management Program
Building Red Teaming Capabilities
Detection Operations in Practice
Purple Teaming in Practice
How to Win at CCDC
Building an Agentic Framework
AI Security Engineering
Zero-Trust Security Engineering

The general price is $3k* per student, per course. Prices may vary depending on the circumstance. Generally this is pricing designed for people to spend their corporate training budget but directly with me on a topic I have deep knowledge in and they are interested in. This could also be a good opportunity to spend corporate budget on some dedicated, 1:1, career coaching. One of the best parts about a training like this is we can tailor the material to exactly what you want to work on as opposed to a pre-canned course where you have to follow a conversation or program on rails. These courses are designed with some of the following groups in mind:

Security orgs looking to level up a team's operational capability
Engineers transitioning into new offensive or defensive security roles
Engineers looking to develop a 'senior' perspective or skill set
Organizations wanting tailored, expert-led training
Professionals using corporate education or training budgets

If you are interested in a course like this you can email me at: training@lockboxx.org
* These prices are subject to change based on demand and circumstance

Book Review: "The Infosec Survival Guides"

2026-04-30T16:55:00.000-07:00

Welcome back y'all. I recently read all four of the current Black Hills Infosec Survival Guides: The Green (intro), Yellow (meta), Blue (SOC and blue team operations), and Orange (incident response) Books. The following is a quick review of all four, to see if they are right for you. Each book shares the same format: sections running 1-3 pages each, community contributor bylines, Loggy the cartoon log mascot in the margins, and a multi-page Antisyphon course catalog at the back. Once you've seen the pattern in one, you've seen the structural shape of all four. The main differences are in the focus of their contents, which you can see in their individual table of contents below. That said, the articles seem too shallow and unfocused to be practical desk references to me. As I explored all four books, I found them to contain great intro articles, but it didn't feel like they held a ton of value in revisiting. Each article barely has enough time to introduce the subject, then it has time to make about 2-5 points, which come with some heavy assumptions about how the user is operating. The end result is a bunch of introductory articles with essentially a subject intro and 1-2 nuggets of useful information. This actually lends itself best to the early articles in my opinion because it works best as an intro to Security as a whole, as opposed to technical reference or something you will revisit time and time again.

The first book in the series is the Green book, which I think was really just their first attempt at trying this, which you can kind of see later refined into the Yellow book material. Let's take a look at the contents of the green book:

Choose Wisely
Common Cyber Threats
How to Set Smart Goals
Use Your Home Lab
OSINT
Understanding GRC
Malware Analysis
Cloud Security
Lead Effective Tabletops
Backdoors & Breaches
Network Engineering
IT Help Desk
Hire The Right Person
Secure Small Business
AI for Good
Umm, Actually...
Who is BHIS?
Antisyphon Course List

I think this zine really is a great introduction to information security. Letting people dive into common cyber threats, setting up a home lab, helping them through early career advice is all great stuff. That said I don't think someone should be running malware only several pages after learning what some fundamental infosec threats and risks are. The Malware Analysis article assumes you already understand what a normal execution environment looks like. If you don't know that svchost.exe should always have services.exe as a parent, or why explorer.exe spawning cmd.exe is worth flagging, running a sample in a sandbox just produces output you can't interpret. You'll see registry modifications and process injections and have no baseline to compare them against. The article teaches you that the discipline exists, not how to practice it This isn't the worst if you are just getting introduced to the subject, but it can set you up for a bad time if you are actually trying to execute on these things as a novice practitioner. The OSINT article has the same problem in a different domain. Real OSINT is pivot chains, source corroboration, and knowing when you've actually exhausted a lead versus when you've just hit a wall. Two pages gets you a definition and a tool list. That's not a methodology, and a tool list without methodology is how people spend three hours in Maltego and conclude the subject doesn't exist online. I also think this is a bit of a Black Hills marketing play, as opposed to a real handbook. Don't get me wrong, there is a lot of value in these texts. But the big tell for me is in the page allocation, and we will see this again and again throughout each book. Backdoors & Breaches, a card game BHIS produces and sells, gets as much real estate as any single technical article. Often times it gets almost double the length of a normal article. The Antisyphon course catalog at the back also runs several pages. These aren't editorially neutral inclusions, they're product placements in a format that implies field reference. A real field guide doesn't sell you a companion course.

The Yellow Book on the other hand seems to acknowledge this a little bit and be another attempt at tightening the scope. I believe this book is kind of a refinement of the Green book as there is a decent amount of overlap between the two. Each part is about 1-3 pages long:

Choose Wisely
5 Phase Plan
Quality Training
Build a Home Lab
To Cert or Not to Cert
How to Get a Job
Social Engineering
Blue Team
Security Operations Center (SOC)
Threat Hunting
Red Team
Pentesting
Backdoors & Breaches
How to Tell a Client No
How to Get a Yes
Purple Team
Incident Response
Digital Forensics
How to Write Reports
Trials and Joys
Mental Health
Protect Your Privacy
How to Put Yourself Out There
Who is BHIS?
Antisyphon Course List

This book honestly still comes off as a little too Black Hills skewed, and frankly a little unfocused. I was never really sure if the guide was on infosec skills, or the meta of the industry, or jobs within the industry. "How to Get a Job" and "Mental Health" are sitting in the same guide as "Threat Hunting" and "Digital Forensics." I suppose what connects them isn't subject matter, it's audience. People new to the industry who want orientation on everything at once. This also creates its own limitations though, the 'Threat Hunting' article can't go deep when the book's implicit audience is someone fresh to the industry. I think if it was more logically organized it would come off as cleaner and easier to grok. That said, I do think it's a strong resource for people who are brand new to the industry. It's a good way to orient oneself and it's a decent primer on some different fields in infosec. This again comes off as if it would be helpful for new people, but I'm not sure it's the desk reference that they were looking to make. By the time you crack open the Blue and Orange books, you've seen the pattern. BHIS has a template for each book that isn't changing. Each article runs 1-3 pages, written by community contributors, doodled over by Loggy the cartoon log mascot, and closed out with a multi-page advertisement for their training catalog and Backdoors & Breaches. These two books are the most technically focused of the series, and that's both their strength and the clearest lens through which their structural problems become visible. They are still going for the desk reference text, like a replacement for the red team or blue team field manual. I just don't think it quite nails the reference text. None of the format really feels like cheat sheets, or things I would reference over time. Rather it still feels like intro material. I think I would get the most value out of it the first time I read it or understood the ideas but not much after that.

Let's look at the the Blue Book's table of contents:

One Fight, Many Defenders
The Life of a SOC Analyst
Threat Hunter Methodology
EDR 101
Log Analysis Field Guide
SIEM 101
Network Traffic Analysis
Cloud Basics and Investigation
Backdoors & Breaches
Report As You Go
SOAR 101
AI in SOC
SOC & GRC
Detection Engineering
Application to Advancement
John Strand's 11 Core Skills
Who is BHIS?
Antisyphon Class List

The Blue Book is the most focused of the four guides. The focus is clearly SOC work. It includes topics such as monitoring, detection, triage, and the daily grind of the analyst. If you're new to the field or trying to understand the SOC ecosystem conceptually, this is a genuinely decent orientation. The best articles in this book are the ones where someone sat down and tried to teach a skill rather than describe a role. The Network Traffic Analysis piece by Troy Wojewoda is really solid, sensor placement, flow directionality, east-west traffic as the canary in the coal mine. These are things a working defender actually needs to understand. And John Strand's 11 Core Skills at the tail end is probably the single most condensed piece of useful framing in the book. His point that we're training analysts to find needles in haystacks when they don't know what hay looks like is exactly right, and it's the kind of insight that belongs at page 2, not page 34. But those articles are surrounded by a lot of ambient noise. There's a lot of articles that can be read once or skipped all together. The format forces every article to stay at the same altitude, intro-level, regardless of whether the topic demands more. The biggest structural tell in the Blue Book, again is the page count. Backdoors & Breaches gets four pages, tied with zero other articles for most space. Meanwhile, articles on SIEM, SOAR, cloud security, and detection engineering only get two pages. The card game advertisement gets double the real estate as anything technical in the book. That's not a neutral editorial decision. Further, Backdoors and Breaches is largely a novelty. Don't get me wrong, it's cool they made an interactive card game and made it competitive. This is coming from an avid Magic the Gathering player. That said, I have never once used Backdoors and Breaches in a serious work scenario or in place of a real tabletop exercise. The critique is that a guide for SOC analysts is spending a large chunk of its technical page budget on a self produced novelty rather than a concrete Windows Event ID cheat sheet or an example of a SIEM correlation rule. Again this isn't something RTFM or BTFM would do. Ben Clark's Blue Team Field Manual gave you 180 pages of commands, one-liners, and reference tables.

Moving on to The Orange Book's table of contents:

IR Manifesto
"Goldilocks" Alert Review
Your Critical First-Hour Response
KAPE 101
A Simple, Useful IR Plan
Incident Investigations
Hayabusa 101
Know Your Enemy
The MITRE ATT&CK® Framework
Common IR Findings
Backdoors & Breaches
Containment & Eradication
DeepBlueCLI 101
Forensic Data
Business Impact Planning
Velociraptor 101
Calling Reinforcements
After the Dust Settles…
Who is BHIS?
Antisyphon Class List

The Orange Book is the best guide in the series. It's the one that most closely resembles what these publications were trying to be. The focus is tight, incident response from initial triage through post-incident review and several of the articles here are pretty good. The tool "101" articles are where this book earns its keep. KAPE 101, Hayabusa 101, DeepBlueCLI 101, and Velociraptor 101 all follow the same, practical structure. They are solid, usable guides. If you're a junior responder who has never run Velociraptor, you can read that article, download the binary, and run a VQL query against a process list inside of thirty minutes. That is field utility. That is the spirit of a field manual. I do wish these field guides contained more commands and maybe a few tips and tricks, but these articles are the closest thing to it, imo. One of the most shocking parts is the entire book is on the incident response life cycle but never really includes the incident response life cycle in full. There is an early article that shortcuts the lifecycle to 3 stages rather than 6. But I don't think you can teach shortcuts without first teaching the full process, otherwise you are shortcutting the lesson rather than the process. The 'Containment & Eradication' section delivers one of the most honest pieces of IR guidance in either book: disconnect the internet and rotate all credentials. "Simple doesn't mean easy." That framing is correct. Most organizations that get hit with ransomware don't fail because they didn't know these were the right moves. they fail because nobody mapped out the blast radius in advance and nobody practiced what credential rotation at scale actually looks like under pressure. The article drives this home effectively. The First-Hour Response article is the longest piece in the book at four pages (notably, the same as Backdoors & Breaches). It earns those pages. The structure, preserve evidence first, then triage, then communicate, is really good, and the escalation tiers (supervisor, security leadership, exec leadership, legal) are genuinely useful scaffolding for someone who has never run a real incident before. Incident Investigations by Patterson Cake is also strong. The MIND framework (Memory, Identity, Network, Disk) is a clean mental model for attack surface categorization, and the Windows/Linux artifact lists are the kind of concrete reference that belongs in a book like this. "Prioritize ASEPs, scheduled tasks, the MFT, and event logs" is something a responder can actually execute on. Compare that to the Blue Book's "know what's normal" advice, and the difference in operational altitude is clear. Where the Orange Book stumbles is the same place the Blue Book does, but the failures are more visible because the subject matter demands more depth. The MITRE ATT&CK® Framework gets exactly one page. One page, for a framework that entire training courses are built around. The article correctly identifies ATT&CK as a gap analysis goldmine and gives three full technique examples but then it's over. This is the exact wrong inversion: Backdoors & Breaches gets four pages; the framework that should inform every detection rule and hunting hypothesis gets one. Common IR Findings is also squeezed into a single page. It's structured around the Backdoors and Breaches (B&B) cards: Crisis Management, Isolation, UEBA, Endpoint Analysis, none of which I've ever seen as top level findings in an incident response report. These are like informational things you would bring up w/ the client, not real IR findings (like systemic malware or active attackers) which is a clever idea in theory, but in practice means the article is half-organized around teaching you about B&B rather than about the findings themselves. The marketing apparatus shapes the editorial even when it doesn't intend to. Finally they should just toss the IR Manifesto at the beginning of the book. The IR Manifesto that opens the book is two pages of "we are incident responders, we are curious, we serve and protect." It reads like the kind of thing you read at a conference keynote and then forget on the drive home. It takes up space that could have been a triage checklist. It's largely useless and no one is even remembering it, let alone ever repeating it.

The Blue and Orange books are the best two in the BHIS Survival Guide series, which is both a compliment and a qualification. The Orange Book in particular has genuine field value, primarily from the tool 101 articles and the Incident Investigations MIND framework. If you're a junior IR analyst who has never touched KAPE or Velociraptor, this book will save you time. Overall these books are more like intro texts for someone totally new to the topic. The Blue Book is a decent orientation for someone entering the SOC world, but it's more of a career primer than a field guide. John Strand's 11 Core Skills is the article everyone should probably read in my opinion. Neither book is what the RTFM/BTFM was. They don't have the density, the command-line specificity, or the format-as-function that made field manuals like the RTFM genuinely reference-worthy. They're closer to a well-curated blog anthology, which is essentially what they are, because the model is community contributors writing 2-page articles. That's not a bad model for building an audience and generating content at scale. But that model has several drawbacks. The most noticeable drawback to me is there is very little cohesiveness between the articles. Some articles approach the defensive posture like a SOC analyst looking at a well instrumented environment, and other articles approach the subject like a researcher or hobbyist working on single log files or samples. Worth buying at the paper price point for someone new to infosec? Sure, these are good and cheap zines to read if you are new to infosec. Is this worth buying as a desk reference or as a seasoned infosec practitioner? Check out the pdf and you can probably skip the hard copy. Think of them as a well-intentioned sampler platter, it's little nuggets of each idea to expose you to a topic. It's great for new people, but the value rapidly diminishes the longer you've been in the industry.

Don't Run This Game: Inside the Myth Journey Malware Campaign

2026-04-19T17:05:00.000-07:00

TL;DR Don't run random "games" sent over Discord, even from friends. Even if they're hosted somewhere that looks legitimate. Google it. Upload it to VirusTotal first. The campaign we are going to look at today steals credentials, crypto wallets, and browser sessions, then spreads through the compromised accounts it just emptied. We are going to look at Myth Journey or https://myth-journey.com

If you already ran it skip to the bottom, to "If You Ran It" immediately.

A friend messages you on Discord.

They want you to test their game.

You run it. Nothing happens.

48 hours later, your accounts are gone.

This is already happening. The Myth Journey campaign is active as of April 2026, and it costs the attacker nearly nothing to operate. I learned about it because my personal friend was compromised and asked me to "test his new game". Talk about sending your malware to the wrong person.

Why This Attack Is So Cheap to Run

This malware doesn't need its own infrastructure anymore. It borrows legitimacy.

The Myth Journey landing page at myth-journey.com runs on Vercel's free tier, the same platform used by thousands of legitimate developers. The payload is hosted on GitHub Releases, with gofile.io as a fallback. The installer is 86 MB, indistinguishable in size from a real game download. Every piece of this sits on infrastructure the attacker didn't build and doesn't pay for.

The underlying malware, Myth Stealer, is sold as a subscription service on Telegram, payable in crypto and Razer Gold. A buyer doesn't write code. They pay a fee, get access to a builder, and assemble their own delivery chain. The Myth Journey campaign is one operator's version. There are others running parallel variants under different names with the same payload kit.

The result is a fully functional, professional-looking attack with near-zero overhead. A GitHub link or a Vercel URL is not a trust signal. The HTTPS padlock, the polished page, the fast download speed, none of these mean anything.

Why People Fall For This (And Still Will)

The attack spreads through social engineering on common chat platforms.

Everyone wants to play new games with their friends. Especially games their friends wrote. But your friend didn't send you that. Their hijacked account did.

Discord tokens, the session credentials that keep you logged in, are a primary target for this entire malware category. Once stolen, a token gives an attacker full account access without the password, without triggering 2FA. They can read your message history, understand your relationships, and craft targeted messages that fit the conversation. The account looks and sounds like your friend because it has your friend's entire history to draw from.

The lure is a game because it's plausible. Indie development is common. Beta testing is expected. The ask is small. There's no obvious red flag in the request itself, which is the point.

The download page adds one more nudge. The instructions read: "do not extract the ZIP file and run it in the zip for the best experience." That's not advice, it's an anti-inspection technique. The instruction discourages the one thing that might reveal what's inside before you execute it.

If anyone on Discord asks you to run a game or an installer, verify through a second channel before you do anything. Call them. Ask what the game is about. A stolen account cannot answer that question in real time.

What Happens After You Click Run

Here is the full execution sequence, based on direct binary analysis and sandbox observation of our exact sample.

Stage 1: The Installer

"MythicJourney Setup 1.1.2.exe" is an 86MB NSIS self-extracting installer, the same format used by Notepad++ and VLC. It shows a progress bar. It looks like every other installer you've ever run.

SHA-256: 1567e11339c9dd227691111007a2021a90195f28a1d4b7766c1baee961953324

The PE header checksum is zeroed. The compile timestamp reads 2018, a seven-year-old NSIS stub, reused because the timestamp mismatch confuses automated triage. The 86 MB of malicious content doesn't live in the standard PE sections at all, it's compressed in the overlay, invisible to plain string scanning. This malware hides its real payload so well that basic antivirus tools don't even see it. AV detection at time of analysis was effectively zero.

Inside the compressed overlay:

app-64.7z - an 85 MB archive containing a complete Electron application (the actual stealer)
Five other NSIS plugin DLLs for extraction and execution
A fake uninstaller for cover

The install sequence:

NSIS decompresses everything to "%TEMP%\nsd557A.tmp\"
Installs the Electron payload to "%LOCALAPPDATA%\Programs\MythicJourney\MythicJourney.exe"
Drops a persistence copy to "%LOCALAPPDATA%\mythicjourney-updater\installer.exe"
Then deletes the temp directory.

By the time installation finishes, there's no trace of how it got there, and the malware is running under a name that looks like a normal app.

Stage 2: The Electron Payload

MythicJourney.exe is a full Electron application, the same framework used by VS Code, Slack, and Discord itself. Packaging a stealer as an Electron app is deliberate. It looks like legitimate software, behaves like legitimate software, is interpreted at runtime, and its network traffic looks like legitimate software.

The bundled native addons tell you exactly what it's after:

Module	What it does
`@primno/dpapi`	Calls Windows "`CryptUnprotectData"` to decrypts browser master keys
`better-sqlite3`	Reads browser SQLite databases directly (`Login Data`, `Cookies`, `Web Data`)
`robotjs`	Screen capture, keyboard input
`archiver`	ZIP packages stolen data for exfiltration
`aes-js`	Encrypts staged data locally

Stage 3: Three Layers of Obfuscation

The stealer logic lives in a single 2.9 MB JavaScript file (main.js) inside the Electron application archive. It is layered three levels deep.

The outer layer uses a custom base91 encoding scheme: 321 string table entries, shuffled by a rotation function, the entire script wrapped in an obfuscated Function() constructor so nothing executes or resolves until runtime. No plaintext URLs. No IPs. Nothing for a scanner to catch.

Decoding that reveals an AES-256-GCM encrypted second payload embedded directly in the file:

Key:   26303b532115653c49f950999fa94f1af5abb94a8f39b113e102025adbc6ba4b
Nonce: 74fa957219626ffb885f5aec

Decrypting yields 1.1 MB of a second obfuscated JavaScript payload, itself with 679 encoded string table entries and state-machine control flow. The only thing that leaked in plaintext was the module list passed to require() — "axios" (HTTP client for C2), "crypto", and "bytenode" (bytecode loader, likely a third stage we haven't reached). The C2 address is still locked inside those 679 encrypted strings.

NSIS installer (86 MB)
  └─ app-64.7z (Electron runtime)
       └─ app.asar → main.js (basE91 obfuscated)
            └─ AES-256-GCM decrypt → stage2.js
                 └─ [bytenode .jsc] → possible Stage 3

The malware is a set of nested encrypted containers. You have to crack each layer before you can even see the next one. I got through two layers. There may be more beyond the third I haven't decoded.

Stage 4: Reconnaissance and Evasion

I also uploaded a sample to VirusTotal for their sandboxes and HybridAnalysis. HybridAnalysis was good at teasing out the various layers of anti-reversing at play. Before touching credentials, the payload verifies the environment. It spawns 17+ PowerShell instances and queries system hardware, disk drives, BIOS, motherboard, RAM, GPU, monitors, then runs explicit evasion checks:

"(Get-CimInstance Win32_ComputerSystem).HypervisorPresent" — are we in a VM?
"[System.Windows.Forms.SystemInformation]::TerminalServerSession" — are we in an RDP/sandbox session?
"echo %COMPUTERNAME%.%USERDNSDOMAIN%" — are we on a corporate domain-joined machine?
"WHERE smartctl" — are disk forensics tools installed?

It immediately kills watcher.exe and mitmdump.exe, common dynamic analysis and MITM proxy tools, by name. If something is watching it, it kills the watcher.

The sandbox we submitted this to scored it 100/100 malicious. But AV marked it clean at runtime! That gap is the entire point of this evasion layer.

Stage 5: Credential Theft and Exfiltration

With the environment cleared, the payload:

Opens a handle to "lsass.exe", the Windows process holding cached authentication credentials in memory (NTLM hashes, Kerberos tickets)
Injects 4,024 bytes into 18 PowerShell processes, with PAGE_GUARD memory protection on the injected code to block memory dumping
Reads "desktop.ini" from every user shell folder (Desktop, Documents, Downloads, Music, Pictures, Videos, OneDrive), mapping the filesystem for data staging
Attempts to access "%APPDATA%\EXODUS\EXODUS.WALLET\SEED.SECO", the Exodus wallet master seed phrase
Decrypts browser credential stores via DPAPI and reads browser SQLite databases directly
Packages everything into an encrypted archive for exfiltration

It's suspected that the C2 communication goes out over TLS:443 to CDN-fronted endpoints (104.16.124.96 on Cloudflare and 142.251.210.35 on Google's network). These are not the attacker's servers, they're CDN edge nodes. This enables the C2 to automatically rotate IPs without touching the malware binary and the network traffic is indistinguishable from normal HTTPS to Google.

The runtime also installs a certificate into the Windows certificate store, allowing its own TLS to be trusted without triggering OS warnings.

By the time this finishes, your browser passwords, session cookies, Discord token, and wallet seed are on a remote server. Your Discord account is now delivering the same message to your contacts.

If You Ran It

Assume full compromise. Don't spend time figuring out what was taken, move immediately.

Change every password from a clean device.
Revoke all active sessions: Discord, Google, GitHub, any financial service.
Enable hardware 2FA: TOTP app or phone verification minimum
Regenerate cryptocurrency wallets from a clean seed on a clean machine, then move funds before anything else.
Delete "%LOCALAPPDATA%\Programs\MythicJourney\" and "%LOCALAPPDATA%\mythicjourney-updater\"
Check "%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup" for unexpected shortcuts
Report "github.com/ryprs" and "myth-journey.com" to GitHub and Vercel abuse teams

The attacker who built this campaign is almost certainly a MaaS customer, not the malware author. The author sells the kit; operators build their own delivery chains. The same payload runs under different names in parallel campaigns. If a friend on Discord asked you to test a game and something feels off, their account may already be hacked.

Book Review: "Agentic Artificial Intelligence"

2026-04-04T09:03:00.000-07:00

"Agentic Artificial Intelligence: Harnessing AI Agents to Reinvent Business, Work, and Life" by Pascal Bornet. This book was written at the very beginning of the agentic AI wave, looking at early adopters of using LLMs in agents to have generic language models drive computer tools. It has some great lessons learned on implementing agentic systems, but it’s largely non-technical, likely because it was written before these systems became standardized.. I listened to this on Audible at about ~$15 for roughly 10 hours (on 1.5x speed). At nearly 500 pages it's a pretty heavy read, although I personally found the first two parts the most impactful in terms of AI theory and implementation insights. The final three parts shift toward business building, enterprise adoption, and long-term societal impact. While the end of the book seemed to depart from reality a bit (talking about Universal Basic Income once agents take over the majority of jobs), I thought the beginning was fascinating and eye opening in terms of planning and reasoning with agents. Overall I'm going to give this 5 out of 10 stars. I recommend this to people wanting to get more theory and guidance when building out agentic systems, although I'm not sure I would recommend this if you were looking for a technical book. The book has no real mention of actual technology needed to implement these ideas. There is almost no mention of specific models, structures, or even the types of agents that could be run to automate these goals. In that sense the book left a lot to be desired, it was almost purely theory. That said, I did enjoy the first two parts of the book. The following are the chapters of the book so you can get a better idea of it's contents before picking it up:

Introduction
Part 1: The Rise of AI Agents
Chapter 1: Beyond ChatGPT: The Next Evolution of AI
Chapter 2: The Five Levels of AI Agents: From Automation to Autonomy
Chapter 3: Inside the Mind of an AI Agent
Chapter 4: Putting AI Agents to the Test
Part 2: The Three Keystones of Agentic AI
Chapter 5: Action: Teaching AI to Do, Not Just Think
Chapter 6: Reasoning: From Fast to Wise
Chapter 7: Memory: Building AI That Learns
Part 3: Entrepreneurship and Professional Growth with AI Agents
Chapter 8: A Practical Guide For Building Successful AI Agents
Chapter 9: From Ideas to Income: Business Models for the Agent Economy
Part 4: Enterprise Transformation Through Agentic AI
Chapter 10: Human-Agent Collaboration: Leadership, Trust, and Change
Chapter 11: Scaling AI Agents From Vision to Reality
Chapter 12: Case Study and Use Cases of Agents Across Industries
Part 5: Future Horizons For Work and Society
Chapter 13: The New World of Work
Chapter 14: Society in the Age of Agents
Conclusion

I struggled with parts of the book, because it repeatedly argues that agents should take action, but rarely explains how that action is implemented.. Should agents be calling APIs in a microservice architecture, or should we be giving agents full control of systems with local tools like ClawdBot? Is it better to give agents skills on how to use specific tools, or should we continue using MCP servers for up-to-date information on the tools? There is a ton of implementation details the book conveniently glosses over. The book also glosses over memory in a similar way, which in my experience if done wrong can make an agentic system much worse. Memory has lots of core questions, like storage location and structure, as well as retrieval quality and embedding limitations. It's a pretty hard thing to get right, so I'm surprised it didn't dive into any of the technical edge cases there. One technique the book does mention in depth is using an extensive RAG library or even a wiki or onboarding documents to support the agentic system if it needs to lookup context or understanding around a process. The book is also very idealistic. From it's estimations on agentic reasoning capabilities (nearly a year after it was written and these models still make regular mistakes) to it's predictions around Universal Basic Income when many common jobs are automated, it honestly makes me a little worried what a more grounded future might look like when I see these as proposed solutions.

On The Rise of AI Augmented Writing

2026-04-01T09:25:00.000-07:00

Welcome back Internet people! Lately I've seen a rise in AI generated articles, blog posts, and even book content. I need to say loudly, as a reader, this is a major turnoff. If a reader can tell that something was written by AI, then the tools are being used poorly. Please don’t pass off LLM output directly as human writing. It makes your work output feel cheap. AI should be used as a writing tool, it shouldn't be replacing human writers altogether.

When writers use LLM output verbatim the result is often stale and lacks clarity. In many cases, it actually makes ideas harder to understand. Current LLMs struggle to maintain consistent, logical models of complex ideas. So while the writing may sound polished at first, it sometimes misrepresents concepts or drifts into conflicting, multiple definitions. Moving past the coherency issues, it's often obvious when a writer has an overreliance on verbatim LLM output. There are many obvious tells. From the overuse of the em dash, to the nonsensical use of the colon; AI generated content sticks out to those who use frontier models often. Certain phrasing patterns also stand out. For example: “It’s not X, it’s Y.” As a writer, this often feels like filler. Just write about Y. Just because these are the current form of these tells doesn't mean these are universal or ubiquitous tells. Quite the opposite, these will change over time as the models change, but the heavy users of the models will very likely recognize their output when used verbatim.

Don't get me wrong, I'm not saying don't use LLMs to help you write. I previously wrote about how to use AI in your technical writing, such as creating templates, voice files, and dynamic prompts to generate rich content. It also makes for a great editor! But one of the key takeaways there is in the last paragraph, where I emphasize heavily modifying and adapting the output. You can't use the output verbatim; frontier LLM output is just too recognizable.

I recently read this great and thoughtful article titled "Don't Let AI Write For You", where Alex Woods lays out that the point of writing is to develop and cement thoughts worth communicating, not simply generating words or content. I couldn't agree with this more. I often use LLMs to help expand on ideas, or think about edge cases I might be considering. I use it to help me refine my writing prompts and generate starting points. But very rarely do I use the ideas or output verbatim. It's an incredibly useful tool, but in my opinion it shouldn't replace the art all together.

So I'll repeat it, and I hope somewhere out there other writers take it to heart. When writers use LLM output verbatim it comes across as incredibly lazy. And frankly, why would anyone read that? A reader could just prompt the model themselves and get the same result.

Book Review: "Adverserial AI Attacks, Mitigations, and Defense Strategies"

2026-03-18T04:07:00.000-07:00

I recently finished "Adversarial AI Attacks, Mitigations, and Defense Strategies: A Cybersecurity professional's guide to AI attacks, threat modeling, and securing AI with ML/SecOps" the book by John Sotiropoulos. The book is a deep dive into adversarial machine learning, focusing heavily on how AI models can be attacked across their lifecycle, from training and supply chain to deployment and inference, using techniques like poisoning, perturbations, and model extraction. The book is a great deep dive on model-level security and the various mode-level adversarial attacks. I grabbed the book for about ~$30, mostly because they were a fellow Packt author, and read it over the course of a long weekend. Overall, I'm give this book 6 out of 10 stars. At over 600 pages, it’s a dense read, and even though it’s divided into a handful of major parts, the structure doesn’t always make it easier to navigate. I'm sure this type of content is useful to some kind of academic or maybe a company actually making and hardening the models themselves, but I'm not sure general security practitioners could apply most of this book. It strikes me as the difference between cryptography security (attacking the algorithms) and applied cryptography security (attacking systems using crypto). The former would have an extremely small audience and applicability, whereas the later is very useful to most security engineers. This book is the former, but for AI. The quality isn’t the issue, it’s more that the content doesn’t map well to the needs of most security practitioners.The book is clearly focused on a model-centric view of AI security, and that shapes both its strengths and weaknesses. A large portion of the content is focused on algorithmic attacks against machine learning models themselves, using techniques such as poisoning, evasion, extraction, inversion, etc. The book goes into meaningful depth on how these techniques work, which is neat, but these feel very much like academic attacks to me as a practitioner. That makes it a strong resource for understanding how models can fail at a mathematical or behavioral level, and introduced me to a lot of resources to that extent, such as the Adversarial Robustness Toolbox. This gives the material a practical edge and makes it easier to reproduce attacks in a controlled environment, allowing you to see the techniques for yourself. In my typical style, here are the chapters of the book, so you can get a better understanding of the content:

Chapter 1: Getting Started with AI

Chapter 2: Building Our Adversarial Playground

Chapter 3: Security and Adversarial AI

Chapter 4: Poisoning Attacks

Chapter 5: Model Tampering with Trojan Horses and Model Reprogramming

Chapter 6: Supply Chain Attacks and Adversarial AI

Chapter 7: Evasion Attacks against Deployed AI

Chapter 8: Privacy Attacks: Stealing Models

Chapter 9: Privacy Attacks: Stealing Data

Chapter 10: Privacy-Preserving AI

Chapter 11: Generative AI: A New Frontier

Chapter 12: Weaponizing GANs for Deepfakes and Adversarial Attacks

Chapter 13: LLM Foundations for Adversarial AI

Chapter 14: Adversarial Attacks with Prompts

Chapter 15: Poisoning Attacks and LLMs

Chapter 16: Advanced Generative AI Scenarios

Chapter 17: Secure by Design and Trustworthy AI

Chapter 18: AI Security with MLSecOps

Chapter 19: Maturing AI Security

Where the book does particularly well is in drawing a clear distinction between recognition-based AI systems and generative AI systems, and explaining how different their attack surfaces really are. The treatment of traditional models, image classifiers, NLP predictors, and similar systems, focuses on adversarial examples and perturbation-based attacks that manipulate outputs without changing the underlying model. In contrast, the discussion of generative AI shifts toward prompt injection, jailbreaking, indirect attacks through retrieved content, and abuse of tool integrations. This distinction is important, because it highlights how generative AI expands the attack surface to language driven control of systems, which is particularly relevant today with the rise of agents. There is also a really good history of the evolution of these systems throughout the book, which is helpful historical context to get. Another thing the book does pretty well is give the reader a hands-on introduction to generative models, particularly GANs, before moving into how they can be abused. It doesn’t just stay conceptual, it walks through building a GAN from scratch, explaining the generator–discriminator dynamic and how they’re trained against each other. The book walks you through building your own naive GANs for things such as deep fakes, which is certainly fun although I'm not sure how truly useful. Further the defensive content is generally less detailed and less operational than the attack content. There is comparatively little guidance on how to implement security controls in real GAN pipelines, how to monitor for abuse, or how to use AI to improve existing cloud and application security practices. To me, the audience is clearly people doing research in adversarial machine learning or teams that are actually building and hardening their own models from the ground up. For that group, the depth on model-level attacks is probably useful. But for a general security practitioner, especially someone working on real-world systems that integrate AI rather than build it, much of the content feels difficult to apply. The scenarios are often centered on directly attacking or manipulating models in ways that just aren’t relevant if your organization is primarily consuming third-party models through APIs like ChatGPT or Claude. That model-centric perspective is also reflected in the book’s implicit assumption that organizations own and train their models. Much of the threat model revolves around scenarios like model theft, training data poisoning, and tampering with serialized model artifacts. While these are valid concerns, they are less aligned with how many enterprises actually use AI today. Most organizations are consumers of models rather than builders, relying on APIs from foundational models. In those environments, the attack surface shifts away from model internals and toward things like API misuse, prompt injection, data leakage through retrieval, and access control failures. As a result, some of the book’s most detailed attack scenarios can feel somewhat removed from the day-to-day risks faced by most security teams out there. We can hear John talk below about the OWASP top 10 AI risks :

Defensive Refusal Bias in LLMs is Hurting Infosec

2026-03-05T07:49:00.000-08:00

Last year a few of us in infosec met up for the National CCDC competition and did some LLM research while at the competition. We gathered data from both the defenders and the attackers on their ussage of LLMs and how well the technology aided them in the competition. This research goes on to show that these LLMs really aren't helping the blue teams, especially when paired with the evolutionary direction i've seen ALCCDC go. We can see a clear bias towards tools like Claude Code enabling offensive tool development and helping attackers, to the extent that the agent swarm will activly hack for the attacker, whereas the blue teams are struggling to get simple questions answered from LLMs like ChatGPT for the fear that the information may be abused.

What we found became the basis of our research on Defensive Refusal Bias, but the story has only gotten more interesting since then. During the competition we collected thousands of real prompts from both blue and red interacting with LLMs. The results showed something counterintuitive: modern safety-aligned models were far more likely to refuse legitimate defensive questions than they were to block creative offensive questions. Blue teams trying to analyze malware, harden systems, or investigate suspicious processes were frequently blocked because their requests looked “too much like hacking.” Meanwhile, attackers could often get what they needed simply by framing their prompts as experimentation, scripting help, or development work.

Fast forward to spring 2026, and the gap appears to be widening. After just completing events like ALCCDC last weekend, tools like Claude Code are rapidly evolving into autonomous agent swarms, capable of developing deep offensive capabilities, writing exploit scripts, chaining reconnaissance tools, and iterating quickly on offensive tooling. Not only that, with some clever prompting and abstraction, Claude Code will drive an offensive agent swarm as an operator, doing fully automated hacking and post-exploitation of victim systems. In practice, what this means is that attackers can now spin up something closer to an offensive agent collective, a set of automated assistants that will explore systems, write exploits, refine attacks collaboratively, and move to post exploitation activities. They will also leverage their own post exploitation for creative attack chains moving forward. Instead of just answering questions, the model increasingly acts as a co-developer and operator for offensive tradecraft, where defenders are getting stopped when asking simple analysis questions.

Defenders are often stuck fighting the safety rails. Ask a model to help analyze a piece of malware or break down how an exploit works, and there’s a decent chance you’ll get a refusal or a heavily sanitized answer. The irony is hard to miss: the same guardrails meant to prevent misuse are often slowing down the people trying to defend systems, where the attackers are using the readily available, professional models today at blazing speeds to auto-hack.