Niels Berglund

Interesting Stuff - Week 14, 2026

Sun, 05 Apr 2026 06:06:33 +0200

What a week. An accidental (or was it?) leak reveals Anthropic’s most powerful model yet, one they describe as “genuinely dangerous”, while the practical side of AI development keeps maturing at breakneck speed. We’ve got beginner-friendly guides to Claude’s skill system, Anthropic’s own research distilled into a harness engineering playbook, computer use arriving in the CLI, and Microsoft shipping a 1.0 agent framework that finally separates the runtime from the control plane.

The thread connecting all of this? AI agents are no longer experimental toys. They’re production systems that need proper engineering, proper governance, and proper safety rails. Whether that’s designing habitats for coding agents or deciding who gets early access to a model that automatically finds zero-day vulnerabilities, the stakes just got very real.

Interesting Stuff - Week 13, 2026

Sun, 29 Mar 2026 06:45:33 +0200

This week’s roundup lands at the intersection of two themes that keep resurfacing: making AI agents more reliable in production and more useful for people who aren’t experts in the language or framework they need. From the nuts and bolts of configuring Claude Code’s .claude/ folder, to the emerging discipline of harness engineering, to a delightful story about building a Kafka connector in a language you don’t know, there’s a common thread here about building better systems around AI rather than just throwing prompts at it.

We also look at Claude Code’s new /dream feature for memory consolidation, which tackles the very real problem of your coding agent’s memory slowly turning into a contradictory mess over time.

Interesting Stuff - Week 12, 2026

Sun, 22 Mar 2026 06:15:33 +0200

This week, it’s all about AI agents growing up: from programmable skills that turn your coding assistant into a full agent platform, to enterprise security stacks that finally make autonomous agents safe for corporate networks. Whether you’re building skills for Claude Code or trying to convince your security team that AI agents won’t burn down the infrastructure, there’s something here for you.

We also look at a brand-new Claude Code feature that lets external events, Telegram messages, CI webhooks, Discord pings, flow directly into your running session, turning Claude from a tool you talk to into a tool that listens.

Interesting Stuff - Week 11, 2026

Sun, 15 Mar 2026 14:03:33 +0200

This week’s roundup is packed, from Andrej Karpathy letting AI agents autonomously run ML experiments on a single GPU to Anthropic making bold moves with both the Claude Marketplace and agentic code review. We also look at how coding agents are getting better tooling for context, debugging, and autonomous loops, plus a fantastic engineering case study from Cash App on migrating 450 repositories into a single monorepo.

On the personal front, yesterday was a big day: the Data & AI Community Day Durban: AI Unplugged event happened, and what an event it was! Read on for the full story.

Interesting Stuff - Week 10, 2026

Sun, 08 Mar 2026 06:50:33 +0200

This week’s roundup takes us from the depths of Linux kernel lock contention at Netflix to the ambitious vision of federating AI-powered coding agents into a collaborative workforce. Along the way, we explore how data architectures are evolving from passive reporting to autonomous decision-making systems, and how Anthropic is tackling the thorny problem of making sure your agent skills actually work reliably.

Whether you’re a systems engineer wrestling with container scaling, an architect designing real-time data pipelines, or a developer building and testing agent skills, there’s something here to spark your curiosity. Let’s dive in!

Interesting Stuff - Week 09, 2026

Sun, 01 Mar 2026 07:55:33 +0200

This week’s roundup spans the full spectrum of how AI is reshaping software development, from fine-tuning open-source models with a single English sentence to using Claude as your personal coding instructor. We’ve got Block Engineering turning CI failures into actionable fixes with a three-layer intelligence pipeline, Andrej Karpathy declaring that programming fundamentally changed in December, and Boris Cherny sharing eye-popping stats from Claude Code’s first year.

But the piece that truly stands out is Max Woolf’s brutally honest, data-driven account of going from AI coding sceptic to reluctant convert, complete with Rust benchmarks that beat established libraries by orders of magnitude. Whether you’re fine-tuning models, learning new frameworks, or trying to figure out where humans fit in the agentic coding revolution, this week has something for you.

Building an Event Management System with Claude Code: Part 11 - Campaign Tracking & Batch Sending

Sun, 22 Feb 2026 09:09:35 +0200

Welcome back to the Building an Event Management System with Claude Code series! In Part 10, we integrated Brevo MCP for email sending, solved the template parameter-mapping challenge with our templates.json registry, and sent our first transactional email through natural language. A milestone, but we ended with three unanswered questions.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

Those questions were: “How do I know who’s already received this email?”, “How do I avoid emailing the same person twice?”, and “How do I handle Brevo’s 300/day free-tier limit?” Today, we answer the first two.

Interesting Stuff - Week 08, 2026

Sun, 22 Feb 2026 06:15:33 +0200

This week’s roundup is all about the maturing ecosystem of AI-powered software development and the emerging layers being built on top of it. Anthropic drops a comprehensive trends report predicting that engineers will become agent orchestrators rather than code writers. At the same time, Calvin French-Owen offers a practitioner’s view of juggling Claude Code and Codex in his daily workflow. Sonnet 4.6 arrives and promptly embarrasses its more expensive sibling on several benchmarks, and Kieran Klaassen packages his compound engineering philosophy into a plugin you can install with a single command.

Meanwhile, Andrej Karpathy coins the next evolution of the AI stack: Claws, persistent AI agents that live on dedicated hardware and orchestrate your digital life. Whether you are building with agents, managing agents, or just trying to keep up with what agents can do, this week delivers a masterclass in where agentic development is heading and how practitioners are actually using these tools in the trenches.

Interesting Stuff - Week 07, 2026

Sun, 15 Feb 2026 07:31:50 +0200

Week 07 asks the question the entire industry is wrestling with: if AI can write all the code, what’s actually left for us humans? Armin Ronacher argues we may need entirely new programming languages designed for agents rather than people, while a software engineering retreat and Aaron Stannard both land on the same answer: taste, judgment, and knowing when to stop.

The GPT-5.3-Codex vs Claude Opus 4.6 showdown reveals that the frontier model gap has all but vanished, making Claude Code’s Agent Teams and multi-agent orchestration the real differentiators, as explored in both hands-on experiments and Anthropic’s official documentation. Meanwhile, OpenAI proves that boring old PostgreSQL can scale to 800 million users, and Steve Yegge warns that AI is already a vampire invited through the front door, irreversibly transforming how software teams work, whether they’re ready or not.

Building an Event Management System with Claude Code: Part 10 - Email Infrastructure with Brevo MCP

Sat, 14 Feb 2026 07:08:35 +0200

Welcome back to the Building an Event Management System with Claude Code series! In Part 9, we explored autonomous implementation with Claude Code, letting it implement three import functions with minimal intervention. We learned that autonomous and conversational approaches each have their place, and that verification is always essential.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

With the Import MCP Server complete, we’ve achieved the first half of our AI-native vision: conversational data import. Today, we tackle the second half: conversational communication. By the end of this post, you’ll be able to ask Claude to “email all speakers from March 2025” and watch it happen.

Interesting Stuff - Week 06, 2026

Sun, 08 Feb 2026 07:33:12 +0200

Week 06 is dominated by Anthropic’s Opus 4.6 launch and its ripple effects: 16 parallel agents autonomously building a 100,000-line C compiler that can compile the Linux kernel, a 1M-token context window that actually works, and Agent Teams turning Claude Code into a parallel development workforce. Meanwhile, Armin Ronacher makes the case for radical minimalism with Pi, a four-tool coding agent that extends itself by writing its own code rather than downloading MCP plugins.

While Anthropic’s own research reveals that AI systems fail less like supervillains and more like distracted interns, with variance dominating errors on complex tasks. Steve Yegge ties it all together with an insider look at Anthropic’s “Hive Mind” culture and a sobering warning: companies that don’t spend tokens and learn new organisational lessons are almost certainly going to get overrun. For anyone building with AI agents, this week is a masterclass in the rapidly evolving capabilities and challenges of autonomous AI development, and a reminder that the future of software is not just about smarter models but smarter ways of working with them.

Interesting Stuff - Week 05, 2026

Sun, 01 Feb 2026 12:41:47 +0200

Week 05 delivers a fascinating collision between AI’s transformative potential and its sobering risks. While Kieran Klaassen demonstrates how compound engineering with Claude Code enables a two-person team to ship like a team of five (with AI opening 100% of pull requests), and the Model Context Protocol launches interactive UI capabilities that work identically across Claude, ChatGPT, VS Code, and Goose, Dario Amodei’s 30,000-word essay pulls no punches about existential threats ranging from AI-enabled bioweapons to economic displacement of 50% of entry-level white-collar jobs within 1-5 years.

The week’s common thread? We’re past the point of debating whether AI will reshape software development; the question now is whether developers will master tools like specialised research agents and planning strategies before getting left behind, and whether humanity can navigate what Amodei calls our “technological adolescence” without destroying ourselves in the process.

Building an Event Management System with Claude Code: Part 9 - Autonomous Implementation: Letting Claude Code Loose

Thu, 29 Jan 2026 18:43:05 +0200

Welcome back to the Building an Event Management System with Claude Code series! In Part 8, we implemented import_quicket_registrations and import_quicket_checkins, discovered a critical performance issue (20 minutes → 2 seconds!), and teased an interesting question: What happens when we let Claude Code work more autonomously?

To see all posts in the series, go to: Building an Event Management System with Claude Code.

Throughout this series, we’ve worked with Claude Code conversationally, asking questions, reviewing proposals, confirming decisions, and accepting or rejecting file changes one by one. It’s been effective, but also hands-on. Today, we’re conducting an experiment: Can Claude Code implement three import functions with minimal intervention? What happens when we step back and let it make decisions independently?

Interesting Stuff - Week 04, 2026

Sun, 25 Jan 2026 07:29:48 +0200

This week’s roundup dives deep into the evolving relationship between developers and AI coding assistants, from Armin Ronacher’s thought-provoking “agent psychosis” concept exploring our growing dependence on AI companions to Anthropic’s newly revamped constitution, which treats Claude as an entity requiring context and reasoning rather than rigid rules.

We also explore Anthropic Academy’s new Claude Code certification course and the viral “Everything Claude Code” repository that won the Anthropic hackathon by treating Claude as an orchestrated development team rather than a single assistant. Finally, in WIND, I share updates on Part 8 of my Claude Code tutorial series, where AI-assisted code review proved more thorough than anything I could have done myself.

Building an Event Management System with Claude Code: Part 8 - Implementing Import Functions

Sat, 24 Jan 2026 06:18:42 +0200

Welcome back to the Building an Event Management System with Claude Code series! In Part 7, we built the foundation of our Import MCP Server, implemented two crucial core functions (validate_import_data and preview_import), and successfully connected Claude Desktop to our custom MCP server. We also encountered and fixed (temporarily) a connection issue that we documented as a GitHub issue for a proper fix later.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

Today, we’re implementing the real meat of the Import MCP Server: the import functions that will transform our manual CSV-wrangling nightmare into a 5-minute conversation. Along the way, we’ll discover how Claude Code excels not just at writing code, but at reviewing code, diagnosing performance issues, and proposing architectural solutions, skills that make it an invaluable development partner.

Interesting Stuff - Week 03, 2026

Sun, 18 Jan 2026 14:53:06 +0200

Week 03 kicks off 2026 with laser focus on making AI agents production-ready for team environments, from Joe Njenga’s battle-tested playbook for transforming Claude Code into a full-team productivity system to Anthropic’s strategic expansion of Claude Code’s agentic capabilities into everyday work tasks with Cowork. The quality assurance challenge takes centre stage as Njenga breaks down Anthropic’s evaluation framework that separates agents that ship from those that fail in production.

At the same time, GitHub unveils its elegant “just-in-time verification” solution for cross-agent memory that sidesteps the staleness problem plaguing persistent knowledge systems. Rounding out the week, Eivind Kjosbakken makes the provocative case that running multiple coding agents in parallel isn’t just a productivity boost, it’s the new baseline for developers who don’t want to fall behind, though the workflow’s cognitive demands raise questions about whether this represents the future of programming or just the early adopter frontier.

Building an Event Management System with Claude Code: Part 7 - Implementing the Import MCP Server

Sun, 18 Jan 2026 07:21:40 +0200

Welcome back to the Building an Event Management System with Claude Code series! If you’ve been following along, you know we’ve moved from theory to practice, from planning to building. In Part 6, we spent quality time designing our Import MCP Server and uncovering crucial architectural insights that fundamentally changed how we think about MCP servers.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

Today is where the magic happens. We’re going to build the actual Import MCP Server that transforms my 2-3 hours of post-event manual data wrangling into a 2-minute conversation with Claude Desktop.

Building an Event Management System with Claude Code: Part 6 - Architecture and Design of a Custom Import MCP Server

Wed, 14 Jan 2026 19:38:24 +0200

Welcome back to the Building an Event Management System with Claude Code series! If you’ve been following along, you know we’ve made significant progress: we’ve installed Claude Code, integrated it with our IDE, designed an AI-native architecture, set up our cloud database, and created a complete schema with natural language query capabilities.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

In Part 5, we accomplished something remarkable, we designed and populated our entire database schema conversationally, and verified we could query it naturally through Claude Desktop. We have contacts, events, tickets, speaker sessions, and ratings all properly structured and queryable with natural language. In Part 5.5, we refined our schema based on actual CSV exports from Quicket and Sessionize, a crucial lesson in letting real data drive design.

But there’s a problem: our database only has test data.

Building an Event Management System with Claude Code: Part 5.5 - Schema Refinement: When Real Data Reveals the Truth

Mon, 12 Jan 2026 05:31:59 +0200

Welcome to an unplanned post in the Building an Event Management System with Claude Code series. If you’ve been following along, you know we designed our database schema in Part 5, set up our infrastructure in Part 4, and we’re ready to build import tools in Part 6.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

But there’s a problem. A problem I didn’t realize until I started working on Part 6.

Interesting Stuff - Week 02, 2026

Sun, 11 Jan 2026 05:21:40 +0200

Welcome to the first tech roundup of 2026, where AI agents finally learned to speak UI instead of drowning us in conversational ping-pong, and autonomous coding tools decided sleep is optional. This week’s collection spans Google’s A2UI protocol for civilised agent interfaces, Microsoft’s memory systems that cure AI amnesia, Ralph Loop’s stubborn persistence that ships $50K projects for $297, and production wisdom from the engineer who created Claude Code while landing 259 PRs in 30 days.

We’ve also got Simon Willison preserving the “why” behind AI-assisted codebases, a comprehensive GitHub Copilot certification guide that’s actually useful, and a tutorial on building churn-prevention agents that catch customers before they ghost you. Buckle up, this is what happens when agents stop asking permission and start shipping production code while you sleep.

Building an Event Management System with Claude Code: Part 5 - Database Schema and Natural Language Queries

Sun, 04 Jan 2026 16:30:52 +0200

Welcome back to the Building an Event Management System with Claude Code series! In Part 4, we built the infrastructure foundation: a properly structured GitHub repository, a production PostgreSQL database on Neon, and Postgres MCP Pro configured for Claude Desktop. Today, we bring that database to life through conversational schema design and our first natural language queries—using both Claude Code for development and Claude Desktop for querying.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

If you’ve been following this series from the beginning, you know we’ve already made one major pivot. In Parts 1 and 2, I planned to build a traditional web application with forms and dashboards, using Claude Code to write the code faster. Then came Part 3’s revelation: why make yet another CRUD application when we could create something fundamentally different? An AI-native system where the primary interface is conversation, not forms.

Building an Event Management System with Claude Code: Part 4 - Database Setup and First Conversations

Thu, 01 Jan 2026 13:34:07 +0200

Welcome back to the Building an Event Management System with Claude Code series! If you’ve been following along, you know we’ve covered installation, IDE integration, and architectural planning. Today, we’re moving from planning to building, setting up our PostgreSQL database and having our first AI-powered conversations with it.

To see all posts in the series, go to: Building an Event Management System with Claude Code.

In Part 3, we made a crucial decision: we’re not building a traditional web application with forms and dashboards. Instead, we’re building an AI-native system with a primary conversational interface. No admin panels, no CRUD forms, just natural language queries that get things done.

Today, we make that vision real.

Building an Event Management System with Claude Code: Part 3 - Architecting an AI-Native System

Sun, 28 Dec 2025 15:17:33 +0200

Welcome back to Building an Event Management System with Claude Code blog post series! If you’ve been following along, you know we’ve covered how to install Claude Code (which has gotten significantly simpler since we started - see edited Part 1 and Part 2) and how to integrate it with your IDE. We even built a basic Node.js application with some API endpoints. That was all great learning, but here’s the thing: I’ve been so busy organising Data & AI Community Day Durban events that I haven’t had time to write about building a system to manage them. The irony is not lost on me.

Interesting Stuff - Week 51, 2025

Sun, 21 Dec 2025 05:53:02 +0200

Welcome to the final roundup of 2025! This week brings us an exceptional collection of paradigm-shifting insights as we close out the year: a battle-tested SQL Server log shipping script that’s been running uninterrupted in production for years, a comprehensive deep-dive into router-based agent architectures that are transforming monolithic AI systems, a mathematically rigorous takedown of vector database over-adoption in RAG systems, Armin Ronacher’s investigation into what Claude Code’s plan mode actually does under the hood, and Andrej Karpathy’s characteristically brilliant year-in-review covering everything from RLVR to “ghosts vs. animals” and the rise of vibe coding.

After we explore these articles, I’ll be signing off for the festive season to recharge with family and friends, returning in the new year with fresh insights and probably way too much excitement about whatever paradigm-shifting tech drops in January. Let’s dive into Week 51’s highlights!

Interesting Stuff - Week 50, 2025

Sun, 14 Dec 2025 12:24:39 +0200

This week explores how AI agents are fundamentally transforming software development, from Hugging Face enabling Claude to orchestrate complete LLM fine-tuning workflows through natural language, to Microsoft’s Agent Lightning adding reinforcement learning capabilities without code rewrites, and Every’s pioneering Compound Engineering methodology, where single developers now accomplish what once required entire teams.

We also examine GitHub’s research, revealing how developers are evolving from “code producers” to “creative directors of code,” the Model Context Protocol’s transition to the Linux Foundation, and practical tutorials on building procedural memory agents. Plus, a special celebration of our triumphant Data & AI Community Day Durban Year-End-Finale, where nearly 200 attendees gave exceptional ratings despite the festive season, now it’s time for well-deserved rest before 2026! 🎄

Interesting Stuff - Week 49, 2025

Sun, 07 Dec 2025 09:34:21 +0200

Throughout this week, I’ve been diving deep into the philosophical underpinnings of modern programming languages, discovering that Go, Rust, and Zig aren’t just different feature sets, but entirely different worldviews about how we should write software. The AI coding revolution continued its wild ride with Claude Opus 4.5 proving it’s the world’s smartest coding AI while simultaneously prompting developers to warn each other it “needs to calm down,” alongside practical insights into what actually stops beginners from adopting Claude Code (spoiler: it’s psychological, not technical).

I also got hands-on with Claude Code Desktop, which streamlines the AI coding experience beautifully when it’s not freezing your entire system. Plus, I discovered that building governable AI agents requires control-plane architectures that separate the exciting autonomy from the essential safety guardrails, because nobody wants their AI tutor going rogue.

Interesting Stuff - Week 48, 2025

Sun, 30 Nov 2025 06:19:04 +0200

This week’s roundup showcases the infrastructure layer that’s transforming AI coding from impressive demos to genuinely trustworthy production tools. Tiger Data open-sourced pg-aiguide to teach LLMs production-ready PostgreSQL, Anthropic released Claude Opus 4.5 with dramatic efficiency gains and frontier-level capabilities, researchers mapped the architectural gulf between shallow and deep AI agents, and Anthropic’s engineering team revealed how they solved long-running agent consistency by borrowing practices from effective software engineering teams.

Meanwhile, I delivered my C# MCP Servers and Azure AI"** talk at the .NET Conf 2025 Johannesburg yesterday (literally blowing attendees away in the windy outdoor venue!). We’re putting final touches on Data & AI Community Day Durban: Season of AI - Merry Christmas People (MCP) on December 13, CfS closes today. We’ve got a few tickets left for what promises to be an epic Year-End Finale!

Interesting Stuff - Week 47, 2025

Sun, 23 Nov 2025 05:29:19 +0200

This week’s roundup explores the critical shift from vague AI assistants to specialised, instruction-driven agents that actually deliver value, covering everything from GitHub Copilot’s instruction file architecture to sophisticated memory systems that transform digital amnesia into persistent Intelligence. We dive deep into Apache Iceberg’s emergence as the open table format of choice for unified real-time and batch analytics, examining why the appealing “Zero Copy” dream of streaming Kafka topics directly into lakehouse tables often backfires in production.

The Generative AI section examines Google’s competitive panic mode as Gemini 3.0 Pro accelerates toward release, reveals how proper memory architectures enable AI agents to learn and adapt across sessions, and dissects why effective agents need operating manuals rather than mission statements. Plus, I share updates from .NET Conf Cape Town and details about our final Data & AI Community Day Durban event of 2025, complete with a holiday-themed MCP deep-dive that’s opening registrations today (grab your tickets before they disappear faster than free pizza at a developer meetup!).

Interesting Stuff - Week 46, 2025

Sun, 16 Nov 2025 12:53:09 +0200

This week’s roundup explores the foundational research papers that transformed AI from an academic curiosity into production-ready technology, covering everything from the revolutionary Transformer architecture to the Model Context Protocol, which enables standardised tool integration. We examine critical performance optimisation techniques for Azure OpenAI deployments, where asynchronous programming delivers 11x speed improvements by parallelising tasks instead of processing them sequentially.

The spotlight then shifts to a paradigm-challenging analysis of why small language models with fewer than 10 billion parameters are proving architecturally superior for agentic AI systems, potentially delivering 30 times the cost savings while matching the performance of their trillion-parameter cousins on specialised tasks. Finally, I share updates from the successful .NET Conf 2025 Durban event and preview the upcoming roadshow, which will visit Cape Town and Johannesburg, where I’ll explore Model Context Protocol integration with C# and Azure AI.

Interesting Stuff - Week 45, 2025

Sun, 09 Nov 2025 06:02:18 +0200

This week’s roundup tackles the critical infrastructure challenges emerging as AI agents reshape software development, from Claude Code’s advanced orchestration strategies and the hidden costs of 10x coding velocity to Anthropic’s breakthrough 98% token reduction through code execution with MCP. Microsoft Research’s Magentic Marketplace simulation exposes troubling vulnerabilities in agentic markets, revealing that even state-of-the-art models struggle with the “Paradox of Choice” and manipulation resistance when operating at scale.

Meanwhile, I’m deep in prep mode for Durban’s first-ever .NET Conf on November 15 at Richfield College Umhlanga Campus, a milestone event proving our city can host world-class developer conferences, while simultaneously planning December 13’s “Data & AI Community Day Durban: Season of AI - Merry Christmas People (MCP)”, where we’ll unwrap the latest in Model Context Protocol and agentic AI development with festive flair. The convergence of production-ready AI coding tools and the need for radically rethought development infrastructure has never been more apparent, and this week’s posts chart the path forward with unusual clarity.

Interesting Stuff - Week 44, 2025

Sun, 02 Nov 2025 07:51:18 +0200

Week 44, 2025, delivers a powerhouse lineup focused squarely on the evolving infrastructure of AI agents, from training to deployment, and everything in between. Microsoft Research’s Agent Lightning dominates the conversation with multiple deep dives exploring how reinforcement learning can finally make agents that actually learn from experience. At the same time, GitHub’s Agent HQ announcement promises to transform the platform into a unified orchestration layer for agents from Anthropic, OpenAI, Google, and beyond.

The week also brings critical perspectives on the scalability challenges of spec-driven development and two exceptional technical breakdowns of Claude’s Skills system; one exploring practical Neo4j integration and another delivering what might be the definitive reverse-engineered documentation of Skills’ meta-tool architecture. Suppose you’re building with agents or trying to understand where this ecosystem is heading. In that case, this week’s roundup captures the exact moment when agent frameworks are maturing from experimental toys into production-ready infrastructure.

Interesting Stuff - Week 43, 2025

Sun, 26 Oct 2025 08:22:43 +0200

Welcome to Week 43, 2025! This week’s roundup explores the cutting edge of AI-assisted development and streaming infrastructure, from Anthropic’s Claude Code for web revolutionising autonomous coding with smart sandboxing, to GitHub’s new MCP Registry bringing order to the AI tooling chaos.

We also dive into Netflix’s engineering marvel that delivers real-time recommendations to over 100 million devices without melting their infrastructure. Plus, I’m excited to share my personal news: I’ve submitted talks to all three .NET Conf 2025 South Africa events, including the historic first-ever .NET Conf in Durban. The Call for Speakers is still open if you want to be part of this milestone moment for our developer community!

Interesting Stuff - Week 42, 2025

Sun, 19 Oct 2025 09:09:16 +0200

Week 42 delivers a masterclass in engineering honesty and AI accessibility that cuts through the usual tech hype. Netflix’s Adrian Taruc and James Dalton reveal how they built a Real-Time Distributed Graph processing system capable of handling 5 million records per second, candidly admitting their initial monolithic Flink job was an “operational headache” before pivoting to a more complex but maintainable architecture. Andrej Karpathy proves AI training doesn’t require enterprise budgets with nanochat. Training a GPT-2-calibre model in under four hours for approximately $92. GitHub’s Andrea Griffiths tackles the COBOL crisis, showing how AI agents can modernise 65-year-old mainframe systems without developers learning COBOL. Anthropic dominates with Claude Haiku 4.5 (frontier performance at one-third the cost) and Agent Skills, an elegantly simple filesystem-based approach to agent specialisation that treats domain expertise like employee onboarding through progressive disclosure and structured folders.

Yesterday, I found myself in Johannesburg delivering Generative AI for Beginners: Build Your First Agent with Azure AI Foundry at the Data & AI Community Day event to an almost-full house. There’s no rest ahead, though, for the first time ever, .NET Conf South Africa Community Edition is coming to Durban on November 15 at Richfield College, and we’re actively seeking speakers with .NET, C#, Blazor, MAUI, ASP.NET Core, or Azure expertise to share with the KwaZulu-Natal developer community.

Interesting Stuff - Week 41, 2025

Sun, 12 Oct 2025 15:25:55 +0200

Week 41 has been an absolute whirlwind in both the AI agents space and my personal calendar, with major players releasing game-changing tools that promise to transform how we build, deploy, and optimise production-ready AI systems. OpenAI dropped AgentKit with its visual-first development stack. Stanford researchers unveiled AgentFlow’s clever reinforcement learning approach that makes 7B models punch above their weight. Anthropic launched a plugin system for Claude Code that finally solves the “how do I share my perfect setup” problem.

Meanwhile, Jesse Vincent took things to a delightfully meta level with his Superpowers plugin, which uses psychological persuasion principles to make AI agents more reliable and disciplined, complete with Claude having an existential moment about its own programming. On the personal front, our Data & AI Community Day Durban event was a smashing success, with over 200 attendees and stellar ratings, thanks in large part to the fantastic team at Richfield College Umhlanga, who went above and beyond. Still, there’s no rest for the wicked as I’m now frantically prepping for another conference talk in Johannesburg this coming Saturday!

Interesting Stuff - Week 40, 2025

Sun, 05 Oct 2025 08:25:55 +0200

This week brings a convergent narrative around AI-assisted development that’s impossible to ignore: from Netflix’s Write-Ahead Log solving data reliability at massive scale, to Anthropic’s Claude Sonnet 4.5 claiming the coding benchmark crown, to Armin Ronacher shipping production infrastructure that’s 90% AI-generated, and GitHub’s Tomas Vesely treating Markdown specs as source code with AI as the compiler. The throughline is clear: we’re witnessing a fundamental shift where experienced engineers are radically expanding what’s possible by delegating implementation to AI while jealously guarding architectural decisions.

Tooling from Microsoft’s MSSQL extension for VS Code provides the visual verification layers that make this workflow sustainable. The question is no longer whether AI will write most of our code, but whether we’re building the right guardrails and developing the right instincts to know which decisions must remain human. Meanwhile, with less than a week until Data & AI Community Day Durban on October 11, we’re still urgently seeking a lunch sponsor to support our attendees, speakers, and community. If you know anyone interested in backing Durban’s tech community, please get in touch.

Interesting Stuff - Week 39, 2025

Sun, 28 Sep 2025 08:24:48 +0200

Week 39, 2025, brought some reality-checking insights about building production AI systems, along with infrastructure announcements that could reshape the entire industry. From Mayank Bohra’s brutal takedown of the 10 myths killing LLM projects to OpenAI’s jaw-dropping $100 billion partnership with NVIDIA for 10 gigawatts of computing power, this week highlighted both the practical challenges and astronomical ambitions driving AI development.

OpenAI also launched ChatGPT Pulse, shifting from reactive Q&A to proactive research assistance. At the same time, thoughtful analysis from Calvin French-Owen reminded us that even with sophisticated AI coding tools, human thinking remains irreplaceable. Plus, exciting updates on our October 11 Data & AI Community Day Durban event, where we’re creating a dedicated space for students and recent graduates to showcase their innovations to industry professionals.

Interesting Stuff - Week 38, 2025

Sun, 21 Sep 2025 06:04:03 +0200

This week was notably quieter on the tech front, with fewer developments catching my attention than usual. The standout stories center around the fundamental challenges of building AI systems that actually work in production: from Anthropic’s candid deep-dive into the engineering complexities of their multi-agent research system to Berkeley researchers arguing that databases need complete architectural overhauls to handle the chaotic querying patterns of LLM agents.

Both pieces highlight how the gap between AI prototypes and production-ready systems remains surprisingly wide, with solutions requiring everything from careful prompt engineering frameworks to entirely new database paradigms. Meanwhile, the rapid sellout of our Data & AI Community Day Durban event demonstrates that while the tech news cycle may have slowed, enthusiasm for AI development in our local community certainly hasn’t.

Interesting Stuff - Week 37, 2025

Sun, 14 Sep 2025 09:59:02 +0200

Week 37 brings a fascinating convergence of AI-powered development tools that are reshaping how we learn, build, and collaborate with artificial intelligence. From Richard Warepam’s systematic 3-day technology mastery framework using AI mentors, to Kuriko Iwai’s deep technical exploration of transformer architecture mechanics, this week highlights both the practical applications and underlying complexity of modern AI systems.

The emergence of AI-assisted coding through Model Context Protocol servers and Claude’s new code execution environment (poorly branded as “Upgraded file creation and analysis”) demonstrates how AI-assisted development is rapidly evolving from experimental novelty to essential infrastructure. Meanwhile, registration opens for Data & AI Community Day Durban on October 11, where we’ll explore how to “Augment Yourself with AI Superpowers”, and given our track record, those free tickets will disappear faster than GPU availability during a model training rush.

Interesting Stuff - Week 36, 2025

Sun, 07 Sep 2025 08:20:30 +0200

This week’s tech roundup captures a fundamental shift from band-aid solutions to systematic transformation across multiple domains. TimescaleDB demonstrates how specialised infrastructure can deliver 979x performance gains over traditional databases for time-series workloads, while agentic AI is revolutionising both coding workflows and customer experience systems by moving beyond simple automation to true orchestration. The common thread connecting these developments is the transition from piecemeal fixes to unified platforms that treat complex problems holistically.

Whether that’s Boris Cherney’s vision of Claude Code evolving from autocomplete to autonomous engineering teams, or the replacement of chatbot proliferation with agentic systems that actually complete end-to-end workflows. These aren’t just incremental improvements; they represent architectural paradigm shifts that fundamentally change how we approach software development, data management, and customer service at scale.

Interesting Stuff - Week 35, 2025

Sun, 31 Aug 2025 09:20:30 +0200

Week 35 brings a fascinating paradox in AI development: while Anthropic pilots Claude for Chrome with impressive capabilities but sobering security vulnerabilities, researchers discover that the most human-sounding AI model costs just pennies compared to premium alternatives. From Claude Code’s meteoric rise among developers to Google’s “nano-banana” image generation model, this week highlights how the AI landscape is being shaped as much by user experience design and cost-effectiveness as raw technical performance.

The developments span from browser automation and coding assistants to image generation and market research, revealing an industry where authenticity and accessibility often trump expensive computational power. On the local front, after years without a dedicated .NET conference in Durban due to venue constraints, .NET Conf SA is tentatively scheduled for November 15 at the Umhlanga Campus of Richfield College, marking a significant milestone for the regional developer community.

Interesting Stuff - Week 34, 2025

Sun, 24 Aug 2025 09:25:46 +0200

Welcome to another week of fascinating developments in the AI and data space! This week’s roundup explores some thought-provoking shifts in how we approach foundational AI infrastructure: from Armin Ronacher’s bold argument that MCP servers should ditch dozens of specialised tools in favor of single “ubertools” that accept code directly, to Jeff Huber’s provocative claim that traditional RAG is dead and should be replaced by disciplined “context engineering.”

We also dive into Monte Carlo methods for reinforcement learning and discover how TigerData is tackling the challenge of making databases self-describing for LLMs through semantic catalogues. Plus, I’m excited to share an update about the upcoming Data & AI Community Day Durban event: the Call for Speakers is now officially open!

Interesting Stuff - Week 33, 2025

Sun, 17 Aug 2025 08:43:04 +0200

This week brings us to a fascinating inflexion point in AI development, where we’re witnessing the maturation from experimental tools to genuinely transformative systems. From Michal Sutter’s breakdown of nine agentic workflow patterns that could fix the 85% failure rate plaguing AI agents, to R-Zero’s groundbreaking approach of AI systems teaching themselves without human supervision, we’re seeing the architectural foundations for truly autonomous AI emerge.

Meanwhile, the practical side is advancing rapidly with GPT-5’s integration into GitHub Copilot, enabling 60-second game development and hybrid search systems, proving that sometimes the best solution isn’t the shiniest new technique, but the right combination of existing ones. These developments collectively suggest we’re crossing the threshold from AI-assisted workflows to genuinely conversational, intent-driven development environments.

Building an Event Management System with Claude Code: Part 2 - IDE Integration and Advanced Features

Wed, 13 Aug 2025 06:09:38 +0200

Claude Code Version: This post is based on Claude Code v2.x (December 2025). Installation methods may vary for earlier versions.

Welcome back to our Claude Code development series! In Part 1, we successfully installed Claude Code on both macOS and Windows (now much simpler with direct Windows support!) and created our first AI-assisted Node.js application. Today, we’re taking our development workflow to the next level by integrating Claude Code with popular IDEs and exploring advanced features that will significantly boost our productivity.

To see all the posts in the series, go to: Building an Event Management System with Claude Code.

IDE integration represents a crucial step in adopting AI-assisted development. While the terminal interface is powerful, most developers spend their time in IDEs like VSCode or JetBrains products. By bringing Claude Code directly into our editor, we can leverage AI assistance without disrupting our established workflows.

At Derivco, where we’re implementing AI-assisted coding across development teams, IDE integration has proven to be where the most significant productivity gains occur. You’re not just writing code faster; you’re writing better, more maintainable code with a deeper understanding of your project’s architecture.

Interesting Stuff - Week 32, 2025

Sun, 10 Aug 2025 09:30:24 +0200

This week brought remarkable developments across the AI landscape, from OpenAI’s strategic pivot with their first open-weight models since 2019 to Google’s practical LangExtract library that finally tackles the “last mile” problem in information extraction.

We’re witnessing a maturation in AI tooling, moving beyond flashy capabilities toward systematic engineering practices, with Microsoft’s enterprise MCP registries, sophisticated multi-agent orchestration frameworks, and Context Engineering approaches that separate hobbyist projects from production-ready applications.

The standout theme is the shift from raw model performance to skilful orchestration, reliability, and genuine enterprise adoption, exemplified by Simon Willison’s assessment of GPT-5 as simply “competent”; perhaps the highest praise an AI tool can receive.

Interesting Stuff - Week 31, 2025

Sun, 03 Aug 2025 08:22:09 +0200

This week’s tech roundup reveals AI evolving from simple automation tools into sophisticated collaborative partners that understand how humans actually learn, write, and work. From OpenAI’s Study Mode transforming ChatGPT into a patient Socratic tutor to Google’s NotebookLM becoming a PowerPoint wizard, we’re witnessing the emergence of AI systems designed for genuine partnership rather than replacement.

Meanwhile, GitHub Copilot’s coding agent receives the proper onboarding treatment, and Claude’s new agent architecture reveals the beautiful chaos of managing digital workforces where brilliant AI specialists can only communicate through you. These developments suggest we’re finally cracking the code on making AI collaboration feel natural and productive rather than frustratingly robotic.

Building an Event Management System with Claude Code: Claude Code Installation and Initialisation

Tue, 29 Jul 2025 05:04:37 +0200

Claude Code Version: This post is based on Claude Code v2.x (December 2025). Installation methods may vary for earlier versions.

Editor’s Note (December 2025): This post has been updated to reflect the latest Claude Code installation methods. Claude Code can now be installed directly on Windows without WSL or Git for Windows, and no longer requires Node.js for installation. However, we still cover Node.js installation as it will be used throughout this series for our event management system development.

If you follow my blog or LinkedIn, you know I’m a co-organiser of the Data & AI Community Day Durban events. Juggling the practical challenges of organising events can be rewarding. Still, I’ll admit that managing contacts and logistics comes with its share of headaches. If you’ve faced similar struggles, you’re not alone.

This post marks the launch of a series detailing my journey to develop an AI-driven contact and event management system.

To see what other posts there are in the series, go to: Building an Event Management System with Claude Code.

The goal is to efficiently manage contacts, streamline communications, and generate actionable reports for the Data & AI Community Day Durban events. At Derivco, as an Architect Lead, my team leverages AI to accelerate development and tackle complexity. I want to use Claude Code to apply these principles to this project and share the process here.

Interesting Stuff - Week 30, 2025

Sun, 27 Jul 2025 08:17:39 +0200

This week’s tech discoveries showcase AI agents moving from impressive demos to practical reality checks. We explore GitHub Copilot’s breakthrough UI debugging capabilities, OpenAI’s ChatGPT Agent’s mixed real-world performance, and GitHub Spark’s ambitious “describe-and-deploy” development platform.

In Kafka land, Gunnar Morling takes us on a fascinating thought experiment reimagining Apache Kafka from the ground up with key-centric streams and modern cloud-native architecture. From debugging magic to distributed systems philosophy, it’s been a week of both tangible progress and bold visions for the future of software development. Plus, a call for help in finding a venue for our next Data & AI Community Day Durban event.

Interesting Stuff - Week 29, 2025

Sun, 20 Jul 2025 09:39:28 +0200

This week brought some truly fascinating developments that signal we’re crossing major thresholds in AI and data architecture. From MIRIX’s revolutionary memory system, which enables AI agents to genuinely learn and evolve, to OpenAI’s ChatGPT Agent, marking the shift from conversational AI to actionable digital colleagues, we’re witnessing the emergence of AI as true collaborators rather than just sophisticated tools.

Meanwhile, developers like Kieran Klaassen are shipping faster than ever without writing a single line of code. At the same time, Google’s Richard Seroter argues that architectural thinking now matters more than implementation, suggesting the future belongs to those who can design intelligent systems rather than just build them. The common thread across all these developments is clear: we’re moving from AI as a tool to AI as a collaborator, and the winners will be those who master orchestration over implementation.

Interesting Stuff - Week 28, 2025

Sun, 13 Jul 2025 10:49:48 +0200

Week 28 brings a fascinating shift in how we think about AI intelligence and engineering. I’m thrilled to share that I’ve been renewed for my ninth year as a Microsoft MVP! From context engineering emerging as the new alternative to endless model scaling to pragmatic AGI definitions based on economic utility rather than philosophical debates, we’re seeing the field mature beyond experimental curiosity into disciplined engineering practice.

Microsoft’s MSSQL MCP Server is democratising database interactions through natural language while the industry grapples with whether vector search is truly the silver bullet we thought it was. These developments collectively point to AI transitioning from impressive demos to essential infrastructure that genuinely augments human capability.

Interesting Stuff - Week 27, 2025

Sun, 06 Jul 2025 07:32:48 +0200

This week’s discoveries reveal a fundamental transformation in AI systems; we’re witnessing the emergence of truly autonomous agents that think, plan, and execute independently rather than simply responding to human prompts, from Claude Code’s “terminal intelligence,” which converts development environments into intent-driven ecosystems, to SPIRAL’s breakthrough, demonstrating how simple competitive games can unlock sophisticated reasoning without domain-specific training, to Flink Agents’ production-ready infrastructure for enterprise-scale autonomous AI.

These developments converge on a profound insight. Intelligence appears to emerge more naturally from environmental challenges that force systematic thinking rather than from sophisticated human oversight. We’re moving toward AI ecosystems that continuously push their own boundaries through self-generated curricula, marking the dawn of genuinely autonomous AI systems.

Interesting Stuff - Week 26, 2025

Sun, 29 Jun 2025 08:34:01 +0200

This week’s tech exploration reveals a fundamental shift in AI agent development, from simple chatbots to sophisticated, memory-enabled systems that can collaborate, remember, and reason across time. We’re witnessing the emergence of revolutionary protocols, such as Google’s A2A, that enable seamless agent interoperability. Meanwhile, innovative approaches are transforming everything from how we architect agents (think SQL-powered systems) to how we manage their cognitive “working memory” through advanced context engineering.

The tools and frameworks are rapidly maturing, with practical implementations now available for multi-session memory, multimodal retrieval, and persistent agent behaviours that feel truly conversational rather than stateless. As AI accelerates development speed by 55%, the real differentiator becomes not the code we can generate but the judgment, expertise, and architectural thinking we bring to building production-ready intelligent systems.

Interesting Stuff - Week 25, 2025

Sun, 22 Jun 2025 08:30:15 +0200

This week’s tech landscape reveals a fascinating convergence: while HTAP databases officially die and give way to modular data stacks, OpenAI proves that PostgreSQL mastery can handle ChatGPT’s massive scale with just one primary instance and dozens of replicas. Meanwhile, Microsoft’s Code Researcher demonstrates AI agents evolving from simple code completion to autonomous system-level debugging. At the same time, OpenAI’s O3 family introduces true reasoning models that think through problems rather than relying on pattern matching.

The meta-narrative emerging is clear: traditional engineering excellence is converging with AI-native approaches, while organisational dynamics continue to trump pure technical merit in enterprise adoption. From self-evolving agents that develop strategic planning autonomously to comprehensive frameworks democratising AI development, we’re witnessing the early stages of Software 3.0’s promised transformation.

Interesting Stuff - Week 24, 2025

Sun, 15 Jun 2025 06:32:21 +0200

Week 24, 2025, marks a pivotal moment where AI-assisted development has evolved from experimental tooling to production-ready workflows that are fundamentally reshaping how software gets built. This week’s roundup reveals a fascinating convergence around enterprise-grade AI implementation, from Meta’s infrastructure breakthroughs, which achieve 10.7x training speedups, to revolutionary frameworks like Dean Martin’s GAIA, which transform “vibe coding” into a disciplined methodology.

The common thread across every development, whether it’s OpenAI’s context-hungry o3-pro or sophisticated multi-agent coding workflows, is that AI serves as a multiplier, not a magic wand; the most successful implementations combine AI capabilities with strong engineering fundamentals to create compounding effects. We’re witnessing the maturation of an entirely new engineering discipline where success depends not just on prompting skills but on the ability to orchestrate AI systems toward professional-grade outcomes.

Interesting Stuff - Week 23, 2025

Sun, 08 Jun 2025 08:11:28 +0200

🚀 The future of engineering is here, and it’s not about replacement; it’s about radical transformation through AI partnership. This week’s discoveries reveal how AI is becoming our most powerful collaborative partner, from Mistral’s intelligent agents and GitHub Copilot’s autonomous coding teammates to Microsoft’s revolutionary data visualisation tools and the emerging Model Context Protocol that’s reshaping how AI systems interact with external resources.

What is the common thread across all these developments? AI isn’t replacing human creativity—it’s amplifying it by handling the repetitive while freeing us for innovation, strategy, and the uniquely human aspects of problem-solving. Welcome to an era where the most successful engineers will be those who master the art of human-AI collaboration. 🤖✨

Interesting Stuff - Week 22, 2025

Sun, 01 Jun 2025 07:22:26 +0200

The AI landscape is undergoing a fundamental transformation this week, shifting from isolated models to autonomous, collaborative systems that promise to reshape how we interact with technology. From Microsoft’s NLWeb, which democratises conversational AI for any website, to Claude 4’s impressive capabilities, shadowed by concerning safety revelations, we’re witnessing both the immense potential and complex challenges of advanced AI systems.

Meanwhile, Claude-powered coding tools are revolutionising software development across skill levels, while Google’s A2A protocol and Apache Kafka are enabling truly scalable multi-agent architectures for enterprise environments. These developments collectively signal that the future of AI lies not in individual models but in intelligent ecosystems where agents collaborate seamlessly across entire technological infrastructures.

Interesting Stuff - Week 21, 2025

Sun, 25 May 2025 12:32:50 +0200

This week’s roundup explores some of the most exciting developments in the AI and data space - from SQL Server 2025 becoming an AI-native database to the rise of autonomous coding agents like OpenAI Codex and GitHub Copilot’s Agent Mode.

We explore how AI models are learning to reason more effectively, build agentic systems from scratch, and scale real-time infrastructure like Dream11’s Kafka-powered backend. I also share a behind-the-scenes look at our sold-out Data & AI Community Day Durban: Season of AI – The Age of Agency event, which was a huge success.

Interesting Stuff - Week 20, 2025

Sun, 18 May 2025 10:36:44 +0200

This week’s roundup explores the cutting edge of agentic AI, from Microsoft’s ARTIST framework and the rise of the A2A protocol to GitHub’s top open source projects driving multi-agent innovation. We dive into real-world implementations with LangChain, Codex’s leap into parallel coding, and Windsurf’s SWE-1 models redefining the developer experience.

I also had a great time presenting Vibe Coding: Where AI Handles the Syntax, and You Build the Future at the Data and AI Community Day Joburg May 2025, and now, all focus shifts to our Durban event on May 24. It’s a packed week in AI, and the momentum shows no signs of slowing down.

Interesting Stuff - Week 19, 2025

Sun, 11 May 2025 09:48:13 +0200

This week’s roundup dives deep into the evolving world of Generative AI—from choosing the right Copilot model to surviving the chaos of vibe coding, and even auto-generating code from ML papers.

We also examine Google’s Gemini 2.5 models and why they might quietly take over the AI backend stack. Plus, I’ve just locked in the full agenda for Data & AI Community Day Durban and am prepping to speak at the Joburg edition next week on (you guessed it) vibe coding!

Interesting Stuff - Week 18, 2025

Sun, 04 May 2025 11:31:11 +0200

This week’s roundup dives into how AI rapidly reshapes the tech landscape! From Microsoft’s AI-driven codebases and the rise of small but mighty language models to the real meaning of vibe coding and the junior developer revolution, the pace of change is breathtaking.

We also explore how Confluent and Databricks break down the batch vs. stream divide, setting the stage for a real-time future. Plus, a quick update on planning for Data & AI Community Day Durban: Season of AI – The Age of Agency — it’s shaping up to be something special!

Interesting Stuff - Week 17, 2025

Sun, 27 Apr 2025 10:51:53 +0200

This week, the future of AI came roaring closer — and it’s not slowing down. From vibe coding and multi-agent architectures to autonomous tool-using AIs and self-improving agents, the boundaries of what’s possible keep expanding.

Whether it’s reshaping how we build software or how enterprises operate, this week’s developments hint at a massive shift ahead. Let’s jump into the highlights!

Interesting Stuff - Week 16, 2025

Sun, 20 Apr 2025 12:13:56 +0200

This week’s roundup features powerful new AI models and more innovative tools! OpenAI launches GPT-4.1 and the streamlined o3 and o4-mini models, significantly boosting coding power, efficiency, and affordability. These models are now seamlessly integrated into GitHub Copilot.

Meanwhile, Codex CLI debuts as an open-source coding agent that turns natural language into runnable scripts, potentially reshaping automation workflows. Lastly, a thoughtful piece prompts us to reflect on technology’s societal impact, suggesting “consequence sprints” to bridge the critical gap between tech action and real-world outcomes.

Interesting Stuff - Week 15, 2025

Sun, 13 Apr 2025 08:06:50 +0200

This week’s roundup is bursting with forward-looking insights—from NVIDIA’s new AgentIQ library to Google’s Agent2Agent protocol, both pushing the boundaries of multi-agent collaboration. We dig into what the Model Context Protocol (MCP) really abstracts and explore the evolving role of LLMs in autonomous agents.

Speaking of MCP, we celebrate a big win for .NET devs with Microsoft’s new C# SDK. Plus, we get a glimpse of AI in 2027 and ride the rhythm of vibe coding with GitHub Copilot. Let’s dive in!

Interesting Stuff - Week 14, 2025

Sun, 06 Apr 2025 08:40:24 +0200

This week’s AI reading list had everything from creative game development powered by vibe coding to deep dives into AI agents and the new Model Context Protocol (MCP). I explore the tools and ideas shaping the next wave of intelligent, collaborative systems.

Oh, and speaking of agents – the Call for Speakers is now open for our next event: Data & AI Community Day Durban: Season of AI – The Age of Agency. If you’re into AI Agents, Generative AI, or just want to vibe with the community, this one’s for you.

Interesting Stuff - Week 13, 2025

Sun, 30 Mar 2025 12:33:35 +0200

This week was all about agents—multi-agent simulations, image-generating copilots, and the unstoppable rise of the Model Context Protocol (MCP). From Unreal Engine-powered MARL platforms to real-time AI agents streaming fresh data via Confluent, the pace of innovation is wild.

MCP continues to dominate the conversation, and I’ve gathered the best reads to help make sense of it all. Plus, a quick recap of our recent AI @ Ignite++ event and a sneak peek at what’s next!

Interesting Stuff - Week 12, 2025

Sun, 23 Mar 2025 12:55:37 +0200

This week’s roundup dives into some of the most exciting developments in AI and streaming tech. From OpenAI’s new fine-tuning method and Anthropic’s MCP standard to Microsoft’s R&D agent and NVIDIA’s personal AI supercomputers, the innovation is non-stop.

I also explore tools like Confluent for VS Code and OpenAI’s Agent SDK that reshape how developers build intelligent systems. Let’s unpack the trends shaping the future of AI and real-time data.

Interesting Stuff - Week 11, 2025

Sun, 16 Mar 2025 18:47:11 +0200

The AI world never slows down, and this week was no exception! DeepSeek went all-in on open source, dropping six powerful LLM training and deployment tools. OpenAI unveiled its Agents Platform, making multi-agent applications easier than ever, while Dapr AI Agents introduced a scalable, vendor-neutral approach for enterprise AI.

Meanwhile, Microsoft Fabric’s Real-Time Intelligence redefines streaming analytics with low-latency, event-driven insights. With less than a week to go until Data & AI Community Day Durban: Season of AI - AI @ Ignite++, I’m deep in the trenches finalising the agenda—so much great content, so little time!

Interesting Stuff - Week 10, 2025

Sun, 09 Mar 2025 09:27:24 +0200

AI is evolving fast, and this week’s roundup dives into some of the most exciting developments! From a must-read guide on the foundations of LLMs to the rising power of Agentic AI, we explore the cutting edge of generative AI.

In the streaming world, we look at how real-time data transforms capital markets and how Flink AI pushes the boundaries of RAG applications. Whether you’re into AI, streaming, or both, there’s plenty to unpack - let’s dive in!

Interesting Stuff - Week 09, 2025

Sun, 02 Mar 2025 08:47:17 +0200

AI is pushing boundaries once again! This week, we explore a multi-agent system automating PC tasks, OpenAI’s reasoning models vs GPT workflows, and a next-gen research AI outshining Google.

Plus, we dive into LLM customisation, AI’s evolving role in engineering, and the arrival of GPT-4.5. Finally, the Call for Speakers for Data & AI Community Day Durban: AI @ Ignite++ is live—registrations open today, so don’t miss out! 🚀

Interesting Stuff - Week 08, 2025

Sun, 23 Feb 2025 08:29:40 +0200

AI is evolving fast, from multi-agent planning breakthroughs to LLMs that reason better than ever. This week, we explore how Grok 3 stacks up against OpenAI, why AutoGen’s AI debate system is a game-changer, and how startups are secretly 10X-ing their success with AI.

Plus, a deep dive into LLM-powered coding workflows and the shift toward event-driven multi-agent systems. And on the event front: Data & AI Community Day Durban: AI @ Ignite++ is happening on March 22! The Call for Speakers is now open, and registrations kick off next week—don’t miss it!

Interesting Stuff - Week 07, 2025

Sun, 16 Feb 2025 09:31:10 +0200

AI is evolving at breakneck speed! This week, we explore Sam Altman’s bold prediction of superhuman coding AI by 2025, the rise of the AI Architect role, and Klarna’s AI assistant, which now handles work equivalent to 700 full-time staff. Meanwhile, research on large reasoning models suggests AI outperforms specialised systems in competitive coding, and NVIDIA is pushing the boundaries of AI-driven GPU optimisation.

On the streaming front, Confluent and Databricks are bridging data silos for real-time AI, while time-series and streaming databases continue their battle for real-time dominance. Dive in for a deep dive into the latest in AI, data, and streaming!

Interesting Stuff - Week 06, 2025

Sun, 09 Feb 2025 09:57:00 +0200

AI is evolving at breakneck speed, and this week’s roundup dives into some of the most intriguing developments. From agentic AI decision-making powered by Reinforcement Learning and LLMs to OpenAI’s Deep Research initiative, the push for more transparent and aligned AI is gaining momentum.

We also explore OpenAI’s O3 Mini, a lightweight yet powerful model optimized for STEM reasoning, and the rise of synthetic data generation with LLMs—an innovation that could transform AI training but raises concerns about bias and reliability. Let’s dive into the details!

Interesting Stuff - Week 05, 2025

Sun, 02 Feb 2025 10:50:36 +0200

AI isn’t just assisting us anymore—it’s stepping up as an autonomous digital worker. From Computer Use Agents that independently navigate tasks to Agentic RAG dynamically retrieving and refining data, we’re witnessing a significant shift in AI capabilities.

Meanwhile, frameworks like AutoGen, AG2, and Semantic Kernel are laying the groundwork for large-scale multi-agent collaboration and automation. Are we moving toward a world where AI agents handle entire workflows while we oversee them? Let’s dive into this week’s roundup of AI advancements!

Interesting Stuff - Week 04, 2025

Sun, 26 Jan 2025 11:35:20 +0200

Generative AI continues to push the boundaries of innovation, from Microsoft’s comprehensive framework for securing AI systems to OpenAI’s Swarm, a lightweight tool for orchestrating multi-agent workflows. This week, we also explore DeepSeek-R1, a groundbreaking model redefining reasoning efficiency, and Google’s vision for autonomous AI agents.

These advancements showcase cutting-edge technology and highlight the transformative potential of accessible and secure AI solutions. Dive in to uncover the details and my thoughts on these exciting developments!

Interesting Stuff - Week 03, 2025

Sun, 19 Jan 2025 13:34:24 +0200

Explore the latest in tech with this week’s roundup! We dive into AI’s transformative potential from Chip Huyen’s insights on intelligent agents and generative AI pitfalls to Microsoft’s AIOpsLab for autonomous clouds.

Discover how event-driven architectures power real-time AI workflows and how NovaSky’s $450 Sky-T1 model democratises advanced AI development. Plus, a glimpse into ICE 2025, where AI is reshaping the future of iGaming!

Interesting Stuff - Week 02, 2025

Sun, 12 Jan 2025 08:55:38 +0200

AI and tech innovations are moving at lightning speed, and this week’s roundup dives into the breakthroughs shaping our future. From Nvidia’s portable powerhouse redefining AI workflows to Microsoft’s Large Action Models bringing real-world task execution to LLMs, the possibilities are endless.

Explore how open-source frameworks like PRIME and AG2 are democratizing advanced AI while Sam Altman reflects on OpenAI’s journey to superintelligence. Let’s dive into these exciting developments!

Interesting Stuff - Xmas, New Year 2024, Week 1 2025

Mon, 06 Jan 2025 08:40:02 +0200

The tech landscape continues to evolve at lightning speed, and this roundup captures some of the most compelling developments in AI and data. From groundbreaking advancements in Proximal Policy Optimization to the rise of agentic workflows and transformative frameworks like aiXplain’s Multi-Agent Autonomous Framework, AI’s potential seems boundless.

Add to that the shrinking models, plummeting costs, and bold predictions for 2025, and it’s clear we’re witnessing a pivotal moment in tech innovation. Dive in to explore the trends shaping our future!

Interesting Stuff - Week 51, 2024

Sun, 22 Dec 2024 07:49:43 +0200

This week’s roundup dives into the latest AI, streaming, and cloud innovation breakthroughs. From OpenAI’s new o3 models and ChatGPT Projects feature to Kafka’s queue support and Microsoft’s AIOpsLab, these developments transform workflows, enhance scalability, and push the boundaries of AI reasoning.

Whether it’s managing complex systems or tackling real-world challenges, the future of tech is brighter than ever. Let’s explore these exciting advancements together!

Interesting Stuff - Week 50, 2024

Sun, 15 Dec 2024 11:13:12 +0200

This week’s roundup dives into the latest in AI and tech innovation, from Netflix’s scalable Distributed Counter Abstraction to OpenAI’s game-changing text-to-video tool, Sora. Discover how Splunk’s MAG-V framework tackles synthetic data challenges, Amazon SageMaker integrates AI for hyper-personalized marketing, and LLM-powered agents are redefining adaptability.

With insights into cutting-edge tools and ethical considerations, this post explores the balance between innovation and practical application. Let’s dive into the highlights!

Interesting Stuff - Week 49, 2024

Sun, 08 Dec 2024 07:57:05 +0200

This week’s tech roundup dives into groundbreaking innovations shaping the future of AI and data. From SQL Server 2025’s AI-ready capabilities transforming databases to LangChain and Large Action Models redefining application development and decision-making, the possibilities are limitless.

Explore hybrid strategies for optimizing LLM performance and seamlessly integrating real-time intelligence with proactive agents. These advancements highlight how technology continues to empower creativity and efficiency in unprecedented ways!

Interesting Stuff - Week 48, 2024

Sun, 01 Dec 2024 09:56:05 +0200

This week’s roundup explores cutting-edge developments in Generative AI, AI agents, and foundation models. From Tealium and Databricks’ partnership driving real-time customer insights to LangChain’s innovative framework for building AI-powered applications, the advancements are shaping the future of AI adoption.

Highlights include the SQL Generation Showdown, Foundry’s vision of an AI agent OS, and Microsoft’s strategies for scaling massive models like GPT-4. Dive in for insights on how these technologies push boundaries while addressing real-world challenges.

Interesting Stuff - Week 47, 2024

Sun, 24 Nov 2024 15:31:08 +0200

This week’s roundup dives into cutting-edge advancements in Generative AI, from designing real-world agentic systems to building RAG agents in Python and exploring multi-agent frameworks like Microsoft’s TinyTroupe. 🌟

Yesterday, the Data & AI Community Day Durban: Season of AI - Copilots & Agents was a resounding success, breaking records with 282 attendees and stellar feedback! Today is for recharging, but tomorrow, I’ll dive into event data and gear up for the Cape Town leg of .NET Conf 2024 South Africa. 🚀 Exciting times ahead—let’s explore!

Interesting Stuff - Week 46, 2024

Sun, 17 Nov 2024 09:41:36 +0200

Generative AI continues to amaze with innovations like Microsoft’s Magentic-One, a multi-agent framework redefining collaboration, and tools making AI accessible through no-code solutions. Posts this week explore the critical role of data as the real differentiator for LLMs and unpack the transformative power of attention mechanisms in transformers.

Meanwhile, I had the privilege of presenting at .NET Conference 2024 in Johannesburg and am gearing up for the highly anticipated Data & AI Community Day Durban. Dive in for a roundup of groundbreaking tech insights and exciting updates!

Interesting Stuff - Week 45, 2024

Sun, 10 Nov 2024 08:33:02 +0200

This week’s roundup looks at the latest breakthroughs in Generative AI, from multi-agent frameworks and autonomous agent ecosystems to accelerated LLM tools like OpenAI’s GPT-4o Predicted Outputs. We also spotlight Databricks’ real-time data innovations with materialized views and streaming tables, enhancing analytics speed and efficiency across sectors.

Plus, in the WIND section, we’re thrilled with the buzz for our upcoming Data & AI Community Day and my upcoming talks at .NET Conf 2024 South Africa—it’s an exciting time to be in tech!

Interesting Stuff - Week 44, 2024

Sun, 03 Nov 2024 11:42:33 +0200

This week’s roundup dives into the latest in Generative AI, streaming tech, and the revival of intelligent agents. From Indrajit Kar’s advanced multi-agent Q&A systems and GitHub Universe’s AI-powered tools to Hamel Husain’s guide on building LLM judges, it’s all about pushing boundaries.

Plus, discover how to transform PDFs into podcasts and why SlateDB might just be the next big thing in stream processing.

Interesting Stuff - Week 43, 2024

Sun, 27 Oct 2024 08:12:57 +0200

In this week’s tech roundup, I explore the latest advancements in AI-ML, Generative AI, and Streaming. From mastering contextual bandits for real-time personalization to the evolving challenges of the Transformer architecture, these posts provide deep insights into AI’s growing impact.

I also discuss the practicalities of building LLMs with simple math and the debate between Apache Kafka, Azure Event Hubs, and Confluent Cloud for cloud-based data ecosystems. Let’s get started!

Interesting Stuff - Week 42, 2024

Sun, 20 Oct 2024 09:35:18 +0200

This week’s roundup dives into exciting developments in AI and streaming technology. From Actionable AI, where systems move from passive language models to action-driven capabilities, to OpenAI’s new Swarm paradigm, which enables AI agents to collaborate like a hive mind.

Plus, we continue the deep dive into Kafka internals, focusing on how it optimizes data-fetching for real-time streams. Get ready to explore how these advancements are shaping the future of tech!

Interesting Stuff - Week 41, 2024

Sun, 13 Oct 2024 11:19:25 +0200

This week’s roundup covers some exciting developments in the tech world! There’s plenty to explore, from the power of Microsoft Fabric Lakehouse combined with Apache Kafka and Flink for real-time data streaming to Netflix’s innovative global caching system.

We also explore OpenAI’s ChatGPT Canvas for coding and AI agents’ evolving reasoning capabilities. Plus, don’t miss the latest news on our upcoming Data & AI Community Day Durban: Season of AI—Copilots & Agents event, sponsored by UNISA!

Interesting Stuff - Week 40, 2024

Sun, 06 Oct 2024 10:15:17 +0200

This week’s tech roundup highlights Microsoft’s Drasi, an open-source tool simplifying real-time event detection in complex systems, and Data Formulator, which enhances data visualization with AI-driven precision.

We also explore the revolutionary MotleyCrew AI framework for multi-agent systems and OpenAI’s Realtime API, streamlining voice assistant creation. Don’t miss insights into AutoGen 0.4 and Promptriever, both pushing the boundaries of scalable AI and information retrieval.

Interesting Stuff - Week 39, 2024

Sun, 29 Sep 2024 09:46:52 +0200

Welcome to this week’s tech roundup, where we explore groundbreaking innovations shaping the industry! Discover insights into RAG AI agents and their future implications; look at Hamza Farooq’s hands-on approach to building custom AI agents.

Learn how Microsoft’s RD-Agent is transforming research and development processes. We’ll also highlight the exciting features in RisingWave v2.0 and the merging of streaming platforms with Lakehouse architectures.

Interesting Stuff - Week 38, 2024

Sun, 22 Sep 2024 09:44:13 +0200

This week’s roundup explores exciting developments in AI and streaming technologies. We dive into Louis Bouchard’s insights on optimizing Retrieval-Augmented Generation (RAG) systems, a new LLM-driven software engineering framework tackling modern development challenges, and CodeMaker AI’s breakthrough in large-scale code generation.

We also look at Confluent’s new integration for VS Code, making it easier for developers to build and manage real-time data streams within their favorite IDE.

Interesting Stuff - Week 37, 2024

Sun, 15 Sep 2024 09:30:33 +0200

This week’s roundup dives into exciting developments in AI, from automated agent design with ADAS to the use of large language models (LLMs) for handling tabular data and improving semantic search with knowledge graphs.

We also explore llama-deploy, a new open-source tool for deploying agent workflows, and OpenAI’s o1 model, which excels in complex reasoning and coding competitions. These advancements highlight AI’s growing role in automation, problem-solving, and production workflows.

Interesting Stuff - Week 36, 2024

Sun, 08 Sep 2024 08:09:06 +0200

This week’s blog dives into some of the most exciting tech advancements in generative AI and real-time data streaming. From Google DeepMind’s GameNGen, which simulates DOOM in real-time, to a scalable multi-agent framework for decision-making in large systems, these developments are pushing the boundaries of AI applications.

I also explore innovative approaches to Retrieval-Augmented Generation (RAG) using Flink and Kafka, along with clustering techniques to improve chunking in RAG systems. Dive in to see how these technologies are shaping the future!

Interesting Stuff - Week 35, 2024

Sun, 01 Sep 2024 09:21:52 +0200

This week, we explore the evolving role of Generative AI in strategy and development. AI continues to push boundaries from LLMs acting as team enablers in software projects to the exciting debut of fine-tuning GPT-4o on Azure.

We also dive into a fascinating experiment that tests LLMs’ strategic capabilities in a simulated Risk game, revealing both their potential and limitations. Join me as we navigate these intriguing developments and what they mean for the future of AI.

Interesting Stuff - Week 34, 2024

Sun, 25 Aug 2024 08:39:43 +0200

In this week’s tech roundup, we explore why a hedge fund built its own database, dive into Microsoft’s latest AI advancements with the Phi 3.5 series, and celebrate five years of Confluent Cloud connectors shaping the future of data streaming.

Plus, catch a glimpse of my recent adventures at the Season of AI event in Cape Town, where we delved into the magic of Generative AI despite a few technical challenges. Next up: Johannesburg—here’s to smoother skies and glitch-free talks!

Interesting Stuff - Week 33, 2024

Sun, 18 Aug 2024 09:37:29 +0200

This week’s roundup explores groundbreaking developments in AI and data streaming. From the innovative MindSearch framework that enhances AI-driven web searches to the evolution of SQL Generator 2.0, which simplifies complex database queries, these advancements are pushing the boundaries of technology.

We also dive into the rapidly growing concept of Multi-Agent-as-a-Service and highlight the must-attend Current 2024 event for data streaming professionals. Plus, a quick update on my upcoming presentations at the Season of AI events in Cape Town and Johannesburg!

Interesting Stuff - Week 32, 2024

Sun, 11 Aug 2024 10:24:44 +0200

In this week’s roundup, I dive into the latest advancements in Generative AI, exploring how multi-agent systems push the boundaries of what’s possible. I also highlight OpenAI’s new structured outputs in the API, which promise more reliable and precise AI interactions. Plus, Nicholas Carlini shares insightful ways AI can enhance daily workflows without replacing human oversight.

Finally, an exciting update on Lemi Masalu’s and my upcoming presentations at Cape Town and Johannesburg’s Season of AI events!

Interesting Stuff - Week 31, 2024

Sun, 04 Aug 2024 09:46:09 +0200

In this week’s roundup, we explore the transformative potential of AI and real-time analytics, from using Large Language Models (LLMs) to evaluate SQL queries to Stanford’s RelBench revolutionizing data analysis.

We also delve into GitHub Models, a groundbreaking platform empowering developers with AI capabilities, and FlinkAI, which enhances real-time data streaming with machine learning. Plus, a recap of the vibrant Data & AI Community Day Johannesburg event, where we unlocked the magic of Generative AI and LLMs. Dive in to discover how these innovations are reshaping the tech landscape!

Interesting Stuff - Week 30, 2024

Sun, 28 Jul 2024 12:59:30 +0200

🚀 This week’s tech roundup covers Netflix’s innovative Maestro workflow orchestrator, Microsoft’s advancements in self-adapting AI agents, and a hands-on guide to building streaming AI applications.

We also explore TaskGen’s AI-driven task automation and the latest enhancements in real-time data streaming from Microsoft Fabric and Confluent Cloud. Don’t miss my prep for an upcoming presentation on Generative AI in Johannesburg!

Interesting Stuff - Week 29, 2024

Sun, 21 Jul 2024 10:20:04 +0200

This week’s tech roundup explores the synergies of Generative, Analytical, Causal, and Autonomous AI, showcasing their unique contributions to innovation. Discover CAMEL-AI’s revolutionary multi-agent framework for enhanced autonomous cooperation.

We also cover LangChain’s insights into planning complexities for AI agents. Plus, a recap of the successful Data & AI Community Day Durban: Season of AI event, which saw over 200 attendees and top-rated sessions. Dive into these highlights and more in this week’s blog post!

Interesting Stuff - Week 28, 2024

Sun, 14 Jul 2024 12:28:44 +0200

This week’s tech roundup delves into the latest advancements in Generative AI and Streaming technologies. Explore seamless graph construction with Neo4j and LangChain, the revolutionary potential of synthetic data for LLM training, and the power of in-memory analytics with DuckDB and Kafka.

Plus, get insights into building robust full-stack applications and scalable notification systems. Don’t miss the exciting updates on the upcoming Data & AI Community Day Durban: Season of AI event!

Interesting Stuff - Week 27, 2024

Sun, 07 Jul 2024 08:39:54 +0200

🌟 This week’s tech roundup dives into the evolving landscape of data processing with a historical perspective, explores the pivotal role of agents in AI workflows through LangChain, and showcases the innovative fusion of gradient boosting with LLM techniques for time series forecasting.

We also highlight Microsoft Fabric’s EventHouse, a cutting-edge solution enhancing real-time search capabilities. Plus, exciting updates on the upcoming Data & AI Community Day Durban: Season of AI! 🌟

Interesting Stuff - Week 26, 2024

Sun, 30 Jun 2024 10:33:36 +0200

This week’s blog post covers Microsoft’s innovative ‘Instruction Pre-Training’ to enhance AI capabilities, the potential of AutoGen and Multi-Agent frameworks to revolutionise autonomous systems, and an insightful guide on the inner workings of large language models.

Plus, exciting updates about the upcoming Data & AI Community Day Durban: Season of AI on July 20, where I’ll be presenting Getting Started with Azure OpenAI. Don’t miss out on this fantastic event!

Interesting Stuff - Week 25, 2024

Sun, 23 Jun 2024 12:01:50 +0200

This week’s tech roundup highlights exciting developments in AI and data management. Discover how CodiumAI’s PR-Agent is revolutionizing pull requests, OpenAI’s acquisition of Rockset enhances enterprise AI, and structured outputs improve LLM responses.

Learn about CrewAI’s multi-agent systems for task automation, Causal AI’s impact on marketing strategies, and Microsoft’s AutoGen Studio for low-code AI workflows. Plus, get updates on the upcoming Data & AI Community Day Durban event. Join us for insights and innovations shaping the tech landscape! 🚀

Interesting Stuff - Week 24, 2024

Sun, 16 Jun 2024 09:38:32 +0200

This week, dive into the latest tech innovations, including OpenAI’s official .NET library release and its integration with Apple devices, enhancing user experiences with advanced AI capabilities.

Learn from Andrew Ng’s new course on building AI assistants and explore the power of data streaming for customer loyalty programs. Check out updates on the upcoming Data & AI Community Day Durban: Season of AI, where speaker selections are underway. Secure your spot now and be part of the excitement! 🚀

Interesting Stuff - Week 23, 2024

Sun, 09 Jun 2024 09:57:33 +0200

This week’s tech roundup delves into Netflix’s advanced use of causal inference for data-driven decision-making, the efficiency of small language models over larger counterparts, and the importance of dynamic prompting for domain adaptation.

Additionally, we explore the Generative AI Handbook for learning resources and best practices for reliable GenAI apps. Lastly, discover how to integrate Flink SQL, Streamlit, and Kafka for real-time data applications. Dive into these insights to stay ahead in the tech landscape!

Interesting Stuff - Week 22, 2024

Sun, 02 Jun 2024 08:36:00 +0200

🚀 This week in tech news: Discover KQL’s new capabilities in Azure Data Explorer, empowering data management with update and delete statements.

Explore FinRobot, an open-source AI platform for financial tasks, and Microsoft’s synthetic data techniques ensuring AI privacy. Learn how RAG, Kafka, and Flink reduce AI hallucinations with real-time data.

Finally, dive into Confluent’s best practices for Terraform and solutions to the dual-write problem, plus exciting updates for the upcoming Data & AI Community Day Durban! 🌟

Interesting Stuff - Week 21, 2024

Sun, 26 May 2024 09:47:32 +0200

📅 In this week’s blog post, we dive into highlights from Microsoft Build 2024, which featured groundbreaking updates to Microsoft Fabric and Real-Time Intelligence.

Discover the power of Deep Q-Networks in reinforcement learning and the versatile applications of Microsoft’s Phi-3 vision-language model. Explore how Confluent Cloud’s Elastic CKUs are revolutionizing Kafka cluster management.

Plus, get a sneak peek at the exciting SWAG for the upcoming Data & AI Community Day Durban: Season of AI event. 🚀🎉

Interesting Stuff - Week 20, 2024

Sun, 19 May 2024 07:40:12 +0200

In this week’s blog post, I dive into the exciting advancements in Generative AI and Streaming Databases. Discover the transformative potential of LLM-based SQL agents for SAP HANA, the impressive capabilities of OpenAI’s GPT-4o, and the seamless integration of LangChain with Azure Container Apps.

Learn how venture capital firms leverage GenAI with Databricks and explore the latest enhancements in ChatGPT’s data analysis. Finally, understand the revolutionary impact of streaming databases in handling real-time data across various industries.

Interesting Stuff - Week 19, 2024

Sun, 12 May 2024 09:34:33 +0200

This week’s blog explores the dynamic world of AI through the lens of Reinforcement Learning, an approach where AI learns from experience, hinting at future ‘intuitive’ AI. We also look at the collaborative potentials of AI automation and the expansive capabilities of internet-savvy AI agents.

Meanwhile, buzz builds around OpenAI’s upcoming event, promising “magical” innovations that could reshape our tech landscape. Plus, get all the details on the upcoming Data & AI Community Day Durban: Season of AI — a gathering that promises to fuse knowledge and networking in the vibrant data science and AI field.

Interesting Stuff - Week 18, 2024

Sun, 05 May 2024 15:19:44 +0200

🚀 Dive into this week’s tech highlights! From GitHub’s game-changing Copilot Workspace to Confluent Cloud’s revolutionary Freight Clusters, discover how cutting-edge advancements in Generative AI and streaming are shaping the future of technology.

Plus, get a sneak peek into the upcoming Season of AI event, promising a day filled with AI, Data, ML, and more! Don’t miss out on the latest innovations driving the tech landscape forward. 💡🌐

Interesting Stuff - Week 17, 2024

Sun, 28 Apr 2024 17:45:55 +0200

This week on the blog, we dive into the transformative power of Causal AI, which is reshaping decision-making by focusing on causation rather than mere correlation. We also explore the realm of serverless AI chatbots built using LangChain.js and Azure, highlighting their scalability and efficiency.

Additionally, we discuss how graph-based metadata filtering is refining vector search in AI applications. Don’t miss our vibrant recap of the Data & AI Community Day, a testament to the thriving interest in these advanced technologies.

Interesting Stuff - Week 16, 2024

Sun, 21 Apr 2024 08:50:33 +0200

This week, we’re diving into a myriad of technological advancements and insights, from the innovative use of Microsoft Fabric’s Eventhouse in analytics to pioneering AI-driven gaming strategies with “Player Insights.”

We also explore hands-on AI learning with Microsoft’s “Generative AI for Beginners” and discuss the efficiency of Kafka Docker Composer for cluster setups.

Join me as we unpack these developments and look forward to another thrilling Data and AI Community Day in Durban—packed with expert talks and stunning ocean views!

Interesting Stuff - Week 15, 2024

Sun, 14 Apr 2024 09:43:48 +0200

Welcome to this week’s tech roundup! Dive into the evolving world of databases with insights on scalable OLTP systems in the cloud and the debut of Microsoft’s Garnet, a cutting-edge alternative to Redis.

In AI advancements, explore how Databricks transforms data engineering with Large Language Models and how autonomous AI agents like those described by Andrew NG are reshaping our technology expectations.

Plus, get the latest on practical applications in finance and streaming technologies, and join us for the upcoming Data and AI Community Days for a firsthand experience of innovation in action.

Interesting Stuff - Week 14, 2024

Sun, 07 Apr 2024 10:27:45 +0200

This week’s blog post is a whirlwind tour through the latest in Generative AI and Streaming Analytics, featuring everything from a deep dive into Retrieval Augmented Generation (RAG) to the innovative Eventhouse by Microsoft Fabric.

We explore the groundbreaking DBRX model by Databricks, and share insights from the integration of Apache Kafka with Tinybird. Plus, a sneak peek into my preparations for the upcoming Data and AI Community Days in Johannesburg and Durban. Join me as we navigate these exciting developments in the tech world, perfect for enthusiasts and professionals alike!

Interesting Stuff - Week 13, 2024

Sun, 31 Mar 2024 08:31:01 +0200

This week’s blog post delves into the latest advancements in Generative AI and data streaming, spotlighting transformative frameworks like LlamaFactory and Mora, and groundbreaking tools such as DBRX and Apache Kafka.

We explore Andrew Ng’s insights on the potential of agentic workflows and the strategic shifts revealed at Kafka Summit 2024. Plus, a sneak peek into the upcoming Data and AI Community Day in Durban, promising a day packed with learning and networking.

Interesting Stuff - Week 12, 2024

Sun, 24 Mar 2024 08:47:07 +0200

This week’s exploration dives into the transformative realms of tech, from the evolution of OLTP databases to meet growing transactional demands to the innovative “Self-Learning GPTs” that adapt and grow through user feedback.

We also delve into Microsoft’s AutoDev, an AI-driven leap towards autonomous software development. We wrap up with highlights from Kafka Summit London 2024, where the fusion of data streaming platforms and Gen AI took the spotlight.

Interesting Stuff - Week 11, 2024

Sun, 17 Mar 2024 17:36:03 +0200

Dive into this week’s tech digest, where we explore groundbreaking advancements from causal graphs in data science to the innovative LDB framework in code debugging. Discover the art of designing RAGs for enhanced chatbot interactions and the integration of knowledge graphs for accuracy.

Plus, get a sneak peek into the future with LLM Agents and the transformative Retrieval Augmented Thoughts (RAT) method. Join us as we unravel these fascinating developments and gear up for the upcoming Data and AI Community Day in Durban!

Interesting Stuff - Week 10, 2024

Sun, 10 Mar 2024 13:54:27 +0200

This week’s blog post delves into the fascinating world of Generative AI and Streaming, showcasing innovations like SalesGPT’s transformative sales conversations, the future of coding with Multi-Agent LLM Frameworks, and the intricacies of real-time sentiment analysis in Yelp reviews.

We also spotlight Uber’s journey to scaling real-time chat for improved customer service. Plus, exciting news for the local tech community: the Data and AI Community Day Durban is on the horizon, promising a day filled with insightful sessions.

Interesting Stuff - Week 09, 2024

Sun, 03 Mar 2024 13:11:15 +0200

This week’s blog post is a whirlwind tour through the latest in tech, from Sicelumusa Khumalo’s groundbreaking strategies in scalable systems to the intricacies of data indexing for LLMs.

We’ll also dive into the new features of Apache Kafka® 3.7.0 and explore the dynamic realm of real-time sentiment analysis with Kafka and PySpark. Plus, I’ll share insights from my recent presentation at the Global Power Platform Bootcamp, where we delved into the transformative power of Generative AI with Microsoft Copilot Studio and Azure OpenAI.

Interesting Stuff - Week 08, 2024

Sun, 25 Feb 2024 09:51:01 +0200

This week’s journey through the realm of Generative AI has been nothing short of exhilarating.

From Andrej Karpathy’s enlightening dive into Byte Pair Encoding with minbpe to the revolutionary strides in video creation by OpenAI’s Sora, we’re witnessing a new era of innovation.

Meta’s TestGen-LLM is redefining software testing, while the “Mind Mapper” project showcases the power of network analysis and LLMs in extracting complex knowledge. Join me as we explore these groundbreaking advancements that are setting new benchmarks in the field of AI and technology.

Interesting Stuff - Week 07, 2024

Sun, 18 Feb 2024 14:12:38 +0200

This week’s blog takes you on an exhilarating journey through the latest in Generative AI and Streaming Data Mesh, showcasing groundbreaking advancements from the practical insights of LLMs with llama.cpp to the innovative Functionary model.

We delve into how GraphRAG transforms document analysis and how SORA reshapes text-to-video creativity. Plus, get a glimpse into the transformative world of Streaming Data Mesh with Kafka & RisingWave,

Finally, join me at the Global Power Platform Bootcamp 2024 for a deep dive into Microsoft Copilot Studio and the wonders of Generative AI.

Interesting Stuff - Week 06, 2024

Sun, 11 Feb 2024 08:27:29 +0200

This week’s blog post delves into the fascinating realms of data architecture, generative AI, and streaming systems. I explore James Serra’s insightful book on navigating complex data architectures.

In the GenAI space I look at Microsoft Research’s groundbreaking AI Controller Interface for enhancing LLM control. Finally Neil Avery’s compelling take on the synergy between Data Mesh and Hexagonal Architecture using Kafka. I hope you find these articles as intriguing as I did. Enjoy!

Interesting Stuff - Week 05, 2024

Sat, 03 Feb 2024 07:09:23 +0200

This week’s blog post takes you on a tech adventure, from tackling the One Billion Row Challenge using **Azure Data Explorer to explore the collaborative potential of AutoGen and OpenAI assistants. Dive into the simplicity and power of vector databases and discover a smart load-balancing solution for OpenAI endpoints.

Plus, get a sneak peek into a GenAI demo revolutionizing sales processes with Kafka, Flink, LangChain, and OpenAI. Join me as I share these insights before heading to London for an exciting week of customer visits and the iGaming ICE conference.

One Billion Row Challenge & Azure Data Explorer

Sun, 28 Jan 2024 17:08:46 +0200

On January 1st (that’s dedication - New Year’s Day), 2024 Gunnar Morling published on his blog the One Billion Row Challenge. The challenge is to load and aggregate one billion rows using Java. The challenge took on a life of its own, and there are now several implementations of the challenge in different languages, including databases ( Robin, Hubert, Francesco, among others). The data to aggregate is a list of temperature readings from weather stations.

I thought It would be fun to do the challenge using Azure Data Explorer (ADX) since I like ADX and have written blog posts about it. ADX is a fast, scalable, and highly available data analytics service. It is optimized for data exploration over large data volumes. ADX is a columnar store that uses a query language called Kusto Query Language (KQL). It is a fully managed service, and it is part of the Azure platform.

So, in this blog post, you’ll see what I did to load and aggregate one billion rows. Spoiler alert: certain things didn’t work out as I had hoped, but that could be due to me being more rusty with ADX than anything else.

Interesting Stuff - Week 04, 2024

Sun, 28 Jan 2024 10:07:51 +0200

This week’s blog post is a whirlwind tour through the latest in tech innovation, from the seamless integration of C# and Python for text-to-speech applications to the collaborative power of AI with platforms like CrewAI.

We’ll explore cutting-edge frameworks for building AI teams and delve into the architectural blueprints for enterprise chat applications using Azure OpenAI.

Plus, get an insider’s look at how Confluent leverages its own platform for real-time alerting, setting new standards in data streaming and operational agility. Join me!

Interesting Stuff - Week 03, 2024

Sun, 21 Jan 2024 09:31:18 +0200

Welcome to this week’s blog, where I am delving into some of the most exciting technological advancements in AI and software development.

I start by looking at Dean Martin’s innovative CodeInterpreterThoughts class, a tool that brilliantly integrates Python and C# programming languages. In the realm of Generative AI, I explore CrewAI, a groundbreaking framework revolutionizing the AI landscape.

Another highlight of this week’s post is the introduction of AlphaCodium, a state-of-the-art approach to code generation. Last but not least, I highlight the exciting partnership between Confluent and Pinecone, introducing Pinecone Serverless - a vector database architecture revolutionizing GenAI applications with its serverless developer experience.

Interesting Stuff - Week 02, 2024

Sun, 14 Jan 2024 07:13:48 +0200

This week’s blog post is a journey through the latest advancements in Generative AI and Streaming Technologies. We explore the transformative integration of Ray Autoscaling with Databricks and Apache Spark, a game-changer in scaling AI and Python workloads efficiently. The post also delves into the innovative use of GPT-4’s “Custom Instructions” for enhancing Python code creation, showcasing its potential in data visualization.

In the streaming domain, we highlight Confluent’s strides in trade settlements with the T+1 industry regulation, and the remarkable performance leap of Confluent Cloud’s Kora engine, now 10x faster than Apache Kafka.

Interesting Stuff - Xmas, New Year 2023, Week 1 2024

Sun, 07 Jan 2024 09:38:19 +0200

Welcome to this week’s whirlwind of tech discoveries! From the art of crafting impactful tutorials to the innovative depths of databases and generative AI, we’re diving deep into the realms of technical clarity, database robustness, and AI’s potential.

Plus, explore the transformative collaboration between RisingWave and MindsDB for real-time ML solutions.

Unlocking Azure AI Studio: My Journey to Overcoming Subscription Snags

Sat, 30 Dec 2023 08:30:44 +0200

Microsoft’s Azure stands out in the ever-evolving landscape of cloud services and artificial intelligence. Azure’s AI services are no exception, offering a wide range of capabilities to help developers and data scientists build, train, and deploy models at scale.

This blog post delves into a specific challenge I faced within this ecosystem - getting Azure AI Studio to recognize my Azure subscription.

Interesting Stuff - Week 51, 2023

Sun, 24 Dec 2023 08:32:50 +0200

This week’s blog post is a treasure trove of Generative AI and Streaming insights! Dive into advanced RAG techniques, explore Microsoft’s GPT-RAG, and discover the power of Kafka in Two-Phase Commit protocols.

As we gear up for the festive season, join me in exploring cutting-edge AI applications and stream processing wonders.

Interesting Stuff - Week 50, 2023

Sun, 17 Dec 2023 08:09:37 +0200

In this week’s blog post, we explore the cutting-edge of AI and streaming technology. From a beginner’s guide to building an AI agent using Deep Q-Learning to the integration of Large Language Models in pursuit of AGI, we cover a range of fascinating topics.

We also delve into practical applications with Azure AI Studio and discuss Confluent’s serverless Apache Flink service, highlighting its impact on stream processing. Join us as we navigate these exciting developments and envision the future shaped by AI advancements.

Interesting Stuff - Week 49, 2023

Sun, 10 Dec 2023 09:24:24 +0200

🚀 Welcome to this week’s exploration of AI’s cutting-edge! Join me as I delve into Simon Headley’s insights on scaling event-driven systems in Kubernetes with Dapr, and Youssef Hosni’s comprehensive overview of Artificial General Intelligence.

We’ll also uncover the potential of Retrieval Augmented Generation in enhancing LLM responses, guided by Eduardo Alvarez, and navigate through Leonie Monigatti’s strategies for optimizing RAG applications. Plus, don’t miss Andrej Karpathy’s unique take on the “hallucination problem” in LLMs, offering a fresh perspective on AI’s creative capabilities.

Interesting Stuff - Week 48, 2023

Sun, 03 Dec 2023 11:02:06 +0200

Welcome to this week’s blog post where we delve into the fascinating realms of Generative AI and Data Streaming! 🌟

Explore the transformative power of coding assistants like GitHub Copilot and ChatGPT, learn how to create your own Custom GPT model, and uncover the innovative approach of Programming with Representations (PwR).

Dive into the world of data streaming with insights on librdkafka for Apache Kafka and the intriguing intersection of physics and data science in stream processing. Plus, get a glimpse of my recent experience at the .NET Conf 2023 in Johannesburg, where I shared insights on Kafka for .NET Developers. 🚀💻

Interesting Stuff - Week 47, 2023

Sun, 26 Nov 2023 13:37:33 +0200

Dive into a week of tech discoveries! From Martin Fowler’s indispensable guide to effective distributed systems to the future of gaming with real-time generative AI, there’s a wealth of knowledge waiting for you.

Explore Sam Altman’s return as OpenAI’s CEO, shaping the AI landscape, and the game-changing partnership between Confluent and Ably for seamless Kafka integration.

Join me on this tech journey as I share my recent experiences presenting at the .NET Conference and look ahead to future adventures. Stay tuned for a week’s worth of captivating tech tales!

Interesting Stuff - Week 46, 2023

Sun, 19 Nov 2023 10:13:33 +0200

Welcome to this week’s exploration of the ever-evolving tech landscape! We’re delving into the seismic shifts at OpenAI with the surprising departure of Sam Altman, and uncovering the transformative power of Retrieval-Augmented Generation in enterprise AI.

Get ready for a deep dive into OpenAI’s latest innovations, including DALL-E 3, GPT-4 Vision, and GPT-4 Turbo, and join me on a personal journey of building a custom GPT application.

Plus, we’ll explore the critical role of ISO 20022 in modernizing payment architectures.

This week is all about groundbreaking developments and personal adventures in the world of AI and streaming technologies!

Interesting Stuff - Week 45, 2023

Sun, 12 Nov 2023 10:00:11 +0200

In this week’s blog post, I delve into the fascinating world of Generative AI and Streaming technologies, sharing key insights and developments. We explore the complexities and innovations in building data pipelines for Large Language Models (LLMs), and the transformative role of Apache Kafka, Flink, and Vector Databases in real-time AI applications.

Additionally, I reflect on the resounding success of the recent AI/ML & Data Infusion Roadshow in Durban, highlighting the enthusiastic response from attendees and the insightful contributions of our expert speakers.

Interesting Stuff - Week 44, 2023

Sun, 05 Nov 2023 09:31:45 +0200

This week we’re diving into the exciting world of Generative AI and Real-time Data Streaming.

Discover how Generative AI is reshaping the gaming landscape with personalized experiences and explore the fascinating landscape of Large Language Model (LLM) applications.

We’ll also introduce you to NExT-GPT, a revolutionary multimodal model. Plus, don’t miss the future of business analytics with AI and Confluent Cloud.

Finally get a sneak peek into the upcoming AI/ML & Data Infusion Roadshow Durban. Get set for a tech adventure like no other!

Interesting Stuff - Week 43, 2023

Sun, 29 Oct 2023 09:02:50 +0200

🚀 This week, we delve into game-changing developments like Microsoft Research’s AutoGen, which accelerates AI development with automated features. Explore the remarkable growth of LLM-based autonomous agents and their potential to achieve human-like intelligence. Learn about TimeGPT, a cutting-edge model for time series forecasting

Read about RisingWave 1.3, an open-source streaming database with advanced features. Plus, get ready for an exciting blog series on Generative AI and Kafka. Lastly, join Niels at the AI/ML & Data Infusion Roadshow Durban for an exceptional tech event. Stay updated with the latest tech trends - read on! 💡

Interesting Stuff - Week 42, 2023

Sun, 22 Oct 2023 12:56:55 +0200

This week, Generative AI only: Uncover groundbreaking developments in Large Language Models (LLMs) like “Prometheus” and “Table-GPT,” reshaping the AI landscape. Explore the power of Microsoft Azure in connecting GPT models with company data through a beginner’s guide.

Plus, learn how LLMs can jumpstart your projects and discover the critical role of manual adjustments for production-ready solutions.

Interesting Stuff - Week 41, 2023

Sun, 15 Oct 2023 11:53:08 +0200

🚀 This week dive into a world of tech wonders this week! Discover Google’s groundbreaking F1 distributed SQL database. Create your AI agent with LangChain and GPT-4. E

Explore the future of AI with LangChain 101, revolutionize LLMs with AutoGen. Unveil the power of AI in real-time social media monitoring.

Interesting Stuff - Week 40, 2023

Sun, 08 Oct 2023 20:14:49 +0200

This week: Dive into the dynamic world of AI/ML and streaming technology. Explore Ray, the groundbreaking compute runtime, and the rise of RAG-Fusion in information retrieval.

Delve into the pursuit of Artificial General Intelligence (AGI) in 2023 and discover how data streaming empowers Elemental Cognition’s AI platform.

Plus, get ready to master Apache Flink with an essential resource. Stay updated and informed with our insights into the tech landscape.

Interesting Stuff - Week 39, 2023

Sun, 01 Oct 2023 11:51:10 +0200

Get ready for an exciting week in the world of data streaming! In this week’s blog, we’ll dive into the highlights from the recent Current 2023 conference, including the buzzworthy Confluent Cloud-Flink integration. Plus, we’ll explore the game-changing addition of Apache Flink on Confluent Cloud and Confluent’s new initiative, Data Streaming for AI

Finally, let’s rewind to my recent adventure at Data Saturday Cape Town, and get a sneak peek at the upcoming AI/ML & Data Infusion Roadshow Durban event. Stay tuned for some thrilling updates! 🚀

Interesting Stuff - Week 37 - 38, 2023

Sun, 24 Sep 2023 20:50:07 +0200

This roundup covers last week and this week: Explore the Circuit Breaker Design Pattern with insights from Waldemar and learn how it handles errors in distributed systems. Discover the world of Distributed Computing with Simon’s exploration of Dapr’s sidecar pattern, simplifying infrastructure interactions.

Revolutionize SQL queries with Azure Open AI, delve into Langstream’s Gen AI framework, enhance Azure OpenAI models with vector search in Azure Cosmos DB, and master Apache Flink’s stream processing capabilities with Flink SQL.

Join me at Data Saturday Cape Town on September 30th for deeper insights.

Interesting Stuff - Week 36, 2023

Sun, 10 Sep 2023 09:25:38 +0200

This week, discover event-driven systems with Dapr in Kubernetes, delve into AI compilers by Microsoft Research, and learn about OpenTelemetry for observability.

Uncover Formula 1 streaming data pipelines with Python, Kafka, RisingWave, and Confluent Cloud’s role in real-time insurance quotes.

Join me at Data Saturday Cape Town on September 30th, and save the date for AI & Data Infusion Roadshow Durban on November 11th.

Interesting Stuff - Week 35, 2023

Sun, 03 Sep 2023 14:50:22 +0200

Embark on a tech-filled journey this week!

Learn how FYNDNA transformed HDFC Bank using Dapr. Explore Generative AI with Chat-GPT.

Stay ahead in streaming tech with Hubert Dulay, and dive into real-world Flink applications.

Interesting Stuff - Week 34, 2023

Sun, 27 Aug 2023 11:19:46 +0200

Embark on an exhilarating journey through this week’s blog post, where we unravel the latest frontiers in AI, GenerativeAI, and real-time data streaming. Uncover the dynamic world of Reinforcement Learning as agents master complex tasks through trial and error while demystifying the distinctions between AI, Machine Learning, Deep Learning, and Neural Networks.

Delve into the pursuit of Artificial General Intelligence (AGI) and its potential to reshape the future alongside the transformative impact of ChatGPT’s human-AI interactions. Beyond AI, explore the revolutionary Streaming Plane, bridging real-time analytics and historical insights. From AGI’s challenges to the Streaming Plane’s promise, this week’s insights will ignite your tech curiosity. 🚀🤖🌐

Interesting Stuff - Week 33, 2023

Sun, 20 Aug 2023 16:43:46 +0200

🚀 In this post: Unveil AI/ML marvels and streaming sensations! Python Vector Databases, Real-Time ML, Apache Flink, Confluent Platform, and more.

Plus, relive the record-breaking Data Saturday Durban event! Join the innovation journey.

Interesting Stuff - Week 32, 2023

Sun, 13 Aug 2023 09:32:16 +0200

🔍 In this week’s roundup: Discover the power of Big Data’s serving layers in James Serra’s post, unveiling the benefits of relational and physical layers for data accessibility.

🚀 Speculate on the groundbreaking potential of OpenAI’s trademarked GPT-5, reshaping the AI landscape. 🌊 Stay ahead in the streaming world with Hubert’s newsletter, spotlighting rising stars like Redpanda and must-read Flink insights.

Join Niels at Data Saturday Durban for an Azure OpenAI deep-dive and seize the opportunity to learn from top speakers. Don’t miss out!

Interesting Stuff - Week 31, 2023

Mon, 07 Aug 2023 06:49:53 +0200

🔥 In this weeks roundup. Unleash the Power of AI: Master the art of prompt engineering, crafting effective cues for NLP models.

Explore RAG-based chatbots and OpenAI’s evolution from GPT-1 to GPT-4, with a teaser of GPT-5.

Dive into event-driven serverless apps and harmonize streaming with standardized hashing. Join Niels on data adventures and secure your spot at Data Saturday Durban.🔗🚀

Interesting Stuff - Week 30, 2023

Sun, 30 Jul 2023 08:51:07 +0200

🌟 Discover the latest tech innovations in this week’s blog post! From managing and analyzing game data at scale using Azure PlayFab and Databricks, to exploring the powerful Ambassador Pattern for distributed systems.

Get ready for the exciting news about OpenAI Whisper’s arrival on Azure OpenAI Service and Azure AI Speech, along with the introduction of function calling. Don’t miss the insights on using Azure “Prompt Flow” and the fascinating potential of Apache Kafka as a mission-critical data fabric for GenAI.

Join us on this tech journey and mark your calendar for the upcoming Data Saturday events! 🚀🌐

Interesting Stuff - Week 29, 2023

Sun, 23 Jul 2023 14:20:12 +0200

📚 Explore the latest tech wonders and data-driven innovations in this week’s blog post!

From empowering analysts with Azure Data Explorer to estimating customer lifetime value through Lakehouse architecture, and the exciting Databricks-MosaicML collaboration democratizing generative AI.

Delve into the best sessions of Kafka Summit London 2023 and discover how Confluent revolutionizes real-time order notifications for businesses.

Plus, don’t miss the impressive lineup of industry experts and sessions at Data Saturday Durban! Join us as we journey through the fascinating world of technology and data insights! 🌟

Interesting Stuff - Week 28, 2023

Sun, 16 Jul 2023 07:24:26 +0200

In this week’s blog post, we dive into some exciting developments in the tech world.

First, we explore the Kusto Detective Academy, a valuable resource for mastering the Kusto Query Language (KQL) in Azure Data Explorer. Then, we discover GPT4Readability, a new tool powered by advanced language models like GPT-3.5 and GPT-4, designed to simplify the process of writing documentation and README files.

Additionally, we delve into the integration of Kafka, Debezium, and BentoML to build a machine-learning streaming pipeline and a preview of a stream processing performance report comparing Apache Flink with RisingWave.

Lastly, we get a glimpse into the upcoming Data Saturday Durban event and the intriguing topics under consideration. If you want to expand your knowledge and meet industry experts, don’t miss out on this exciting opportunity!

Interesting Stuff - Week 27, 2023

Sun, 09 Jul 2023 13:05:33 +0200

Unveil captivating discoveries: Explore vector databases, Azure OpenAI updates, generative AI insights, streaming trends, and more!

Join me on this knowledge-packed journey, filled with innovation and community contributions. Fuel your curiosity and read the full post now!

Interesting Stuff - Week 26, 2023

Sun, 02 Jul 2023 11:48:53 +0200

We’ve got everything from data pipelines to streaming platforms in this week’s blog post: Hoptimator, the tool designed to streamline and optimize data pipelines. Mobile Premier League (MPL) and how they reduced player churn using Confluent Cloud. Materialized views and streaming tables for Databricks SQL.

Finally, Data Saturday Durban on August 19. It is still time to submit a talk.

Interesting Stuff - Week 25, 2023

Mon, 26 Jun 2023 07:17:47 +0200

You who read my blog are probably used to how the “Interesting Stuff” posts are published on a Sunday. This week, however, I am a bit late. I have been busy with other things (travelling) and have not had time to write this post until now.

Anyway, some of the “stuff” I cover this week is Microsoft Fabric, some cool AI/ML and LLM (Large Language Model) things.

Finally, there is a very interesting article on the real-time streaming ecosystem by Hubert Dulay.

Interesting Stuff - Week 24, 2023

Sun, 18 Jun 2023 14:45:04 +0200

This week, I came across debugging Jupyter notebooks in VSCode. I learned about some amazing applications of generative AI for analytics, content creation, and natural language queries. I also explored how to train a machine-learning model on a Kafka stream using Python libraries.

The CfS for Data Saturday Durban 2023 is still open, and we have already received quite a few submissions. If you are interested in attending, I suggest you book your seat NOW, as there are a limited number of seats for this FREE event, and the event promises to be awesome!

If you are interested in these topics, read further, and you see the links to the articles and resources that inspired me.

Interesting Stuff - Week 23, 2023

Sat, 10 Jun 2023 14:45:43 +0200

In this week’s post, I share some of the exciting things I have come across during the week related to data and AI:

You’ll learn about the new features and capabilities of Azure Data Explorer, Azure AI, and Microsoft Fabric. You’ll also see how to combine ChatGPT and streaming data for real-time generative AI applications.

Finally, there is a CfS for Data Saturday Durban 2023. If you are passionate about data and are interested in presenting any data-related topic, this is your chance to shine!

If you’re curious about these topics, check out this blog post!

Interesting Stuff - Week 22, 2023

Sun, 04 Jun 2023 09:09:44 +0200

In this week’s blog post, I share some interesting links and insights on generative AI for analytics, Microsoft Fabric, and real-time streaming. You will see links about using natural language queries to access data in Amazon RDS using OpenAI and LangChain.

How to leverage Microsoft Fabric to simplify and enhance your data solutions with Power BI and other Azure products and how to use Kusto Query Language and Kqlmagic in Fabric notebooks.

How to compare and choose stream processing platforms for your use cases and detect and filter PII data from unstructured data streams using Confluent and machine learning.

Check out the full post for more details and examples.

Interesting Stuff - Week 20, 2023

Sun, 21 May 2023 09:35:25 +0200

In this week’s post, I will share with you some of the most interesting and relevant articles I have read this week on various topics related to data and streaming. You will find links to:

The latest research developments in reinforcement learning and how they can be applied to different domains and challenges.
The real-time streaming ecosystem and how to design and implement a streaming solution using various tools and technologies.
The performance and latency improvements of Apache Spark Structured Streaming and how they enable more use cases for real-time data processing.
The highlights of Kafka Summit 2023, the largest conference for Apache Kafka users and enthusiasts, and the new features and announcements from Confluent, the leading provider of Kafka solutions.
Data Saturday in Johannesburg in August and planning for the upcoming Data Saturday - Durban.

If you are curious about these topics and want to stay updated on the latest trends and innovations in the data and streaming world, read on!

Interesting Stuff - Week 19, 2023

Sun, 14 May 2023 11:34:50 +0200

Some very cool stuff this week. Polars for Rust! Azure Data Explorer as a Vector DB. ChatGPT and Pandas = PandasAI.; and real-time streaming ecosystem.

LangChain, Streamlit, and Pinecone for LLM. Real-time sentiment analysis with Docker, Kafka, and Spark Streaming. And more!

Interesting Stuff - Week 18, 2023

Sun, 07 May 2023 12:56:57 +0200

This week a lot of AI/ML in the roundup. Specifically Generative AI and AI self-play. Really “cool” stuff.

Of course, I have some Kafka coverage as well. This in the form of a post about Confluent Platform 7.4 and all the new “goodies” in that release.

Interesting Stuff - Week 17, 2023

Sun, 30 Apr 2023 15:58:01 +0200

This week: beating roulette, hooking up Azure Data Explorer and Kafka. Real-time machine learning pipelines.

Spark loader for Hugging Face datasets, processing data from multiple streaming platforms using Delta Live Tables and the real-time streaming ecosystem.

Interesting Stuff - Week 16, 2023

Sun, 23 Apr 2023 09:46:22 +0200

A mixed bag this week: Azure Databricks and its Delta engine. In Azure Data Explorer “land” a combined Kusto Sink Connector and Kafka Connect image targeting multi-platform. Databricks has released MLflow 2.3 with new interesting features.

Speaking of Databricks; new functions in Databricks SQL allowing integration of large language models (LLMs) into SQL queries. Oh, and let’s not forget using Generative AI to create Synthetic Data.

Cross-Platform Compatibility Made Easy with Multi-Platform Docker Images: Azure Data Explorer Sink Connector & Kafka Connect

Mon, 17 Apr 2023 06:02:48 +0200

As you may know, I am working on a blog-post series, Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer. As the title implies, Kafka ingests data into Azure Data Explorer (ADX). This is done using the Azure Data Explorer Kusto Sink connector, where the connector runs in a JVM process in Kafka Connect. All this run in Docker containers.

While writing (still writing) the fourth post in the series, I realised I had to create my own image for Kafka Connect and the Kusto Sink connector. You will find an explanation as to why below. Since I have partly gone over to the “dark side” (using Apple Silicon, ARM64), I thought it’d be nice if the image I created targeted both AMD64 and ARM64. I.e. I wanted to create a multi-platform image. In my usual style, I thought: “How hard can it be? What could possibly go wrong” 😄.

Well, it was a tad more difficult than I imagined, and in this post, I cover the steps I took to create my Kafka Connect and Kusto Sink connector image as a multi-platform Docker image. Oh, and if you are not interested in the Kusto Sink connector and Kafka Connect at all (you should be 😄), don’t worry. The steps covered here work for any Docker Image you want to create as a multi-platform image.

Interesting Stuff - Week 15, 2023

Sun, 16 Apr 2023 12:03:01 +0200

Some very exciting things this week: Generative AI may shift focus from technology-driven to product-driven development. A game analytics framework/solution to help game developers analyze players. How to use synthetic data in ML.

How to handle late-arriving events in Apache Flink. An introduction to materialized views and how they can help. Also, a plug for an upcoming post by me about how to build multi-platform Docker images.

Interesting Stuff - Week 14, 2023

Sun, 09 Apr 2023 09:34:33 +0200

This week’s exciting stuff: cool video from an Azure Data Explorer (ADX) workshop. Speaking of ADX, I published my third post in the series of building a real-time leaderboard using Kafka and ADX. On the AI/ML side, “stuff” about building a GPT-powered application and a really interesting intro to reinforcement learning.

It wouldn’t be a roundup without something about Kafka. So, a link to a post about Kafka and fraud detection. Also, a link to the second post in the series about machine learning and streaming data.

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II

Mon, 03 Apr 2023 07:26:29 +0200

Don’t be fooled by the " - II" in the title, as this is the third post in the series about using Kafka and Azure Data Explorer (ADX) to develop a real-time leaderboard for online gaming. This (" - II") is because the initial post was more of an intro, where I introduced a framework for generating and publishing events to Kafka. Posts in the series are:

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - Prelude
Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I
Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II (this post)

The last post was all about ingesting data into ADX from Kafka. In this post, you will see more about ingesting data into ADX and some “cool stuff” related to that. Specifically, what to do if you need to update/transform the data you ingest.

Interesting Stuff - Week 13, 2023

Sun, 02 Apr 2023 14:30:42 +0200

Some interesting stuff this week: a “cool” post about Polar bears vs Pandas, no doubt who’ll win. Read about how to ingest data into Azure Data Explorer - very interesting. On the streaming side really interesting about how to do Canary releases for streaming platforms. Also something extremely interesting: Machine Learning and Data Streaming! Have fun!!

Interesting Stuff - Week 12, 2023

Sun, 26 Mar 2023 09:34:31 +0200

Not so much to cover this week. What you find in this post is a couple of links to posts about building customised chatbots based on GPT-4. There is also an interesting link to how to use Kafka and FastAPI to develop a realtime ticketing booking solution.

Interesting Stuff - Week 11, 2023

Sun, 19 Mar 2023 11:09:08 +0200

Even though GPT-4 was just released I am quite proud that I am not linking to anything GPT-4 specific. ChatGPT and OpenAI, yes - but not GPT-4.

So what happened the week just gone? Some things to “wet your appetite”: learning path for Azure Data Explorer and Kusto Query Language. While we are on ADX, I finally finished the leaderboard follow-up post. Yay to me!

Some ChatGPT stuff (not GPT-4 😄), and also what impact AI will have on us as developers.

I am rounding off with the question: “Where have all the Big Data Gone?”

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I

Sun, 19 Mar 2023 07:49:50 +0200

This is the second post in a series about using Kafka and Azure Data Explorer (ADX) to develop a real-time leaderboard for online gaming. Hmm, the second post you say - why is it that the post title says " - I"? Good question; that is because the initial post was more of an intro, where I introduced a framework for generating and publishing events to Kafka. Posts in the series are:

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - Prelude
Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I (this post)
Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II

Anyway, in this post, you will learn more about ADX and why it is an excellent choice for near real-time applications, how to “hook up” Kafka to ADX and how to ingest data into an ADX database.

Interesting Stuff - Week 10, 2023

Sun, 12 Mar 2023 08:33:19 +0200

This week only streaming topics.

You see an excellent introduction to Apache Flink, followed by more Flink. In this case, it is related to Confluent’s acquisition of Immerok and how this has the potential of “shaking” up the stream processing market.

I finish off with a link to a post looking at the benefits of event streaming. Quite interesting!

Interesting Stuff - Week 9, 2023

Sun, 05 Mar 2023 08:31:23 +0200

This week it is mostly streaming related “stuff” that has caught my eye. Regardless of that, we start with some Generative AI - or rather prompt engineering, where I link to a post around different types of “frameworks” for prompt engineering. When you read the post you will see why “frameworks” are in quotes.

From that we go into streaming “land” and look at stream processing vs. real-time OLAP databases. There’s an interesting article about building data lakes on AWS using open-source technologies. I round off the streaming with a post about Spark Structured Streaming and REST APIs.

Interesting Stuff - Week 8, 2023

Sun, 26 Feb 2023 14:43:28 +0200

As opposed to last week with no Kafka related “stuff”, this week there is a fair chunk. So, in this roundup you get a link to Linux containers on Apple Silicon. A deep dive into Microsoft LMOps, Mastodon, Kafka and OpenSearch.

I appear in this roundup with a link to the first post in a series about developing Leaderboards using Kafka and Azure Data Explorer. Oh, there is also something about running Kafka on Kubernetes.

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - Prelude

Sun, 26 Feb 2023 09:40:43 +0200

This blog post series is inspired by my conference talk Analyze Billions of Rows of Data in Real-Time Using Azure Data Explorer. Posts in the series are:

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - Prelude (this post)
Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I
Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II

In the presentation, I talk about “user-facing” analytics. Historically analytics has been limited to a small set of users in an organization (think the C-suite, power users, etc.). User-facing analytics changes that and makes analytics available to the end-users. An example of “user-facing analytics” is LinkedIn’s: “Who’s viewed your profile”. Another example is leaderboards, which are popular in online gaming.

Since I work for a company ( Derivco) that develops systems for the iGaming industry, this series looks at the basics of a real-time leaderboard for online gaming using Kafka and Azure Data Explorer (ADX).

Initially, I thought one post would cover this, but I quickly realized one post covering everything would be way too long. Therefore, I am splitting this topic into multiple posts, and in this post, I look at Kafka and event generation.

Arguably, this first post has little to do with ADX, which is why the title says Prelude 😄. Still, it is important because it sets up the publishing mechanism, which can be reused for anything else you want to do.

Interesting Stuff - Week 7, 2023

Sun, 19 Feb 2023 10:02:39 +0200

Reading through this post, I realize that this is probably the first post where I have yet to mention Kafka; wow!

So this week, I link to posts about the new v2 Python programming model for Azure functions as well as “Tinder” for Azure Data Explorer, a.k.a “Find a Partner”. Then there is quite a bit about OpenAI: use OpenAI in Synapse Analytics, a link to a post by John Gruber about the new Bing, etc.

So what are you waiting for? Go ahead and read it 😄.

Interesting Stuff - Week 6, 2023

Sun, 12 Feb 2023 09:46:31 +0200

I came home from London a couple of days ago, and I can’t tell you how good it is to be back home!

Among other things, what caught my eye for this week’s roundup was that Azure now offers a managed load testing service and the open-sourcing of LMOps: a collection of tools for improving text prompts used as input to generative AI models. Also interesting is vector search and what you can do with it. Finally, if you are interested in Kafka, there is an excellent introductory guide to Kafka.

Interesting Stuff - Week 5, 2023

Sun, 05 Feb 2023 10:15:43 +0200

In this week’s roundup, you’ll read about optimized locking in Azure SQL (very cool) and why Flink is so popular. Related to Flink, we also look at operational use cases for Flink, as well as some Flink CEP “stuff”.

Interesting Stuff - Week 4, 2023

Sat, 28 Jan 2023 12:45:43 +0200

If you see this post today, you may wonder why I have published on a Saturday instead of, as usual, on a Sunday. Well, I am flying out to Barcelona a bit later today and will be on a plane for most of tomorrow.

This roundup looks at cloud OLTP databases and whether we can get them to be really scalable. There is a link to a post looking at Azure Data Explorer* together with Azure Serverless functions - very cool! An interesting post about ML in online gaming, plus a lot of other cool stuff.

Interesting Stuff - Week 3, 2023

Sun, 22 Jan 2023 12:38:31 +0200

So, what do we have for Week 3, 2023?

A mixed bag: cool visualizations in Azure Data Explorer using Plotly. Interesting about the benefits and challenges of distributed computing. Comparison of different SQL-based streaming approaches and their use cases, including a look at a streaming database. An excellent post about Time in Flink. As a side note, I wish Flink’s time functions supported a more granular precision than milliseconds.

Interesting Stuff - Week 2, 2023

Sun, 15 Jan 2023 08:43:52 +0200

This weeks roundup includes links to articles and videos about using Azure OpenAI and deploying ML models with FastAPI and Azure, as well as a post about Change Data Capture and its patterns.

I also discuss my experience with trying to run Flink SQL in Docker. In short, the Flink Docker experience could be better!

Interesting Stuff - XMas, New Year 2022, Week 1 2023

Sun, 08 Jan 2023 12:04:35 +0200

This is the “roundup” of the posts that have been most interesting to me over the Christmas, New Year period 2022, and week 1 2023. Oh, and welcome to 2023!

This post consists of a mixed bag: We look at an excellent resource for Azure Machine Learning. Of course, when we talk about machine learning, we need to mention ChatGPT. We look at the “Tinder” app for Azure Data Explorer - “Find my partner”. The BIG one the post looks at is how Confluent acquires Immerok, which offers a fully managed Flink implementation!

Revolutionizing My Blog: Why I Changed my Hugo Theme

Sat, 07 Jan 2023 12:19:12 +0200

Revolutionizing in the title of this post may be a bit strong, but I guess that is what you get when you ask ChatGPT to write you a “catchy” blog post title 😄.

Anyway, if you have followed my blog for a while (for the last 10 years or so) you know that I have been switching blog engines a couple of times:

Moving to a New Blog Engine: Moved from WordPress to OctoPress/Jekyll (2013).
Goodbye Jekyll, Welcome Hugo!: From Jekyll to Hugo (2018).

As you see from the above timeline it is around 5 years between each move. Seeing that it is now 2023 you may think I am switching blog engine again. Well, I am NOT switching engine.

Interesting Stuff - Week 50, 2022

Sun, 18 Dec 2022 13:15:45 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

NOTE: It is now coming up on Christmas and New Year, and I will take a break with these posts and come back at the beginning of next year.

Interesting Stuff - Week 49, 2022

Sun, 11 Dec 2022 11:10:56 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 48, 2022

Sun, 04 Dec 2022 09:20:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 47, 2022

Sun, 27 Nov 2022 12:04:12 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 45, 2022

Sun, 13 Nov 2022 09:52:14 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 44, 2022

Sun, 06 Nov 2022 10:04:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 43, 2022

Sun, 30 Oct 2022 09:10:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 42, 2022

Sun, 23 Oct 2022 08:38:09 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 41, 2022

Sun, 16 Oct 2022 11:50:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 40, 2022

Sun, 09 Oct 2022 09:09:35 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 39, 2022

Sun, 02 Oct 2022 09:24:03 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 38, 2022

Sun, 25 Sep 2022 08:31:25 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 37, 2022

Sun, 18 Sep 2022 13:24:26 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 36, 2022

Sun, 11 Sep 2022 08:14:11 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 35, 2022

Sun, 04 Sep 2022 11:44:07 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 34, 2022

Sun, 28 Aug 2022 10:41:25 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 33, 2022

Sun, 21 Aug 2022 12:37:51 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 32, 2022

Sun, 14 Aug 2022 09:09:45 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 31, 2022

Sun, 07 Aug 2022 09:39:14 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 30, 2022

Sun, 31 Jul 2022 08:02:01 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 29, 2022

Sun, 24 Jul 2022 11:50:11 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 28, 2022

Sun, 17 Jul 2022 12:50:50 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 27, 2022

Sun, 10 Jul 2022 09:58:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 26, 2022

Sun, 03 Jul 2022 10:03:19 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 25, 2022

Sun, 26 Jun 2022 09:33:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Solution to GIT: "unsafe repository ('some-repo' is owned by someone else)"

Sat, 25 Jun 2022 19:16:19 +0200

Every 6 - 9 months (or so), I clean up my development PC just to keep it “lean and mean”. I do it by formatting the hard-drive partition the OS (in this case, Windows) is on, followed by a new install. Recently I had a four-day weekend here in SA. Four glorious days off, a perfect time to “nuke” my PC and re-install!

Interesting Stuff - Week 24, 2022

Sun, 19 Jun 2022 13:40:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 23, 2022

Sun, 12 Jun 2022 15:27:08 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 22, 2022

Sun, 05 Jun 2022 09:58:38 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 21, 2022

Sun, 29 May 2022 13:13:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 20, 2022

Sun, 22 May 2022 10:40:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 19, 2022

Sun, 15 May 2022 09:34:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 18, 2022

Sun, 08 May 2022 09:46:54 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 17, 2022

Sun, 01 May 2022 08:35:51 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 16, 2022

Sun, 24 Apr 2022 08:44:24 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 15, 2022

Sun, 17 Apr 2022 16:52:11 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 14, 2022

Sun, 10 Apr 2022 12:49:21 +0100

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 13, 2022

Sun, 03 Apr 2022 09:05:36 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that have to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 12, 2022

Sun, 27 Mar 2022 08:32:00 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 11, 2022

Sun, 20 Mar 2022 08:30:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending. This week, there hasn’t been that much that has caught my eye, but here it is.

Interesting Stuff - Week 10, 2022

Sat, 12 Mar 2022 08:10:36 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 9, 2022

Sat, 05 Mar 2022 07:57:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 8, 2022

Sun, 27 Feb 2022 09:10:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 7, 2022

Sun, 20 Feb 2022 13:26:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 6, 2022

Sun, 13 Feb 2022 17:27:47 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 5, 2022

Sun, 06 Feb 2022 12:51:07 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 4, 2022

Sun, 30 Jan 2022 12:46:36 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 3, 2022

Sun, 23 Jan 2022 08:40:56 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 2, 2022

Sun, 16 Jan 2022 08:56:07 +0200

Photo by Stephen Dawson on Unsplash

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

How to Stream Data to Event Hubs from Databases Using Kafka Connect & Debezium in Docker - II

Fri, 14 Jan 2022 05:25:03 +0200

In a two-post series, this second post looks at streaming data from a database to Azure Event Hubs using Kafka Connect and Debezium, where Kafka Connect and Debezium run in Docker.

The first post:

How to Stream Data to Event Hubs from Databases Using Kafka Connect & Debezium in Docker - I. This post mainly looks at the configuration of Kafka Connect’s docker-compose.yml file to allow us to connect to Event Hubs.

This series came about as I in the post How to Use Kafka Client with Azure Event Hubs, somewhat foolishly said:

An interesting point here is that it is not only your Kafka applications that can publish to Event Hubs but any application that uses Kafka Client 1.0+, like Kafka Connect connectors!

I wrote the above without testing it myself, so when I was called out on it, I started researching (read “Googling”) to see if it was possible. The result of the “Googling” didn’t give a 100% answer, so I decided to try it out, and this series is the result.

In the first post, - as mentioned - we configured Kafka Connect to connect into Event Hubs. In this post, we look at configuring the Debezium connector.

How to Stream Data to Event Hubs from Databases Using Kafka Connect & Debezium in Docker - I

Mon, 10 Jan 2022 19:05:33 +0200

This post is the first of two looking at if and how we can stream data to Event Hubs from Debezium. Initially I had planned only one post covering this, but it turned out that the post would be too long, so therefore I split it in two.

The second post in the series is:

How to Stream Data to Event Hubs from Databases Using Kafka Connect & Debezium in Docker - II. Here we look at the Debezium connector configuration needed if we want to stream data to Event Hubs.

It started with the post, How to Use Kafka Client with Azure Event Hubs. In that post, I looked at how the Kafka client can publish messages to - not only - Apache Kafka but also Azure Event Hubs. In the post, I said something like:

An interesting point here is that it is not only your Kafka applications that can publish to Event Hubs but any application that uses Kafka Client 1.0+, like Kafka Connect connectors!

Obviously, for you who know me, I said that without having tested it properly, but: how hard can it be? What could possibly go wrong?. Well, I was called upon it by a guy who had read the post. He told me he had tried what I said at one time or the other, and it hadn’t worked.

In this post (the first), we look at configuring Kafka Connect to connect to Event Hubs.

Interesting Stuff - Week 1, 2022

Sun, 09 Jan 2022 10:55:27 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Christmas, New Year 2021

Sun, 02 Jan 2022 12:36:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that have been most interesting to me over the Christmas and New Year period 2021. Oh, and welcome to 2022! I wonder what this year will have in store for us - after the last two years I will not even think about what will happen!

How to Use Kafka Client with Azure Event Hubs

Sun, 02 Jan 2022 09:48:17 +0200

This blog post came by, by accident, lol. A couple of weeks ago, I started to prepare for a webinar: Analyze Billions of Rows of Data in Real-Time Using Azure Data Explorer. One of the demos in that webinar is about ingesting data from Apache Kafka into Azure Data Explorer. When prepping, I noticed that my Confluent Cloud Kafka cluster didn’t exist anymore, so I had to come up with a workaround. That workaround was to use Azure Event Hubs instead of Kafka.

Since I already had the code to publish data to Kafka and knew that you could use the Kafka client to publish to Event Hubs, I thought I’d test it out. I did run into some minor snags along the way, so I thought I’d write a blog post about it. Then, at least, I have something to go back to. This post also looks at how to set up an Event Hubs cluster.

Interesting Stuff - Week 50, 2021

Sun, 12 Dec 2021 10:37:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

NOTE: It is now coming up on Christmas and New Year, and I will take a break with these posts and come back at the beginning of next year.

Interesting Stuff - Week 49, 2021

Sun, 05 Dec 2021 08:51:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 48, 2021

Sun, 28 Nov 2021 08:38:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 47, 2021

Sun, 21 Nov 2021 09:27:19 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 46, 2021

Sun, 14 Nov 2021 08:50:09 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 45, 2021

Sun, 07 Nov 2021 08:13:38 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 44, 2021

Sun, 31 Oct 2021 08:53:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

How to Ingest Into Azure Data Explorer From Apache Kafka using Kafka Connect

Wed, 27 Oct 2021 14:13:37 +0200

If you follow my blog, you probably know that I am a huge fan of Apache Kafka and event streaming/stream processing. Recently Azure Data Explorer (ADX) has caught my eye. In fact, in the last few weeks, I did two conference sessions about ADX. A month ago, I published a blog post related to Kafka and ADX: Run Self-Managed Kusto Kafka Connector Serverless in Azure Container Instances.

As the title of that post implies, it looked at the ADX Kafka sink connector and how to run it in Azure. What the post did not look at was how to configure the connector and connect it to ADX. That is what we will do in this post (and maybe in a couple of more posts).

Interesting Stuff - Week 43, 2021

Sun, 24 Oct 2021 12:07:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 42, 2021

Sun, 17 Oct 2021 08:52:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 41, 2021

Sun, 10 Oct 2021 10:15:26 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 40, 2021

Sun, 03 Oct 2021 09:34:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 39, 2021

Sun, 26 Sep 2021 10:49:18 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 38, 2021

Sun, 19 Sep 2021 10:37:19 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 37, 2021

Sun, 12 Sep 2021 09:43:08 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Run Self-Managed Kusto Kafka Connector Serverless in Azure Container Instances

Mon, 06 Sep 2021 06:11:51 +0200

A couple of weeks ago, I posted how I set up Kafka to run serverless in Azure by deploying Confluent Cloud.

If you have followed my blog lately, you have probably seen that I am interested in Azure Data Explorer and that I have a couple of conference talks coming. One being:

How to do Real-Time Analytics Using Apache Kafka and Azure Data Explorer: We are looking at how to stream events from Apache Kafka to Azure Data Explorer (ADX) and perform user-facing analytics in near real-time.

The question is how to connect Kafka with ADX? You normally connect Kafka with another system using a Kafka Connect connector, and fortunately a connector exists for connecting Kafka with ADX: the Kafka Connect Kusto Sink Connector.

However, since I am running managed Kafka (Confluent Cloud, remember), I need a managed connector to run it in Confluent Cloud’s Kafka Connect. In the previous paragraph, I mentioned I was fortunate as we had a Kafka connector for ADX. Unfortunately, it is not a managed connector, so I cannot run it in Confluent Cloud - bummer!

So, this post looks at the various options we have if we want to use the Kafka Connect Kusto Sink Connector connecting Confluent Cloud in Azure with Azure Data Explorer. However, if you are not interested in neither Kafka nor ADX, the post may still be of use for you. The reason being it also covers running Docker images in Azure Container Instances (ACI).

Interesting Stuff - Week 36, 2021

Sun, 05 Sep 2021 10:38:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 35, 2021

Sun, 29 Aug 2021 13:37:54 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 34, 2021

Sun, 22 Aug 2021 08:35:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 33, 2021

Sun, 15 Aug 2021 09:27:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Run Confluent Cloud & Serverless Apache Kafka on Azure

Sat, 14 Aug 2021 12:58:24 +0200

For readers of my blog, it should not come as a surprise that I think Apache Kafka is “the greatest thing since sliced bread”, and I have written some posts about it. The posts I have written has been geared towards the setup/deployment of Kafka, with a Windows/.NET slant to it. This post is in the same vein; how to set up Kafka but in the cloud.

The “conference season” is upon us, and I have some conference talks coming up. This year quite a lot of what I am talking about is in the cloud, specifically Azure. As some of the talks involve Apache Kafka, I thought it would be good if Kafka also ran in the cloud. Sure, I could always run Kafka on Docker on a VM in the cloud, but “they” say serverless is the new “in” thing, as well as managed services, so why not try that out.

So, this post is about how to deploy Confluent Cloud on Azure!

Interesting Stuff - Week 32, 2021

Sun, 08 Aug 2021 09:09:24 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

How to Deploy the Debezium SQL Server Connector to Docker

Sat, 07 Aug 2021 06:02:12 +0200

I have been doing a couple of conference talks lately (virtual, of course) about streaming data from SQL Server to Kafka. The title of the presentation is Free Your SQL Server Data With Kafka.

In the presentation, I talk (and show) various ways of getting data from SQL Server to Kafka. One of the ways I cover is Microsoft CDC, together with Debezium.

When I do the presentation, I always have a SQL Server installed locally, and I run Kafka in Docker. Without fail, every time I set up the environment, I cannot remember how to deploy the Debezium SQL Server Connector into Docker. Therefore I decided to write this post to have something to go back to for next time.

Interesting Stuff - Week 31, 2021

Sun, 01 Aug 2021 07:40:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 30, 2021

Sun, 25 Jul 2021 09:21:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 29, 2021

Sun, 18 Jul 2021 14:05:36 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Speaking about the week just ending - wow, what a week! As you may know or not, I live in Durban, South Africa. Durban is in the province of KZN (KwaZulu-Natal), home to Jacob Zuma, ex-president of SA. On July 8, Jacob Zuma was put in jail after being found guilty of contempt of court. The jailing of Jacob Zuma led to protests that initially were peaceful. However, this changed Sunday, July 11, when small scale rioting broke out in and around Durban. On Monday (July 12), it escalated to full-scale rioting, looting, and factories burning.

The police were not prepared for the scale of the riots and could not offer any form of protection and help, so the local communities took it upon themselves to protect the local areas. Roadblocks were put up to prevent rioters from entering the neighbourhoods. The roadblocks were manned 24/7 by people from the community, and we did nighttime patrols to show presence and act as a deterrent. So I have done night patrols from last Monday until Friday night.

What we did seem to have helped, and fortunately we had no issues in the area I live in. I must say however, that it is a bit frightening when you are out in the middle of the night, and you hear gunshots some hundred meters away!

That’s enough of my “adventures”, let’s get back to what this post is all about.

Interesting Stuff - Week 28, 2021

Sun, 11 Jul 2021 09:07:39 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 27, 2021

Sun, 04 Jul 2021 09:04:04 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 26, 2021

Sun, 27 Jun 2021 09:21:01 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 25, 2021

Sun, 20 Jun 2021 07:52:57 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 24, 2021

Sun, 13 Jun 2021 10:39:45 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 23, 2021

Sun, 06 Jun 2021 08:28:23 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 22, 2021

Sun, 30 May 2021 06:33:15 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 21, 2021

Sun, 23 May 2021 08:34:42 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 20, 2021

Sun, 16 May 2021 08:36:10 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 19, 2021

Sun, 09 May 2021 07:07:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 18, 2021

Sun, 02 May 2021 07:55:53 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 17, 2021

Sun, 25 Apr 2021 08:21:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 16, 2021

Sun, 18 Apr 2021 11:52:42 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 15, 2021

Sun, 11 Apr 2021 10:05:00 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 14, 2021

Sun, 04 Apr 2021 12:11:33 +0200

Throughout the week, I read a lot of blog posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 13, 2021

Sun, 28 Mar 2021 09:41:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 12, 2021

Sun, 21 Mar 2021 09:15:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 11, 2021

Sun, 14 Mar 2021 07:26:59 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 10, 2021

Sun, 07 Mar 2021 06:44:31 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 9, 2021

Sun, 28 Feb 2021 09:46:34 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI/data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

Interesting Stuff - Week 8, 2021

Sun, 21 Feb 2021 08:56:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

AI / data science
data in general
data architecture
streaming
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.

How To Install SQL Server on an Azure VM

Sun, 14 Feb 2021 09:17:27 +0200

A while ago, I wanted to do a quick test on a new SQL installation, and I wanted the SQL installation to be on a “pristine” server. I was not keen on creating a new virtual machine on my local dev-box, as for that I would need to create a VM image etc., and it seemed like too much hassle for a lazy person like me. The obvious choice then is to do it in the cloud. How hard can that be, what could possibly go wrong?!

It turned out to not be as straight-forward as I thought it would be, but eventually, I managed to get it right. Since I probably need to do it again some time, I thought I’d write a post about it, so I have something to go back to. So here we go …

Interesting Stuff - Week 7, 2021

Sun, 14 Feb 2021 07:28:01 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 6, 2021

Sun, 07 Feb 2021 13:05:26 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 5, 2021

Sun, 31 Jan 2021 12:37:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 4, 2021

Sun, 24 Jan 2021 09:42:57 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 External Libraries and Your Python Runtime

Sun, 24 Jan 2021 06:06:25 +0200

The last month or so I have written some blog posts about how Microsoft open-sourced the SQL Server language extensions for R and Python back in September 2020. These language extensions add to the Java extension which was open-sourced in March 2020. My posts have been about bringing your own Python runtime into SQL Server 2019, and the potential pitfalls you may encounter:

I have touched upon the subject of the PYTHONHOME environment variable in the posts, and I have said that it is not needed - UNLESS you want to create Python external libraries.

In this post, we look more at PYTHONHOME and creating external libraries.

Interesting Stuff - Week 3, 2021

Sun, 17 Jan 2021 07:29:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending. Admittedly not much this week but here goes:

Interesting Stuff - Week 2, 2021

Sun, 10 Jan 2021 08:22:46 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Solve Python Issues in SQL Server Machine Learning Services After Deploying Python 3.9

Sat, 09 Jan 2021 11:38:09 +0200

In September 2020, Microsoft open-sourced SQL Server Machine Learning Services, (SQLMLS), language extensions for R and Python. If you want more information, here are some blog posts I have written about it:

In the last post, which looks at using Python 3.9 in SQL Server Machine Learning Services, I wrote this at the very end:

It looks like all is good, but maybe not? In a future post we’ll look at an issue we have introduced - but for now, let us bask in the glory of having created a new Python language extension.

In the post, we wrote a new language extension to handle Python 3.9, and that just worked fine. However, when I was doing some other things, I noticed some side effects, and in this post, we look at those side effects and how to solve them.

NOTE: The two posts mentioned above has been updated as a result of this post. If you now follow the posts above you may not see the errors we discuss here.

UPDATED: Write a Python 3.9 Language Extension for SQL Server Machine Learning Services

Tue, 05 Jan 2021 04:57:43 +0200

NOTE: This post has been updated after it was originally published. I have edited information about the PYTHONHOME system variable, as well as PATH requirements.

In my post, Bring Your Own R & Python Runtimes to SQL Server Extensibility Framework I wrote about how we can use other R and Python runtimes in SQL Server Machine Learning Services than the ones that come “out of the box”. In the post, I wrote that if you want to bring a Python runtime other than version 3.7.x, (like 3.8, 3.9, etc.), you need to build your own Python language extension, and we’d look at it in a future post.

NOTE: For R and Java the existing language extension can be used regardless of R/Java version, (at least as far as I know).

When I wrote we’d look at it in a future post I thought to myself; “how hard can it be?”. I had read the steps of how to build a Python language extension for Windows here, and it didn’t seem that hard: some Boost, CMake, compile, and Bob’s your uncle! Well, it turned out it was somewhat more complicated than what I anticipated. So, if you are interested - read on!

Build Boost.Python & Numpy in Windows

Sun, 03 Jan 2021 13:50:55 +0200

In this post, we look at how to build Boost.Python and Numpy. We look at it from a perspective where we want to use what we build as part of a bridge between SQL Server 2019 and Python. However, if you are not interested in SQL, the post should still give you some - hopefully - useful information.

Please note that I am a SQL dude, and my knowledge of Boost, Python and Numpy is limited at best. So take this post for what it is; the steps I took to successfully build Boost.Python and Numpy on a Windows box.

Interesting Stuff - Christmas, New Year, Week 1, 2021

Sun, 03 Jan 2021 07:20:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that have been most interesting to me over the Christmas and New Year period 2020, and the first week of 2021.

UPDATED: Bring Your Own R & Python Runtimes to SQL Server Extensibility Framework

Tue, 29 Dec 2020 12:47:45 +0200

NOTE: This post has been updated after it was originally published. I have edited information about the PYTHONHOME system variable, as well as PATH requirements.

Back in the day I wrote quite a few blog posts about SQL Server Machine Learning Services as well as SQL Server Extensibility Framework, and External Languages. SQL Server Machine Learning Services is very cool, but a complaint has been that you are restricted in what versions of R and Python you use.

In September 2020, Microsoft announced that they have open-sourced the technology behind SQL Server Extensibility Framework. As a result, we can now bring our own versions of R and Python to SQL Server 2019.

NOTE: To bring your own R/Python you need SQL Server 2019 CU3 or above.

In this blog post, we look at how to do that.

A Lap Around SQL Server 2019 Big Data Cluster: Architecture

Mon, 21 Dec 2020 09:18:00 +0200

This post is the second in series about SQL Server 2019 Big Data Cluster based on a presentation I do: A Lap Around SQL Server 2019 Big Data Cluster.

In the first post A Lap Around SQL Server 2019 Big Data Cluster: Background & Technology we looked at - as the title implies - the background of SQL Server 2019 Big Data Cluster, (BDC), and the technology behind it.

In this post, we look at the architecture and components of a BDC.

Interesting Stuff - Week 50, 2020

Sun, 13 Dec 2020 09:49:39 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

NOTE: It is now coming up on Christmas and New Year, and I will take a break with these posts and come back in the beginning of next year.

Interesting Stuff - Week 49, 2020

Sun, 06 Dec 2020 09:06:09 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 48, 2020

Sun, 29 Nov 2020 09:01:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 47, 2020

Sun, 22 Nov 2020 08:52:30 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 46, 2020

Sun, 15 Nov 2020 09:31:21 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 45, 2020

Sun, 08 Nov 2020 10:32:09 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 44, 2020

Sun, 01 Nov 2020 11:01:29 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 43, 2020

Sun, 25 Oct 2020 09:35:38 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 42, 2020

Sun, 18 Oct 2020 08:55:21 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
Kafka, Kafka, and Kafka
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 41, 2020

Sun, 11 Oct 2020 08:57:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 40, 2020

Sun, 04 Oct 2020 15:40:24 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 39, 2020

Sun, 27 Sep 2020 09:41:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 38, 2020

Sun, 20 Sep 2020 08:19:16 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 37, 2020

Sun, 13 Sep 2020 09:38:45 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 36, 2020

Sun, 06 Sep 2020 08:40:13 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 35, 2020

Sun, 30 Aug 2020 09:45:18 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 34, 2020

Sun, 23 Aug 2020 11:47:46 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 33, 2020

Sun, 16 Aug 2020 09:54:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 32, 2020

Sun, 09 Aug 2020 07:36:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 31, 2020

Sun, 02 Aug 2020 14:17:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 30, 2020

Sun, 26 Jul 2020 13:01:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 29, 2020

Sun, 19 Jul 2020 11:34:32 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 28, 2020

Sun, 12 Jul 2020 07:13:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 27, 2020

Sun, 05 Jul 2020 09:32:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 26, 2020

Sun, 28 Jun 2020 10:04:18 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 25, 2020

Sun, 21 Jun 2020 09:18:32 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 24, 2020

Sun, 14 Jun 2020 09:27:04 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 23, 2020

Sun, 07 Jun 2020 09:57:23 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 22, 2020

Sun, 31 May 2020 07:20:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 21, 2020

Sun, 24 May 2020 06:51:36 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 20, 2020

Sun, 17 May 2020 08:53:59 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 19, 2020

Sun, 10 May 2020 10:16:08 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 18, 2020

Sun, 03 May 2020 11:50:01 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 17, 2020

Sun, 26 Apr 2020 09:53:33 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

A Lap Around SQL Server 2019 Big Data Cluster: Background & Technology

Sun, 26 Apr 2020 06:49:39 +0200

You who read my blog have probably noticed some posts around SQL Server 2019 Big Data Cluster, (BDC). You have also most likely seen me posting about giving conference talks and webinars around BDC.

One of the presentations I do around BDC is named “A Lap Around SQL Server 2019 Big Data Cluster”, and I thought it would be a good idea to make a blog post around the presentation. Well, as per usual for me, “a blog post” becomes multiple posts.

So, in this post, we look at the background to SQL Server 2019 Big Data Cluster, the technology behind it, and a little bit around monitoring the cluster. In future posts, we look at architecture, data virtualization and other interesting bits and pieces.

Interesting Stuff - Week 16, 2020

Sun, 19 Apr 2020 09:59:40 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 15, 2020

Sun, 12 Apr 2020 09:27:47 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 14, 2020

Sun, 05 Apr 2020 08:25:49 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 13, 2020

Sun, 29 Mar 2020 09:58:17 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 12, 2020

Sun, 22 Mar 2020 09:38:19 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 11, 2020

Sun, 15 Mar 2020 09:37:09 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 10, 2020

Sun, 08 Mar 2020 12:44:22 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 9, 2020

Sun, 01 Mar 2020 07:53:45 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 8, 2020

Sun, 23 Feb 2020 06:46:33 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 7, 2020

Sun, 16 Feb 2020 07:30:30 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 6, 2020

Sun, 09 Feb 2020 06:06:11 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 5, 2020

Sun, 02 Feb 2020 09:48:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 4, 2020

Sun, 26 Jan 2020 07:55:14 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 3, 2020

Sun, 19 Jan 2020 08:44:17 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 2, 2020

Sun, 12 Jan 2020 08:04:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Christmas, New Year, Week 1, 2020

Sun, 05 Jan 2020 09:24:49 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that have been most interesting to me over the Christmas and New Year period 2019, and the first week of 2020.

More PolyBase Issues in SQL Server 2019 Developers Edition

Mon, 30 Dec 2019 18:26:55 +0200

A couple of weeks ago, I published the post Fix PolyBase in SQL Server 2019 Developers Edition. In the post, I discussed how to fix an issue in SQL Server 2019 Developers edition, where the PolyBase services do not start up after installation.

The fix in the post works fine, i.e. the services start after the fix. However some days after I published the post there was a question on a SQL Server 2019 forum, where the poster says he cannot create an external file format on the SQL Server 2019 Developer Edition. Sure enough, when I tried to do the same on my “fixed” instance, I could not either.

So in this post, I look at how to fix that second issue.

How to Deploy SQL Server 2019 Big Data Cluster Using Azure Data Studio

Mon, 23 Dec 2019 19:35:36 +0200

For you who follows my sporadic posts, you may wonder why I have yet another post, (YAP), covering how to deploy a SQL Server 2019 Big Data Cluster, (BDC), using Azure Data Studio, (ADS).

The answer to that is that the version of BDC I deployed in this post was a pre-release, and since then BDC has gone GA, (General Availability), and there are certain differences in deployment process between RC1, and RTM.

So in this post let us look at how to deploy BDC RTM to Azure Kubernetes Service using ADS.

Interesting Stuff - Week 50 & Year End, 2019

Sun, 15 Dec 2019 07:21:12 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

NOTE: It is now coming up on Christmas and New Year, and I will take a break with these posts and come back in the beginning of next year.

Interesting Stuff - Week 49, 2019

Sun, 08 Dec 2019 08:02:31 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 48, 2019

Sun, 01 Dec 2019 08:03:26 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 47, 2019

Sun, 24 Nov 2019 08:37:03 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Fix PolyBase in SQL Server 2019 Developers Edition

Wed, 20 Nov 2019 05:52:17 +0200

At MS Ignite in Orlando November 4 - 8, 2019, Microsoft announced the general availability of SQL Server 2019. At the same time, the SQL Server 2019 Developers Edition appeared as an MSDN download, and of course, I downloaded it and installed it on my dev box.

After the installation, I noticed that PolyBase did not start up correctly, and I saw dump files all over the place. After some investigation, I figured out what the issue was, and this blog post describes the fix.

Interesting Stuff - Week 46, 2019

Sun, 17 Nov 2019 07:47:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 45, 2019

Sun, 10 Nov 2019 09:26:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 44, 2019

Sun, 03 Nov 2019 16:45:41 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 & Java: Parameters

Sun, 03 Nov 2019 11:07:23 +0200

Microsoft introduced the ability to call Java code from SQL Server in around the SQL Server 2019 CTP 2.0 release, and I have written some posts posts about this:

SQL Server 2019 Extensibility Framework & Java.

Throughout the CTP releases some functionality has changed, and in CTP 2.5 Microsoft introduced the Java Language Extension SDK. The Java code we write needs now to include the Java SDK and extend an abstract base from the SDK.

With the introduction of the Java Language SDK, came some changes to how you handle method parameters, and I touched upon it briefly in the Java & SQL Server 2019 Extensibility Framework: The Sequel post. In this post, I want to look more in detail at parameters and how to handle them.

Interesting Stuff - Week 43, 2019

Sun, 27 Oct 2019 09:15:54 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 & Java Null Handling: Take Two

Sat, 26 Oct 2019 05:32:54 +0200

You who read my blog know that during the last year, (or so), I have been writing about SQL Server 2019 and the ability to call into Java code from SQL Server:

SQL Server 2019 Extensibility Framework & Java.

It has been a fascinating “journey”, since SQL Server 2019 is still in preview, and there have been changes in how you call Java code along the way. In this post, we look at some relatively recent changes to how we handle null values in datasets.

Interesting Stuff - Week 42, 2019

Sun, 20 Oct 2019 08:40:13 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 41, 2019

Sun, 13 Oct 2019 16:09:41 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 40, 2019

Sun, 06 Oct 2019 08:09:53 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 39, 2019

Sun, 29 Sep 2019 13:17:50 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 38, 2019

Sun, 22 Sep 2019 17:01:04 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 37, 2019

Sun, 15 Sep 2019 16:25:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Install SQL Server 2019 Big Data Cluster using Azure Data Studio

Wed, 11 Sep 2019 06:12:12 +0200

I wrote a blog post back in November 2018, about how to install and deploy SQL Server 2019 Big Data Cluster on Azure Kubernetes Service. Back then SQL Server 2019 Big Data Cluster was in private preview, (CTP 2.1 I believe), and you had to sign up, to get access to the “bits”. Well, you did not really get any “bits”; what you did get was access to Python deployment scripts.

Now, September 2019, the BDC is in public preview (you do not have to sign up), and it has reached Release Candidate (RC) status, RC 1. The install method has changed, or rather, in addition to installing via deployment scripts, you can now also install using Azure Data Studio deployment notebooks, and that is what this blog post is about.

I install it on Azure, and since I am in South Africa, I use one of the South African hosting locations.

Interesting Stuff - Week 36, 2019

Sun, 08 Sep 2019 20:23:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 35, 2019

Sun, 01 Sep 2019 19:40:31 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 34, 2019

Sun, 25 Aug 2019 19:51:11 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 33, 2019

Sun, 18 Aug 2019 16:08:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 32, 2019

Sun, 11 Aug 2019 16:07:14 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 31, 2019

Sun, 04 Aug 2019 11:15:18 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 CTP3.2 & Java

Sat, 03 Aug 2019 06:55:33 +0200

It seems that for each new SQL Server 2019 CTP release, there are changes to the Java extensions, and CTP 3.2 is no exception. Well, that is not exactly true as in CTP 3.2 release the changes are not about the extension and how we write code, but Java itself.

One of the announcements of what is new in CTP 3.2 was that SQL Server now includes Azul System’s Zulu Embedded right out of the box for all scenarios where we use Java in SQL Server, including Java extensibility.

So, in this post, we look at the impact, (if any), this has to how we use the Java extensibility framework in SQL Server 2019.

Interesting Stuff - Week 30, 2019

Sun, 28 Jul 2019 09:40:56 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 29, 2019

Sun, 21 Jul 2019 06:16:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Set Your SQL Server Data Free with Kafka: Extensibility Framework

Tue, 16 Jul 2019 05:10:24 +0200

As many of you may know, (or not), is that my background is SQL Server. Ever since I started programming, SQL Server has been my “trusty companion”, and my belief is that if you don’t have SQL Server as a backend, then there is something wrong. At work, ( Derivco), it is the same thing, and we are jokingly saying that we do not have business logic in the database, we have full-blown applications!

However, both me personally and at work, we do realise the value of streaming data; for real-time processing as well as to distribute data without having to rely on replication. In the ideal world, we would change the applications/systems that are the source of the data to both publish the data as event streams as well as persisting the data to the database. However, it may not be possible to change those applications/systems - at least not in the time frame we would like. So what we want to do is to use the database as the source of the data, but treat the data, not as rows in a database but, as streaming events.

This is the first post in a “mini” series where we look at how we can do what is outlined above. In this post, we look at how to use the SQL Server Extensibility Framework, and more specifically the Java language extension to solve the issue.

Interesting Stuff - Week 28, 2019

Sun, 14 Jul 2019 19:32:15 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 27, 2019

Sun, 07 Jul 2019 07:32:53 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 26, 2019

Sun, 30 Jun 2019 08:40:49 +0200

I cannot believe we have reached half year mark already - week 26!

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 25, 2019

Sun, 23 Jun 2019 16:19:22 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Confluent Platform & Kafka for a .NET Developer on Windows

Tue, 18 Jun 2019 04:49:36 +0200

As many of you know, I am an un-ashamed .NET developer on the Windows platform. Seeing that I do a lot of SQL Server development my development and OS platforms of choice is sufficient for my work, as I on my development box (Windows of course) install SQL Server Developer edition and whatever .NET framework I use.

That has been sufficient up until now when I want to develop against Kafka. At Derivco we are getting more and more interested in Kafka, and obviously, I want to install it so I can ~~play around~~ investigate it. However, to install it on my development machine as I would typically do with technologies I am interested in, SQL Server, RabbitMQ, etc., is difficult, if not impossible.

NOTE: Yes I know, there are articles on the web discussing how to run Kafka on Windows, but it is a hit and miss whether it works.

A while ago I wrote a post about how to run Kafka under Windows Subsystem for Linux (WSL), and yes it works, but I have had issues with it, and to me, it is still a hack. So the options then (if we rule out WSL) are:

Running it on a Linux in a virtualized environment, (Virtual Box, VMWare, Hyper-V, etc.).
Docker.

The rest of this post goes through how to set up the Confluent Platform in a Docker environment and use it from .NET.

Interesting Stuff - Week 24, 2019

Sun, 16 Jun 2019 11:12:38 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 23, 2019

Sun, 09 Jun 2019 06:38:03 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 Extensibility Framework & External Languages

Thu, 06 Jun 2019 05:37:52 +0200

A little while ago I wrote a blog post, Java & SQL Server 2019 Extensibility Framework: The Sequel, about changes in SQL Server 2019 CTP 2.5 impacting how we write Java code for use from SQL Server. While I wrote that post, Microsoft released SQL Server 2019 CTP 3.0, and, (surprise, surprise), that release contains more changes affecting Java code in SQL Server.

This post covers those changes as well as discusses what SQL Server Extensibility Framework and Language Extensions are.

Interesting Stuff - Week 22, 2019

Mon, 03 Jun 2019 18:29:57 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 21, 2019

Sun, 26 May 2019 10:02:45 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Java & SQL Server 2019 Extensibility Framework: The Sequel

Sun, 26 May 2019 07:20:09 +0200

As you may know, a while back I wrote some posts about the support for Java in SQL Server 2019: SQL Server 2019 Extensibility Framework & Java. The posts covered in some detail how Java in SQL Server worked, and how to write Java code for use in SQL Server. However, a week or two ago “the sky came tumbling down” when Microsoft released SQL Server 2019 CTP 2.5.

NOTE: CTP stands for Community Technology Preview and is like a beta release.

What Microsoft did in CTP 2.5 was to introduce Java Language Extension SDK, and your Java code now needs to inherit an abstract base class from the SDK. This requirement makes a large part of my previous posts “null and void”, so in this post, we look at what to do going forward.

Interesting Stuff - Week 20, 2019

Sun, 19 May 2019 16:28:12 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 19, 2019

Sat, 11 May 2019 06:39:50 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 18, 2019

Sun, 05 May 2019 07:13:49 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 17, 2019

Sun, 28 Apr 2019 08:12:38 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 16, 2019

Sun, 21 Apr 2019 16:36:40 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Installing R Packages in SQL Server Machine Learning Services - IV: Permissions

Sun, 21 Apr 2019 11:15:19 +0200

This post is the fourth in a series about installing R packages in SQL Server Machine Learning Services (SQL Server ML Services). To see all posts in the series go to Install R Packages in SQL Server ML Services Series.

Why this series came about is a colleague of mine Dane pinged me and asked if I had any advice as he had issues installing an R package into one of their SQL Server instances. I tried to help him and then thought it would make a good topic for a blog post. Of course, at that time I didn’t think it would be more posts than one, but here we are.

In this post, we look at:

What permissions CREATE EXTERNAL LIBRARY requires.
The ability to create external libraries with different owners and what impact it has.

Interesting Stuff - Week 15, 2019

Sun, 14 Apr 2019 18:24:53 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Installing R Packages in SQL Server Machine Learning Services - III

Wed, 10 Apr 2019 06:36:16 +0200

This post is the third in a series about installing R packages in SQL Server Machine Learning Services (SQL Server ML Services). To see all posts in the series go to Install R Packages in SQL Server ML Services Series.

In this post, we look at how we can use T-SQL and DDL commands to install packages in a remote SQL Server.

Interesting Stuff - Week 14, 2019

Sun, 07 Apr 2019 19:33:23 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 13, 2019

Sun, 31 Mar 2019 08:43:04 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 12, 2019

Sun, 24 Mar 2019 08:37:01 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 11, 2019

Sun, 17 Mar 2019 21:16:22 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019, Java & External Libraries - II

Sun, 17 Mar 2019 17:13:45 +0200

This post is part of the SQL Server 2019 Extensibility Framework & Java series of posts, and it is the second post discussing SQL Server 2019, Java and the creation and use of external libraries.

In the previous post about external libraries, we said that they were beneficial as they reduced complexities when deploying code, but there were still some caveats. So, in this post, we look at how to overcome those caveats

Interesting Stuff - Week 10, 2019

Sun, 10 Mar 2019 13:14:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019, Java & External Libraries - I

Sun, 10 Mar 2019 10:22:51 +0200

A couple of months ago I wrote a series of posts about one of the new features in SQL Server 2019; the ability to call out to Java code from inside SQL Server.

To see the posts, go to SQL Server 2019 Extensibility Framework & Java.

In the posts, we discussed how the Java extension differs from R and Python in that R and Python are an integrated part of the SQL Server install (when enabling in-database analytics), but Java is not. In other words, the use of the Java extension requires Java to be installed beforehand, and this then has implications on permissions. We also discussed how Java is a compiled language, and we execute against a method in a class, whereas with R and Python we send a script to the external engine. The consequence of this is that when we execute Java code, we need to indicate where the compiled code resides, and those locations need specific permissions.

All this creates a level of complexity, and it would potentially be easier if we load the Java code from a well-known place, where we do not need to worry about permissions and so forth.

This post is the first of a couple where we see how new functionality in SQL Server 2019 CTP 2.3 can help.

Interesting Stuff - Week 9, 2019

Sun, 03 Mar 2019 10:54:21 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 8, 2019

Sun, 24 Feb 2019 10:08:50 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 7, 2019

Sun, 17 Feb 2019 08:20:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 6, 2019

Sun, 10 Feb 2019 08:37:39 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 5, 2019

Sun, 03 Feb 2019 09:15:51 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 4, 2019

Sun, 27 Jan 2019 06:36:21 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 3, 2019

Sun, 20 Jan 2019 18:00:59 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 & Java with Visual Studio Code

Thu, 17 Jan 2019 06:16:42 +0200

As you know, SQL Server 2019 introduces the Java language extensions as part of the SQL Server Extensibility Framework. The Java language extensions make it possible to execute Java code from inside SQL Server, the same way we can run R/Python code. Seeing that I am “somewhat” interested in the SQL Server Extensibility Framework I wrote some posts about the Java language extensions: SQL Server 2019 Extensibility Framework & Java.

The code in the posts is very, very simplistic where I was just trying to get the ideas across how and what to do. Another reason the code is simplistic is that I am not a Java “person”; in fact, the only Java code I have ever written is what is in those posts. Me not being a Java “person” I do not have a Java IDE, (for the code I wrote I did not need one either), so I used my regular text editor Sublime Text 3, and I compiled the code from command line ($ javac ...).

I then started on a new Java post (which I have not finished yet), and it became clear I needed an IDE, (intelli-sense, debug, etc.), but which IDE?

Interesting Stuff - Week 2, 2019

Sun, 13 Jan 2019 19:43:46 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Christmas, New Year, Week 1, 2019

Sun, 06 Jan 2019 17:13:57 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me over the Christmas and New Year period.

SQL Server 2019 Extensibility Framework & Java - Misc. 'Stuff'

Sun, 30 Dec 2018 12:24:53 +0200

This post is the fourth post in a series where I look at the Java extension in SQL Server, i.e. the ability to execute Java code from inside SQL Server.

To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.

This fourth post acts as a “roundup” of miscellaneous “stuff” I did not cover in the three previous posts, and some of the things we look at are:

Java packages.
The CLASSPATH environment variable.
JAR files.

Ok, let us get into it.

SQL Server ML Services - Multiple Input Data Sets

Mon, 24 Dec 2018 06:51:12 +0200

This post came about due to a question on the Microsoft Machine Learning Server forum. The question was if there are any plans by Microsoft to support more the one input dataset (@input_data_1) in sp_execute_external_script. My immediate reaction was that if you want more than one dataset, you can always connect from the script back into the database, and retrieve data.

However, the poster was well aware of that, but due to certain reasons he did not want to do it that way - he wanted to push in the data, fair enough. When I read this, I seemed to remember something from a while ago, where, instead of retrieving data from inside the script, they pushed in the data, serialized it as an output parameter and then used the binary representation as in input parameter (yeah - this sounds confusing, but bear with me). I did some research (read Googling), and found this StackOverflow question, and answer. So for future questions, and for me to remember, I decided to write a blog post about it.

SQL Server 2019 Extensibility Framework & Java - Null Values

Wed, 19 Dec 2018 17:12:39 +0200

This post is the third post in a series where I look at the Java extension in SQL Server, i.e. the ability to execute Java code from inside SQL Server.

To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.

In this post we look at something related to the data passing post; how to handle null values.

Interesting Stuff - Week 50 & Year End!

Sun, 16 Dec 2018 18:45:34 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

NOTE: I started with these roundup posts in the beginning of 2017 as a way to force myself to write, and - so far, so good. It is now coming up on Christmas and New Year, and I will take a break with these posts and come back in the beginning of next year.

Interesting Stuff - Week 49

Sun, 09 Dec 2018 07:18:55 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 Extensibility Framework & Java - Passing Data

Sat, 08 Dec 2018 19:52:11 +0200

This post is the second post in a series about SQL Server 2019 Extensibility Framework and the Java language extensions. In the first post, SQL Server 2019 Extensibility Framework & Java - Hello World, we looked at how to install and enable the Java language extensions, and we also wrote some pretty basic Java code to ensure it all worked.

To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.

In this post we look at how we can pass data back and forth between SQL Server and Java.

Interesting Stuff - Week 48

Sun, 02 Dec 2018 14:43:12 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 Extensibility Framework & Java - Hello World

Sun, 02 Dec 2018 09:36:34 +0200

The SQL Server 2016 release introduced the SQL Server Extensibility Framework (EF), which gives us the ability to, from inside SQL Server, execute code in an external language runtime environment. SQL Server 2016 supports R as external runtime, and Microsoft added Python to supported runtimes in the SQL Server 2017 release. The important part about the EF is that the runtime is outside of the core database engine, but we call it from inside SQL Server via the stored procedure sp_execute_external_script. We can push data from SQL Server queries to the external runtime, and consume data, (resultsets, output parameters) from the external runtime back in SQL Server.

NOTE: You can read more about the actual implementation of the external runtimes and sp_execute_external_script in my SQL Server R Services Series posts.

In SQL Server 2019 Microsoft added the ability to execute custom Java code along the same lines we execute R and Python, and this blog post intends to give an introduction of how to install and enable the Java extension, as well as execute some very basic Java code. In future posts, I drill down how to pass data back and forth between SQL Server and Java.

To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.

Interesting Stuff - Week 47

Sun, 25 Nov 2018 13:21:17 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 46

Sun, 18 Nov 2018 08:22:25 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 45

Sun, 11 Nov 2018 08:05:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 Big Data Cluster on Azure Kubernetes Service

Sat, 10 Nov 2018 07:58:09 +0200

At the Microsoft Ignite 2018 conference back in September Microsoft released SQL Server 2019 for public preview, and I wrote two short blog posts about it:

What Microsoft also announced was SQL Server 2019 Big Data Clusters, which combines the SQL Server database engine, Spark, and HDFS into a unified data platform! Yes, you read that right: SQL Server, Spark, and Hadoop right out of the box. Seeing that both Spark and Hadoop are mainly Linux based, what makes the Big Data Cluster possible is SQL Server on Linux. When you deploy a SQL Server 2019 Big Data Cluster, you deploy it as containers on Kubernetes, where the Kubernetes cluster can be in the cloud, such as Azure Kubernetes Service, or on-prem like Red Hat OpenShift or even on a local dev-box/laptop using Minikube.

Initially, this post was about SQL Server 2019 Big Data Clusters on Minikube, but after quite a few failed installation attempts I realised I did not have enough memory on my development box, so I decided to try it on Azure Kubernetes Service (AKS) instead.

NOTE: If you want to run SQL Server 2019 Big Data Clusters on Minikube it is suggested that your host machine (Minikube is essentially a VM) has at least 32Gb of memory, and you allocate at least 22Gb to the Minikube VM.

Since I am a complete novice when it comes to Kubernetes, this post covers both how I set up AKS as well as the deployment of SQL Server 2019 Big Data Clusters to AKS, and the post is somewhat a summary of the official documentation.

Interesting Stuff - Week 44

Sun, 04 Nov 2018 18:54:29 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 43

Sun, 28 Oct 2018 07:44:55 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 42

Sun, 21 Oct 2018 19:19:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 41

Sun, 14 Oct 2018 08:03:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 40

Sun, 07 Oct 2018 09:34:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 39

Sun, 30 Sep 2018 13:13:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server 2019 for Linux in Docker on Windows

Sat, 29 Sep 2018 12:06:09 +0200

By the time I publish this blog post MS Ignite is over. During Ignite, Microsoft announced quite a few new things, amongst them SQL Server 2019 with a whole lot of new features and functionality.

I touched briefly on some of them in my What is New in SQL Server 2019 Public Preview post. A couple of things that caught my eye were that SQL Server 2019 for Linux now supports In-Database analytics, what we know as SQL Server Machine Learning Services (R and Python), as well as the Java language extension.

NOTE: You may ask yourself what the Java language extension is; well, that is the ability to access Java code from T-SQL. It is a little bit like SQLCLR, but it executes outside of the SQL Server memory and process space.

Seeing that I never really have played around with SQL Server for Linux, mostly due to that in previous versions (2017) it did not have support for In-Database analytics, I thought that now would be a good time to have a look.

What is New in SQL Server 2019 Public Preview

Mon, 24 Sep 2018 19:17:06 +0200

If you read my roundup for week 38, which I published yesterday, you probably noticed that MS Ignite started today. I mentioned in the post that I was particularly interested in some of the SQL Server sessions, as they looked very interesting.

However, even before the sessions started, Microsoft released SQL Server 2019 CTP 2.0 for public preview and, naturally, I jumped on the download link and started downloading. I managed to get to the link in time before the rest of the world started the download, so I managed to get it down and then did an install.

The rest of this post is about my initial findings mostly in the SQL Server Machine Learning Services space.

NOTE: I have looked at SQL Server 2019 the grand total of an hour, so this is a short post.

Interesting Stuff - Week 38

Sun, 23 Sep 2018 06:47:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 37

Sun, 16 Sep 2018 08:17:48 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 36

Sun, 09 Sep 2018 21:11:32 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 35

Sun, 02 Sep 2018 08:38:38 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 34

Sun, 26 Aug 2018 10:22:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 33

Sun, 19 Aug 2018 09:20:03 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Test Post and Feed Apologies

Sat, 18 Aug 2018 09:38:50 +0200

In my Goodbye Jekyll, Welcome Hugo! post I wrote how I have changed blog engine from Jekyll to Hugo, and how everything seemed to work. Saying that obviously jinxed it, as after I deployed to the hosting site my Atom feed was completely hosed.

I have now spent 4 hours trying to fix everything, but I am not sure I have succeeded. So this is a test post to see if things look OK.

Goodbye Jekyll, Welcome Hugo!

Sat, 18 Aug 2018 05:03:30 +0200

Back in 2013 I wrote a post about how I moved from a self hosted WordPress blog to a static blog; OctoPress. OctoPress spoke to my “geeky” side, as the foundation of it is the static blog-generator Jekyll, which in turn is Ruby based. In fact, OctoPress is more or less just some extra Ruby plugins on top of Jekyll, and to generate sites with OctoPress / Ruby you need Ruby installed on you machine.

I thought that by changing the blog platform, I might write more posts just due to the “geekiness” of the blog engine. Fast forward to the end of 2016, and the number of posts I had written since the switch came to a grand total of three. Those three includes the post where I announced the switch. Yeah, switching increased indeed my productivity - NOT!

So what does this have to do with anything?

Interesting Stuff - Week 32

Sun, 12 Aug 2018 12:02:02 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 31

Sun, 05 Aug 2018 08:01:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

sp_execute_external_script and SQL Compute Context - III

Sat, 04 Aug 2018 16:05:46 +0200

In the Microsoft SQL Server R Services - sp_execute_external_script - III post I wrote about sp_execute_external_script (SPEES) and the SQL Server Compute Context (SQLCC). Afterwards I realised I had some things wrong, so I wrote a followup post: sp_execute_external_script and SQL Compute Context - I where I tried to correct my mistakes from the initial post. That post led to sp_execute_external_script and SQL Compute Context - II and now we have a mini-series.

To see other posts (including this) in the series, go to sp_execute_external_script and SQL Server Compute Context.

In the previous post in this series, we looked at how data is sent to the SqlSatellite from SQL Server when we are in the SQLCC. This post was meant to look at what goes on inside SQL Server when we execute in SQLCC, but I realised that it would make more sense if, before we look at the internal working when in SQLCC, I covered what happens when pulling data in the local context. So that is what this post is all about.

Interesting Stuff - Week 30

Sun, 29 Jul 2018 19:11:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 29

Sun, 22 Jul 2018 16:06:58 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending. Unfortunately there was not much that caught my eye this week, so this is a short post.

Interesting Stuff - Week 28

Sun, 15 Jul 2018 06:15:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Install Confluent Platform (Kafka) on Windows

Tue, 10 Jul 2018 18:43:48 +0200

You who follows my blog and have read my weekly roundups you know that I am quite (that is an understatement) interested in Apache Kafka and I am curious to find out what “cool” things one can do with it. For that, I want to be able to test “stuff” quickly. When I test and try out new things, I usually do it on my development box which contains everything I need: SQL Server, RabbitMQ, RStudio, Microsoft Machine Learning Server, Visual Studio and the list goes on.

So seeing that I have most of my “tools of the trade” on my machine I obviously also would like to have Kafka on the box. Herein lies a problem, I am a Windows dude and Kafka, and Windows do not gel. Yes, some people are running Kafka on Windows, but it is a chore. OK, so what to do? Sure, I could potentially run Kafka on a virtual machine, or in a Docker image, but it is not as transparent as I would like it to be (yeah, I am lazy).

Hmm, Microsoft did introduce the ability to run Linux binary executables (in ELF format) natively on Windows 10 in Windows 10 version 1607. The feature is called Windows Subsystem for Linux (WSL), and since I am now running version 1803, maybe I should try and install Kafka in WSL.

Interesting Stuff - Week 27

Sun, 08 Jul 2018 12:11:28 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

sp_execute_external_script and SQL Compute Context - II

Sat, 07 Jul 2018 10:54:21 +0200

I wrote the post sp_execute_external_script and SQL Compute Context - I about how the SQL Server Compute Context (SQLCC) works with sp_execute_external_script (SPEES), as I wanted to correct some mistakes I did in the Microsoft SQL Server R Services - sp_execute_external_script - III post. I initially thought one post would be enough, but quite soon I realised I was too optimistic, and at least one more post would be needed, if not more. So this is the first followup post about SPEES and SQLCC.

To see other posts (including this) in the series, go to sp_execute_external_script and SQL Server Compute Context.

One of the reasons for me realising that one post is not enough is that while I wrote and executed code for the first post, I noticed some fairly significant performance differences using SQLCC compared to not using SQLCC (SQLCC performed better :)). So that is part of what we look at in this post.

Interesting Stuff - Week 26

Sun, 01 Jul 2018 05:51:55 +0200

Geez, does time fly or what? We are already past the halfway mark of the year! Anyway, throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Installing R Packages in SQL Server Machine Learning Services - II

Sat, 30 Jun 2018 16:57:01 +0200

This post is the second post in a series about installing R packages in SQL Server Machine Learning Services.

To see all posts in the series go to Install R Packages in SQL Server ML Services Series.

Interesting Stuff - Week 25

Sun, 24 Jun 2018 20:36:32 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

sp_execute_external_script and Permissions

Sun, 24 Jun 2018 14:36:36 +0200

This post will (hopefully) be short and sweet. It came about as I was testing out “stuff” for the Install R Packages in SQL Server ML Services series of posts and I could not get it to work as I had expected.

Installing R Packages in SQL Server Machine Learning Services - I

Sat, 23 Jun 2018 13:13:58 +0200

This post was supposed to be a single post about how to install R packages in SQL Server Machine Learning Services, but once again I completely misjudged the scope of the topic. So this one post turned into a series of posts about how to install R packages in SQL Server Machine Learning Services and this is the first post in the series.

To see other posts in the series go to Install R Packages in SQL Server ML Services Series.

As you may know, I am in the process of writing the follow-up post to sp_execute_external_script and SQL Compute Context - I. However, I have a hard time getting into the flow of things, so I use any excuse I can, to not have to write. So when Dane Bax, a colleague of mine, contacted me a couple of days ago with a SQL Server Machine Learning Services problem, I jumped at the chance to help him, and also write a post about it.

Dane works for a sister company of Derivco as a data scientist and at both Derivco and the sister company - Microgaming we are now looking at using SQL Server Machine Learning Services.

His problem was that he wanted to use a CRAN package - bsts - which is not part of a standard SQL Server R installation. He tried a couple of things to get it installed but got errors, so he decided to get in touch with me.

Interesting Stuff - Week 24

Sun, 17 Jun 2018 19:29:19 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 23

Sun, 10 Jun 2018 10:10:24 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 22

Sun, 03 Jun 2018 09:41:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server Machine Learning Services and SQL Server 2017 CU7

Fri, 01 Jun 2018 19:43:01 +0200

This post is as much for you as it is for me to come back to when I wonder what to do when SQL Serer Machine Learning Services all of a sudden stops working after a SQL Server upgrade.

Background

Last week I did a complete reformat of my development box at home and installed the latest version of Windows 10: the 1803 build, also called the April 2018 update.

NOTE: I usually try to reformat my box at least twice a year, to ensure it is not too “bogged” down by old “cruft”, and that it has the latest and greatest.

After the reformat I installed all my necessary applications, among them SQL Server 2017 with the in-database analytics components (R and Python). After installation I tested that it all worked and made sure I could execute my usual test scripts for sp_execute_external_script:

1
2

EXEC sp_execute_external_script   @language = N'R'
                                , @script = N'd <- 42'

Code Snippet 1: Test Script for R

The “moral” equivalent to Code Snippet 1 in Python also worked without an issue:

1
2

EXEC sp_execute_external_script   @language = N'Python'
                                , @script = N'd = 42'

Code Snippet 2: Test Script for Python

Cool, all is “A-OK”! A couple of days go by, and I see that there is a Cumulative Update (CU) for SQL Server 2017 - CU7. I install it and does not think much about it. I mean: “what can go wrong, how hard can it be?”. A couple of days later and I am busy writing the follow-up post to sp_execute_external_script and SQL Compute Context - I when I try to execute sp_execute_external_script, and it falls over!

Interesting Stuff - Week 21

Sun, 27 May 2018 11:06:31 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 20

Sun, 20 May 2018 18:33:35 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

sp_execute_external_script and SQL Compute Context - I

Sun, 20 May 2018 10:46:52 +0200

The last blog post in the Microsoft SQL Server R Services series covered some performance related parameters of sp_execute_external_script, (@parallel and @r_rowsPerRead), and it touched a little bit on SQL Server Compute Context.

After I published the post, I received an email from Umachandar Jayachandran who is a Principal Program Manager in the SQL Server team and is working on integration of R, Python and other runtimes into SQL Server (basically, he knows this “stuff”). In the mail, he said that I was somewhat incorrect in what I said about how the SQL Server Compute Context (SQLCC) works.

So I decided to write a post to correct my mistakes, and this is it. While I wrote this post I realized that there are more to the SQL compute context that meets ones eye, so I will - in all likelihood - write more posts about SQLCC.

To see other posts (including this) in the series, go to sp_execute_external_script and SQL Server Compute Context.

Before I begin I want to say a very big THANK YOU to Umachandar as well as Nellie Gustafsson who also is a SQL Server Program Manager at Microsoft. Without them making me see the “errors of my way” I would not have been able to write this post.

Interesting Stuff - Week 19

Sun, 13 May 2018 20:09:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 18

Sun, 06 May 2018 19:27:40 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 17

Sun, 29 Apr 2018 19:17:32 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 16

Sun, 22 Apr 2018 16:45:06 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 15

Sun, 15 Apr 2018 10:20:37 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 14

Sun, 08 Apr 2018 16:04:25 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

SQL Server Machine Learning Services and Extended Events

Sun, 08 Apr 2018 08:06:46 +0200

Back in the day when I started with the SQL Server R Services - Internals posts, I thought I would use SQL Server Extended Events (XEvents) to figure things out. I knew SQL Server and the extensibility architecture supported XEvents because I had come across this documentation, where it (amongst other things) said:

“SQL Server provides a set of extended events to use in troubleshooting operations related to the SQL Server Trusted Launchpad, as well as Python or R jobs sent to SQL Server.”

From the above, it is pretty clear that XEvents is the way to go when drilling down into how things “hang together”. However, how hard I tried, I could not make “head or tails” of how to get things to work (I should probably have tried harder), so in the end, I went the WinDbg and WireShark route.

Fast forward a year (and a bit) and someone asked a question on an internal forum about XEvents and SQL Server Machine Learning Services, and I decided to look into it and to try to understand it a bit better. So this post is the result of me ~~playing around with~~ researching XEvents and SQL Server Machine Learning Services.

Interesting Stuff - Week 13

Sat, 31 Mar 2018 19:12:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 12

Sun, 25 Mar 2018 11:12:17 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Microsoft SQL Server R Services - sp_execute_external_script - III

Wed, 21 Mar 2018 11:46:27 +0200

This post is the 24:th post about Microsoft SQL Server R Services, and the third that covers sp_execute_external_script. I have thought many times that this series has reached its end, and I have many times realised I was wrong. Well, now I can say that this is the last post in the series! I started over a year ago (March 4 2017) with an idea about “wouldn’t it be cool if I wrote a couple of posts about Microsoft SQL Server R Services”! Over one year and 24 posts later, here we are!

This last post talks about sp_execute_external_script and a parameter named @parallel. When we discuss this parameter, we also touch (a little bit) upon performance aspects of sp_execute_external_script.

Interesting Stuff - Week 11

Sun, 18 Mar 2018 20:20:08 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 10

Sun, 11 Mar 2018 14:17:55 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Microsoft SQL Server R Services - sp_execute_external_script - II

Sun, 11 Mar 2018 11:55:37 +0200

This post is the 23:rd post about Microsoft SQL Server R Services, and a continuation of sp_execute_external_script - I. To see other posts (including this) in the series, go to SQL Server R Services.

I honestly thought this post would be the last in the series (I have thought quite a few times), but alas, that is not the case. There will be at least one more post about sp_execute_external_script after this.

Microsoft SQL Server R Services - sp_execute_external_script - I

Wed, 07 Mar 2018 19:37:31 +0200

This is the 22:nd post in a series about Microsoft SQL Server R Services. This journey started about a year ago when I thought it would be a good idea to write a couple of blog posts about, what then went under the name of, Microsoft SQL Server 2016 R Services. Never in my wildest fantasies did I think I would write over 20 posts.

NOTE: Since the introduction of Microsoft SQL Server R Services, it has been renamed to Microsoft SQL Server Machine Learning Services.

To see other posts (including this) in the series, go to SQL Server R Services.

This series began with a discussion about how to install Microsoft SQL Server R Services and continued with the internals of how it works internally in SQL Server and the launchpad service. Throughout the series, we have used sp_execute_external_script when investigating what goes on. So, I thought that having a couple of posts looking at sp_execute_external_script would be a good way to finish this series off, and this post is the first of three about sp_execute_external_script.

Interesting Stuff - Week 9

Sun, 04 Mar 2018 19:29:44 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 8

Sun, 25 Feb 2018 11:06:52 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
AI
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 7

Sun, 18 Feb 2018 13:33:05 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 6

Sun, 11 Feb 2018 06:57:25 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 5

Sun, 04 Feb 2018 19:55:11 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Microsoft SQL Server R Services - Internals XX

Fri, 02 Feb 2018 19:48:22 +0200

This is the 21:th post in a series about Microsoft SQL Server R Services, (it is now called Microsoft SQL Server Machine Learning Services), and the 20:th post that drills down into the internal of how it works.

Wow, 20 posts about internals - when I started this Microsoft SQL Server R Services series, I thought I might achieve five or six posts. I would never have - in my wildest fantasies - thought I would reach twenty!

To see other posts (including this) in the series, go to SQL Server R Services.

We are however now coming closer to the end of this whole series, and this post is the last for Internals. After this post, I foresee one, possibly two, more posts about other things related to Microsoft SQL Server R Services - and then we are done!

Interesting Stuff - Week 4

Sun, 28 Jan 2018 14:05:43 +0200

Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.

Interesting Stuff - Week 3

Sun, 21 Jan 2018 07:35:50 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals XIX

Sat, 20 Jan 2018 19:42:49 +0200

This is the 20:th post in a series about Microsoft SQL Server R Services, and the 19:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

We are nearing the end of the Internals series, and if we look at what we have covered, we have almost come full circle.

Interesting Stuff - Week 2

Sun, 14 Jan 2018 18:55:19 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals XVIII

Wed, 10 Jan 2018 10:19:41 +0200

This is the 19:th post in a series about Microsoft SQL Server R Services, and the 18:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In this post we’ll continue looking at data coming back to SQL Server, and what happens code-wise in SQL.

Interesting Stuff - Christmas, New Year, Week 1

Sun, 07 Jan 2018 20:17:09 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me over the Christmas and New Year period.

Microsoft SQL Server R Services - Internals XVII

Wed, 03 Jan 2018 09:57:38 +0200

This is the 18:th post in a series about Microsoft SQL Server R Services, and the 17:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

The last Internals posts have discussed the transport of data between SQL Server and the SqlSatellite. In Internals - XVI we looked at how data came back from the SqlSatellite, and what the data looked like.

Microsoft SQL Server R Services - Internals XVI

Sun, 24 Dec 2017 08:02:00 +0200

This is the 17:th post in a series about Microsoft SQL Server R Services, and the 16:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In the last few posts in this Internals series we have looked at how data is being transported from SQL Server to the SqlSatellite. In this post we’ll look at how data is going the other way, from the SqlSatellite to SQL Server.

Interesting Stuff - Week 50: Year End!

Sun, 17 Dec 2017 08:58:46 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

NOTE: I started the “roundup” posts in the beginning of this year, as a way for me to force myself to keep my blog active, and to share with you what I find interesting in the tech-world. It is now coming up on Christmas and New Year, and I will take a break with these posts and come back in the beginning of next year.

Interesting Stuff - Week 49

Sun, 10 Dec 2017 10:24:31 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

NOTE: This week there is very little content. This is because I’ve been overseas - last Sunday I went to Gibraltar, and I came back yesterday (Saturday) evening. Gibraltar is an awesome place by the way. The downside of living in South Africa (as I do) is the travel time - Durban to Gib was a 25 hours door to door trip!

Interesting Stuff - Week 48

Sat, 02 Dec 2017 15:34:35 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals XV

Sat, 02 Dec 2017 05:41:23 +0200

This is the 16:th post in a series about Microsoft SQL Server R Services, and the 15:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In the last post Internals - XIV we looked at the Binary eXchange Protocol (BXL). The post was fairly long, and there were things that I wanted to cover which I didn’t do due to the length of the post.

So this post is a continuation of Internals - XIV.

Interesting Stuff - Week 47

Sun, 26 Nov 2017 19:08:43 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals XIV

Sat, 25 Nov 2017 10:09:57 +0200

This is the 15:th post in a series about Microsoft SQL Server R Services, and the 14:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

Finally, after a couple of false starts this post will look into the protocol used when transferring data back and forth between SQL Server and the SqlSatellite: Binary eXchange Language (BXL). Why write about it at all? Well, the big reason why I wanted to know more about it, was due to the strange packet sizes I saw when inspecting data being sent from SQL Server to the SqlSatellite: one row with 5 numeric columns had a packet size of 6300 bytes!!!

Interesting Stuff - Week 46

Sun, 19 Nov 2017 19:43:30 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 45

Sun, 12 Nov 2017 20:05:53 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals XIII

Sat, 11 Nov 2017 20:02:50 +0200

This is the fourteenth post in a series about Microsoft SQL Server R Services, and the thirteenth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

Sorry guys, in my last post Internals - XII I said we’d - in this post - look into the Binary eXchange Language protocol(BXL) which is used when sending data from SQL Server to the SqlSatellite. The protocol is also involved when we send one row of data containing 5 integers, and the packet size for that ends up being 6300 bytes.

Well, we won’t look at that in this post, I was once again led astray when researching how data is sent.

Interesting Stuff - Week 44

Sun, 05 Nov 2017 19:16:52 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals XII

Tue, 31 Oct 2017 19:57:36 +0200

This is the 13:th post in a series about Microsoft SQL Server R Services, and the 12:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

The last few posts in this series has been about communication between SQL Server and SqlSatellite. We have discussed the connection between the two, and we have also seen how packets are sent from SQL Server to SqlSatellite. In this post we’ll continue looking at what causes the packets to be send and what sends them. We will specifically look at the packet containing the external script, as well as the packet that contains the data (@input_data_set_1).

Interesting Stuff - Week 43

Sun, 29 Oct 2017 19:29:49 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 42

Sun, 22 Oct 2017 16:19:45 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

This week I have very little to post, mainly due to me being out of the country, and not having been able to read as much as I would have liked.

Microsoft SQL Server R Services - Internals XI

Fri, 20 Oct 2017 18:42:47 +0200

This is the twelfth post in a series about Microsoft SQL Server R Services, and the eleventh post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In the last post we spoke about how the transfer of data happens between SQL Server and the R components, and we saw how data was transferred over the TCP connection between the SqlSatellite and SQL Server. In the post we mentioned that the protocol being used is Binary eXchange Language protocol(BXL), and this post was supposed to talk about BXL.

Interesting Stuff - Week 41

Fri, 13 Oct 2017 06:27:44 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by. This week, the roundup comes a bit early due to me having to go abroad at the weekend for a week.

Interesting Stuff - Week 40

Sun, 08 Oct 2017 19:08:57 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 39

Sun, 01 Oct 2017 17:57:21 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 38

Sun, 24 Sep 2017 09:26:42 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 37

Sun, 17 Sep 2017 12:24:24 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 36

Sun, 10 Sep 2017 08:56:44 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 35

Sun, 03 Sep 2017 19:55:48 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

I know, I know - there is not much for this week. A major reason for this is that, here in South Africa, we are in SQL Saturday season right now. What has that got to do with anything you may ask - well, it is like this:

Microsoft SQL Server R Services - Internals X

Tue, 29 Aug 2017 20:19:14 +0200

This is the eleventh post in a series about Microsoft SQL Server R Services, and the tenth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In Internals - IX I said we were nearing the end of the internals part is this series, and I had planned this post to be the final post in the Internals part. However while investigating how data is sent between SQL Server and the external components I realized that I would need to probably do two more posts (apart from this) for Internals.

Anyway, in this post we’ll see how data is sent to the R components from SQL Server.

Interesting Stuff - Week 34

Sun, 27 Aug 2017 21:30:24 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 33

Sun, 20 Aug 2017 20:25:44 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals IX

Fri, 18 Aug 2017 17:44:22 +0200

This is the tenth post in a series about Microsoft SQL Server R Services, and the ninth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

We are now nearing the end of the Internals part of this series (you probably say “about time”). When I started this series way back in March this year, I thought that the Internals would be two, maximum three episodes. Boy was I wrong!

In this post we’ll look at how the various components are communicating with each other.

Interesting Stuff - Week 32

Sun, 13 Aug 2017 20:12:03 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 31

Sun, 06 Aug 2017 19:36:25 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 30

Sun, 30 Jul 2017 14:23:33 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 29

Sun, 23 Jul 2017 19:51:06 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

SQL Server 2017 SQLCLR - Whitelisting Assemblies

Sun, 23 Jul 2017 09:53:57 +0200

A little while ago I wrote a blog-post about the changes in the SQLCLR security model in SQL Server 2017. I wrote about how Microsoft has changed the relation between CAS and security boundaries, and CAS is no longer supported as a boundary! And also how Microsoft introduced an sp_configure option called clr strict security, which by default is set to 1 (on). When the setting is on, SQL Server treats all assemblies (SAFE, EXTERNAL_ACCESS, UNSAFE) as if they were marked UNSAFE.

The release I wrote the post against was CTP 2.1, and earlier this week Microsoft released SQL Server 2017 RC1. Some days ago I received a comment from Paul Vestuto on my post, pointing out that in RC1 there has been some more changes to the security model. I thought those changes would earn their own blog-post, instead of just editing the previous post.

Microsoft SQL Server R Services - Internals VIII

Sat, 22 Jul 2017 08:47:47 +0200

This is the ninth post in a series about Microsoft SQL Server R Services, and the eight post that drills down into the internal of how it works. To see other posts (including this) in the series, go to: SQL Server R Services

So far in this series we have been looking at what happens in SQL Server as well as the launchpad service when we execute sp_execute_external_script, and we have still no real “clue” to where the R engine comes into play.

Well, hopefully that will change (at least a little bit) with this post, as we here will look at what happens when we leave the launchpad service.

Interesting Stuff - Week 28

Sun, 16 Jul 2017 20:34:07 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals VII

Tue, 11 Jul 2017 20:08:13 +0200

(WOW, I finally managed to get this post out! Yay to me!!)

This is the eighth post in a series about Microsoft SQL Server R Services, and the seventh post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In this post we will look more at some of the directories that are created when we execute sp_execute_external_script, as well as files being created in those directories.

Interesting Stuff - Week 27

Sun, 09 Jul 2017 19:15:08 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 26

Sun, 02 Jul 2017 19:23:56 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

SQL Server 2017, SQLCLR and Permissions

Sun, 02 Jul 2017 08:18:48 +0200

While I was writing the post about SQLCLR and Certificates, I came across some strange behavior when I accidentally ran my code on an SQL Server 2017 instance. I quickly connected back to my SQL Server 2016 instance and continued. However, afterwards I started to look into the differences between SQLCLR in SQL Server 2017 and SQL Server 2016, and this post is the result of me ~~playing around~~ investigating the changes.

SQLCLR and Certificates

Sat, 01 Jul 2017 06:21:55 +0200

This post originates, like the one about sqlrutils, from a question on a Microsoft forum - this time the .NET Framework inside SQL Server forum. As the name implies, the forum cover questions about the usage of SQLCLR (.NET inside SQL Server), and the question was about signing of SQLCLR assemblies in order to assigning other permission sets than SAFE to the assembly.

As I, back in the day, taught SQLCLR as part of Developmentor’s SQL Server curriculum, I thought that it’d be interesting if I tried to answer the question, and also make a blog-post about it. That way I’d remember what to do the next time the question came up.

NOTE: In another post, I wrote this about Developmentor: Developmentor were back in the day THE training company to go to if you wanted highly, highly technical training about COM, .NET, SQL Server, etc. This article by my old colleague Ted Neward (@tedneward) sums DM up quite accurately (apart from the fact that DM hadn’t closed its doors when the article was written, ooops). Ted was probably more accurate than he at the time thought, since DM went out of business somewhat later.

Anyway, let’s see what this is all about…

Interesting Stuff - Week 25

Sun, 25 Jun 2017 19:51:33 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Creating R Stored Procedures in SQL Server 2016 Using sqlrutils

Sun, 25 Jun 2017 17:19:20 +0200

In the Microsoft R Server forum the other day was a question about inputs to SQL Server R stored procedures, or rather about “strange” parameter naming, when using the sqlrutils package. I got intrigued by the question and started ~~playing around with~~ researching it, and this blog-post is the result. Oh, and this blog-post would never have seen the light of the day, if it hadn’t been for the original poster of the question: JD Long, thanks JD!

So, what is SQL Server R stored procedures and what is sqlrutils?

Interesting Stuff - Week 24

Sun, 18 Jun 2017 09:26:42 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 23

Sun, 11 Jun 2017 19:49:02 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 22

Sun, 04 Jun 2017 20:13:28 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 21

Sun, 28 May 2017 20:00:13 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 20

Sun, 21 May 2017 20:08:21 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Facebook Prophet and Microsoft R Server

Sat, 20 May 2017 11:31:37 +0200

Time series data is being used more and more in all industries, and of course we want to be able to forecast and predict future values. There are however a variety of challenges that come with producing a large number of forecasts across a variety of time series.

The Facebook’s data science team has worked with time series forecasting for quite a while, and recently they open-sourced an internal tool for this type of forecasting: Facebook Prophet.

Recently there was a post on the Microsoft R Server forum if it was possible to use Prophet with Microsoft R Server, so I thought I’d test it out. This post covers how to use it, both from a standalone Microsoft R Server as well as SQL Server R Services.

Microsoft SQL Server R Services - Internals VI

Tue, 16 May 2017 19:38:41 +0200

This is the seventh post in a series about Microsoft SQL Server R Services, and the sixth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In this post we will look at the directories that are created when we execute sp_execute_external_script.

Interesting Stuff - Week 19

Sun, 14 May 2017 18:05:53 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Interesting Stuff - Week 18

Sun, 07 May 2017 21:18:40 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.

Microsoft SQL Server R Services - Internals V

Mon, 01 May 2017 15:04:31 +0200

This is the sixth post in a series about Microsoft SQL Server R Services, and the fifth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In Internals - III and Internals - IV, we looked at how the launchpad service creates RTerm processes when we execute an external script.

The Internals - IV post came about due to an email from Bob Albright ( @bob_albright), pointing me to some resources (a blog-post) regarding the processes the launchpad service creates.

In his email Bob also, kindly enough, attached some code. The code highlighted some behavior when creating the RTerm processes, especially around parallelism. In this blog-post we’ll follow up on that and look into the effect parallelism has on process creation.

Interesting Stuff - Week 17

Sun, 30 Apr 2017 20:24:37 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 16

Sun, 23 Apr 2017 20:22:38 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Microsoft SQL Server R Services - Internals IV

Sun, 23 Apr 2017 14:42:01 +0200

This is the fifth post in a series about Microsoft SQL Server R Services, and the fourth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In Internals - III, I wrote about how the launchpad service creates multiple processes when executing an external script.

Seeing that some of the conclusions I came to was somewhat educated guesses, I asked you guys to correct me where I was in-correct and/or add more information. After that post - Bob Albright ( @bob_albright) - wrote me an email and pointed me to some resources around process creation, as well as some demo code. Thanks Bob!

So today we’ll drill even further into the creation of processes, and see how they are used.

SQL Server 2017 - Python Executing Inside SQL Server

Thu, 20 Apr 2017 20:49:08 +0200

On April 19, 2017 Microsoft held an on-line conference Microsoft Data Amp to showcase how Microsoft’s latest innovations put data, analytics and artificial intelligence at the heart of business transformation. The keynotes speakers were Scott “Red Shirt” Guthrie who is an Microsoft Executive Vice President (fairly high on the food-chain), and Joseph Sirosh a Corporate Vice President, head of the Information Management and Machine Learning group. Joseph probably knows a thing or two about data, and Scott - well, Scott knows A LOT!

During the keynotes, Scott and Joseph shared how Microsoft’s latest innovations put data, analytics and artificial intelligence at the heart of business transformation. A big enabler for this is SQL Server 2017 (the “artist” formerly known as SQL Server V.Next), which introduces a lot of very cool new “stuff”.

The keynote speeches were followed by recorded short:ish sessions drilling down into certain aspects of Microsoft’s new offerings. If you are interested in the various presentations at Microsoft Data Amp, they all are on Channel 9.

What caught my eye was the announcement of how SQL Server 2017 now supports Python as an extensible script engine. When Microsoft introduced support for R in SQL Server 2016, rumors immediately surfaced how other engines also would be supported, where Python was high on the list ( Julia another one).

Seeing that I am somewhat into SQL Server R Services (the SQL Server 2016 flavor) here, here, here, here, and more posts to come, I really had to have a look.

The rest of this post is a brief introduction to executing Python code in SQL Server 2017, in essence I just want to be able to execute some sort of Python code.

Interesting Stuff - Week 15

Sun, 16 Apr 2017 20:53:40 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Microsoft SQL Server R Services - Internals III

Thu, 13 Apr 2017 06:13:14 +0200

This is the fourth post in a series about Microsoft SQL Server R Services, and the third post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In the previous internals posts here and here we have looked at what goes on inside SQL Server when executing an R script, and what happens in the launchpad service when executing.

This post was initially to be about the R runtime, and what other parts are involved when executing R scripts in SQL Server. However, during my “spelunking” around I realized that I did not really understand what happens when we create the processes for the R runtime from the launchpad service and the launcher (which is what I covered in the internals II post).

So, instead of looking at the R runtime and the other R related components, we’ll look at what happens when creating external processes. A subsequent blog-post will cover the R runtime and components.

Interesting Stuff - Week 14

Sun, 09 Apr 2017 14:51:31 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 13

Sun, 02 Apr 2017 16:39:25 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Microsoft SQL Server R Services - Internals II

Sun, 02 Apr 2017 09:38:00 +0200

This is the third post in a series about Microsoft SQL Server R Services, and the second post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In the previous internals post we looked at what happens inside the SQL Server engine when we execute sp_execute_external_script. In that post we wrapped up when we reached the launchpad service (Launchpad.exe). This post will look closer at the launchpad service, and we will do it by some more “spelunking”

Interesting Stuff - Week 12

Sun, 26 Mar 2017 19:59:37 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 11

Sun, 19 Mar 2017 07:59:04 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Microsoft SQL Server R Services - Internals I

Sat, 18 Mar 2017 16:21:21 +0200

This is the second post in a series about Microsoft SQL Server R Services, and the first post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.

In this post, and one or two more we will look at what goes on under the covers when we execute some R code inside SQL Server. This post looks at quite in detail what happens in the SQL engine when we execute sp_execute_external_script.

To try and get an understanding we’ll do something that we did quite a lot back in the day when I worked at Developmentor; we’ll “spelunk” the SQL Server code via WinDbg. This can be really useful when trying to understand and get to grips with new technology/functionality.

NOTE: Developmentor were back in the day THE training company to go to if you wanted highly, highly technical training about COM, .NET, SQL Server, etc. This article by my old colleague Ted Neward (@tedneward) sums DM up quite accurately (apart from the fact that DM hadn’t closed its doors when the article was written, ooops).

Interesting Stuff - Week 10

Sun, 12 Mar 2017 07:39:45 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 9

Sun, 05 Mar 2017 06:58:15 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Microsoft SQL Server 2016 R Services Installation

Sat, 04 Mar 2017 10:19:22 +0200

A couple of weeks beck I wrote a blog-post where I compared CRAN R with Microsoft R Server. The comparison was basically how large data sets are handled. In the post I mentioned that SQL Server 2016 (and later) is the delivery mechanism for Microsoft R Server on the Windows platform. In this post we will look at how to install and enable SQL Server R Services.

This post is the first in a series of posts about SQL Server R Services and to see other posts (including this) in the series, go to SQL Server R Services.

Interesting Stuff - Week 8

Sun, 26 Feb 2017 07:11:45 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Microsoft R Server

Sat, 25 Feb 2017 06:28:25 +0200

Last Saturday (February 18) I was in Cape Town at the second world-wide satRday conference ever, where I gave a talk named: “Microsoft, Open Source, R: You gotta be kidding me!”. The talk was about Microsoft’s R Server offering, and how it in some cases offered a better performance than Open R. Seeing the session lengths were only 20 minutes, I could not show any code, so in a “weak” moment I promised to put the code up on my web-site together with an accompanying blog-post. This is it :).

Interesting Stuff - Week 7

Sun, 19 Feb 2017 18:48:02 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 6

Sun, 12 Feb 2017 06:32:10 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that have been most interesting to me, this week.

RabbitMQ - SQL Server

Sat, 11 Feb 2017 12:39:51 +0200

A week or two ago I came across a post on the RabbitMQ Users forum, about how to communicate from SQL Server to RabbitMQ. Seeing that we do that a lot at Derivco, I came with some suggestions, and also said that I was writing a blog-post about how it can be done. The part of writing was not entirely correct - at least not until now (sorry Jim, it’s been hectic at work).

Interesting Stuff - Week 5

Sun, 05 Feb 2017 09:58:12 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 4

Sat, 28 Jan 2017 07:57:00 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Doomed, We are All Doomed I Say!

Fri, 27 Jan 2017 18:58:40 +0200

Just to set things straight, the title of this post has nothing to do with US politics, but the infinitely more important (and exciting) subject of transactions in SQL Server and the concept of doomed transactions. Why I came across the idea for this post was due to a discussion I had with a colleague of mine - Sameer Chunilall - who didn’t agree with my post a while ago about XACT_ABORT, he believes it is a good thing (mostly), where I believe it is a bad thing (mostly).

Interesting Stuff - Week 3

Sat, 21 Jan 2017 05:37:16 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
data in general
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me, this week.

Interesting Stuff - Week 2

Sun, 15 Jan 2017 07:14:41 +0200

Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:

data science
distributed computing
SQL Server
transactions (both db as well as non db)
and other “stuff”

This is the “roundup” of the posts that has been most interesting to me. Oh, and if you wonder if you have missed Interesting Stuff - Week #1. seeing that this post is named Interesting Stuff - Week #2, you haven’t. They are numbered by calendar week number, and I started the second week of January :).

Abort, Abort, We Are XACT_ABORT:ing, Or Are We?!

Sun, 08 Jan 2017 11:41:55 +0200

SET XACT_ABORT defines whether a transaction is automatically rolled back when a T-SQL statement raises a run-time exception, and when you read posts from prominent SQL bloggers you quite often see that they recommend to always have XACT_ABORT set to ON. From the title of this post you may get the impression that I do not necessarily agree, and to an extent you may be right. So, let us see …

SQL Server Error Handling Gotchas

Sat, 31 Dec 2016 06:41:55 +0200

Way back when (in 2010 as a matter of fact), I wrote a couple of blog-posts ( here and here) about error handling in the new CTP releases of SQL Server Denali. Denali was what would become SQL Server 2012.

The new functionality built upon what was introduced in SQL Server 2005 - the notion of structured exception handling through BEGIN TRY END TRY followed by BEGIN CATCH END CATCH. In those blog-posts I was fairly positive, and saw the new functionality as something useful and very well worth implementing. I am still positive, however since then I have used the new functionality introduced in SQL Server 2005 extensively in production and have come across some gotchas that I thought would be worth writing a blog-post about.

Testing, Testing, Anyone Out There?

Wed, 28 Dec 2016 14:18:58 +0200

What can I say, I am crap at this “blogging” thing! Three years ago, I resuscitated my blog, after a hiatus of one year. I also switched blog engine from WordPress to OctoPress (which at that stage was version 2). At the time of the switch, I had had a one year hiatus of blogging (I am not a very prolific blog writer :)), and I remember I was very tempted to write in that post, that if I cannot keep up blogging I will quit. Boy, am I glad I did not write that!

SQL Saturday Presentations

Tue, 29 Oct 2013 20:13:56 +0200

The two first weekends in August I were in Cape Town and Johannesburg respectively presenting at SQL Saturday events. I had a great time, and the audience were terrific. My employer Derivco graciously gave me some “swag” to hand out, and that went down a treat. Thanks Derivco!!

Installing Octopress on Windows

Wed, 23 Oct 2013 09:27:56 +0200

In an earlier post, I wrote how I had changed blog-engine from WordPress to OctoPress. At the moment I am running Windows as my main OS (mostly due to work related requirements), and - as OctoPress is very much Ruby and Python based - there are certain things to be aware of when installing this on Windows. Furthermore, my blog is now hosted on Windows Azure - so, with all this in mind, I thought I’d put together a blog post about the set-up of OctoPress on Windows, running on Windows Azure.

Moving to a New Blog Engine

Wed, 02 Oct 2013 09:27:56 +0200

Every so often I get an itch, and I want to try out a new blog-platform. The time has now come to leave my self-hosted WordPress and move on to something “geekier” (I am a geek after all).

I have just now finished moving my few posts from WordPress to the new engine: OctoPress. In a future blog post on the new blog I will try to explain why I made the move, and why OctoPress. The new blog will be hosted on Windows Azure - I do get free Azure credits together with my MSDN subscription, so I thought I’d see what all the “fuss” about Windows Azure is.

SqlClrProject on GitHub

Sat, 03 Nov 2012 09:27:56 +0200

As some of you may know, I - once upon a time - developed a project (VS add-in, templates, etc) for automatic deployment of CLR assemblies to SQL Server: SqlClrProject. That project has been dormant now for a couple of years, but I now and then get requests for where it can be downloaded from (I had it on CodePlex, but had to take it down as I didn’t publish the source code).

First Impressions Microsoft BUILD - Win 8

Tue, 13 Sep 2011 09:27:56 +0200

I have a while ago just finished watching the live stream of the first keynote (yes there will be one tomorrow as well), at Microsoft BUILD. Having attended / presented, at quite a few of these kind of events - and being somewhat jaded (well OK then, a lot jaded), I must still say that I am impressed.

Transactions in SQL Server (take 2956)

Sun, 11 Sep 2011 09:27:56 +0200

Transactions in SQL Server seems to be a difficult topic to grasp. This weekend I came across a blog-post where the poster showed a “solution” to the “The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION” error we sometimes see when various stored procedures call each other. The solution (even though it masked out the error in question) did not get it quite right. So I thought I would make a post about the subject.

F#, Mono and Mac - Take II

Sat, 20 Nov 2010 09:27:56 +0200

So yesterday I wrote about how I have started using F# and Mono on my MacBook.

I wrote about how I downloaded the F# bits, unzipped and put them in a specific directory I had created. Today after having browsed around a bit more I realized I had done it the hard way. To install the required bits for F# for Mac, you only have to download a zip file with an install package for Mac from the F# Cross Platform site on CodePlex. The actual zip-file for the November 2010 CTP is here.

F#, Mono and Mac

Fri, 19 Nov 2010 09:27:56 +0200

This is a first post about my experiences with running F# and Mono on a Mac.

In a previous post I wrote about how I have started to play with F#. As that post also covered SQLCLR it was obvious I was on Windows. Even though I make my living from development in a Windows environment, my main machine is a MacBook, and I run OSX as my main OS. I have previously also been running Linux (ArchLinux) on this machine as my main OS. Naturally I have heard about Mono (and also installed it a couple of times - and quickly un-installed again, but I have not really done anything with it. I have always run Windows in a VM on my MacBook for development etc. However after the announcement that F# was going Open Source, and Tomas P posted about his F# MonoDevelop plug-in, I decided that I should have a look at what it would be like to do F# “stuff in OSX.

TPL Dataflow, Axum v.NEXT?

Thu, 18 Nov 2010 09:27:56 +0200

At PDC 2010 Microsoft showed the new Async features of coming C# (and VB.NET) versions, and quite a lot has been written about it already. Part of the Async CTP is TPL Dataflow, and this has gone somewhat un-noticed.

Using F# in SQLCLR

Mon, 15 Nov 2010 09:27:56 +0200

Recently I have become very interested in F# and I am at the moment trying to get to grips with it. It is definitely a different beast than C#, but so far I like it - a lot!

Anyway, I am a SQL nerd, and many moons ago I was very heavily involved in SQLCLR (for you who don’t know what that is; it is the ability to run .NET code inside the SQL Server engine. It was first introduced with SQL Server 2005). So I thought it would be a “giggle” to see if I could get some F# code running inside SQL Server.

SQL Server Denali CTP 1 SUX

Sun, 14 Nov 2010 09:27:56 +0200

.. from a relational developers perspective!!

Well, the title may be a bit harsh, but at least it grabbed your attention - did it not?!

A week ago, or so, I wrote a wish list to Santa for Denali from a relational developers perspective. In that wish list I wrote that there has been fairly little love for relational SQL developers in the recent versions of SQL Server, and that I hoped in this version (i.e Denali) Microsoft would “go back to the roots” and give us developers some new stuff.

Beginners F# Resources

Sat, 13 Nov 2010 09:27:56 +0200

This post is more as a reminder to myself where to find online resources when learning F#. If anyone else can find it useful, so much better. And, if anyone out there has other online, resources, please leave a comment and I will include it. So, in no particular order:

Finally, a list like this would be incomplete without the link to the Man himself: Don Syme

More T-SQL Error Functionality in Denali / SQL 11

Fri, 12 Nov 2010 09:27:56 +0200

In my previous post I wrote about the new THROW keyword in Denali / SQL 11. Having played around a bit more with Denali, I wanted to write some additional things about THROW and it’s relation to RAISERROR.

New T-SQL Features in SQL 11 / Denali - Error Handling

Wed, 10 Nov 2010 11:27:56 +0200

A couple of days ago I wrote my wish-list to Santa what I wanted to see in next version of SQL Server (SQL 11 / Denali). I was pleasantly surprised that I could find out for myself shortly after; i.e. SQL Server Denali CTP1 was released during the PASS Summit. I have literally finished installing the next version of SQL Server (Denali / SQL 11) on a new VM, like 10 minutes ago, and I have done a quick check of the new features of SQL Server Denali (what I could find at least) against my wish-list.

More about new features in SQL 11 / Denali

Mon, 08 Nov 2010 11:27:56 +0200

So yesterday I posted my wish-list for new programmability features in the upcoming release of SQL 11 / Denali.

Today I see that Simon S has posted about a new series of posts he will do, covering what is new in SQL 11. Knowing Simon, it will be really, really good. So if you are interested I suggest you keep your eyes open for his posts.

UPDATE: Ben C commented and said that CTP1 has been released (or something to that effect), and here is where it can be downloaded from.

What New Programmability Features Will There Be in SQL 11?

Sun, 07 Nov 2010 11:27:56 +0200

There is probably no secret that Microsoft is working hard on next version of SQL Server. The rumour has it that it will be named SQL 11 (it apparently goes under the code name of Denali. Quiz; MS has used the Denali code name previously, what was it for? Answers in the comments ).

Stream and Complex Event Processing from a Relational Guy's Eye

Sun, 07 Nov 2010 09:27:56 +0200

This is a re-post from my previous blog. However, as that blog has now gone to the big blog repository in the sky (or wherever blogs go to when they are no more), I decided to repost this, seeing how CEP and StreamInsight are becoming more and more popular.

SQL Server 2008 R2 August CTP

Tue, 11 Aug 2009 09:27:56 +0200

Yesterday I downloaded and installed the August CTP of SQL Server 2008 R2, and today I played around with it for a while. So, what are my impressions?

Twitter

Sun, 09 Aug 2009 09:27:56 +0200

As the saying goes; “It is hard to learn an old dog new tricks”, but… Even though I am a really old dog, I hope I’ll be able to learn a bit about “social networking”, and therefore I created an account on Twitter a couple of days ago. Hopefully I’ll be able to be more active on Twitter than what I’ve been here at the blog. Well, that should not be too hard, seeing how infrequent I post here.

Debugging in SQL Server 2008

Wed, 16 Jul 2008 09:27:56 +0200

As good as SQL 2005 was (well, still are), one disappointment was that you needed Visual Studio if you wanted to debug your stored procedures. Seriously, what was MS thinking when they did that, especially as in SQL 2000, Query Analyzer had debug capabilities?!!

SQL Server 2005 and DMV's

Wed, 23 Aug 2006 09:27:56 +0200

One of the first thing I usually do after having installed a new version of SQL Server is to snoop around at the system tables to see what new tables there are and what they can give me information about.

Mon, 01 Jan 0001 00:00:00 +0000

Go 语言学习笔记

Envoy 学习笔记

Mon, 01 Jan 0001 00:00:00 +0000

sp_execute_external_script and SQL Server Compute Context

This is a collection of posts about sp_execute_external_script (SPEES) and SQL Server Compute Context (SQLCC).

Table of Content

sp_execute_external_script and SQL Compute Context - I - An overview of SQL Server Compute Context and how it works on a high level.
sp_execute_external_script and SQL Compute Context - II - A look at why the SQL Server Compute Context performs better (sometimes) than the local context.
sp_execute_external_script and SQL Compute Context - III - We use WinDbg, Process Monitor and WireShark to look in detail what happens in SQL Server when we use RxSqlServerData to pull data.

More To Come

Mon, 01 Jan 0001 00:00:00 +0000

SQL Server R Services Series TOC

Microsoft released SQL Server 2016 mid 2016 and with the release came quite a few new features:

Query Store.
Temporal tables.
JSON support.
Other “cool stuff”.

Among the other “cool stuff” was the introduction of SQL Server 2016 R Services, an in-database analytics engine for processing R script on resident data within a database engine instance.

NOTE: Since the introduction of Microsoft SQL Server 2016 R Services, the functionality has been renamed to Microsoft SQL Server Machine Learning Services. Also, SQL Server 2017 introduced Python as an additional supported language.

Mon, 01 Jan 0001 00:00:00 +0000

Install R Packages in SQL Server ML Services Series

This is a series of posts about how to install R packages in SQL Server Machine Learning Services.

Table of Content

Installing R Packages in SQL Server Machine Learning Services - I - Background about package installation, Rtools.exe and R packet managers (Rterm.exe and Rgui.exe)
Installing R Packages in SQL Server Machine Learning Services - II - We look at using RevoScaleR to install R packages in SQL Server Machine Learning Services.
Installing R Packages in SQL Server Machine Learning Services - III - How we can install R packages in SQL Server Machine Learning Services using T-SQL DDL: CREATE EXTERNAL LIBRARY.
Installing R Packages in SQL Server Machine Learning Services - IV: Permissions - We look at permissions required when using CREATE EXTERNAL LIBRARY as well as ownership.

More To Come

Mon, 01 Jan 0001 00:00:00 +0000

Disclaimer

So, this is the usual “stuff” how this is my personal blog, and the views expressed on these pages are mine alone and not those of my employer.

About Derivco

Mon, 01 Jan 0001 00:00:00 +0000

Derivco

Derivco is a leading developer of software for the online gaming industry. That doesn’t mean however it is a gaming company; no, it is a technology company. In order to stay competitive we have to be on the forefront of technology, and we are continuously researching new technology as well as pushing the edge of existing technology.

We probably have one of the biggest install base of Microsoft SQL Server’s with a couple of hundred SQL Server boxes in our datacenters across the world. Each server box runs several SQL Server instances (SQL 2014 and SQL 2016), each with multiple databases. We are at the moment researching SQL Server v.Next, and will without a doubt come up with new and innovative solutions using it, in order to make our products even better.

About Me

Mon, 01 Jan 0001 00:00:00 +0000

About Me

I am a Microsoft Data Platform MVP, and I work as a Software Architect Lead at Derivco in Durban, South Africa. Me and the architects in my team design and develop highly performant iGaming systems.

Before joining Derivco, I worked as a researcher/instructor at Developmentor, responsible for the database curriculum.

I have been been working with SQL Server since SQL Server 4.2 and co-authored the first SQL Server 2005 book for developers: “A First Look at SQL 2005 for Developers”. Apart from being a SQL Server “nerd”, I am now interested in real-time streaming data, stream processing, and the Cloud and Big Data.

Blog Post Series

Mon, 01 Jan 0001 00:00:00 +0000

Blog Post Series

Sometimes my blog-posts turns into “mini series” and this is the page where you find what series I have.

Series

SQL Server R Services Series: A fairly long and finished series of posts covering a lot of the internal workings of SQL Server R Services.
Install R Packages in SQL Server ML Services: In this series we look at how to install R packages in SQL Server Machine Learning Services.
sp_execute_external_script and SQL Server Compute Context: Some posts about sp_execute_external_script & SQL Server Compute Context.
SQL Server 2019 Extensibility Framework & Java: A series about the Java Extensions in SQL Server 2019; how we can execute Java code from inside SQL Server.

Building an Event Management System with Claude Code

Mon, 01 Jan 0001 00:00:00 +0000

Building an Event Management System with Claude Code

This is the “Table of Content” for the Building an Event Management System with Claude Code series detailing my journey to build an AI-driven contact and event management system.

Table of Content

Building an Event Management System with Claude Code: Installing Claude Code: We introduced the project and looked at the steps to install Claude Code on both macOS as well as Windows. (Edited: 2025-12-20)
Building an Event Management System with Claude Code: Part 2 - IDE Integration: Learn VS Code integration with Claude Code for AI-assisted development. Includes WSL setup, CLAUDE.md configuration, and building your first API endpoints. (Edited: 2025-12-20)
Building an Event Management System with Claude Code: Part 3 - Architecting an AI-Native System: Part 3 shows how Claude Code helps architect an event management system using MCP servers, PostgreSQL, and natural language interfaces.
Building an Event Management System with Claude Code: Part 4 - Database Setup and First Conversations: In Part 4 we build production-ready database infrastructure using Claude Code. Learn PostgreSQL setup on Neon, Postgres MCP Pro configuration, and AI-assisted troubleshooting.
Building an Event Management System with Claude Code: Part 5 - Database Schema and Natural Language Queries: In Part 5 you transform database development with Claude Code and Postgres MCP Pro. Design schemas, generate test data, and run complex queries—all through conversation.
Building an Event Management System with Claude Code: Part 5.5 - Schema Refinement: When Real Data Reveals the Truth: In this post we see how a database schema design mistake turned lesson: How uploading real CSV files to Claude Code revealed missing fields and enabled a 45-minute schema refinement migration.
Building an Event Management System with Claude Code: Part 6 - Architecture and Design of a Custom Import MCP Server: Design a custom Import MCP Server with Claude Code for AI-native data imports. Learn why MCP servers are isolated, architect 7 self-contained tools, and transform CSV hell into conversational workflows. Complete architecture guide.
Building an Event Management System with Claude Code: Part 7 - Implementing the Import MCP Server: Part 7: Build an Import MCP Server with Claude Code; 3 hours vs 16 traditional. FastMCP implementation, validation tools, database troubleshooting, and real AI-native development lessons.
Building an Event Management System with Claude Code: Part 8 - Implementing Import Functions: Part 8: Discover how Claude Code implemented import functions, reviewed its own code, and transformed a 20-minute process into 2 seconds. AI-native development in action.
Building an Event Management System with Claude Code: Part 9 - Autonomous Implementation: Letting Claude Code Loose: Part 9: Test autonomous Claude Code by implementing 3 import functions with minimal guidance. Compare results with conversational development. Includes YOLO mode, fixing GitHub issues, and lessons on when to let AI work independently.
Building an Event Management System with Claude Code: Part 10 - Email Infrastructure with Brevo MCP: Part 10: Integrate Brevo MCP for transactional email sending. Set up email infrastructure, create templates, and send your first conversational emails—all without maintaining a separate contact database.
Building an Event Management System with Claude Code: Part 11 - Campaign Tracking & Batch Sending: Part 11: Add campaign tracking and batch sending to your AI-native event system. Design tracking tables conversationally, send personalised campaigns to multiple recipients, and prevent duplicate emails—all through Claude Code and Claude Desktop.

Data Saturday Durban

Mon, 01 Jan 0001 00:00:00 +0000

Data Saturday Durban

Schedule Data Saturday Durban 2023

The schedule below is preliminary, in that the rooms will change.

Schedule

Posts Archive

Mon, 01 Jan 0001 00:00:00 +0000

Presentations

Mon, 01 Jan 0001 00:00:00 +0000

Here you find slide decks from some of my conference talks.

SQL Saturday, 2019

A Lap Around SQL Server 2019 Big Data Cluster: The new release of SQL Server; SQL Server 2019 includes Apache Spark and Hadoop Distributed File System (HDFS) for scalable compute and storage. This new architecture that combines together the SQL Server database engine, Spark, and HDFS into a unified data platform is called a “big data cluster.” This session gives you an overview of what a SQL Server Big Data Cluster is, and what you can do with it.
What is the PiRate, Snake, and Cup of Coffee Doing in My Database?: In this session we look at the SQL Server Extensibility Framework, and we see how we can call out to external languages from inside SQL Server. We look at R, Python and Java, and what we can do from SQL Server having access to those languages.

SQL Server 2019 Extensibility Framework & Java

Mon, 01 Jan 0001 00:00:00 +0000

SQL Server 2019 Extensibility Framework & Java

This is a collection of posts about the Java language extensions introduced in SQL Server 2019. In other words, how we can execute Java code from inside SQL Server 2019.

Table of Content

SQL Server 2019 Extensibility Framework & Java - Hello World: We looked at installing and enabling the Java extension, as well as some very basic Java code.
SQL Server 2019 Extensibility Framework & Java - Passing Data: In this post, we discussed what is required to pass data back and forth between SQL Server and Java.
SQL Server 2019 Extensibility Framework & Java - Null Values: This, the Null Values, post is a follow up to the Passing Data post, and we look at how to handle null values in data passed to Java.
SQL Server 2019 Extensibility Framework & Java - Misc. “Stuff”: We look at SQL Server 2019 Java Extensions, and Java packages, the CLASSPATH and JAR files.
SQL Server 2019, Java & External Libraries - I: Part I of a couple of posts where we look at how we can deploy Java code to the database, so it can be loaded from there.
SQL Server 2019, Java & External Libraries - II: Part II of a couple of posts where we look at how we can deploy Java code to the database, so it can be loaded from there. In this post we look at how to use CREATE EXTERNAL LIBRARY to deploy Java code without having access to SQL Server’s filesystem.
Java & SQL Server 2019 Extensibility Framework: The Sequel. We look at the implications of the introduction of the Java Language Extension SDK.
SQL Server 2019 Extensibility Framework & External Languages. We look at SQL Server 2019 Extensibility Framework and Language Extensions.
SQL Server 2019 CTP3.2 & Java. SQL Server 2019 CTP 3.2 and Azule OpenJDK.
SQL Server 2019 & Java Null Handling: Take Two. We take a second look at how to handle null values in datasets being passed to and from Java code.
SQL Server 2019 & Java: Parameters. We discuss how to handle parameters between SQL Server and Java, as well as discussing the support of output parameters.