Mostly Maths

2019#20,21,22 Readings of the week

2019-07-28T18:39:00.000+02:00

I’ve been pretty busy lately, and although reading doesn’t stop, my writing sometimes takes a hiatus.

NOTE: The themes are varied, and some links below are affiliate links. Data engineering, adtech, history, apple. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Photo by Benjamin Wong on Unsplash

The Launch

And it’s not about a technology product.

How I practice at what I do

Working on improvement is a full time job, that you usually need to take during your free time.

The Good Employee, a story about how you can explain modern companies with graph theory

This was a weird read. But now I wonder if I can use Prim’s algorithm to improve my decision making?

The Washington Post is preparing for post-cookie ad targeting

The cookiepocalypse is coming, and if you work in adtech you should be thinking about it.

Dry Stone Walls – Principles of structurally sound construction

Around my hometown (L’Arboç) there’s an abundance of dry stone wall huts, presumedly from around 18th century. Now I know how to build one.

Book Review: Impro by Keith Johnstone

You may be aware that I read anything that can improve me in any way. So, Impro is now on my currently reading.

Introducing Dagster

Anything that is not Airflow is a win on my book, but I’m not super-thrilled about how Dagster works either.

Here Dragons Abound: Iskloft Mountain Style

I have always loved maps and map-drawing, so how to draw fantasy maps is an interesting enough subject on its own.

Why Category Theory Matters

I’m ramping up my category theory knowledge lately (and spreading through to sheaves and maybe schemes).

Roger Federer as Religious Experience

We’ll miss him (and Rafa Nadal, and Novak Djokovic) when he retires.

Did Functional Programming get it wrong? - Noteworthy

I’ve been thinking about “spreadsheets” as in “data and code mixed” lately, specially in terms of category theory, so this post was... close to mind-reading.

If you’re playing EVE online you basically already have an MBA

I’ve never been drawn to that game, but given the amount of hours some people give it, it makes total sense.

Haskell - An Experience Summary

As part of my categorification, I’m back again at learning Haskell (using Haskell Programming from First Principles). So I’m reading anything reasonably non-technical about Haskell.

'These kids are ticking time bombs' -- The threat of youth basketball

The amount of bone and tissue stress a young basketball player has gone through is astonishing.

‘I don’t see jeans in my future’: the people who wear complete historical dress – every day

I like that 30s look. I even have a similar hat.

Preserving Laptop Stickers on MacBooks - Graham Stevens

Seems too late for my batch of stickers, but I always pick 2 of a kind if possible, now I’m thinking where should I show the duplicates.

For 40 Years, Crashing Trains Was One of America’s Favourite Pastimes

Watching things explode has always been fun?

Pearls Before Breakfast

Would you recognize genius if it was in front of you?

📚 Finite and Infinite Games

I’ve seen many people recommend this book... And I didn’t get it. It was interesting, but not so much as I expected.

🎥 Approaching the Yoneda Lemma

Some ways of seeing the Yoneda lemma. I’m still trying to wrap my head categorically, without forgetting anything. I want to be free.

Newsletter?

These weekly posts are also available as a newsletter. These days (since RSS went into limbo) most of my regular information comes from several newsletters I’m subscribed to, instead of me going directly to a blog. If this is also your case, subscribe by clicking here.

2019#19 Readings of the week

2019-07-08T21:52:00.001+02:00

NOTE: The themes are varied, and some links below are affiliate links. History, haskell. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here. You can also get these as a weekly newsletter by subscribing here.

Photo by Chris Marquardt on Unsplash

When Pepsi Had a Navy

From the title, I thought it was related to the “Sugar Wars”, but the reality is weirder.

Your Work Peak Is Earlier Than You Think

Mildly depressing. A quote I found interesting:

Careers that rely primarily on fluid intelligence tend to peak early, while those that use more crystallized intelligence peak later. For example, Dean Keith Simonton has found that poets—highly fluid in their creativity—tend to have produced half their lifetime creative output by age 40 or so. Historians—who rely on a crystallized stock of knowledge—don’t reach this milestone until about 60.

Why I (as of June 22 2019) think Haskell is the best general purpose language (as of June 22 2019)

Although I can somewhat read Haskell code, I still can’t write much. So, can’t really agree... Yet.

Easy Parsing with Parser Combinators

I have a small project I wrote with AWK that I want to rewrite “properly” in Scala, and I’d need to do some real parsing (not the ad-hoc parsing you always end up writing in AWK), so learning to use FastParse is worth it. Also Li is a very clear writer.

Sam and Max Hit the Road Development History

So many fond memories from this game, I’m getting it again from GOG any day soon. Also this post made me look up where my copy of the ??? was.

Comonads for Life

Implementing the Game of Life using comonads (the dual of monads)

Real-world dynamic programming: seam carving

I have always been fascinated by seam carving. It’s so amazing

The surprising story of the Basque language

Uhm, Armenia is not even close.

History Will Not Be Kind to Jony Ive

Flexgate, antennagate, keyboardgate. There have been many f-ups that can be traced back to looking for a specific design. I hope Apple gets back to top-design and top-usability now.

'It's getting warmer, wetter, wilder': the Arctic town heating faster than anywhere

Longyearbyen (Svalbard’s capital) is in danger.

Reasons of State: Why Didn't Denmark Sell Greenland?

An analysis by Gwern Branwen about why Denmark didn’t sell Greenland to the US after WWII (they wanted it as a base).

'Like a military operation': restoration of Rembrandt's Night Watch begins

The weight of the painting will surprise you.

📚 Sprint

The nitty gritty details of how to run design sprints (a Google practice). Interesting.

📚 Diaspora

This is the second Egan book I read (after Clockwork Rocket) and I found Diaspora much better, although with a weak ending and a somewhat erratic plot. The book has some strong Star maker vibes. By the way, when adding the link above I thought the narrator for Audible was this Adam Epstein (a pretty darn good mathematician I know)

📚 Thinking in bets

The general idea is sound, but it’s one of those books that could be written as a long blog post or short essay.

📚 Set your voice free

Exercises to improve your voice. I have recently started to do the daily warm-ups, but so far have felt no difference. Of course, other people should be the ones to tell me.

🎥 Wardley mapping interview

I was recently interviewed by John Grant and Ben Mosior as part of a set of interviews leading up to MapCamp London 2019. If you are interested in my thoughts about mapping the tech landscape, I wrote my train of thought in this blog post as well.

Newsletter?

A (section) of a map of the data engineering space

2019-07-07T13:34:00.002+02:00

Table of contents

Problem I'm trying to solve
Looking for axis metrics
The map
Questions and further ideas

The map and problem described here were part of my presentation Mapping as a tool for thought, and mentioned in my interview with John Grant and Ben Mosior (to appear sometime soon in the Wardley Maps community youtube channel). I’m looking for ideas on how to make this map easier to understand and useful, so I posted it to the Wardley Maps Community forums requesting comments.

Problem I'm trying to solve

As a consultant (and as someone always trying to keep up with technology) I'm interested in being able to answer three questions of a language or technology:

How easy it is to find work/workers in the area right now?
How hard it is to learn?
How easy is it going to be to find work/workers in the area once I'm proficient enough?

Also, I need to know the relationship between any of them.

This problem has been on the back of my mind for many years, and upon getting proficient with Wardley mapping, I thought I could just map it. Of course, it's not a Wardley map, because the axes are completely different, but having anchors and movement, it is spiritually close enough for me.

In the diagram I will show in a while, I have placed technologies I am proficient in, currently learning, or looking forward to learn. In all of them I am at least a beginner in the sense that I know what they are used for and have done some minor PoC (proof of concept) to get an idea of how the work.

Looking for axis metrics

There are several ways you can address this technology space to answer the questions above. The first and easiest metric, and one of the axes I have used (Y axis) is Difficulty. Since I know something about each technology I can rank them on Difficulty, at least in relationship with each other. It's only a qualitative metric of difficulty, because in any new technology there are always unknown unknowns. There is no movement assumed in this axis, because Difficulty is supposed to be consistent throughout (leaving aside the more you know the easier it is to learn as well as familiarity with similar concepts that offset that, you could think of these two concepts as doctrine in such a map).

One natural metric for the other axis could be popularity, as measured by any of the several programming language/framework popularity rankings. You can use popularity as one of the axes, and use arrows to indicate whether it is growing in popularity or diminishing in popularity. But, popularity alone does not help in answering questions 1 and 3. What we need is knowing how large the market for this technology is, and how large the pool of workers in this market is. Could we use either as an axis?

If we were to use market size as X axis, we would probably have large markets on the right and small markets on the left, we would likely use arrows to indicate growing markets and shrinking markets. But, market size alone won't answer the questions either. A small thought experiment: imagine we have the largest market possible, it is growing... but the pool of workers for that technology is 2x the size of the market. It would be impossible to find work there (but, would be easy to find workers). This suggests that a possible correct for the X axis is market saturation, i.e. the ratio of market size with worker pool. Highly saturated markets are uninteresting to look for work, but are very interesting if you are starting a company: you'd have an easy time finding hires for that technology. Market saturation is related to flows (as in Systems thinking flow analysis) of users into a variable-sized container.

Markets become saturated in one of 3 ways:

Market is growing, but the pool of workers grows faster
Market is stagnating with a growing pool of workers
Market is shrinking faster than the pool of workers is shrinking

Cases 1 and 2 are the most usual (I'd put Python as type 1 and Java as type 2), but 3 is an interesting situation: it would indicate a technology that has died in favour of another. Workers in that pool have retrained in the new technology, but are still in pool for the dying technology (for instance, traditional MapReduce).

Markets become desaturated in one of 3 ways as well:

Market is growing faster than the pool of workers is growing
Market is stagnating with a shrinking pool of workers
Market is shrinking slower than the pool of workers is shrinking

Likewise, here, cases 1 and 3 may be the most interesting. I'd put Kubernetes in 1, and Scala in either 2 or 3. Please note that not only are these subjective evaluations, but are not meant to be negative. Scala has been my preferred language for a long while.

We can represent all these with slanted arrows: slant up covers growing markets, slant down covers shrinking markets. And then the arrow points left or right whether it is becoming saturated or desaturated.

With market saturation as an axis and arrows to indicate evolution of a technology, we can now almost answer questions 1 and 3. There is still the question of market size, which can't be represented with such a relative measure. Although we could add circles to represent current market size, that would bring an already weird map to more weirdness. Hence, market size is not considered.

The map

Here you can see the map. Before I get a bit into the topography of it, let me quickly define some of the technologies:

APL: programming language based on non-ASCII symbols designed in the 70s. Not extensively used, but in use
Airflow: workflow scheduler for data operations
Akka: Scala/JVM actor framework used for reactive programming, clustering, stream processing, etc
Arrow: cross-language platform for in-memory data. Used in Spark, Pandas, etc
Awk: special-purpose programming language designed for text processing
Beam: unified model for data processing pipelines. Can use Spark, Flink and others as execution engines
Dask: cluster-capable, library for parallel computation in Python.
Datafusion: rust-based, Arrow-powered in-memory data analytics
Docker: containerisation solution
FP: as in Functional Programming. Software development paradigm based on immutable state, among other things. Scala and Haskell are some the most mainstream languages for it
Flink: cluster computing framework for big data, stream focused
Forth: stack based, low level programming language. Not in common use.
FoundationDB: multi-model distributed NoSQL database, offering "build your own abstraction" capabilities
Go: statically typed, compiled programming language
Haskell: statically typed, purely functional programming language
Hive: data warehousing project over Hadoop, roughly based in "tables"
Kafka: cluster based stream processing platform (often used as a message bus) written in Scala
Kubernetes: container orchestration system for managing application deployment and scaling. Written in Go, depending (non-strictly) on Docker
Presto: distributed SQL engine for big data
Python: interpreted high level programming language, very extended in data science and engineering
Rust: memory safe, concurrency safe programming language. Has some functional capabilities
Scala: JVM based language offering strong typing and functional and OOP capabilities
Spark: cluster computing framework for big data, batch and stream (stronger in batch)

These cover a range of the data engineering space (Flink, Spark), as well as technologies I want to get better at and are close enough (Kubernetes, FoundationDB) and technologies I know but are not directly related (AWK, APL, Forth) and are used as anchors.

Anchors

To display relative positions, I needed to anchor some of the technologies. For instance, Haskell and APL set the bar for difficulty, with AWK setting the minimum, and Python and Forth set the extremes for saturation. Everything else is placed in relation with these.

Links and colours

In the map, I have used colours to distinguish languages, frameworks, libraries, containerisation and databases. Colour is not fundamental though, links are. Related technologies are linked: Spark is written in Scala, and can be used with Scala, Python and other languages. Hence, changes in market saturation for Spark indirectly affect market saturation for Scala.

Empirical map division

We can think of the map as divided in 2 areas vertically (high barrier to entry and low barrier to entry) and 3 areas horizontally (saturated, accessible, desaturated).

Quick overview

If you were a CTO, you'd probably be interested in:

Low barrier to entry, saturated market for high turnover positions (easier and cheaper to hire and train)
Accessible currently becoming saturated for more stable positions (becoming easier to hire in the future)
High barrier to entry, with growing markets for proof of concepts, exploration (exploring new technologies that may make an impact in the future)

A map like the one above can help make these decisions.
For deciding on future work, I'd be (as a consultant) interested in:

First, growing market, high barrier to entry for current learning. If the barrier to entry is too low and the market is lucrative enough any return on time investment will tend to 0 in the end otherwise.
Second, accessible with low-to-mid barrier to entry for imminent work opportunities.
Finally, to hedge bets on a long term plan, anything which is very low saturation, and where the market is unknown or may be in growth.

With this map (which is suited to my current knowledge and interests) I can definitely answer the question of where I should put my efforts right now.

Curiosities

Haskell and Rust raise interesting questions. Haskell has a very high barrier to entry, the market is not very large and might not be growing fast, but there are developers working in other languages (Scala, Rust, Kotlin, Go, even Python) that would love the opportunity to work in Haskell. This makes the Haskell job market actually saturated (or at least saturated if you consider worldwide market). Thus, starting a company focused on Haskell might not be as bad of an idea as it might sound. Similarly with Rust: Rust is growing as a side project language, the amount of developers familiar with the language is growing faster than the market and thus is an interesting target for a starting company.

We'd have Python on the other side: since it is taking over as the lingua franca of data science and engineering, and becoming one of the teaching languages at universities, the amount of developers with enough knowledge to become part of the work pool is growing faster than the market (even if the market for Python is growing at a fast pace). It makes it an ideal language for creating a company or a consultancy company (large pool of candidate workers), but not so interesting for being an independent consultant, since competition could be too large

Questions and further ideas

This is the approach and train of thought I have followed to trace these ideas, but it’s still a work in progress. I’d like to hear what you have to say about this: what would you change? What would you do different? I'm still unsure about using market saturation and arrows to show market and pool of workers behaviour, but I have not found anything easier to represent. Ideas? And here are some areas I’m unsure or where I have more questions.

What other approaches would you have taken to explore this questions?

There are probably many other ways you can take to approach these questions. What would be yours?

Do you think market size would need to be shown?

It could be shown with a circle (different sizes to be able to compare) below the arrow indicating behaviour of saturation but that could make the map way too complex. Any other ideas? Do you think it is that important, as long as saturation is taken into account?

Links between related technologies are a bit hazy

The links between technologies are a bit too abstract. An "increase" in "Python" moves "higher" Airflow, Spark, Dask and any related technologies... but in what sense? Popularity? Market share? Market saturation? I suspect the link is useful to see, and it is supposed to bring some dynamic/movement, but I'm still unsure how.

Flows

An interesting approach I didn't pursue is using flow maps. For each programming language, there is a set of flows into other languages. For instance, developers in Scala have a tendency to be interested in Kotlin, Rust and Haskell, with some making the jump as soon as market is able to absorb them (and for each of these flows we can assume there is a non-zero flow to the other side). Similarly, we'd have flows from Python to Go and Scala, from Go to Rust. These could inform on market trends and behaviours, but they are not only hard to show on a map (what would be the axes? what would be the anchors?) but also might not be interesting enough on their own. What do you think?

2019#18 Readings of the week

2019-06-25T20:01:00.002+02:00

Photo by Nik Shuliahin on Unsplash

NOTE: The themes are varied, and some links below are affiliate links. Software engineering, history, planning, data engineering. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Fresh look at mysterious Nasca lines in Peru

An analysis of what kind of birds they represented.

4 Simple Steps To Set-up Your WLM in AWS Redshift

Good suggestions on workload management and process queues for Redshift

Adventure Games and Eigenvalues

Finding dead ends in a game using Markov processes (instead of a formal language approach)

When pigs fly: optimising bytecode interpreters

Quite a meaty post about... well, optimising interpreters (incidentally, bytecode based)

See through words

Did you know metaphor design is a thing? Read this article for more.

Decision tables

Calling decision tables a formal method may be a stretch, but they can clarify your thinking. And that is one of the powerful things formal methods bring.

Monad Transformers aren’t hard!

No, they aren't, but your heap can suffer in Scala!

🎥 Wardley Maps Saved The Day - How Stack Overflow Enterprise automated all the things...

I’ve been into Wardley mapping for several months (even gave a presentation on Mapping at SoCraTes UK 2019) and I’m basically consuming any content related to it. This short video is a very good intro to Wardley mapping, by the way.

🎥 Cartoons are about how drawing and writing work together on the page

A quite funny video presentation by Tom Gauld

📚 The goal: a process of ongoing improvement

It’s like Sophie’s World but for Theory of Constraints (with a whiz of the Toyota Production System probably)

📚 Make your contacts count

Networking tips. It describes very directed ways of building a network and actually being useful in it, not just a parasite.

Newsletter?

2019#17 Readings of the week

2019-06-17T17:24:00.002+02:00

NOTE: This week is a bit light on technical content because I was attending Scala Days 2019 in Lausanne and I had enough with the talks.

NOTE: The themes are varied, and some links below are affiliate links. Software engineering, psychology, history. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Photo by Eduardo Romero on Unsplash

The mindfulness conspiracy

It’s a trap!

One Diagram To Mind Them All: Hyperspace in the 1970s

The beginnings and history of mind mapping, and Tony Buzan in particular.

A tale of lost WW2 uranium cubes shows why Germany’s nuclear program failed

Splitting your efforts won’t lead to faster splitting of the atom.

Exploring Domain-Driven Design at CircleCI

DDD is an interesting concept, and knowing how companies implement it is always good. This lacks a bit of more material though.

A Year’s Worth

An interesting way of visualizing a year’s worth of technical effort, by Kent Beck.

The Making of Lemmings

I enjoy a lot tales of the dark ages of gaming. If you haven’t, read The Making of Prince of Persia (you may enjoy the “prequel”, The Making of Karateka) or Masters of Doom).

Feynman’s Breakthrough, Disregard Others!

If you focus too much in what others do, you’ll lose your uniqueness.

📚 Tribe of mentors

A book by Tim Ferris, where he quick-interviews a lot of people, asking tips. It’s a kind of a not-amusing read, because it’s not designed to be “read”, I suspect.

📚 How to Draw Fantasy Art and RPG Maps: Step by Step Cartography for Gamers and Fans

I had a lot of fun with this one. Super-readable and easy to follow.

Newsletter?

2019#16 Readings of the week

2019-06-10T21:18:00.001+02:00

Photo by Roman Mager on Unsplash

NOTE: The themes are varied, and some links below are affiliate links. Wardley mapping, data engineering and big data, maths. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Meet the Money Whisperer to the Super-Rich N.B.A. Elite

It surely has to be an interesting job

Thought as a Technology

An essay by Michael Nielsen. Very recommended.

A tale of lost WW2 uranium cubes shows why Germany’s nuclear program failed

Curio

Counting to infinity at compile time – The Startup – Medium

You can do really weird things with Scala at compile time, and base 3 is pretty cool.

A visual introduction to Morse theory

I was surprised to see this posted on Hacker News, so decided to read it for the "good old times". Learnt something (since I never had a class on Morse theory, it was just one of those intriguing sections in the library)

The cutting-edge of cutting: How Japanese scissors have evolved

I have some weird scissors, but these go beyond weird.

A different kind of string theory: Antoni Gaudi

That idea was clearly genius.

Formally Specifying a Package Manager

By using Alloy. I really like Alloy.

Maker's Schedule, Manager's Schedule

I was talking with a friend earlier about a previous post in my Weekly Readings series (Do I truly want to become a manager?) and he hadn't read this one, so I'll share it here too.

Schema evolution in Avro, Protocol Buffers and Thrift

I kind of like the sound of how Avro handles schemas. Seems an efficient way, although seems... prone to possible disasters.

Improve Apache Spark write performance on Apache Parquet formats with the EMRFS S3-optimized committer

This is optional in EMR 5.19 and will be the default in 5.20 so we won't see much difference, except in performance.

Why Enthusiasm is a Bad Thing in a CTO

Hey, look a squirrel is 10x more dangerous when it's the CTO spotting squirrels.

🖥 Mapping as a Tool for Thought

A presentation I gave on Wardley mapping and "mapping" in general at SoCraTesUK 2019. I really enjoy this conference.

🎥 Skip the first three months of development for your next app

This ties very well with some of the Wardley mapping concepts.

Newsletter?

2019#15 Readings of the week

2019-06-04T20:34:00.001+02:00

Tom Gauld (for New Scientist, from his blog, sorry!)

NOTE: The themes are varied, and some links below are affiliate links. Data engineering, adtech, ZIO, Rust, writing, some miscellaneous stuff. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Hacking the Casio F-91W to Handle 1000+ PSI

This was an amusing find. I wonder where my F-91 is.

The Day You Became a Better Writer (2nd Look)

Top notch, short and sweet writing tips by Scott Adams (of Dilbert fame)

Rebuffing the Attack of the Clones

I’m guilty of unnecessary cloning. Shame, shame.

Generating geometric image placeholders client-side

Neat! I had written something similar in C a long time ago. It was slow! This is blazingly fast.

Behavioral Ad Targeting Not Paying Off for Publishers, Study Suggests

This is surprising. I’d expect CPM to be higher for high value programmatically bid users.

The Embroiderer's Story: Hand Skills

An unexpected consequence of the digitalisation of our leisure

No frills data warehousing with dbt

I’ve been considering this tool (the open source one, not the cloud offering, I have Airflow for that) for a while.

Why I'm Stepping Back from Scalaz (John De Goes)

A very well reasoned writing. I’m all for ZIO.

Apache Flink: Flux capacitor, huh? Temporal Tables and Joins in Streaming SQL

I don’t use Flink, but it’s there, staring at me.

Using Poetry in production

It sounds powerful in combination with my dear pyenv.

How I run a company with ADHD

I somehow can relate, I wonder why OH HEY A SQUIRREL

CQL: A categorical query language

The categorical hype rises. I’d like to see scheme-based data analysis at some point (that could be a thing at some point, schemes as in generalized algebraic manifolds)

📚 Baking with Kafka

An amusing book by cartoonist Tom Gauld.

🎥 Crossing the River by Feeing the Stones

I've been doing Wardley maps for close to a month, and they have been really influential in how I approach a lot of things. I will be giving a short introduction to them this weekend at SoCraTesUK 2019, if you are there say hello (and I'll post the slides soon)

AWKrdeck: Deckset to Reveal.js

I wrote a simple converter from Deckset style presentations to similarly-themed Reveal.js presentations using some AWK and Pandoc.

Newsletter?

2019#14 Readings of the week

2019-05-26T11:48:00.000+02:00

Photo by Darius Soodmand on Unsplash

NOTE: The themes are varied, and some links below are affiliate links. Data engineering, adtech, functional programing, formal specification. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Schema Management With Skeema

How SendGrid manages schema updates internally. A pity it’s focused on MySQL and family and I prefer Postgres

How to do hard things

I wasn’t aware, but this is exactly how I approach anything I don’t know how to do. Don’t miss this one.

The bullshit I had to go through while organizing a software conference

I’ve been an organiser, luckily didn’t encounter this. Crossing fingers, since we are preparing stuff at PyBCN.

The reason I am using Altair for most of my visualization in Python

I usually like having the possibility of maximum power and expressiveness... But eventually I just want to make easy things easy. I’ll try Altair next time I need to plot anything. Lately I’ve gone a lot to gnuplot, to be fair: nothing beats it to just plot a text file you have lying around.

Introducing Argo — A Container-Native Workflow Engine for Kubernetes

This is what is now being used at BitPhy, after I recommended them to... well, not use Airflow. To be fair they were considering Argo and Airflow, and given they are heavy on Kubernetes, Argo sounds a better fit.

Do I truly want to become a manager?

Six questions you should consider before thinking of making the leap out of the IC route. Thanks to CM for sharing it.

Ask HN: What overlooked class of tools should a self-taught programmer look into

You can find some suggestions in these answers.

Give meaning to 100 Billion events a day

How Teads leverages AWS Redshift.

Performant Functional Programming to the max with ZIO

You were wondering: which ZIO post is he going to share this week?

Meet Matt Calkins: Billionaire, Board Game God And Tech's Hidden Disruptor

A friend of mine is designing and producing a board game (I’ll share the Kickstarter when is ready), so this was a fun read. He’s not a billionaire though.

Open-sourcing the first OpenRTB Scala framework

I’m not sure what the performance of a RTB can be in Scala, but I’m definitely interested.

Zero Cost Abstractions

Rust is always sold as a zero-cost abstraction language. What does that exactly mean?

A novel data-compression technique for faster computer programs

Don’t know, this sounds pretty much what Blosc does.

🎥 Fast Data with Apache Ignite and Apache Spark

(this is an oldie) I tried Ignite+Spark around 1 year ago, and couldn’t get it to work properly (segmentation fault!). I’ll try again: it can open up a lot of things if it works as promised.

🎥 Thinking for Programmers

Leslie Lamport selling you the why of specifications. Very recommended, specially for people who think specifying is complex, long, unnecessary or anti-agile.

🎥 Live Coding with Rust and Actix

I was very impressed by this video (I watched it as “background”, not actively). Very focused, and all written in one go.

🎥 Solving every-day data problems with FoundationDB

FoundationDB is on my list of technologies to watch/follow and learn. Get a glimpse about why here.

📚 Soft Skills

It’s not a bad book, but is basically a summary of stuff I got from other places already. You can give it a go.

🎼 Time Out by the Dave Brubeck Quartet

Saw this recommended somewhere, and I’m liking it a lot (won’t remove Bill Evans as my favourite jazz pianist though)

Newsletter?

LSP for Python and Scala

2019-05-21T23:41:00.000+02:00

If you have have known me for any length of time you'll know I write mostly Python and Scala lately (Rust is getting into the mix slowly). And you should know, I am a heavy emacs user. I have been using emacs for close to 15 years, for the past 3 my emacs of choice has been spacemacs. I used to have a very long, customised and complex .emacs and with spacemacs I get a mostly-batteries-included package. That's nice after a while, and I also have gotten really proficient at using evil.

One problem of using emacs is the integration with some languages. If you write Scala, with IntelliJ you get super-fancy completion, refactoring, code analysis, jump-to-definition... Many goodies. In emacs, the best-in-class system used to be ensime. It worked, but it was not really supported for spacemacs (since I'm an old emacs user I could play around that), but the main issue was that my old MacBook was short on memory for running ensime and a lot more. So, I wrote most of my Scala code in hardcode mode. No completion, documentation or jump to definition.

This is why I learnt how to set up GNU global, jump to definition is just too handy. Luckily, the people at Scala Center not only are smart, but also try to improve developer experience, and had been working in a language server for Scala for a while, called metals. I got it working recently, and it's great. You get documentation on hover, error messages, jump to definition. Oh, I forgot to mention, the language server protocol is an invention from Microsoft to standardise how editors handle language completions and all that. They probably introduced it for Visual Studio Code (I actually use it from time to time, it has some remote pair programming capabilities I'll talk someday), but now it's extending across all editors.

After using LSP in emacs for Scala for a while I decided to set it up for Python as well, in preparation for our next PyBCN podcast, about tools we use. I was pretty happy with the completions I was getting, but semantic completions from a language server are usually better. So far, lsp with python is ok. Oh, you'll see screenshots at the end!

You'll need to install the language server. I usually have a high level Python environment with all my tools, for things I am just starting to work on:

pyenv virtualenv 3.7.1 tools
pyenv activate 3.7.1/envs/tools
pip install "python-language-server[all]" bpython mypy flake8

After this, some configuration is needed in emacs. Here you can find parts of my configuration, commented. These sit in my dotspacemacs/user-config

;; First come the configurations for Scala language server 
;; thingies. sbt is the Scala build system.

(use-package scala-mode
  :mode "\\.s\\(cala\\|bt\\)$")

(use-package sbt-mode
  :commands sbt-start sbt-command
  :config
  (substitute-key-definition
   'minibuffer-complete-word
   'self-insert-command
   minibuffer-local-completion-map))

;; This is the main mode for LSP
(use-package lsp-mode
  :init (setq lsp-prefer-flymake nil)
  :ensure t)

;; This makes imenu-lsp-minor-mode available. This minor mode 
;; will show a table of contents of methods, classes, variables. 
;; You can configure it to be on the left by using `configure`

(add-hook 'lsp-after-open-hook 'lsp-enable-imenu)

;; lsp-ui enables the fancy showing of documentation, error 
;; messages and type hints
(use-package lsp-ui
  :ensure t
  :config
  (setq lsp-ui-sideline-ignore-duplicate t)
  (add-hook 'lsp-mode-hook 'lsp-ui-mode))

;; company is the best autocompletion system for emacs (probably)
;; and this uses the language server to provide semantic completions

(use-package company-lsp
  :commands company-lsp
  :config
  (push 'company-lsp company-backends))

;; I use pyenv to handle my virtual environments, so when I enable  
;; pyenv in a Python buffer, it will trigger lsp. Otherwise, it 
;; will use the old systems (I think based on jedi)

(add-hook 'pyenv-mode-hook 'lsp)

;; Flycheck checks your code and helps show alerts from the linter
(use-package flycheck
  :init (global-flycheck-mode))

;; Show flake8 errors in lsp-ui
(defun lsp-set-cfg ()
  (let ((lsp-cfg `(:pyls (:configurationSources ("flake8")))))
    (lsp--set-configuration lsp-cfg)))

;; Activate that after lsp has started
(add-hook 'lsp-after-initialize-hook 'lsp-set-cfg)

;; I like proper fonts for documentation, in this case I use the 
;; Inter font. High legibility, small size

(add-hook 'lsp-ui-doc-frame-hook
          (lambda (frame _w)
            (set-face-attribute 'default frame 

                 :font "Inter" 

                 :height 140)))

;; Configure lsp-scala after Scala file has been opened
(use-package lsp-scala
  :after scala-mode
  :demand t
  :hook (scala-mode . lsp))

You can see how it looks now.

Traditional completion (non-LSP)

LSP-powered completion. Way more information!

Fancy inlined documentation

Sadly, inlined documentation doesn't look as good as it should: compare with Scala with metals and lsp-scala:

LSP mode in Scala with metals

If you've gotten this far, I share my most interesting weekly readings in tag here . You can also get these as a weekly newsletter by subscribing here.

2019#13 Readings of the week

2019-05-19T19:21:00.001+02:00

NOTE: The themes are varied, and some links below are affiliate links. Functional programming, adtech, history. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Photo by Mgg Vitchakorn on Unsplash

Ancient Chinese Buildings Are Held Together With Rice

As everyone who has prepared sticky rice and forgotten to clean the pan quickly knows.

'I'd Have These Extremely Graphic Dreams': What It's Like To Work On Ultra-Violent Games

Fatality burnout.

The Artistry of China’s Ivory Puzzle Balls

I’ve seen one from really close and they are amazing

Bristol academic cracks Voynich code

There was a rebuttal of this approach close to 2 years ago, but it still sounds interesting. Also, I’d expect phys.org to be somewhat trustable.

Technical Debt? Probably not.

Pay it off. Or not.

Scala's Future.successful: Do Not Block Your Future Success

Beware the future!

The Functional Scala Concurrency Challenge

Some of the solutions look neat, and there is something to be learnt in the diversity.

Technical overview of ads viewability measurement methods

I have always been interested into how ad viewability (if an ad is seen by a user or not) works. Now I know.

GraalVM installation and setup on macOS

GraalVM is a new Java runtime and compiler, which is somewhat faster than the normal JDK, and offers some really fancy cross-language options. I had been using RC9 or 10. It was time to update. I usually use Enterprise Edition for anything local, but can switch per-terminal with several aliases I have (j8, j11, jgree and jgrce) in case something breaks or I want to try another.

Caire - a content aware image resize library

Seam carving is so cool. I wrote a cropping system once, using PILs Haar cascade based object detection. It works pretty well for automated creation of ads, but seam carving is way better for almost all other cases.

🎥 Flare - Optimizing Apache Spark for Scale-Up Architectures and Medium-Size Data

I’m scared this will become a closed source, or business-on-top. Code is not available, but results are awesome. Luckily, the papers are (why the code is not as part of the papers is another question).

🎥 helm-edit

I had used multiple-cursors (and expand-region) in emacs many times, although I had a hard time making it work properly in spacemacs for a while. helm-edit works better already in spacemacs, so, big win!

🎥 Understanding Query Plans and Spark UIs

I knew everything in this video already, but it covers lots. Give it a look.

🎥 What the ƒ is a monad

Although focused on Java, it’s very well explained for any language. Of course, if you have do syntax or for comprehensions, better.

📚 The Great Mental Models

The audiobook by Shane Parrish, of FS Blog. It was good, but I expected something longer.

📚 The art of thinking in systems

If you have read Donna Meadows Thinking in Systems, this won’t give you anything new. And I’d rather recommend TiS.

Newsletter?

2019#12 Readings of the week

2019-05-13T00:31:00.000+02:00

Photo by Zsolt Palatinus on Unsplash

NOTE: The themes are varied, and some links below are affiliate links. Big data and data engineering, metallurgy, Rust (not the metallurgy-related rust). Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Testing Incrementally with ZIO Environment

I’ve been reading a lot about ZIO lately, even to the point of writing a few simple thinks with the help of a more advanced Scala friend. As I’ve mentioned before, it looks solid

Medieval Africans Had a Unique Process for Purifying Gold With Glass

No plans on smelting anything soon, but this was interesting.

Rust Runtime

The proposed approaches for async-await in Rust are shaping up.

Moving from Ruby to Rust (at Deliveroo)

It should surprise no-one, except maybe hardcore rubyists that moving from Ruby to Rust yielded terrific speed-ups. Something similar should happen in Python, though.

Zero-cost futures in Rust

One of the core tenets of Rust is having zero-cost abstractions (as in, being as close to manually written). Here you can read how futures fit that goal.

A quick look at trait objects in Rust

Traits will be familiar to Scala or even Go or Java developers, but there are a few gotchas/differences you need to be aware.

Druid Design

A friend of mine has used Apache Druid before, and has been really happy with its performance, so I’ve been looking into possible use cases to road-test it.

Make your own GeoIP API (in Python)

I’ve linked before to GeoIP approaches, this one is interesting as long as you only want country-level resolution, I usually need city or better.

An in-depth look at the HBase architecture

Similarly to Druid, I’ve been checking several things to speed up some areas. Apache HBase might be one of them, and I wanted to see how it works under the hood, I love these details

🎥 ZIO Schedule

As mentioned above, ZIO is shaping up nicely, and the scheduling helpers presented here are one of the selling points, depending on what you do.

🎥 Pure functional programming in Excel

Duh... but... to be fair... there is a point in here.

🎥 Gradual typing of production applications

Incremental approach suggestions for typing in Python, at Facebook (Instagram to be precise)

🎥 Effects as Data

Everything here should sound familiar if you are anything into functional programming

Newsletter?

gtags (GNU global) in emacs for Scala

2019-05-09T17:27:00.000+02:00

Photo by thom masat on Unsplash

As you may know, I’m a heavy emacs user, and a frequent Scala developer. Scala tooling for emacs was restricted to mostly ensime until recently. Although ensime is an excellent piece of software, it made my old Macbook suffer a lot (it only had 8gb of RAM). So, most of the time I just went hardcore mode developer, and worked with no autocompletion, no jump to definition, no-nothing. A pervasive use of ripgrep and good memory were sometimes enough, but I was envious of many things I could see in my colleagues using IntelliJ. Of course, switching editors was not an option.

I looked up what solutions were available, and the only option that seemed good enough was using the classic ctags/etags/gtags I hadn’t used since my C days.

Having a tags implementation can bring an almost-decent jump-to-definition essentially for free: gtags (and the others) work by pre-analysing the data and generating symbol maps which are usually stored as files, the overhead in speed and memory is minimal.

Installing it can get somewhat unintuitive (specially on Mac), since you need very specific settings for it to work with emacs and especially, Scala code.

Start by installing GNU global:

brew install global --with-pygments --with-ctags --with-exuberant-ctags

The key parts is having pygments.
Aside from this you will need to export the following environment variables:

GTAGSCONF=/usr/local/share/gtags/gtags.conf
GTAGSLABEL=pygments

Finally, you need to install (or activate) ggtags-mode. If you use spacemacs, you only need to activate the gtags layer.
All this has become moot as soon as metals has reached a usable state: now you can have a really fast language server for Scala, written in Scala with low memory overhead.

2019-11 Readings of the week

2019-04-28T17:39:00.000+02:00

NOTE: The themes are varied, and some links below are affiliate links. Software engineering, Spark, history, python. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Testing and debugging Apache Airflow

We've been using Airflow for almost a year (on my suggestion). I'm not super-happy with it, and testing it is one of the pain points. Knowing how to do it is half the battle.

Mind control, levitation and no pain: the race to find a superman in sports

You must lose

I can't Do Anything for Fun Anymore; Every Hobby Is an Attempt to Make Money

You've probably felt the same

Databricks open-sources Delta Lake to make data lakes more reliable

I'm looking forward checking the code for this (should be equivalent to Databricks' Delta product)

Tolkien was right: Scholars conclude Beowulf likely the work of single author

In off the moors, down through the mist beams, god-cursed Grendel came greedily loping.

Don’t inherit Python built-in dict type

Remember kids, what you want to do is either Mapping (read-only) or MutableMapping abstract base classes from collections.abc

How to Integrate Your Stuff into Spark SQL

A pity you need a custom build of Spark in this case (and fighting with EMR to install it)

Using Direct Writes to Significantly Increase the Performance of Spark Workloads

If you happen to use Qubole and Hive, this can be a winner.

Low-memory symbol indexing with bloom filters

I love bloom filters, and have been using the metals Scala language server for a while now in emacs (spacemacs).

CONSTANT Anti-Patterns : tlaplus

That's no constant

Why software projects take longer than you think -a statistical model

We all know we are bad at estimates as humans, mathematical models just can confirm this

How to knit a sports car with carbon fibre - The new black arts of manufacturing

Rapid prototyping and construction with carbon fibre can change a lot of things in manufacturing

🎥 Build a roguelike in Rust

Using libtcod. Also a blogpost series available here.

Newsletter?

2019-10 Readings of the week

2019-04-18T11:39:00.002+02:00

NOTE: The themes are varied, and some links below are affiliate links. Software engineering, psychology, generative art. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

If you are wondering why the list is shorter this week (and is so late) is because last week I said I was going to play Stardew Valley, and I complied. Oh, I also played a bit with creating fun models with Alloy, maybe I'll explain something soon.

Could very well be named
Autumn Rhythm, Letter C

The Day Feynman Worked Out Black-Hole Radiation on My Blackboard

The way Richard Feynman found interesting stuff and just had a go (have you read Surely you’re joking, Mr Feynman?) is astonishing.

How to Set Up VSCode for Remote Pair Programming

I’ve been researching better ways to help my coworkers while I’m remote. This VS Code setup looks interesting, although I’ll need to get an emacs/vim plugin if I am to use it.

SIGBOVIK

If you run OCR on paint splashes, you get mostly valid Perl code.

Multi-Scale Truchet Patterns

I always enjoy a good generative art finding, and Truchet tiles have always claimed a spot in my heart (I loved Cameron Browne’s Che game, even if it's a bit thin after a while)

How Mental Models Unlock Your Thinking

I follow Shane’s podcast (and newsletter) regularly, so I’ve been looking forward the release of his audiobook. Now, it’s here, and I got it already.

Newsletter?

2019-9 Readings of the week (x4)

2019-04-07T19:37:00.000+02:00

You know how you slip once on a habit and everything goes crazy? Well, I’ve been 4 weeks without writing these, so here’s the accumulated reading from 4 weeks. Because, even if I don’t write it, I read a lot anyway. Also, there’s lot of interesting content this “week”.

NOTE: The themes are varied, and some links below are affiliate links. This edition contains Rust, food, Scala, functional programming, psychology, Python, likes to nice images, and a bit on procedural generation. Also the books I read these days. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Stardew Valley and real life labour

I bought Stardew Valley for iOS during Christmas, after hearing how much Mike Hurley (from the Cortex podcast, and all around the Relay.fm podcasting network) likes it. But, still haven’t played, maybe this week? (although, Cultist Simulator appeared on iOS last week, and it’s very engaging, specially on an iPad Pro with a Pencil).

📚 Superforecasting

I’m undecided on this book. The premise was excellent, the analysis as well, but it was lacking a bit on the “how” area of superforecasting.

To Be Lucky, Believe That You Are

There is something to this, also, more on luck further down. I’m a lifetime subscriber to Nautilus, and you should consider becoming one. Their print magazines are beautiful, and the articles always insightful.

What Working At Stripe Has Been Like

Anything Patrick McKenzie (patio11) writes is worth reading. Don’t miss his Salary Negotiation post.

Cloud Management with Prolog

Prolog, and its query ability sound ideal for a lot of scenarios. This could very well be one of them, although dumping to SQL could be more useful when working with a team.

Addition Font

How to exploit the depths of font description languages to compute. Also has a minor proof on how font rendering (removing some implementation blocks to avoid overflows in font engines) is Turing complete.

Twitter thread about immutability in Python by Yury Selivanov

Some ways you can get pretty much immutability. If you are a functional-ish programmer you’ll value this.

White gold: the unstoppable rise of alternative milks

I like the occasional oat milk (I like the taste of oats) or a specific brand of Catalan walnut milk (pairs perfectly with coffee)

📚 Bad blood

To be fair, I got the audiobook version, but it is thrilling. I listened to it in one weekend, and I didn’t finish earlier because AirPods need recharging.

Apache Spark 2.4.0 features - array and higher-order functions

Aside from the move to Scala 2.12, this is for me one of the most significant changes to Spark. Although I haven’t used these yet, there have been several moments in the past where I needed mapping capabilities over arrays and had to resort to a UDF. They are not hot paths, but knowing how UDFs behave, I’d get a decent speed up of these parts by moving to a HOF.

Faster File Distribution with HDFS and S3

How to load your data into HDFS is always a key question. Pre-load it? Load it directly with a specific (say, spark.read) construct from S3? Here there are some benchmarks of what is more effective, at least for Presto. Results probably extend to Spark et al, since being based over HDFS is the key part.

Mumble: the open source VoIP solution

I learnt about Mumble and the Murmur server a few weeks ago. They are a system for very low latency, low bandwidth audio-only (optionally a bit of text) communication. I wonder if it can help me work remotely more effectively (helping with pairing, or with random “quick talk” while avoiding the overhead of a video call under Slack, Skype or Meet). This Hacker News comment seems to cast it in the correct light.

La belleza matemática de la naturaleza, en ilustraciones de proporciones áureas

Note this is in Spanish. Served as an excuse to find out about Rafael Araujo and his books, The Golden Ratio: The Divine Beauty of Mathematics (with text by Gary Meisner) and Golden Ratio Colouring Book. I have now some of his drawings as backgrounds of my devices. They are gorgeous.

Barcelona from Above (set of pictures by Márton Mogyorósy on Behance)

These were my previous device backgrounds. You can also follow the author on Instagram (you can also follow me but I’m not that interesting)

Space Colony Art from the 1970s – The Public Domain Review

The 70s were special

Laws for Communicating Parallel Processes

A classic paper by Hewitt, the seminal work on actor systems.

Roguelike Tutorial in Rust + tcod

I enjoy playing ASCII based roguelikes from time to time (DCSS, CDDA or Dwarf Fortress), writing one would be cool. And yet another excuse/way to learn more Rust.

Fast and easy high resolution fractals with a pixel shader

I stopped generating fractal stuff before using shaders for it was common, so I never got to learn it. Now, I want it back. It was so much fun.

Python Data Landscape Diagram

A super-thorough diagram showing the data analysis library landscape in Python, with some words on each technology/library.

How the UK lost the Brexit battle

From Politico. It is super-long, but well worth the read if you are anywhere related to Brexit. This could easily be described as how not to manage your negotiation deals.

Procedural Generation with Wave Function Collapse

Like fractals, procedural generation is something I have always enjoyed (as a concept, as a result and as a programming problem). WFC is a good approach for generating maps, images and anything with “rules”

Rust Lifetimes for the Uninitialised

A thorough introduction to how Rust lifetimes work and how to use them

Controlling the Spice, Part 3: Westwood’s Dune

A short history about how Dune II came to be, and to appear almost at same time than Dune. Also, how Westwood (and Dune II) shaped real time strategy from then on.

Python Patterns

An analysis of some common patterns (à la Gang of Four, patterns) when applied to Python

The Unexpected Philosophical Depths of the Clicker Game Universal Paperclips

In the clicker realm, I much prefer SpacePlan (or A Dark Room, but that’s not 100% clicker), but UC was interesting.

Spread the word: the rise of craft butter – and how to make your own

Craft beer, craft bread, craft butter. It’s hard to come with good quality cream in Spain, though.

📚 Platt’s How to Draw Cartoons

A very short and sweet book on drawing cartoonish people. Highly recommended. And I found out the author lives in Hay-on-Wye, given the amount of times we’ve been there I wonder if we’ve crossed our paths.

Oh, shit, git!

If your local git guru is away, this can save the day.

UDFs vs Map vs Custom Spark-Native Functions – Farox2q – Medium

Some benchmarks and code comparisons between UDFs, map and custom spark functions. Sadly, you can’t implement the latter without using a custom build of Spark, and good luck deploying that to EMR...

M-16: A Bureaucratic Horror Story

The surprisingly interesting story about a weapon, the M-16, and how a lot of bad decisions happened because “committees & politics”

Talent vs Luck: the role of randomness in success and failure

A paper. This model sounds super-interesting, and I want to replicate it in Python as soon as I have a few hours available (meh)

How to Deal with Difficult People on Software Projects

It’s funny because it’s true

Machine learning teaches me how to write better AI

This is an old post, but I have always found it really interesting. I used Normalised Compression Distance once, but I no longer remember the setting (I think it was as part of a decision tree metric, but the details escape me, it was 6 or7 years ago)

Supervision and error handling in zio, akka, monix

Zio, Monix and Akka can be used for similar things (asynchronous execution of streams/“stuff”). How you handle errors is different though, and sometimes using one for its error handling can make a difference.

From 48s to 5s - optimizing a 350 line pathtracer in Rust

If you’ve been counting, I’ve been reading a lot of Rust stuff lately. I have always been interested in raytracers/path tracers (wrote one in Lisp ages ago, got up to refraction), so this was a natural fit.

📚 Software Abstractions

This “week” I finished the second edition of Software Abstractions, the book on the Alloy modeling language. It is pretty good. Starts as a light read, but after a while how the language works starts to be a bit too cumbersome. Luckily there’s an extensive set of examples throughout the book, and an examples section at the end. I’m itching to model something in Alloy, but I’m still unsure what.

Remastering Star Trek: Deep Space Nine With Machine Learning

The result is impressive. And got me to consider installing the app used.

The Pyrex Glass Controversy That Just Won't Die

Surprisingly fascinating story about why Pyrex is of lower quality lately in the States. Lucky for me, in Europe it’s still fine

Pinot Joins Apache Incubator

YAQEE (Yet Another Query Execution Engine). But sounds interesting, and has a nice logo. I’ve seen worse

How Bluetooth got its name, an interview with Jim Kardach

Ye olde Harald!

A Pi-Powered Plan 9 Cluster

Plan9 is what Linux could-have-been. It was built for clusters, with file, process, sharing made easy. A long time ago I had a 2-machine cluster (1 Raspberry and a very old laptop), which I used as a storage/sharing files across two Macbooks using Plan9 from User Space syncing into a Venti file server. Ah, those were the days!

Modern art was CIA 'weapon'

TL;DR: The CIA funded art tours to show the capitalist West still had creativity

Maya ritual cave ‘untouched’ for 1,000 years stuns archaeologists

My girlfriend majored in History, with a focus on pre-Columbian meso-american history, so I always pay more attention to articles about the area.

Remembering a Programming Language that Helped Shape the Digital New York Times

An interesting history tidbit. And a good lesson on why custom-built languages and DSLs are a blessing and a curse.

📚 Guns, Germs and Steel

I’ve had this book on my Audible account since... 2014 I think. And actually, I also have the print copy. It was recommend to me by my thesis advisor, and it is indeed a very interesting subject, but there are points when the Audible version is as dull as watching paint dry. I haven’t read the print version, but consider that before audiobook.

Final Tagless seen alive

This is a response to John de Goes’ talk The death of final tagless. I don’t have a definite opinion here. I like free monad approaches, since they make interpreters easy to understand and optimisable “easily”, on the other hand Zio looks clean, but that’s as far as I go: I don’t have a strong opinion either way on final tagless, judge yourself from both sources

Newsletter?

2019-8 Readings of the week

2019-03-04T21:11:00.001+01:00

NOTE: The themes are varied, and some links below are affiliate links. Software engineering, formal methods, fraud, Scala. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Messaging startup Slack is secretly an acronym for...

No spoilers, you'll need to click. It may or may not surprise you

Beautiful, Simple, Testable Functional Effects for Scala

Next thing I write in a functional style in Scala, it will be using Zio for sure. This was a follow-up to a talk given at Skills Matter, and shortly followed by a podcast interview in Scala Love. Haven't watched the talk yet, but the podcast was good.

Apple highlights best photos shot on iPhone around the world

They look good. And that may be an understatement.

Formally Specifying UIs

If you've been following my readings of the week you'll know I've been converted into a fan of formal methods. And alloy is pretty cool.

Building the Hangman Game using ZIO

An approachable example (in addition to the details in the post above by John himself) based in a talk John gave, detailed by Abhishek Srivastava.

Revolut insiders reveal the human cost of a fintech unicorn's wild rise

This week the CFO of Revolut resigned his position, after (possibly) an investigation about fraud. That was an amber flag, but reading how Revolut's CEO treats employees... As a worker in the same space (tech) that's a red. I had been using Revolut for more than a year (before I used Monzo) and I'll be using N26 as soon as I get my card. By enrolling by the N26 link here I think I get some kind of affiliate return, I'm not sure.

The hard part in becoming a command line wizard

In the end, it's the hard part in becoming a wizard in anything. Math, programming, playing the ukulele.

Casper’s clever little nightlight

I don't read many hardware breakdowns, but somehow I clicked on this one and now I want one of these nightlights.

📚 The Storytelling Animal: How Stories Make Us Human

I was attracted to the premise of this book, but it felt pretty underwhelming. It has pretty high reviews, so it may have been just me.

Newsletter?

2019-7 Readings of the week

2019-02-26T22:54:00.000+01:00

NOTE: The themes are varied, and some links below are affiliate links. Formal methods, Scala, productivity. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Solving Knights and Knaves with Alloy

Once you start your journey down the formal methods rabbit hole (which I started with TLA+) you can never stop. This is a very good introduction to Alloy, a modelling language which seems well suited for data structure descriptions (not procedural/step-time models)

Seeking the Productive Life: Some Details of My Personal Infrastructure

As much as I don't like Stephen Wolfram, his obsessive take on being productive echoes mine. And that worries me.

Encryption Key Hierarchies in Alloy

Next after the intro above, this is a short post about how you would set up a reasonable hierarchy of keys in an organisation. Something like "Infrastructure team owns infrastructure keys, developers own GitHub" but with more layers. Then you can automatically check somebody has access to stuff, etc. Neat.

Is your Scala object always a singleton?

The guys at SoftwareMill (excellent technical blog and people) stumbled upon this. The kind of bug that could defeat you, but they succeeded, and documented it for the rest of us.

Proving Games are Winnable with Alloy

And the final instalment in this week's formal method extravaganza, how to prove a randomised game (say, Zelda) can be winnable using Alloy.

Don't Let the Internet Dupe you, Event Sourcing is Hard

Yep, can totally agree, I've hit some of the roadblocks and fun moments the author shares. As usual, some HackerNews comments can be enlightening.

On Being A Senior Engineer

There are many definitions of what being senior is, but you can't go wrong trying to follow these suggestions

Hold the front pages: meet the endpaper enthusiasts

I'm pretty sure you didn't know there are collectors of endpaper.

Beating hash tables with trees? The ART-ful radix trie

A synopsis of a data structure paper, about ART radix tries. They are kind-of-like tries, but try to use less memory.

Scala pattern matching: apply the unapply

In case you didn't know how pattern matching works (hint: unapply), this post will tell you.

buffer-expose emacs package

A package released late last week, it helps you navigate your open buffers in a visual way. Pretty neat, and even with my usual 20+ buffers seems to work seamlessly.

Newsletter?

Apache Hive and java.lang.ClassCastException on start

2019-02-19T23:47:00.000+01:00

Photo by Annie Spratt on Unsplash

A couple of days ago I installed Hive from Homebrew on my Mac. Sadly, when I tried to run the hive command, I got the weird-looking error

Exception in thread "main" java.lang.ClassCastException: 
  class jdk.internal.loader.ClassLoaders$AppClassLoader 
   cannot be cast to 
  class java.net.URLClassLoader 
  (jdk.internal.loader.ClassLoaders$AppClassLoader 
    and 
  java.net.URLClassLoader are in module java.base of loader 'bootstrap')

That looked like a JVM incompatibility, so I switched from GraalVM (the one I use by default) to Java 8 (I have aliases jgrce, jgree, j8 and j11 to switch JVMs). Still, the same error regardless. Weird. Maybe Java 11 (the other JVM I have installed)? Nope, same error.

A quick Googling confirmed that this was related to Hive picking up Java 11, but only working with 7, 8 or 9 (not sure about 9). This in turn is due to the Hive boot scripts looking for the latest JRE which is at least 7, like the hive command here:

JAVA_HOME="$(/usr/libexec/java_home --version 1.7+)" \ 
HIVE_HOME="/usr/local/Cellar/hive/3.1.1/libexec" exec \
"/usr/local/Cellar/hive/3.1.1/libexec/bin/hive" "$@"

This will pick 11, which no longer has URLClassLoader (I think this was changed in Java 9). So, won't start.

Sadly the only reasonable fix is modifying the scripts after installation, unless you want to just uninstall Java > 1.8. For me this was not an option, so I just modified the scripts by removing the JAVA_HOME condition (since I set my JAVA_HOME globally when I switch between JVMs). And crossing fingers to remember I did so next time I upgrade HomeBrew.

2019-6 Readings of the week

2019-02-17T18:54:00.005+01:00

NOTE: The themes are varied, and some links below are affiliate links. Softwareengineering, adtech, psychology, python. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Finding Lena Forsen, the Patron Saint of JPEGs

I have written about Lena Forsen (previously Soderberg) before. This is an article explaining the story behind the classic picture, the issues that have arisen with it after the years and a bit of the take of the subject, Lena.

Incrementally migrating over one million lines of code from Python 2 to Python 3

Did you know, Dropbox (the client) is written in Python? Well, I did, but wasn't aware of its scale. And, now it's mostly Python 3, which is good to know. Bonus points for Dropbox for using pytest as well.

How Criteo is trying to navigate GDPR

Since I work in adtech, I need to keep up with what's up and how everything is going. This article explains a bit the impact of GDPR on Criteo, one of the largest players in programmatic advertising in Europe. By the way, Criteo have a superb engineering blog, although it can be a bit hard to read through an adblocker. By the way, the linked post is behind a paywall. Paste the URL of the link on Google and click on the first search result to be able to read it if interested.

10 Reasons Why GTD Might Be Failing

The title of the linked article should end in for you. It lists the common problems a GTD newbie-intermediate can face. Been there, done that. Still am, probably.

Reflecting on My Failure to Build a Billion Dollar Company

The story of GumRoad, the publishing site. A very interesting view on startups.

StranglerApplication

One of Martin Fowler's refactoring approaches. I'm pretty sure you've done this, turns out it has a name.

The de Havilland Comet

The story of the first passenger jet disasters, and why they happened.

Simple dependent types in Python

They may be a bit too simple for what you may be looking for, but the potential is there. Also, linked to the following, which I found ideal.

1-minute guide to real constants in Python

There's actually a Final type coming, someday, to Python, which will make constants real constants. Looking forward to that.

The Dunning-Kruger effect, and how to fight it

There is a quote in this interview of David Dunning which is just brilliant:

The first rule of the Dunning-Kruger club is you don’t know you’re a member of the Dunning-Kruger club. People miss that.”

📚Atomic Habits

This week I have finished Atomic Habits (affiliate link). I didn't expect much, after having read so many productivity books during my life. And indeed, there was nothing new I hadn't read before, but the presentation, examples and overall feel of the book have been excellent, I have rated it 5 stars in Goodreads, which is not something I do frequently (once per year, approximately).

Newsletter?

2019-5 Readings of the week

2019-02-10T22:38:00.000+01:00

NOTE: The themes are varied, and some links below are affiliate links. Software/data engineering, psychology, formal systems. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Playing with Rust

I've known Pere for several years, and I'm glad to see his views on Rust. Looks like not every Scala developer has to move from Scala to Haskell.

Try out walrus operator in Python 3.8

I am the egg man, they are the egg men, I am the PEP 572: Goo goo g'joob. Well, or not. This feels very un-pythonic, and we all know the amount of flak BDFL got for that. But oh well, it's in Python 3.8-dev. Let this serve as the weekly reminder that pyenv is excellent.

Cognitive Journaling: A Systematic Method to Overcome Negative Beliefs

Last year I had been journaling on and off, and this year I have been consistent, every day so far since New Year. This has interesting approaches to understand your mind better.

How Jeff Bezos Turned Narrative into Amazon's Competitive Advantage

This idea is gold, but the HackerNews comments are better than the original article.

Model Metropolis

I read about Forrester's Urban Dynamics in the excellent Thinking in Systems from Donella Meadows. And this also talks about SimCity, what else could I want?

A Humility Training Exercise for Technical Interviewers

The idea presented here is powerful: learn to remove biases from your interviews. As an occasional interviewer (and interviewee), I can totally see how this can help.

pandas blog – Pandas Extension Arrays

Extension arrays show a lot of promise, and right now there is work underway for Pandas extension arrays based on Arrow (fletcher). Although the cool work will happen when the underlying memory layout and building blocks of Pandas are actually Arrow... And that is underway as well.

Demystifying JOIN Algorithms

A very good explanation of basic join algorithms, with a pseudocode that looks pretty much like Python

Newsletter?

2019-4 Readings of the week

2019-02-05T22:56:00.002+01:00

Sorry for the delay, Sunday was my birthday (also, Elmo’s, and The Day The Music Died as well) and I spent the day without access to a computer.

NOTE: The themes are varied. Software/data engineering, psychology, formal systems. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

The Art of Decision-Making

An overview of (what looks like) pop-sci books on decision making. There are some interesting ideas, especially regarding aspiration vs ambition.

The Soviet license plate game and Kolmogorov complexity

Excellent post as usual by John D. Cook mixing something interesting with a bit of mathematical analysis (which is also obviously interesting). Also makes Soviet license plates sound more interesting than they should.

Spark surprises for the uninitiated

I’ve known about the pain of non-deterministic Spark operators (went down some of the codebase rabbit holes while investigating a weird predicate push down rule I suspect is conceptually wrong). This post will actually get you scared: how do you test this?

Regrettable Code, Episode 1

A history about profiling a Python application and the accumulation of small decisions that lead to a large performance bottleneck.

High-Performance in Python with Zero-Copy and the Buffer Protocol

I wasn’t aware of the buffer class/protocol in Python, but it looks like something I should be aware of.

Simulating blobs of fluid

A long long time ago, I can still remember, I loved creating visualizations and animations. I still do, I just don’t have the time. Otherwise, I’d implement this.

Ask HN: Top three questions for a startup before accepting a job offer?

Not the first time a similar question has been asked in Hacker News, but it’s good to keep a few of this stored for future reference.

Ray: Application-level scheduling with custom resources

Dask and Ray are next in my list of stuff I want to try. I love Spark, and love Scala, but the more tools in my belt, the better an engineer I can be.

Rust on iOS

An updated tutorial on using a Rust library in a Swift application for iOS. I have tried, it actually works. I need a time machine or something.

Thread Pool Best Practices with ZIO

The idea reminds me of how the Go (Golang) scheduler handles the mapping between goroutines and OS threads.

The Kubernetes Scheduler

An awesome post about the Kubernetes scheduler and its internal workings, with partial TLA+ specs of the moving pieces. It may be a bit over-the-top if you don’t know a lot about K8s internals or any TLA+, but if you know a bit of two, you’re in for a treat.

Shadows used to peer around corners

Technique to peek behind corners from a still picture.

Queueing theory: The science of waiting in line

Another one by John (I think he has won the readings of this week? There’s no prize, though). I’m a mathematician by training, so I didn’t get to study some of the cool (and, according to CS major friends, boring) stuff I love to hear about, like... queuing theory.

Newsletter?

2019-3 Readings of the week

2019-01-27T20:01:00.000+01:00

NOTE: The themes are varied. Software/data engineering, history, formal systems. Expect a similar wide range in the future as well. You can check all weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Fermentation and Daily Life

I’m not a fan of fermented food, but my girlfriend is. The article is interesting even for me.

Why Are Young People Pretending to Love Work?

The “grind and hustle” gets old pretty quick.

📚 Specifying Systems

The book for learning TLA+ (and, free to download from the link above). I’m reading it right now, step by step. You can also get a paperback version from Amazon (affiliate link) but it's kind of expensive.

The Perils and Pleasures of Bartending in Antarctica

It kind of makes sense.

“O Uommibatto”: How the Pre-Raphaelites Became Obsessed with the Wombat

Whomwhat?

Why Don’t People Use Formal Methods?

As you may have realized, I’m interested in formal methods and verification. I’m not the only one, and since I now pay more attention to articles on the subject, I find more articles to share. Hillel is the author of Practical TLA+ (affiliate link), the book that finally got me to write specs.

The 1859 Carrington Event

The idea of this happening “now” is actually scary.

Ask HN: What are your “brain hacks” that help you manage everyday situations?

You may pick up one or two tricks that can be useful.

What if the Placebo Effect Isn’t a Trick?

Here, take this sugar pill. You’ll be cured.

Kafka at Criteo

Slides from Slideshare. The scale is astounding. Note that the engineering blog at Criteo is top notch, but your adblocker is probably going to give you a hard time reading it.

When Rust is safer than Haskell

I’m closer to doing useful stuff in Rust than in Haskell, so it’s always good to know Rust has some nice tricks up its sleeve.

Inductive invariants

More TLA+ goodness, from Lorin Hochstein.

Spark Barcelona Meetup: Speeding up PySpark with Arrow

This Thursday I’m speaking about how PySpark got faster by using Arrow internally. If you are around Barcelona please join us! Note that the slides for this talk are not up yet!

📚 Meditation for Fidgety Skeptics by Dan Harris

(note there are affiliate links in here) This is the follow-up to 10% Happier. MfFS is good, offering a more practical take than the previous one. As books to stand on its own, 10% Happier is better though.

📽 10 tips for failing badly at microservices

This is a very fun talk about what you should do if you want to prevent (in an ironic way) your company from moving to a microservices-based architecture. You may get flashbacks to the Simple Sabotage Field Manual from the CIA.

Newsletter?

I’m considering converting this into a weekly newsletter in addition to a blog post. These days (since RSS went into limbo) most of my regular information comes from several newsletters I’m subscribed to, instead of me going directly to a blog. If this is also your case, subscribe by clicking here and if enough people join I’ll send these every Sunday night or so.

2019-2 Readings of the week

2019-01-20T16:32:00.001+01:00

NOTE: The themes are varied. Software/data engineering, languages, writing. Expect a similar wide range in the future as well. You can check all weekly readings by checking the tag here.

Goodbye, Mid-Range Shot

I love a good data visualization, for a long while, long, long time ago I was a paid subscriber to Nathan Yau’s Flowing Data. As usual, a short and interesting visualization where we can discover something about the world.

Speed up your Python using Rust

I need to try this ASAP. Write dylibs with Rust that you can use straight from Python (via the cpython crate).

Build Your Own Shell using Rust

I’m a lot into Rust lately, and here you can find out how to call subprocesses, how to handle errors, awaiting... It has a little bit of everything, even if you don’t intend to write a shell.

Introduction to TLA+ Model Checking in the Command Line

If you’ve been following my twitter or blog, you’ll know I’m really liking TLA+ for formal verification. It’s not like I have a lot to verify day-to-day as a big data engineer, but as soon as there are “big data” microservices around, TLA+ rocks to confirm all the Lego pieces fit as I think they fit. Cutting the story short, TLA+ runs on the TLC Toolbox, which is a Java application which is not super-modern-looking. If you’d rather use Sublime/Atom/Emacs/Vim/ed using TLC from the command line is possible and not hard. Take heed of Ron Pressler's comments on Reddit, though.

From Data Types to Session Types: A Basis for Concurrency and Distribution

The concept of session types seems very interesting. In a sense, Rust’s ownership/lifetime concept can model part of what session types cover.

How the Artificial Intelligence Program AlphaZero Mastered Its Games

I’ve been playing go, on and off (right now is on) for almost half of my life. Although I knew the basics already (I know some deep learning y’know), this is a good write up anybody could understand.

High-Performance Functional Programming Through Effect Rotation

I try to stay away from the more crazy levels of category theory inspired programming (as in, I use 0 of it and have no huge interest), but this post (Scala) about how to optimize stacked monad transformers was very interesting.

SoftwareMill blog: Free and tagless compared - how not to commit to a monad too early

I heard Adam talk about Free & Tagless at the Scala Exchange 2017 (IIRC), and it was one of the clearest expositions I had seen. This post is probably another, also from him. Totally recommended, with the bonus of getting to understand tagless encoding for “free”

Using spaced repetition systems to see through a piece of mathematics

I’m a big fan of spaced repetition in general and Anki in particular. Michaeel (Nielsen) uses it regularly for deeper understanding. If you want more “social proof”, Derek Sivers and Gwern Branwen also recommend it dearly.

The three-page paper that shook philosophy—a hacker’s perspective

When you can put a name on something, it becomes easier to handle. Gettier cases are such. Go read this.

Methods for Array Initialization in Rust

Just when I thought Scala offered too many ways for doing a thing, there comes Rust.

Experience report: organising a conference

I couldn’t attend Chris’ LambdAle last year (I will try this), but having helped organize PyData Barcelona, all he mentions are very valid concerns and tips.

Automatic categorization of text is a core tool now

It sounds obvious, but getting obvious staring you at the face without realizing is a problem.

Rust Performance Pitfalls

I’m still far from seeing performance issues in any Rust I have written (mainly because it’s a very small amount) but it’s good to know where I can stumble.

The Puzzle of Proto-Elamite

I love language history. If you do so too, you’ll enjoy this.

Understanding Rust Lifetimes

Nailing this is the main issue with Rust. This is a good introduction to lifetimes.

📚 The Haiku Handbook

Note this is an Amazon affiliate link.I have always liked Haiku (poems, the operating system is ok), and wanted to have a more formal model on them, to be able to write some someday. This is a very well-written book that reads quickly (to be fair, I skipped the chapter on Teaching)

Newsletter?

2019-1 Readings of the week

2019-01-14T00:50:00.001+01:00

If you know me, you'll know I have.a very extensive reading list. I keep it in Pocket, and is part of my to do stored in Things3. It used to be very large (hovering around 230 items since August) but during Christmas it got out of control, reaching almost 300 items. That was too much, and I set myself a goal for 2019 to keep it trimmed and sweet. And indeed, since the beginning of the year I have read or canceled 171 articles (122 in the past week, 106 of which were read). That's a decently sized book!

To help me in this goal, I'll (hopefully) be writing a weekly post about what interesting stuff I have read the past week. Beware, this week may be a bit larger than usual, since I wanted to bring the numbers down as fast as possible.

NOTE: The themes are varied. Software/data engineering, drawing, writing. Expect a similar wide range in the future as well.

The Nature of Infinity, and Beyond – Cantor’s Paradise

A short tour through the life of Georg Cantor and his quest for proving the continuum hypothesis. In the end, he was vindicated.

Statistical rule of three

What is a decent estimate of something that hasn't happened yet? Find the answer here.

Apache Arrow: use of jemalloc

A short technical post detailing why Arrow moved to jemalloc for memory allocation.

Subpixel Text Encoding

This is... unexpected. A font that is 1 pixel wide.

Solving murder with Prolog

I have always been a fan of Prolog, and this is a fun and understandable example if you have never used it.

What Parkour Classes Teach Older People About Falling

Interesting. I'm still young, but I'll keep this in mind for the future.

Implementing VisiCalc

The detailed story about how VisiCalc (the first spreadsheet) was written.

The military secret to falling asleep in two minutes

I was actually doing something similar since I was like 12. It might be a stretch to say 2 minutes, but works.

Index 1,600,000,000 Keys with Automata and Rust

Super interesting (and long) post about how FSA and FST are used for fast search in Rust (I'm a bit into Rust lately). Also, BurntSushi's (Andrew Gallant, the author) cat is called Cauchy, something I appreciate as my cat is named Fatou.

How to Draw from Imagination: Beyond References

An excellent piece on gesture drawing and improving your technique.

Anatomy of a Scala quirk

All languages have their WAT, it's harder to find them in Scala though.

Chaotic attractor reconstruction

An easy example in Python of Takens' embedding theorem

Hello, declarative world

An exploration between imperative and functional, and how declarative fits the landscape

Python with Context Managers

Although I have written tons of Python, I never took the time to either write or understand how context managers work. This one was good.

Raymond Chandler's Ten Commandments For the Detective Novel

You never know when you may write a detective novel. Ruben and the case of the dead executor

Seven steps to 100x faster

An optimisation tour of a piece of code written in Go, from data structures to allocation pressure.

Writing a Faster Jsonnet Compiler

A semi-technical post by Databricks about Jsonnet and why they wrote their own compiler. Serves as an introduction to Jsonnet ("compilable JSON") as well.

Bonus

Monoid font and Poet emacs themeToday I switched from solarized dark and Fira Code Pro to the above. It looks interesting

Newsletter?

2018: Year in Review

2019-01-01T15:00:00.000+01:00

Note some links below are Amazon affiliate links. Buy at your own risk 🤣

Work

During the first half of this year I have been leading a migration to Spark of several systems, one relatively straightforward and the other based on GraphFrames and a graph with ~1 billion nodes. The results so far seem excellent. The second half of the year has been mostly GDPR compliance improvements (I've read the whole law... twice!), looking for additional large-scale data providers and preparing our systems for global scale (around 6x event data capabilities). Has been fun and challenging.

Life

In February I moved from East Finchley to Chiswick. This meant I became a Friend of Kew Gardens, and have been there almost every weekend I've been in London. And you could catch me a bit earlier at The Coffee Traveller, enjoying a latte with a carrot cake muffin.

In February, we spent a few days in Puigcerdà for my birthday. Not as nice as we expected. The food was good, though (as expected).
In March we visited Edimburg again, Laia was attending EdiYarn Festival. I spent the days at Waterstones, eating cupcakes and getting my reading list under control.
In May I attended AWS Summit in London. Was ok-ish. Lack of coffee made us grumpy.
In June I attended SoCraTesUK 2018 in Dorking, together with Carlos. I had an excellent time there, and will repeat next year if possible. We presented together Welcome to Apache Spark. I also learnt some Cobol.

In June as well, I presented How does that PySpark thing work? And why Arrow makes it faster? at the Python Barcelona meetup. I think people enjoyed it.

At the end of June we visited Bath for Laia's birthday. Nice town, but a bit packed with tourists. We were lucky England was playing some soccer match and could visit the Roman baths without people.
In July we published the first podcast of PyBCN (in Spanish). It's not monthly, next one was in... December 😆
In August we roasted a bit under the heat wave hitting the UK by visiting Mayfield Lavender field.
In September I attended Scala Italy in Florence, where I got the opportunity to visit Osteria del Cinghiale Bianco again. Excellent.

In September I presented How does that PySpark thing work? And why Arrow makes it faster? at the Spark London meetup, feedback was positive as well.
In October, I attended Spark Summit Europe in London. Met a lot of people and had a terrific time.

In October we took a few days off so we could disconnect, and spent them on Hay-on-Wye, as some tradition we follow says. We bought a bunch of books (I think there are no books by Roger Zelazny left there) and I did nothing. Bliss. Also, best scones ever.
In November I helped a bit from afar with the organisation of PyDay 2018. I couldn't attend, but looked excellent. We also told the world PyBCN is now an organisation you can become a member of.
In December, I attended Scala Exchange for the third time, and gave a lightning talk about TLA+ (Formally Verifying Complex Systems Using TLA+). Looks like people enjoyed my talk, and I had an excellent time as usual.

I made minor contributions (typos, small fixes) to several open source projects (Spark, typeshed, Monix, bloop,...), and also created a couple for my enjoyment: experimenting with Processing in Scala, sbt-ifttt and rb-zsh-theme.

Learning

This year I got a bit deeper into functional Scala, using GraphFrames and (in general) graph operations on big data. Got acquainted and got to hate Apache Airflow. I improved my knowledge of Akka (adding clustering and sharding to the "could do"). I also learned TLA+ and I plan to dive deeper into this rabbit hole the coming year. Started learning Rust. Made some minor stuff using HyperApp, and have been considering learning PureScript so I can write something close to "enjoyable" Javascript. I have also started playing the ukulele.

Reading

My top pick for the year is without any doubt Thinking in Systems: A Primer among books I read this year (19 books). Didn't read much non-fiction, and can't recommend any this year. According to Pocket (which I started using around June, after the GDPR fiasco of Instapaper) I read the equivalent to 18 books in blog posts since then.

Music/events

This year I attended a couple concerts, both in London: Hayseed Dixie and Steve'n'Seagulls. I ended the year listening to a lot of covers by Leo Moracchioli and progressive metal/instrumental metal like Polyphia (which eventually become boring) and Intervals (which are better). All the links in this section are good renditions by each on Youtube

Gadgets

Main one is that Apple finally upgraded the MacBook Air so I can have one with 16GB of RAM, so I got that one as soon as I could. It also implied getting a USB C hub and I also got a power brick, both from Anker, so I could simplify my desktop cable tangles (also several USB A to C and USB C-C cables). In no particular order, I also got

An acoustic ukulele (soprano) and soon after, I got this electric travel ukulele (concert). To go with it I got an AmPlug (clean)

A mountie very early in the year. Sadly it doesn't work with my new Air, but with the old one it was a blast.
A Brydge keyboard for my iPad (heavy, but excellent)
A SoundCore Motion Q speaker from Anker. We're getting another one soon to get stereo. Excellent.
A Microsoft Wedge Mouse. I saw a picture and when I saw the price (just 17€) I got one. It works very well, and is very small. And works with a rechargeable AA battery, which I like as form factor.
A eInk display for my Raspberry Pi Zero W.
An hourglass. I love hourglasses. You can hear the sand from this one.
A cardholder. I already had a wallet by Bellrow, and they deliver quality.
A Luna Display (so far a bit disappointed with the inability to map correctly my keyboard layout between Mac and iPad, so can't type at all, but the stream quality is very good)
A bunch of twisty puzzles. I didn't have any 2x2 and I really like it now. Also, magnetic 3x3 are excellent.

The Little Lisper for my birthday

An excellent compact fountain pen from Kaweco (the liliput)
To go with it, I got a 0.7mm pencil from them as well (the classic brass)
I got also a 2mm titanium pencil from Kickstarter
A knucklebone from AroundSquare. Excellent for manipulating while thinking
A new spinning top, the Rain Drop from NWTops (to join last years' Yakima)
Several (1, 2, 3, 4) cardistry decks. The Mandala is incredibly gorgeous, the Edge is a design I love (I have the deck as well) and when I saw the trainers I knew I wanted a couple, just because.

And mugs!