Ramblings

Weaving with Light Pt. 1

2016-09-01T01:07:25Z

This is the first in a series of posts regarding a recent project which integrated handweaving, fiber optics, and electronics. It’s a part of a costume for a cosplayer at work, but I’ll be discussing my part of it. TL;DR For those who can’t wait, here’s what the project looks like in the dark: And …

Abusing HAProxy: Stupid Simple Easy Dashboards

2016-06-24T20:35:04Z

I wanted a simple way to have a dashboard to show if hosts and services are alive & didn’t want to write much code and/or run up a nagios instance (or anything like that). All I care is whether it’s green or red. I’d already been setting up HAProxy for a proxy forwarder, so I …

Rules for Operations

2016-04-25T01:59:09Z

The following list was compiled in 2012 for a talk on Operations Principles for Developers (Ops4Devs). They are loosely inspired by the list of rules from Zombieland as well as from my experiences and those shared by others. Looking over the list four years later, I believe that they are still (very) applicable for all …

DevOps Creed (Work in Progress)

2016-04-25T02:36:43Z

This is a work in progress of a DevOps Creed. It will always be a work in progress as I and others learn and grow. Suggestions are welcome! I have drunk deep of the DevOps Kool-Aid. From the visions which ensued, I have come to the following…. I Believe: DevOps methodologies lead to systems which …

I am not a Mindreader: a mini-saga

2016-04-11T19:58:29Z

I must confess a severe failing on my part. I am not a mindreader. I am not privy to the thoughts in your head. I do not know your needs or desires. And I am certainly not aware of your expectations. This is why requirement documents exist. Please use them.

Cloudera Manager Disaster Recovery with JSON Deployment Dump

2016-04-07T12:20:23Z

Cloudera Manager is fairly opinionated. In its defence, it pretty much needs to be given that it needs to wrangle multiple underlying Open Source projects. Each of these, in turn, have their own quirks and opinions. The following is a description of how to recover a Cloudera Manager cluster post disaster, assuming that you have …

Interesting Feature of Dockerfile Volume Directives

2016-03-30T17:00:38Z

I’ve been rewriting a cleanroom version of the hadoop-in-a-box — just about finished. And, truth be told, the code, all in all, is a bit tighter than the original encumbered version. However, I ran into an interesting feature of Volumes — I had thought perhaps to optimize things a bit, but it caused some unexpected …

Docker, Cgroups, Memory Constraints, and Java: A Cautionary Tale, or Here be Reapers (sometimes)

2016-03-17T02:54:54Z

TL;DR: Java and cgroups/Docker memory constraints don’t always behave as you might expect. Always explicitly specify JVM heap sizes. Also be aware that kernel features may not be enabled. And Linux… lies. I’ve recently discovered an interesting “quirk” in potential interactions between Java, cgroups, Docker, and the kernel which can cause some surprising results. Unless …

Zombie Apocalypse! Docker AUFS + Java + Low Memory …. Hadoop in a Box Cloudera Manager Cluster

2016-03-15T19:35:10Z

TL;DR — When using AUFS in a memory constrained environment, Java can spawn (lots!) of Zombies. A workaround is to change the storage driver to the device mapper. In working on the Hadoop in a box CDH cluster with Cloudera Manager, I’ve discovered a few interesting things about AUFS. These experiences are with Ubuntu 14.04 …

Cloudera Manager GUI and API Can Step on Each Other

2016-03-10T16:19:57Z

While learning how the configuration worked — in particular which arguments to pass in order to set non-default values, I discovered that I could lose changes by following these steps: Use the GUI to set a value and save it. This is just so that you can find the variable. Keep the GUI open. Dump …