Peter Whittaker

About time: Setting up ctags for vim and git

2026-01-08T00:00:00+00:00

Professionally, I work pretty close to a bleeding edge (not the bleeding edge, but it’s still hairy here): Most of my job involves cross-domain work of one kind or another, and we prize reliability and performance in equal measure - one dropped packet in 10 billion is too many, and single-digit Gbps on 25Gbps interfaces is too slow, even with a complete protocol break and physically guaranteed one-way transmission. Getting all those ducks lined up can be both exhilarating and exhausting.

Personally, though, I am both a creature of habit and slow to change. Most of the rest of the team uses VSCode, and here I am happily trundling along with vim. Why? Because it is ubiquitous and I know it more than well enough to get my job done. Given how much I have to master for the job itself, I am reluctant to change my tools, since they work. But I am jealous of my colleagues’ ability to jump around their source files, from the use of a function, method, or structure to its definition

So I have kept a tab group about vim and ctags hanging around for years. Among other things, this contained the following, all of which are referred to herein, listed in rough order of importance:

the universal-ctags GitHub project tl;dr I installed ctags from my clone of this repo
tpope’s excellent but slightly out of date git hooks for ctags I used these as the basis for my gitHooks script, below
the universal-ctags man page with working references to related man pages
the extraordinarily out-of-date exuberant-ctags project page IMHO, not worth it, but YMMV
one I found while writing this post that aims but IMHO falls just short of covering ctags+git+vim at a “just get started” level, which I find lacking for a few reasons that make it a good example of what’s missing or out of date about a lot of available info:
- it uses tpope’s work, extending it for a few cases, but overlooks a hook for switch
- it uses exuberant-ctags for reasons unclear to me
- the call to ctags seems to blindly adopt options widely published elsewhere without justifying or explaining them

Well, New Year, and all that, and yesterday I decided it was time. I dusted off those tabs, tried to make sense of them, and ran into a wall.

exuberant or universal? (tl;dr: universal)
Which does my main development box have, and which version? (best guess: universal 6.0.0)
No direct Rust support, eh? (tl;dr: true for exuberant, false for universal, the documentation is waaaaay out of date)
How do I get vim to find tags file, given there are several places they could be? (no spoilers, it turns out to be super easy)

Basically, I ran into the perfect intersection of several hacker problems:

Many maintainers and contributors do really, really well at writing code, but write documentation for people who already know what they know.
Many users accept on authority what they have found elsewhere, and share it, sometimes verbatim, without really understanding the choices and motivation behind the original, and often very old, posts/documents/etm.
Some maintainers don’t understand why anyone would need anything spelled out, after all, it’s all right there in the README!
The documentation in the main branch of some repos can be way behind what is actually on web sites, making RTFM - and building TFM in the first place - essentially pointless (so pip uninstall sphinx && sudo dnf remove python3-pip)

In other words, there was plenty of information that assumed you already knew what you were doing, very little for the person trying to find their feet, and none of it that accurate.

Findings as I worked through this:

exuberant-ctags
- hasn’t been updated in over a decade, and
- doesn’t support Rust directly, but only via extensive regex patterns, which can be very, very slow (and good luck finding curated and up-to-date patterns)
universal-ctags is up-to-date and supports Rust and many other languages
- the documentation included in the repo is, uh, poor, and, if you want to RTFM, you will need pip and sphinx, but they don’t add much value
both exuberant-ctags and universal-ctags have many options, with defaults settables in a ctags file that can live in ~/.ctags or .config/ctags or .ctags.d/*, depending on which you are using and what you prefer, but…
- I found that the default behaviour of universal-ctags, without any such file, was sufficient for my main use-cases.
rusty-tags may be useful, but it depends on your use-case:
- if you have universal-ctags and only need tags on your code, that is, your project itself, you probably don’t need rusty-tags
- if you have exuberant-ctags, rusty-tags provides the regexs to tag Rust source, but this can be slow
- rusty-tags shines when you want tagging into your project’s dependencies, and into the Rust source itself
  - the maintainer chose to generate one tag file per project and per dependency, which can generate a lot of tag files, which, personally, I find cluttering and messy, but it does reduce the likelihood of naming conflicts
adding set tags=.git/tags;,tags; to my ~/.vimrc was enough to satisfy my two primary use cases:
- the ; tells vim to start in current directory and recurse upwards until it finds a matching file, e.g., .git/tags at the top of your repo
- I used git hooks, à la tpope, to generate and maintain tags
- the tags; pattern will match tags I generate ad hoc outside of git (provided I specify -ftags when generating tags; cf “existing tags file”, below)
tpope’s well-known git hooks for tagging seem a little out of date: there was no post-switch hook, e.g.

I decided to

build and install universal-ctags from source,
skip its documentation,
adapt tpope’s git hooks, with some updates, creating gitHooks script I can run on any development machine to set things up for me,
keep my ~/.vimrc simple,
skip rusty-tags for now, as it doesn’t add value for me, at least not yet.

Building `universal-ctags`

git clone https://github.com/universal-ctags/ctags.git
cd ctags
./autogen.sh
./configure
make && sudo make install

This installed /usr/local/bin/ctags, and /usr/local/bin was already in my $PATH

My `~/.vimrc`

As noted above, getting vim to find tag files was, for me, as simple as

echo 'set tags=.git/tags;,tags;' >> ~/.vimrc

This tells vim to search for a ctags file called .git/tags or tags starting in the current directory and searching all parent directories until it finds one. If you change the name of the ctags file in the git hooks in gitHooks (cf “existing tags file”, below), change or add it here or in your ./vimrc directly.

If you run the gitHooks script and git ic in your repository, the .git/tags file will exist in your repository and vim when find it whenever you open a repo file with vim.

The ; syntax is explained by vim’s :help file-searching info.

Adopting tpope’s git hooks

For this, I wrote a gitHooks script:

#!/usr/bin/env bash

if ! command -v ctags 2>&1 /dev/null; then
    >&2 echo "No ctags, exiting; consider installing"
    exit 1
else
    >&2 echo "Proceeding with ctags setup"
fi

# idempotent
# shellcheck disable=SC2088 # we want the exact string
git config --global init.templatedir '~/.git_template'
git config --global alias.tags '!git hook run ctags'
git config --global alias.ic '!git init && git tags'

HOOKDIR=~/.git_template/hooks
mkdir -p "$HOOKDIR"
CTAGS="$HOOKDIR/ctags"
POST="$HOOKDIR/post-"

cat << 'EOF' > "$CTAGS"
#!/usr/bin/env bash
set -e
if ! command -v ctags &> /dev/null; then
    >&2 echo "No ctags, exiting; consider installing"
    exit 0 # exit clean so that git commands reflect proper status (maybe)
fi
dir="`git rev-parse --git-dir`"
trap 'rm -f "$dir/tags.$$"' EXIT INT TERM
# process all tracked files in the repository (git ls-files),
# reading tracked file names from stdin (-L -), creating a
# temporary tags file in the repo's .git dir (-f$dir/tags.$$),
# with tags relative to the .git dir (--tag-relative=yes makes
# the tags relative to the directory containing the tags file).
git ls-files|ctags --tag-relative=yes -L - -f$dir/tags.$$
# rename the temporary file to "tags", replacing any existing tags file
mv $dir/tags.$$ $dir/tags
EOF
chmod +x "$CTAGS"

# create hooks for the following git actions
declare -a hookLinks=()
hookLinks+=(applypatch)
hookLinks+=(checkout)
hookLinks+=(commit)
hookLinks+=(merge)
hookLinks+=(rewrite)
hookLinks+=(switch)

for hookLink in "${hookLinks[@]}"; do
    hookFile="$POST${hookLink}"
    # run all commands in sub-statement in order to redirect just once
    {
        echo '#!/usr/bin/env bash'
        if [[ $hookLink == rewrite ]]; then
            # special case
            # shellcheck disable=SC2016
            echo '[[ $1 == rebase ]] && exec .git/hooks/post-merge'
        else
            # general case
            echo '.git/hooks/ctags &> /dev/null &'
        fi
    } > "${hookFile}"
    chmod +x "${hookFile}"
done

With gitHooks in my ~/bin directory, setting up and maintaining tags in a git repository was simple:

# set up the hooks and aliases I wanted
gitHooks
cd aGitRepo
# use the `ic` alias to run git init, which picks up the templates
# defined by `gitHooks`, and git tags, which generates tags in .git
git ic
cd ../anotherRepo
git ic   # ditto

That was enough to satisfy my major use-case, using tags when working in git repositories with vim.

Obligatory "What I wish I knew about git" post

2020-12-30T00:00:00+00:00

tl;dr: If you think ofgit branchas a verb meaningdiverge from this path, split my treeand think of the branch name as a label that tracks the most recent commit on this new path, a lot of stuff about git becomes clearer. You might already understand whatdetached HEADmeans.

Koan1: If I am on a branch but not at a branch, I am detached.

Koan2: I can always get where I need to go, if I am committed.

“Aha!”

Epiphanies. Eureka moments. Call them what you will, they change our world: We perceive and understand things differently. Some come after difficult, tortured paths, others from seemingly nowhere. Some dawn slowly, some disorientingly suddenly. I especially love the slow dawns, when I can feel the red pill soaking in, feel the change of perspective, feel the new world.

The challenge comes when you want to explain the new perspective to those who aren’t there yet – especially when you took the difficult, tortured path: it can be tempting to recite your journey, in the hope that others will be able to follow your footsteps.

It’s not doomed to failure, just verbose, not to mention potentially inaccessible or irrelevant to those starting from different places: the paths of others need diverge only slightly from our own for the intersection of shared experiences to be very small indeed.

Besides, the one thing that you cannot explain is the experience of the red pill itself: Unless and until someone else takes it, they won’t get what that part of the trip was like. (I didn’t intend the pun, but I’m happy with it :->)

So rather than explain how I came to git, or how I used git badly (as I now perceive it), or reference websites and tutorials and blogs and practices, all in an attempt to share pre-flash headspace, allow me one – or maybe two or three – noble lies.

Lie #1: You don’t work ON a branch, you work AT a branch.

Once you understand what I mean by that, you may think, as I do now, that ON Vs AT is mostly irrelevant, that ON is perhaps slightly more conceptually accurate from a code management perspective, but that AT is more useful when thinking about references in general, about what various commands do and how, and about how to solve various problems.

Aside: As soon as you understand Lie #1, you will be able to infer what a detached head is and you will know that you do not need to use branches or the stash to work with them. Both (the nature of detachment, the commit-only solution) will literally just come to you.

Lie #2: The only difference between a branch and a tag is persistence, or perhaps more accurately, mobility: Branches are mobile, tags are sessile. Git itself moves branches for you, and git itself keeps tags in place, unless you force it to move them.

This one is definitely more of a lie, but once you realize why you can pretty much use a tag anywhere you can a branch, and vice versa, you’ll understand how accurate it is. At least from a perspective very much related to Lie #1. (Andcommit-ishwill start to make more sense, too.)

Lie #3: We could just as well callmasterfred or default or bootStrap or justGetStarted and it would mean exactly the same thing.

bootStrap and justGetStarted are actually pretty good names, when you consider the first problem that the branch concept solves. More on this below.

Usually at this point – or far earlier – most articles have brought updirected graphsand other matters mathematical, which I avoid herein not because I do not understand them but because they aren’t strictly necessary to starting up the ladder or taking the first step on or insertFavouriteMetaphorHere and all I really hope to do is get you started.

(Words of the inner pedant: git mastery requires understanding the nature of git’sdirected acyclic graphand how simply signing the root commit can mark an entire tree as “trusted”. This article is not about mastery.)

Think of source code management at one its most fundamental levels: A way to checkpoint your work, easily and cleanly, so that you revert to your last known good state when something goes wrong. Start with just that.

Thegit commitaction creates that checkpoint. You’ve probably seen in logs and man pages and blogs, etc., that each commit has a unique ID – and you’ve probably seen that these IDs are lengthy apparently random strings (they actually are pretty close to random, that’s one of their cryptographic properties, but that is well beyond the scope of this post).

Your next commit creates another checkpoint, with its own long random ID. That first commit is essentially an ancestor of the second one, since the second one built on your work in the first one. Just a bit of foreshadowing.

If you had to keep track of where you were, of the relationship between these commits, git would lose a lot of its value – so it keeps track for you – and it does this in two distinct ways.

The first is with theHEADconcept.HEADis simply where you are working right now. If you only ever go forward, commit after commit after commit,HEADwill always refer to your last commit. You will never have to “solve” the “problem” of working on adetached HEAD.In fact, you will never have to know aboutHEADorHEADs...but after reading this, hopefully it will just make sense to you.

The second way git helps you keep track of where you are is with the branch concept. Chances are you are familiar with it, but just in case….

Let’s say you have been working for a while, commit after commit, and are happy with your work: it works as you hoped, it is stable, it is secure, etc. Your next step is a problem you aren’t sure how to solve, but you have a few ideas and want to experiment with them – git helps you with branches.

In essence, a branch gives you a “place away”, a sandbox, where you can experiment based on your good work without breaking it.

You create a branch for each approach –git branch approach01orgit checkout -b approach01orgit switch...– start coding, make some commits, compare, contrast, decide on the approach that works best. Merging that best approach and carrying on is the next step.

But before we take that step, let’s come back toHEADand to how git helps you keep track of where you are.

Now that you have multiple branches, you have multipleHEADs:There is aHEADon each branch, and each branch’sHEADis the most recent commit on that branch. Think of a tree, coming up from the ground, tall and strong, and at some point splitting, growing two or more branches from the trunk. Each branch ends in its ownHEAD.It’s easy to refer to the tree or to the trunk, it’s the bulk of stuff that happened before you made your first branch. The branches start where things split from the trunk.

Which is the main part of the tree now? Well. That’s actually a deeply philosophical question, but for most of us the answer ismaster:git knows you are likely to branch off at some point, so it simply makes sure that your work, even if perfectly linear, commit after commit without branching, has a branch name, and that name is master.

Whethermasteris a master in any semantic sense is entirely up to you. This is just the name that git gives the shoot that grows as you commit. Like I wrote above, it could just as easily be called bootStrap or justGetStarted or myStuff or shoot: Unless you need to come back to it, you never need to know what it was called and you only need to come back to it if you create a branch. There are git workflows based on the idea of making master “special”, making it mean something (we’ve just started doing this in my main job: master is our generally available public release branch, with other branches being for feature development, penetration testing, etc.).

This is Lie #3: Master is only as special as you make it, and, if you never branch, you never need it. Git just made sure it was there because chances are that you would need it, at some point. We’ll come back to Lie #3, because Lie #1 also applies.

Getting back to our tree: if you look at the tree, and consider your solid, secure, satisfying work to be the trunk, right up to where you started branching for experimental purposes, where does master end and the branches begin?

Time to considerbranchnot as a noun, but as a verb. Some git commands are pretty verb-like, e.g.,git add,git commit,git push,git pull, while others are pretty noun-like, e.g.,git status,git log. Until very recently, I always thought ofgit branchas a noun-like command, as in make a branch, but in a lot of ways, it is far better to think of it as a verb: branch to this new place. Diverge. Blaze new path.

You started with a shoot that grew straight and tall then got to a point where you needed to experiment, so you created a branch, to “work off to the side”. Then you created another one, because you weren’t sure about that path. Repeat ad infinitum, or at least ad bonum or ad satium, i.e., until you have a good enough branch.

Right now, you’ve gotmaster, which refers to the shoot, and master’sHEAD,which is the last commit on master, and several branches, each diverging frommaster,and each of theirHEADs,which are the leaves, if you will, on each branch.

Right?

Well, yeah, OK, that’s how a lot of people view these branch things.

For the sake of this article, for the sake of pedagogy, please try this: Think of master as a human friendly, human-consumable label for the last commit you made before branching. In other words, on master,HEADandmasterrefer to the same commit.

Now do the same for each branch: Consider the branch name, e.g.,approach01,not to apply to the entire branch, but to refer to the last, most recent commit on that branch. In other words, on any branch, including master,HEADand the branch name refer to exactly the same thing, the most recent commit.

Every time you commit, git movesHEADto point to the new commit. And every time you commit, git moves branch name to point to that same commit.

Why is this so important? When you checkout a branch, even master, git doesn’t place you just anywhere on that branch: It places you at the end of that branch, where end means the last commit made on that branch.

This is Lie #1: When you start working on a branch, you start at a very specific place – the end of the branch. And the end keeps moving: each time you make a commit, the branch grows, and git updates location references for you so that you are always at the end, which is almost always where you want to be.

Lie #1 says a branch is a moving reference, a label that refers to the most recent commit of a related set of commits.

We might as well deal with Lie #2 right here. Tags are clearly labels. That is their whole function: To label a commit as having some semantic meaning, some importance to you or your team. The only idea of a tag is to mark a commit that you should always be able to come back to, easily.

A tag is persistent, static, sessile reference, a label that never moves, a marker for an important spot. (Under the covers, full tags are one of the four types of git objects, along with blobs, trees, and commits; I use “full” to contrast with “lightweight” tags, which are just references.)

Of course, you don’t really need tags. You could capture the same information in a commit message, then read through the log, looking for it, get the commit ID, and git checkout that CommitID and there you’d be! But who has time for that?

Git tag saves you that work, by allowing you to mark specific commits as special and to list those special commits withgit tag -l.

To complete the lie, consider tags as labels that move you back in time, to previous somehow important commits, and branches as labels that move you forward in time, to the most recent commit. Git will never move a tag on you, because it has no way of knowing why you made that label: it just isn’t smart enough, it assumes the label meant something to you, and will only move it if you explicitly force it to. But git moves the end-of-branch label with every commit, because most of the time most of us want to be right there, at the end of the branch, at our most recent work.

Lie #2 is important because you can pretty much use a tag anywhere you can use a branch name, indiff,incheckout,inswitch,inmerge,inrebase,etc., etc., and vice versa. Branch and tag refer to specific commits in a tree of commits. A directed graph that starts with the first commit, the seed, and shoots upward until those first branches, and then along each one.

If you like the tree analogy, we are about to blow it up. If you dislike it, this next bit is for you.

So what’s adetached HEAD?Consider that pretty much anywhere you can use a tag you can use a branch name and vice versa, and pretty much anywhere you can use either you can use a commit ID.

git checkout commitThatIsNeitherTaggedNorTheEndOfTheBranchwill take you to adetached HEAD:You reach adetached HEADwhen your current working commit is not otherwise labeled, that is, it has not been tagged and is not the last commit in/on a branch. It doesn’t matter how many descendants it has, it just isn’t the last commit on ANY branch and it isn’t tagged.

If you change something there and make commit it, you will have “created a branch”, in that you will have diverged your tree, but you won’t really have created a branch, because you never labelled it as such. You’ve diverged in the tree, in the directed graph, without branching, as the idea is most often expressed.

In other words, a detachedHEADis git telling you that you didn’t follow the expected rules. But there aren’t really any rules, just conventions, and, if you understand how these commits and labels (these references) work, you can always get there from here or here from there. So call this Lie #4: adetached HEADis git telling you to follow the rules. That don’t exist. Paradox if true. But false, so logically consistent. Right? Right.

When you decide to go back in time or when you end up back in time and decide that you where you are is where you need to be, you make a change. Now you need to integrate that change into your later work. How?

The traditional advice is as follows:

Commit.
Make a branch at that commit.
Jump to where you want that work (usually by checking out another branch and thereby moving to its end)
Merge the branch from step 2 into your current location.
Sometime later, delete the branch from #2, since you do not need it anymore.

But you don’t need to make a branch, which means you don’t need to clean it up, so steps 2 and 5 are irrelevant. You just need to:

A. Commit – and note the commit ID
B. Jump to where you want that work
C. Merge that commit ID into your current location.

Done.

Huh. What does our tree look like now? The tree analogy just died because we reached into its trunk (the detached HEAD, that commit from the distant past) and we pulled it forward in time and married it to the end of one of the branches – or to the top of the trunk, if we merged it into master, and hadn’t yet merged anything else into master.

Well, mostly died. Because we may have usedrebase.If we usedrebase,we just morphed the tree into a bigger tree. Insert favourite SciFi/SpaceOpera metaphor here.

Think about starting our shoot and branching off intoapproach01 and approach02,so that we have two distinct branches heading to the sky from the trunk we are calling master, then realizing that we need both of these approaches, that they solve different problems. We need to combine them somehow. Most commonly, we would use merge, but we could also use rebase. What’s the diff?

If we

git checkout approach01
git merge approach02

we join our two branches so that we have a trunk and two limbs that split off (that diverge from master) only to rejoin, to converge, somewhere above the trunk. If we then

git checkout master
git merge approach1

suddenlymasteris at the top of the tree. The fact that we took two distinct paths to get to this new master, pathapproach01and pathapproach02,is preserved forever, reflected in our commit history. In fact, becausemastergets pulled along to theapproach1commit, you can picture the three paths as a trunk that separates into three branches with empty space between them only to have the three branches converge and start growing as a single branch again.

But what if instead we had done

git checkout approach01
git rebase approach02

and

git checkout master
git rebase approach2

???

In this case,git rebasebasically says do not preserve the distinctiveness of these paths, make it seem as if there was but one path, which we mean to walk all along: The result of doing these rebase operations is that despite the fact that we branched – diverged – and wandered around design space for a time, so that our tree started to get twisty and possibly gnarled, it is now a single shoot, straight and true, from ground to sky (or whatever altitude master happens to be).

So, yeah, the tree analogy is weak. But all analogies are.

git commits are arranged as a directed graph, one that starts from the first commit and continues linearly, unless and until one branches, at which point the graph diverges. Unless and until one branches, each later commit has as its ancestors all earlier commits (each commit has one and only one ancestor commit, except the first one, which has none).

Ancestry diverges when a branch occurs – but each commit still has only one ancestor commit, unless and until there is a merge.

git mergejoins commits on two separate branches into a new commit, one with multiple ancestral paths: That commit has two or more ancestor commits, and, most of the time, those commits will have one or more common ancestors (not always, we won’t go there, but git does allow merging of completely unrelated trees).

git rebasejoins commits on two separate branches into a new commit with but a single ancestral path, returning us to linearity, so each commit has but one ancestor commit, all the way back to the beginning of the branch (which might be the beginning of the tree).

So, Pete, you don’t use branches, then?

I absolutely do. There are excellent articles and man pages on git workflows, e.g., the kernel workflow, and those workflows are enabled by and depend upon the branch concept. I don’t think I could work with others – and even some of my weirderhmm, will this workideas – without branches.

It’s just that thinking about the branch as its HEAD and not as the whole limb helps me understand what git is doing for me and how we can help each other. At a branch, I am at the labelled end of series of a related commits. But I don’t need to be there, I can be anywhere on the branch – as long as I am careful.

If I am on a branch but not at a branch, I am detached.

(Another git koan.)

What about remotes, then?

All of these same concepts apply when working with remote branches, whether those are on your own on server or they are those of others, members of your team or others with whom you are collaborating. Remotes are simply pointers to other commit trees, other directed graphs, out in the world somewhere.

What happens when you do a pull?

Like the man page says,git pullis basicallygit fetchfollowed bygit mergeFETCH_HEAD: YourHEADis your recent commit on your branch,FETCH_HEADis the most recent commit on the copy of the branch you just fetched, andgit pullmerges that commit and its ancestors onto yourHEAD,creating a new commit and movingHEADand the branch name to that commit.

Note an important part buried in the previous paragraph: the copy of the branch you just fetched. You aren’t working at or on the remote, git fetch created a local copy of that other branch, and you work with that local copy. As is noted elsewhere, ultimately, all git is local, git “just” provides various commands for moving copies around so that we can work with them, together.

Of course, as soon as you have remotes, you have the possibility of merge conflicts, since changes can be made to the same code in different places. Fortunately, git has tools for resolving merge conflicts: git developers recognize merge conflict resolution as a key “value add” for git, one of the main ways git makes distributed development so much easier, and they spend a lot of time and effort making remote access and fetching and merging work really well. Subjects for another session, perhaps.

Sincegit pullis basicallygit fetchandgit merge FETCH_HEAD,what if you didgit fetchandgit rebase FETCH_HEADinstead?

You know how I said there weren’t any rules, just conventions? Yeah. Conventions can be very important. If it’s your sandbox, you may do as you please, set the rules for others, if you want them to play with you. If your rule is rebase not merge, well, it’s your sandbox.

Most sandboxes don’t work that way, though: Merge and rebase preserve all commits, which means that both preserve a history, it’s just rebase doesn’t preserve how the history happened, only the events, in sort of the right order but not really. Merge does.

For most projects, the actually history of the history is important because understanding how we got here from there can often help us better understand the nature of the work, the nature of the problems we are trying to solve, the nature of solution space. Rebase hides or obscures potentially important exploration.

You are free to rebase. Especially your own code, in your sandbox. But don’t be surprised if others are inclined to a dim view of those who would rebase the work of others and discard potentially important history, or at least obscure it.

My preferred take onrebaseVsmerge

Rebase	Use`git rebase someOtherCode`to bring that other code into your unshared branch, e.g., a feature you are developing, to make sure that your code integrates well with what others have done since you started.
Merge	Use`git merge ...`to integrate two branches, especially shared branches

Or, to put it more simply:

`git rebase`	Private, never-before-shared code.
`git merge`	Anything published, shared, or otherwise in two or more places, all of which are not under your direct control

Or, even more simply:

What has been seen should never be rebased.

Violate that only when you and your collaborators truly understand why.

With all that in mind, is a remote a branch, is the same way that a branch is a branch?

No. While they can be listed withgit branch -[ar],and while you can usegit branch someNameto make a local branch that tracks a remote calledsomeNamejust appear, they are not branches.

Branches are places you can go. At least from the perspective of this article.

You may work with remotes, but you can never get to them. You fetch or pull from them, you push to them, but you can never be there. You can never be on them or at them.

This is very different from tags, branches, and even commitIDs, which represent places in commit space you can actually get to. Think ofgit checkoutandgit switchas a bit like themvcommand, which takes you from where you are in the filesystem to a new place in the filesystem: checkout and switch take you to named places in the same way that mv does.

Being the master of the bad analogy (have you read my tree work?), git remotes are not like remote filesystems that can be mounted via NFS or SMB and navigated directly as if they were local. They are more like archives, tar or cpio files, that you download, then extract over your local files, but the extraction program is extra clever, and cleanly integrates your local content with the archive content, pausing to let you resolve conflicts it cannot figure out.

At least that’s the analogy forgit pull.git pushis sort of like the reverse, but not really (git push is much closer to being the opposite of git fetch: if you think of fetch as copy from remote to local, push is copy from local to remote… …of course, the remote need not be remote, it can be another local folder or filesystem, but’s that for another time… …unless you grok that right away… :->).

The cool thing is that agit bundleis an archive and can be treated exactly like a remote, pulling from it and pushing to it, for those times when you need to share or move code but cannot do it directly in a connected fashion. Another thought for another time.

So, no: A remote is not a branch, even ifgit branchwill list available/known remotes. My take is that the git developers know that remotes aren’t branches but that they value usability and friendliness and sensible conventions over slavish devotion to intellectual and/or design purity. The more I use git, the more I respect their work.

But if you start thinking of branches as places, not paths, and stop thinking of remotes as branches, a lot becomes clear. Er. Clearer. Yeah, that’s one.

One big step (at least for me) on the road to git clarity….

References and acknowledgements

This article and the perspective behind it owe a great deal to the following. I especially need to reread the second one a few more times….

The thoroughly enjoyable Think Like (a) Git
- The resources section is especially good. Some of my favourites are Scott Chacon’s Git Internals and Ryan Tomayko’s The Thing About Git – I particularly like Ryan’s take ongit add --patchand howgit addis, in general, a command for manipulating theindex,akastaging,aka, thecache.
A thoroughly underrated stackoverflow answer

Paradox-free Time Travel! Yay! Wait, not so fast....

2020-09-29T00:00:00+00:00

A recent paper on Closed Time-like Curves has editors the world over asserting that paradox-free time-travel is now possible! Readers have gotten pretty excited too….

Fantastic! If we travel back in time and cause an accident, we’ll still exist when we get back!

Well, no. Or at least, not quite. These headlines are inaccurate: a more accurate headline might be “Paradox-Free Time Travel Possible, Provided Closed Time-like Curves (CTCs) Exist (in our universe)”.

It’s a bit wordy, and loses some of the buzz, so one can understand why editors have opted for click-bait.

Notice the conditional in there, the proviso? If CTCs exist - or can exist - in our universe, then we get safe time travel. But that is a huge “if”….

CTCs are a set of possible solutions to the equations of General Relativity (GR). The equations of GR are non-linear and it is not always possible to produce analytic (complete) solutions, so physicists often use approximations to get close to a solution. If a solution is interesting, e.g., T=0, it is studied further (oh, the Big Bang (literally: T=0, everything starts)). One can also make assumptions about some elements of the equations (if we set this to P…) and see what pops out (oh, Big Bang! oh, CTCs!).

The equations of GR are studied as much for how interesting is the math as for their potential physical value. CTCs have been studied for over a century, in part as an intellectual exercise (oh, these are weird, could things really work this way), in part to learn if their study can inform open problems in physics.

One such open problem is that GR and Quantum Mechanics (QM) are inconsistent with each other. For example, GR assumes continuous functions and continuous values (it is a classical theory, in that limited respect) while QM assumes that observations can take on only discrete values and, as a result, functions used in QM are those amenable to treating discontinuous ranges of values. This is where the term “quantum leap” comes from: An observable leaps from one value to another without having any of the values in between; in classical mechanics and GR, this just isn’t allowed. (Fundamentally, the maths of GR and QM don’t even share much commonality; I saw a great graphic about this years ago, and I regret not saving it.)

Cosmology is informed by both QM and GR: GR tells us more or less how to interpret the structure of the universe at a large scale, over large time frames (was there a Big Bang; is the universe expanding; is there dark matter; how do stars evolve and black holes form; etc.) while QM gets us the smallest scales and tells how information behaves in the universe, e.g., is information lost at the event horizon of a black hole, can physical processes be irreversible, etc.

One of the most important aspects of this is that of the four fundamental forces (the electromagnetic, the strong and weak nuclear forces, and gravity), three are described by QM (the first three) and form part of the Standard Model (in which every force is transmitted by a “particle”, e.g., the photon for the electromagnetic force, and from which many incredible predictions have been made, e.g., the existence of the Higgs boson, recently confirmed at CERN). The fourth fundamental force, gravity, the most important to the overall structure of the universe, comes from GR. In particular, there is no “graviton” that carries it (or at least not yet).

When physicists talk about a Grand Unified Theory, they are hoping to reconcile GR and QM, e.g., to make quantum gravity. Or more accurately, to quantize gravity.

Since CTCs are a pure-GR thing, whether or not they can exist should depend on whether or not they are consistent with or violate QM. For example, the paper cited above does include as one of its references a paper arguing that CTCs would violate the Heisenberg Uncertainty Principle, which is pretty much as fundamental to QM as the constancy of the speed of light is to GR.

Where things get really interesting is when one combines CTCs with computational complexity….

If CTCs exist, Aaronson and Watrous have shown based on work by Deutsch that quantum and classical computers have exactly the same power - provided that one could engineer a classical computer to operate in a CTC (I’m going out on a limb stating that, a lot of Aaronson’s work makes my head hurt). Deutsch’s work isn’t universally accepted either.

So. What is the article saying?

If CTCs can exist in our universe (if they are consistent with our most-successful-ever theory, our most tremendous intellectual artifact, QM), then paradox-free time-travel can exist.

But that is a very, very, very large IF. Like 96 point bold underlined flashing neon.

What’s the point, then?

The more we learn about GR and what it allows and forbids, and the more we learn about QM and what it allows and forbids, the more clues we obtain to what is wrong with either and what a future synthesis or replacement theory might look like.

Studying paradox-free time-travel via CTC may provide theoretical insights or it may actually predict observables, either confirmatory (if we see it, CTCs are real) or contradictory/falsifiable (if we see it, CTCs cannot be).

So we don’t know anything more for sure, yet. But we have a better idea of how things might be, provided other things are. We just don’t know what those other things are yet.

Stay tuned….

Python Best Practice re methods Vs external functions

2020-09-24T00:00:00+00:00

Q for my Pythonista friends….

I’ve several utility functions for processing complex data in a file, each function being pithy: read from the file, validate the data, display it if in debug mode, process it, etc. Each function exists in its own module. None have output, they either return valid data or raise appropriate exceptions.

Just this morning I created a Class to represent the structure found in the file and moved the calls to the utility functions from __main__ to the class constructor, and rewrote __main__, shortening it quite a bit.

Everything works. The executable imports the Class, the Class module imports the utility functions, all is well. All of my tests pass. Yay me.

I like “all is well” and I am generally a “if it ain’t broke don’t fix it” kind of person, but I am stalled on wondering whether I should make those utility functions class methods: They are used exclusively for processing this very specific data, and don’t and won’t have a life of their own. Do I leave them as-is (everything works) or do I bring them into the class module? (I did this as a test, all works just as well. But I want to decide on a direction before making the initial commit to this branch.)

My argument for doing nothing is a mix of “it ain’t broke” and aesthetics: The class module is short and pretty; bringing them in increases module line count by over 200.

My argument for bringing them is less clear: It will reduce clutter in the folder that holds everything, and they don’t have a separate life, so having them be “private” methods within the class seems sensible, and I think it will make import and module packaging simpler, but I don’t know, hmm, fret….

I know I am over thinking this, but….

What’s the best practice guidance?

I am nothing ⊃ ¬Patient

2020-09-01T00:00:00+00:00

Once, and I don’t recall why, I said, probably to someone else, “I am nothing if not patient”.

Later, with grudging head-bobbing, self noted that I wasn’t (always) (particulary) patient.

Did I mention that my degree’s in physics (meaning I studied more math than anything else) and that I got most of the way to a degree in philosophy?

So, yeah, there is a strong inner logician. He sleeps a lot, but this woke him.

He eeked out “if we are not patient, are we therefore nothing?”

That was a <insert face-slap/cold-water metaphor here> moment.

It worked, though. Many/most/at-least-some moments of impatience were followed by moments of reflection. I am not patient. What am I? I don’t feel like nothing, but I don’t feel really good about this.

It took a long time.

I feel it. And I think most people consider me pretty chill these days, even thems that know me really well and see private me. It’s a good feeling.

FWIW. YMMV. But one never knows, eh?

Doing Python development under Mac OS

2020-08-19T00:00:00+00:00

tl;dr? Jump right to the recipe

For reasons lost in the mists of time, I’ve been a Mac guy since 2013 - but I’ve been a command line (CLI) guy since 1990 (or earlier, hard to remember), and still am. A few years ago, I discovered brew, aka homebrew after finding out how out-of-date were so many of the standard UNIX/Linux tools included with Mac OS, e.g., bash.

Generalizing, the hardest part of working with the CLI on a Mac is having the same experience on all the platforms I use, which range from Windows 10 with WSL (some flavour of Ubuntu, cannot remember which), RHEL 7.6, Fedora 32, Mac OS 10.15, and Cygwin - and that problem persists across those other platforms, which is why I created a GitHub project to keep my dot files in snc.

Well. Until the last few days.

Since last week I’ve been using the most excellent Yamale to validate YAML schema I’ve defined for my SELinux Linear Assured Pipeline. I’ve even made some contributions to it - or, at least, am in the process of so doing…

…and that’s where things get weird.

Yamale depends upon Python and pip, the Python package manager, and upon tox, a test automation framework for Python - but standard Mac OS has neither pip nor tox. OK, so brew install pip and brew install tox, right? Well, almost.

This is where things can get a little weird. Yamale’s tox.ini, for example, depends upon two different versions of Python, 3.6 and 3.8.

How does get multiple versions of Python? The short answer is pyenv, a python-independent shell-script based system for managing multiple Python environments. But that’s not the whole story.

This page intends itself as the correct answer - install pyenv and go! - but it is lacking a step, as I discovered - and I discovered that page from this page, which tells us what we might do wrong, then what we should do to get things right - but a bootstrap problem remains….

The first page wants us to run pip install pyenv - but Mac OS lacks pip. The second page has a few brew-based recipes, but suggests that brew is the wrong way to go, overall, and that we really want pyenv. This third page tries to put things succinctly, but if you’re running tox, there is YAS (Yet Another Step): By default, tox works only with your default Python environment, whether that is standard or something installed via brew.

To get tox to work with multiple Pythons out of the box, you need tox-pyenv - but that page mentions a lot of irrelevant Circle CI stuff.

Before we go any further, I should note that whatever you do
with brew - or with gem, if you get there - do NOT override
system defaults - Mac OS and many utilities produced for the
Mac depend upon them. Let brew manage things and install them
cellar-only. You then need to find them via appropriate
manipulation of PATH, CPPFLAGS, LDFLAGS, etc. Cf my dot file
project for one way of doing this.

At any rate, none of those pages provide the complete answer. You know that you need

Python (and possibly multiple versions of Python)
pip (not included with Mac OS)
tox, which depends upon pip
pyenv, which you can get a number of ways….
tox-pyenv, which depends upon pip….

So what is one to do? Install pyenv and use it to install pip and tox and tox-pyenv several times over? That seems so….

UGH.

I repeat.

UGH.

I don’t know that what follows is a universally-accepted approach, but it is the shortest and cleanest path to where I needed to go (as verified by running make clean; make in my yamale top-level folder and having all tox tests work out-of-the-box with both Python 3.6 and 3.8, no modifications to yamale files required…).

So, YMMV and FWIW, here is….

Just The Recipe

# install brew
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
# use brew to install python3, which comes with pip3
$ brew install python3          # let brew install this cask-only
# figure out your path to pip3, installed with python3
$ <path to pip3> install tox
# install py-env using brew
$ brew install py-env
$ <path to pip3> install tox-pyenv

Once you’ve done that, figure out what versions of Python your particular tox.ini specifies; for yamale, these are 3.6 and 3.8 which I’ve mentioned a few times already…. Assuming these are what you need, the rest of your recipe is as follows:

$ brew install pyenv
$ pyenv install 3.6
# lists multiple versions, choose appropriate minor #, e.g.,
$ pyenv install 3.6.11
$ pyenv install 3.8
# lists multiple versions, choose appropriate minor #, e.g.,
$ pyenv install 3.8.5
# almost there, now we want to use what we just installed
$ cd <where Yamale lives>
# have pyenv set the versions of python tox should use
$ pyenv local 3.6.11 3.8.5
# that command creates a ./python-version file, which
# is already in ./gitignore! Now confirm those....
$ pyenv version

In my case, the latter command shows

3.6.11 (set by /Users/pww/pipelineWork/yamale/.python-version)
3.8.5 (set by /Users/pww/pipelineWork/yamale/.python-version)

pyenv which shows me what I expected:

$ pyenv which python3.6
/Users/pww/.pyenv/versions/3.6.11/bin/python3.6
$ pyenv which python3.8
/Users/pww/.pyenv/versions/3.8.5/bin/python3.8

Now. Run make clean; make and all should be well.

It was for me, at least.

Easy-peasy. Right?

Right.

Buying a used Wrangler JK/JKU: Rubi or Sport?

2020-08-17T00:00:00+00:00

Someone posted in an FB group indicating they were going to buy a 2012-or-later Wrangler and asking whether they should get the Rubicon or the Sport. Well, my comment was longish (surprise, surprise), so I thought I would reproduce it here.

I bought a Rubi new in 2014. It was my daily for five years, and was wheeled hard. Now it is a trail queen, and will, eventually, be a trailer queen (my truck is now my daily).

Pros of the Rubi:

Electronically controlled lockers and swaybar disconnect.
Beefier rear axle and tube; beefier front axle (the front tube is sort of a 37, it’s a shaved 44, not as tough as the rear, tougher than a 30).

Cons of the Rubi:

Price.
The wires for the lockers can be ripped off by branches if one is cavalier about them and replacing the sensor and actuator is a huge PITA (I am less cavalier now :->).
The swaybar disconnect needs regular maintenance (remove, disassemble, clean, grease but not too much) if ever exposed to water, dust, etc., which it will be if you wheel it all.
The SmartBar component that controls the swaybar has NO protection against water or dust intrusion and the harder you wheel the more likely it will fail, eventually, taking your canbus with it - the Christmas Tree display is, uh, surprising. It’s $2500 to replace. I replaced the entire thing with a Heilweig heavy duty and have been disconnected since June, no problems anywhere.

How to choose? It comes down to two factors:

how mechanically inclined you are, and
how soon you want to get to hard wheeling.

Less of the former and more of the latter == Rubicon.

More of the former and less of the latter == Sport (save money now, upgrade as you discover your wheeling style).

Final note: If you are serious about buying a used Rubi, take it for a test drive, get it onto gravel or dirt, put it in 4Hi, drive around a bit, shift back to 2WD, then back to 4Hi, make sure that is smooth and easy. Stop, put it in neutral, shift into 4Lo, drive some more, and try the lockers (one click is rear, two is front, three is both (? not sure, I always go by the indicator lights) and the disconnect (the lockers and disconnect only work in 4Lo at less than 40kph).

Make sure everything works, including all the indicators. (Try using any of the lockers or swaybar in 4Hi or in 2WD: The indicators should be flash, showing that the component is unavailable.)

If any of these tests fail, run away. Or ask for a deep, deep discount.

Well, I'm back

2020-08-07T00:00:00+00:00

Five years is a long time. Why stop? Why start again? And why was restarting so (technically) painful?

tl;dr It was 2014; I thought I should learn git and GitHub - didn't use it, just thought it would be cool. The blog was a practical exercise in both. I'm not sure why I kept it going (not that it went very long). Work changed. Life changed. Now I use git on work's self-hosted GitLab sometimes weekly, sometimes hourly: I needed to step up my git game. Looking up a very specific how-to dropped me into a four day rabbit hole....

Also tl;dr In this very long post, you'll learn (at least part of) how to maintain a Jekyll-based GitHub Pages blog under Mac OS. There is an awful long in here, take your time.... :->

For most of the last 20 years, I’ve been a consultant. Work inputs were meetings, site inspections, emails, phone calls, interviews both formal and informal, and reams of policy and procedure documents. Deliverables were documents and presentations and emails. Mostly.

The backstory - getting started, back then, and stopping, and starting again….

2014 began the same way, but looked to become more technical. I’d been curious about git for some time, but had neither work need nor leisure time (foreshadowing: life changes) to make any effort. With the 2014 work becoming more technical, that looked to change. I’d also always meant to start a blog, so, brace of birds, single rock, and all that. Yay, the blog was born. There was much rejoicing.

Actually, there were pretty much crickets. I didn’t feel badly about that, many blogs are ignored. Many blogs also peter out over time, so I didn’t feel too badly about that, either (well, sort of kind of? More life change foreshadowing, duh duh daaaaah).

Fast forward to earlier this year: A consulting gig has me coding. February and March find me in the basement typing away for 10 to 12 hours a day, 6 or 7 seven days a week. I was in heaven - the beginning of lockdown went largely - but not entirely - unnoticed for me. Yes, I have the most patient girlfriend in the world, she is sweet and adorable and understanding (queue life change foreshadowing minor key).

That coding involved git. Just basic stuff, trying to get my head around git pull, git add, git commit, and git push. Branches were a step too far, as was using SSH.

That was then (has it really only been five months?). That consulting gig morphed into full-time employment, the first time I have been an employee since 2001 (2013-2014 kind-of/sort-of had employment, but not really, that was a strange period… …maybe one day I’ll share an askari story (there’s a pun there; I quite like it; grokking it requires knowing the story; maybe one day; I digress…)).

That full time job is a mix of marketing, marketing management (mostly product/roadmap), client liaison, prospect management, business development, and hands-on. Instead of writing this, I should probably be working on the compliance platform User Acceptance Plan for one customer; or SELinux linear channel controls for two customers; or the cloud-based compliance platform demonstrator; or a few other things… (I’ve got GitLab issues aplenty assigned to me…).

A few days ago, I thought it would be a good idea to upload my SSH public key to the company GitLab and use that instead of passwords. My git/GitHub/GitLab isn’t yet second nature, so a lot of comparatively simple things require research and reminders - and I dislike breaking things through my own ignorance, so off to GitHub I went to remind myself how to upload SSH keys, etc.

In noodling through my account settings, I noticed that all of the security and analysis features were disabled and I said to myself, “Self, why are these off? Surely they are a good idea!”. Self thought this was very reasonable, did a bit of poking about, satisfied itself that CHECKING ALL THE BOXES!!! was a good idea, and did so….

The emails began to arrive seconds later.

I don’t have a lot of code in GitHub, mostly personal stuff, very few forks. Some of that code failed one Dependabot test or another….

This blog is hosted on GitHub. It hadn’t been updated in six years. There were, uh, vulnerabilities and out-of-date code aplenty.

And the blog broke. Pagination broke. Application of CSS broke. A lot of things broke.

It was embarassing, really. I had to fix it. I just had to!!!

There were a few basic issues:

Out of date code, mostly Jekyll
GitHub pages had stopped supporting
- relative links
- the markdown engine I’d been using
- URL specifications such as {{ site.baseurl }}
The IP addresses for custom domains had changed

The last of those generated build warnings; the first few build failures. I’m not a Ruby guy, so it took a while to find everything and figure things out… …but I couldn’t even start figuring things out until I updated my Mac OS development environment… …which I didn’t realize I had to do for a while.

A digression - how to manage `dot files` across multiple machines

On any given day, I might work on any or all of two Macs (my ancient Air, my work Mini); one or more Linux VMs on the Mini (my Fedora SELinux VM; CentOS or Fedora VMs for the compliance automation platform); one or more customer RedHat VMs for one of my contracts; and WSL, the Windows Subsystem for Linux.

A digression within a digression - WSL is awesome

Until PowerShell, Windows shells sucked. PWSH changed that. It’s powerful, well-designed, and can be a lot of fun to use. But if one is mostly using Linux and bash, it can be a pain to make the mental switch from bash to PWSH and back again.

So I installed CygWin on a few Windows machines and it was OK. Pretty good. Not great. Somewhere along the way, WSL popped onto my radar as I was intrigued. “Huh. Real Linux? On Windows? Directly? Can’t be!!!”

A little poking about, a little research, installation of WSL, installation of Ubuntu under WSL, WHAM!, real bash. Heaven!

We now return to our regularly scheduled inception, er, digression

When I switch machines, I like to have the same .vimrc, the same bash prompt, the same ls defaults, etc.

At first, I was mucking about with Google Drive and uploading and downloading .rc files, but that was a real pain. I took a while to realize that I could a) reduce/eliminate that pain and b) learn more git, if I created a GitHub project for my dot files.

One of the challenges of getting there was merging various .rc files. On Air, my .bashrc had a bunch of entries relating to things like Ruby, ICU4C, etc. I had no idea what they were, no idea why there were, so I commented them out: I didn’t want in my environment things I didn’t understand and didn’t know why they were there. Pretty reasonable, I think.

We will now pause the first digression

There was a lot of Googling to rediscover how to manage a Jekyll-based site on the Mac OS command line. IM(NS)HO, the Jekyll documentation runs at two levels, too basic and too complex. Much forensic guess work was involved.

Long story short, I started running various gem and jekyll commands and nothing was working.

Most notably, I could neither figure out nor remember what this bundle thing was nor how to find it.

Another digression - keeping a Mac OS development environment up-to-date

I’d installed brew and a few other things in 2014 as part of getting the blog going in the first place. Sometime after that, for reasons lost in the mists of time, I installed node. Later, doing advisory and content development for ClickArmor, Node was a great engine for testing some JS code I was hacking about. After a few false starts, I wrote a script to simplify my life:

$ cat ~/bin/update
#!/bin/bash

brew update
brew upgrade
read -p "Ready to continue with npm and node (ctrl-c to skip)? " answer
npm-check -u -g
npm-check -u
node -v
npm -v
$

It’s basically an encapsulation of just-in-time reseach I knew I was likely to forget: Run the brew update and upgrade commands, let me decide whether or not to update node, which can take a while, run the necessary npm-check and node commands….

So far, so good.

We will now pause the second digression

In puzzling out restarting blog management, I realized I needed a few gems - and that my gems were out of date. My first thought was to update my script…

...
brew update
brew upgrade
gem update
read -p "Ready to continue with npm and node (ctrl-c to skip)? " answer
...

This didn’t work. At least not well. I kept getting this error:

ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /Library/Ruby/Gems/2.6.0 directory.

More Googling revealed little/nothing of value, so I naively tried:

...
brew update
brew upgrade
gem update || sudo gem update
read -p "Ready to continue with npm and node (ctrl-c to skip)? " answer
...

Yay! No more errors! Great, right?!?!

Yeah, no, not so much.

I didn’t have any errors, but the site still wasn’t building.

I was lost, so I did what I probably should have done to begin with: careful inspection of all update/upgrade messages, wherein I noticed various brew indications about things being installed cask-only.

Distant bells began to ring, echoes from the past. Right. Mac OS includes a few things that one can also get from brew, notably herein, gem, but neither jekyll nor bundle.

More puzzling. Finally, EUREKA!

I needed to

Install Ruby via brew
Update my bash environment so that brew versions of various things appeared first in various paths

In other words, I was using the Mac OS versions of gem, etc., which are older, and wasn’t picking up bundle, et al, at all.

Step the first was easy enough. Step the second got me looking through my .bashrc….

Merging digressions the first and second

I wanted to manage my bash environment conservatively, only adding to it things that made sense on the machine in question. I’d made a start at that some time ago:

...
function isMacOS {
    false
}

function isLinux {
    false
}
...

case $OSTYPE in
    darwin*)
        function isMacOS {
            true
        }
        ;;
    cygwin*)
        function isCygwin {
            true
        }
        ;;
    linux-gnu*)
        function isLinux {
            true
        }
        case $(uname -r) in
            *Microsoft)
                function isWSL {
                    true
                }
                ;;
            *)
                :
                ;;
        esac
        ;;
    *)
        echo; echo "What OS is this?"; echo
        ;;
esac

Nothing terribly complicated, a few utility functions to help manage conditional inclusion of environment variables, paths, etc., based on platform.

Among other things, the isMacOS function appeared in that commented-out block of settings related to ICU4C, et al, mentioned above.

Lightbulb flash

Right, that’s what those were for. All of that code was first added to my Air .bashrc to make sure I was using the brew versions of various utilities, so that I could build and test the blog locally before pushing it to GitHub. Riiiight!

Progress.

Did I enable all that code? No. I was tempted to. But conservative in code application, remember? I realized I wanted it to be conditional, based at least on whether or not various components were present.

This is what replaced the commented-out code (the path-bit bookends are included for completeness):

if [ -z "${PATH}" ]; then
    # there should be a default path, but in some weird cases, perhaps not
    # set a reasonable default
    PATH=/usr/bin:/bin:/usr/sbin:/sbin
    export PATH
fi

# Take full advantage of BREW, if on Mac OS and if it installed
if isMacOS; then

    # linked brew formula
    checkFor PATH /usr/local/bin
    
    # ic4uc items
    checkFor PATH "/usr/local/opt/icu4c/bin"
    checkFor PATH "/usr/local/opt/icu4c/sbin"
    checkFor LDFLAGS /usr/local/opt/icu4c/lib"
    checkFor CPPFLAGS /usr/local/opt/icu4c/include"

    # ruby items
    checkFor PATH "/usr/local/opt/ruby/bin"
    checkFor LDFLAGS "/usr/local/opt/ruby/lib"
    checkFor CPPFLAGS "/usr/local/opt/ruby/include"

    # python items
    checkFor PATH "/usr/local/opt/python@3.8/bin"
    checkFor LDFLAGS "/usr/local/opt/python@3.8/lib"
    checkFor PKG_CONFIG_PATH "/usr/local/opt/python@3.8/lib/pkgconfig"

    # openssl items
    checkFor PATH "/usr/local/opt/openssl@1.1/bin"
    checkFor LDFLAGS "/usr/local/opt/openssl@1.1/lib"
    checkFor CPPFLAGS "/usr/local/opt/openssl@1.1/include"
    checkFor PKG_CONFIG_PATH "/usr/local/opt/openssl@1.1/lib/pkgconfig"

fi

# always prepend my bin, if it exists and is not already there
checkFor PATH ~/bin

The interesting bits:

Set a reasonable default PATH, if PATH is unset
If running on Mac OS, update various paths iff (if and only if) certain packages are present
If it isn’t already there, prepend ~/bin to my PATH

This works really, really well.

But what’s this checkfor thing? I’m really pleased with this:

checkFor () {
    path="$1"
    want="$2"
    extras=$3

    if [[ -z "$path" || -z "$want" || -n "$extras" ]]; then
        echo "$FUNCNAME requires PATH WANT; called with '$path' '$want' $3; doing nothing"
        return 1
    fi

    # if the desired folder doesn't exist, return - nothing to add
    [[ -d "$want" ]] || return

    # if the wanted folder is in the path, return - nothing to add
    isInPath "$path" "$want" && return

    # we only get here if we need to add to the path
    # now we need to know what type of path it is
    case $path in
        PATH)
            export PATH="${want}:${!path}"
            ;;
        PKG_CONFIG_PATH)
            export PKG_CONFIG_PATH="${want}:${!path}"
            ;;
        LDFLAGS)
            export LDFLAGS="-L${want} ${!path}"
            ;;
        CPPFLAGS)
            export CPPFLAGS="-I${want} ${!path}"
            ;;
    esac
}

Fairly simple logic:

Check for the right number of arguments (two):
1. The type of “path” to update (PATH, LDFLAGS, etc.);
2. The folder to add to that “path”
3. If there are three or more arguments, chances are that
  - The name of the folder “want” contains spaces and
  - That name wasn’t quoted when passed to checkFor
If the folder “want” doesn’t exist, do nothing (e.g., on a Mac where I don’t run brew)
If the folder is already in whatever “path” it is, do nothing
- isInPath is explained below
If
1. all arguments are correct,
2. “want” exists, and
3. “want” isn’t already in “path” add it to the appropriate path, with appropriate separation and prefix, if any

checkFor relies on:

isInPath () {
    path="$1"
    want="$2"
    extras=$3

    # we need exactly two arguments; anything else is probably a quoting problem
    if [[ -z "$path" || -z "$want" || -n "$extras" ]]; then
        echo "$FUNCNAME requires PATH WANT; called with '$path' '$want' $3; doing nothing"
        return 1
    fi

    # if $path is empty, $want obviously isn't present
    [[ -z "${!path}" ]] && return 1

    # replace any ~ with $HOME, to be on the safe side
    [[ "$want" == *"~"* ]] && want="${want/#\~/$HOME}"

    if [[ "${!path}" != *"$want"* ]]; then
        # echo "NEED: no '$want' in '${!path}'" 
        return 1
    else
        # echo "GOOD: '$want' is in '${!path}'"
        return 0
    fi
}

Again, fairly simple logic:

Check arguments
Return false if “path” is empty - if it’s empty, we need to create it, which we do by assigning to in checkFor
- Note the use of ${!path} to cause “$path” to be replaced with the contents of whatever variable it represents, e.g., if $path==PATH, ${!path}==$PATH
Replace any ~ in “want” with the contents of $HOME, to simplify matching
Return
1. false if “path” doesn’t contain “want”
2. true if it does

checkFor updates “path” only if isInPath returns false.

With that in place, I could update ~/bin/update (remember that?). Long story short, it is now:

#!/bin/bash

echo Updating brew....
brew update
echo Upgrading brew....
brew upgrade
echo Updating gems
gem update
read -p "Ready to continue with npm and node (ctrl-c to skip)? " answer
npm-check -u -g
npm-check -u
node -v
npm -v

Notice that sudo isn’t there anymore: with various environment variables set correctly, all of the updates take place in the right place, regardless of whether that’s system-wide or keg-only: Whatever commands I run pick up the correct environment and do this right thing. Notably,

$ which gem
/usr/local/opt/ruby/bin/gem
$ which bundle
/usr/local/opt/ruby/bin/bundle

But these depend a on few more things we need to cover….

You have reached the end of the first and second digressions

Somewhere along the line, I realized or remembered that building the site on my Mac required these two commands:

bundle install
bundle exec jekyll serve

Long story short, bundle install reads Gemfile, installs whatever is needed, and creates Gemfile.lock with the current list of dependencies. But I was still getting build errors whenever I did git push githubio.

Again, long story short, I realized a few things:

When resurrecting a six-year-old site, it’s better not to have a Gemfile.lock, as it might confuse bundle, and
I was going to forget this.

Enter the Makefile

I love make. I really miss the O’Reilly make book I lent to someone who quit the Big Nerd Ranch and took it with them a little while later. Anyway. Typing

make

doesn’t save a lot of keystrokes, but

$ cat Makefile
all:
	@bundle install
	@bundle exec jekyll serve

clean:
	@rm Gemfile.lock

reminds me that make actually means bundle install; bundle exec jekyll serve, which means run bundle install to update everything then run bundle exec jekyll serve to launch Jekyll as a server so that I can find out a) if local builds succeed and b) if I like the look of the local version of the blog.

It also reminds me that running make clean can help resolve odd errors by removing Gemfile.lock and letting bundle install rebuild all dependencies.

If all goes well, I can then git push to GitHub and the site will be updated for the world to see.

(It just occurred that I could add a push rule to the Makefile. Cool. Maybe later.)

Resurrecting digressions the first and the second, temporarily

One note: make expects real tabs in its makefiles, and I use all spaces in vim. This led to an .vimrc update:

syntax on
set nocompatible
set tabstop=8 softtabstop=0 expandtab shiftwidth=4 smarttab
set ignorecase
set autoindent
set ruler
set showcmd
set visualbell
set nostartofline
set mouse=nvh

highlight Comment ctermfg=DarkGrey guifg=DarkGrey cterm=underline

*autocmd FileType make set noexpandtab shiftwidth=8 softtabstop=0*

The last line is key: If editing any kind of Makefile, change back to

hard tabs with the correct tab stop, and
a shiftwidth of 8, and
do not expand to tabs to spaces or
use soft tab stops

I was very happy to find this.

The digressions are dead. Long live the main post.

OK, that’s an awful lot. What’s the real tl;dr?

Getting ready to build a Jekyll site on Mac OS

This is the shortest possible recipe, as I understand it:

Install the XCode command line tools if necessary
Install Ruby using brew
Update .bashrc so that your environment gets the correct paths
Update your brew installation and all your gems
Remove Gemfile.lock
Run bundle install
Out of an abundance of caution, do step 4 again
Run bundle install
Run bundle exec jekyll serve

A note on installing the XCode command line tools

To check if the XCode command line tools are installed, run:

$ xcode-select -p

This should return /Library/Developer/CommandLineTools; if it does, you are probably OK. Move onto installing Ruby via brew (but keep an eye out for odd failures in later commands).

If xcode-select doesn’t exit, check your $PATH and, if necessary, install XCode from the Mac OS App Store.

If it runs but doesn’t return /Library/Developer/CommandLineTools, then run

$ xcode-select --install

When that completes, run xcode-select -p again.

If the command returns /Library/Developer/CommandLineTools you are probably OK. But. One can run into problems anyway. If the subsequent site building steps fail with odd errors, try the following

$ sudo rm -rf /Library/Developer/CommandLineTools
$ xcode-select --install
$ xcode-select -p

If this doesn’t a) result in /Library/Developer/CommandLineTools and/or b) doesn’t solve weird problems, I don’t know what other steps to take.

Command-line précis

$ xcode-select -p
$ sudo rm -rf /Library/Developer/CommandLineTools
$ xcode-select --install
$ xcode-select -p
$ brew update
$ brew upgrade
$ brew install ruby
$ # update your paths appropriately
$ which gem
/usr/local/opt/ruby/bin/gem
$ which bundle
/usr/local/opt/ruby/bin/bundle
$ gem update
$ rm Gemfile.lock
$ bundle install
$ brew update
$ brew upgrade
$ gem update
$ bundle install
$ bundle exec jekyll serve
$ git status
$ # git add as necessary
$ git commit -a
$ git push remote

Finally! We can correct GitHub Pages build errors!

Yeah, that was my last few days. Every time I ran git push remote, I would get an email a few minutes later with build errors. Long story short, in the six years (!!!) since I created the blog, GitHub pages has

Updated Jekyll, etc., several times over
Dropped support for redcarpet, which I replaced with kramdown
Dropped support for relative permalinks
Moved to relative path names for included content, e.g., CSS
Changed the IP addresses used for custom domain names

Fixing the first two was easy, the Mac OS update procedure described above and a quick change to _config.yml, respectively.

The next two required searching for all included files, e.g., CSS being pulled in via _includes/head.html, and, more confusingly, removing all references to {{ site.url }}, {{ site.baseurl }}, etc., in all files, then removing them - I literally just deleted them and GitHub Pages figured things out pretty well.

The command

$ find . -path ./_site -prune -o -type f -exec grep -i -H -e 'site.url' -e 'site.baseurl' {} \;

was very useful for this; the grep clause can be extended as necessary with additional ‘-e ' commands to find other offenders. The rest of the command is basic `find`:

-path ./_site -prune causes find to ignore the _site subfolder
-type f -exec... tells find to apply grep only to files, and not to folders
grep -i -H -e... tells grep to
- Ignore case when searching (just in case),
- Output the name of any matching file (since it is being passed one file at a time, the default is to not output the file name, which means grep prints matches but not where it found them, unless you specify -H, which forces it to output the file name)
- Match multiple -e patterns against the same file
-o tells find to apply either the path-ignore logic or the grep-in-files logic

I figured the relative-path-thing out after carefully reading a GitHub Community post that I’m not even sure how I found. Careful and determined and persistent Googling, I expect.

The last fix required updating my custom domain’s DNS with the correct IP addresses for GitHub Pages; this is pretty well described here.

Really, the bulk of the work was the elimination of relative links.

In fact, in validating that find command, I realized I am missing a few, notably in atom.xml and in the reference to atom.xml in _includes/head.html… I’ll get those later, the site pretty much works as designed, now.

Was it worth it?

Yes. While it was frustrating to figure all of this out,

I’m confident that my Mac development environments are correctly configured
I got a lot of practical git experience in a very short time, enough to start locking in things like
- git pull
- git push remote
- git stash (I used this a lot when cycling through some potential fixes)

Most notably, I’ve started to lock in the procedure for publishing new posts, which is excellent practical git:

$ git checkout -b newpost
$ # some editing
$ make # remember my Makefile, above: `bundle install; bundle exec jekyll serve`
$ git commit -a -m "..."
$ git checkout master
$ git merge newpost
$ git branch -d newpost

It was lot more pleasant and a lot more harmless figuring this out on a site that doesn’t matter (hey, almost six years without updates) than on company code in the company repo: If I messed that up, my colleague would have had to fix my mistakes, and he is as busy as me and doesn’t have time to clean up my messes.

I feel a lot more confident that I will mostly use git correctly now. That’s a big win. And writing this up solified a lot of points in my head, another win.

(Heck, I even figured out to write Jekyll posts about Jekyll, so that they include {% raw %}, for example!)

Time to get back to work, I guess….

OK, what about stopping six years ago? And all the foreshadowing?

Long story short, six years ago, if it wasn’t work-related, I really didn’t have time for it. It was, uh, unpopular, to devote leisure time to anything at the keyboard when there were couply things to do - even if they were of asymmetric interest, if you will.

We separated in 2017, with the divorce finalized in 2018. Still neither a lot of leisure time nor interest in picking this back up, too many other things to do. Like working my butt off to save for a down-payment on a place of my own and get out of the post-split rental.

Last year, my girlfriend and I moved into my new place, and much time was spent on things house-related and new relationship-related.

(FSM bless theatre and movies: My girlfriend and I became involved romantically while working pre-production on a film in mid-to-late 2017, and behind-the-scenes work on a few plays with Ottawa’s LGBTQ+ theatre company (she’s on the board, and we have a lot of friends and family (mostly logical, with a nod to Armistead Maupin), some lawful, some biological) in the LGBTQ+ community in Ottawa and elsewhere).)

Is that it?

Nah. I have to

Fix the atom.xml problem described above
Make sure a few other things about the site are working as they should (once I figure out or remind myself what should is in this case
Update all my issues (cf my post on how I was using ghi and GitHub issues back then
Remove specific product versions from my .bashrc, making it more dynamic and robust
Add a push rule to the site Makefile
And probably a few other things, too.

OK, back to work for real.

A final note

When getting your Mac OS environment configured correctly, it’s worth using

CMD T

every now and again to open a new tab: This will launch a fresh instance of bash and source your .bashrc, updating your environment to reflect any recent changes, e.g., as a result of brew install ruby or bundle install. While writing this post, I had make fail because bundle was missing key environment variables: I’m writing this on my Mini and I had done most of the debugging work on my Air; my environment only had CPPFLAGS half-right, so make failed.

CMD T, new instance of the shell, rerun make, success.

Why people think software is so much easier than it really is

2015-10-02T00:00:00+00:00

New software is net new but most people work on maintaining or incrementally improving their status quo. Factor in their lack of understanding of what it takes to develop complex systems, and you might suddently understand why their reaction to your shiny new thing is "What took you so long?"

A colleague works outside IT and seems to have a pretty good grasp of how complex IT really is. I mentioned to this person that sometimes a customer will, upon seeing “what they really wanted”, simply shake their head and say “what took so long”, blithely ignorant of the complexity involved. My colleague was, well, almost aghast that someone could think this way, and said “I’d not have thought people would be that, I don’t know, ridiculous? Why would anyone presume something like ‘oh that, yeah, you can do that!’ if they can’t do it themselves? That seems an awfully strange way to think (using the word very casually there apparently)”. This is my extended answer to that question.

I think it has something to do with how usable and useful our tools have become: I think we have an implicit appreciation for the complexity “under the hood” (and that can be meant quite literally), inchoate and rarely if ever articulated, but we lack any kind of context for that complexity, any kind of appreciation for the years and in some cases decades of work required to hide that complexity under aesthetically pleasing veneers, be they body panels or airbrushed aluminum cases.

For example, how complicated was it for you to start your car when last you drove? Was there risk of injury?

Until Charles Kettering invented the electric car starter, a saga in and of itself, given the complex mechanical and electrical engineering required, car owners were advised to grip the starter crank handle with their fingers and thumb on the same side of the crank and to pull up. Why same side? To reduce the risk of a broken thumb or wrist or worse in case a backfire caused the crank to suddenly reverse. At immensely higher speed and force than the human had applied in the desired direction.

And this was a commercial product that sold really, really well.

Likewise, think of your independent suspension, automatic transmission, fuel injection, etc., etc., etc.

We know it’s all there, we sort of grok that is fantastically complicated, but since we don’t get how hard it was to solve each individual problem, we take it for granted.

And it doesn’t break! That’s practically a freaking miracle. Ask me why my Jeep has solid axles some time.

A computer is possibly thousands of times more complex than a car (though far less deadly). Again, we grok the complexity, are ignorant of the effort, and take it from granted.

Since we take it for granted that some smart person somewhere put all this complexity in a box, we assume it must be easy for one or two more smart people to add even more complicated functions.

We miss the fact that it wasn’t one smart person that put it all in a box, it was a veritable army of armies, over decades. So we underestimate the effort to improve.

Did you know that rocket engines - and all of their supporting infrastructure, fuel tanks, fuel pumps, etc., etc. - are all engineered to resist the destructive effects of fuel? Rocket fuel is so chemically reactive than it destroys the very vehicles it is powering, so part of the engineering is figuring out how to get fine propulsion control while making the parts as close to indestructible as they can reasonably be and still be light enough and cheap enough to put into the vehicle in the first place. Part of maintaining any shuttle was deciding which parts to replace; there were parts that were replaced every single mission, others every few. Very few, perhaps no, elements of the propulsion system would have lasted the lifetime of any specific shuttle.

And there is another factor, a psychological one.

Until someone gave you a computer, you had no idea what you could do with one.

Then you got one, and went, oooh.

Then they gave you Internet, and you went ooooooooh.

Then they gave you a mobile phone, and you went oooooooooooooh.

Each time the next new thing comes along, we make a few leaps of imagination: “Well, if it can do that…”

Then we find the constraints, and are a little disappointed, but we still have leaps.

Then another new thing, a leap, some constraints, some disappointment.

Repeat.

Finally, after years of effort, the work of an army of teams and of a few outstanding individuals comes together, and we finally get a device that does something close to what we imagined but could not articulate, years before. And our reaction is “FINALLY! What took you guys so long?”

Now, tie together the two lines of thought, the complexity of engineering and the user psychology, by remembering that the vast majority of humans, especially the vast majority of managers, senior managers, and executives, deal with problems of managing people or machines as cogs: Learn to manage a team, get good at it, learn to manage a team of team leaders, get good at it, learn to manage managers, get good at it, etc., etc.

There are additional nuances and subtleties at each level, new skills, and new layers of abstraction, but one has to be very high up the chain before one gets to complexity on a scale comparable to what senior technologists deal with. Everyday.

A colonel or brigadier has a lot more riding on their decisions in terms of immediate consequences when directing their regiment or brigade in battle, but they have trained and drilled for weeks and months and years, and the overall operations are not that complex, compared to what happens inside many of our machines. Especially our networked machines.

An engineer signing off on the design of a bridge is dealing with something a lot more complicated. If they get it wrong, almost as many people could die as in a battle.

An engineer signing off on the design of a commercial passenger aircraft is dealing with something even more complicated. If they get it wrong, many, many more people could die than as in a battle.

But we all see and appreciate the complexity of the senior officer’s job, and the immediacy of the impact of error or poor decisions, so we assume it is the harder one, the more stressful one, the more complex one.

And we get that wrong.

Pretty much every time.

And we assume in our ignorance that the really cool thing that we just had delivered and are now charging couldn’t possibly have been that complicated, was just a matter of time, because we reason about using heuristics and yardsticks that just don’t apply.

But wait, there’s more.

Every time an engineer designs a bridge, they are essentially doing what hundreds and thousands of people have done before. There really isn’t anything new in bridge making, it’s incremental change over all previous bridges. Any new tools and techniques are created to enable a specific end, making a better bridge for less money.

Every time a software developer creates a new program, they are creating something net new that never existed before. Sure, they may use code and libraries written by others, but those were written not to enable a specific end, but any and all ends. Often, the thing they write is similar to what others have done, but software developers are lazy, they try to not duplicate what others have done. Everyone wants to work on net new as much as possible, and the newier the better.

The more they work at the pointy end of problem space, the more novel and more newily new is the net new they create.

And that is the crux of the perception problem: Customers and managers without software development experience, whose entire existence is about maintaining or incrementally improving things we’ve been doing for a long time, simply lack the context and understanding to appreciate the novelty and complexity of the net new that comes with every new piece of code. And the newest and shiniest pieces of code are as far from the mundane new as the mundane new is from mundane maintenence-or-increment experience.

So when we finally get the right pieces of code assembled in the right way, a common reaction is “what took you so long?”

And now you know why.

Scott Aaronson, expertise, engineering, and the Bell inequality

2015-09-28T00:00:00+00:00

A thought-provoking blog post by Scott Aaronson provokes more thought than I bargained for....

Scott Aaronson has a rare combination of genius and communication skill: He does first rate research on computational complexity and follows the implications of that work in fields as disparate as drug research and evolutionary theory, AND writes about some of the most complex technical topics in a clear and accessible style, invoking math only when absolutely necessary (and usually you can just close your eyes and skip those parts).

It so happens that on my birthday (thanks, Scott!) he published a great article about Bell inequality violations. The article is great for several reasons.

One, it describes the whole problem clearly, describes previous weaknesses with related experiments, and covers exactly why more recent experiments were so very, very clever.

Two, it suggests that we are in for some wicked advancements in engineering in this space.

Three, it contains that great line, “as well as I understand them”. This isn’t false humility, this is one of the brightest theorists working today admitting that the subject he writing about, while so close to his areas of expertise, is that far from his expertise.

Fourth, the article - and point #3 - go a long way to supporting a point I’ve been trying to express well for a long time, and will inevitably fail to so express once again: The distances between each of common knowledge, learned knowledge, advanced knowledge, and truly leading edge knowledge are each so vast that most of us simply cannot comprehend how ignorant we are, despite our best efforts. Even experts in any particular field rapidly reach common levels of ignorance as they move outside their fields.

Fifth, and last, true scientists recognize that and rarely if ever “speak from authority”, but revert to humility and to applying the basic, repeatable tools of their trade, critical thinking, empirical research, and questioning, questioning, questioning.

The right time to refuse a search is always

2015-05-08T00:00:00+00:00

When should you refuse a search? Always. To do otherwise is to erode privacy rights, one concession at a time.

A friend posted a video of a girl being tased because she refused a search and lipped off to a border agent (this was in the US, the friend is American, the commenters were a mix of Yanks and Canucks).

Predictably, one comment was to the usual effect, that if she had nothing to hide, she should not have refused.

Well, that got me riled. After I took a deep breath, this was my reply.

The only reason to refuse a search is to protect your privacy. The question of whether or not you are hiding something is irrelevant.

There is a reason you (Americans) have a Fourth Amendment and that we (Americans, Canadians, many others) all have prohibitions against unreasonable search and seizure.

The starting point is that any and all searches are unreasonable, invasions of privacy. It is then up to legislators and the courts to specifically and reasonably allow searches under specific and reasonable conditions, balancing the privacy right against judicial oversight and probable cause.

Refuse all searches, because you are American. I will refuse all searches because I am Canadian. This is what it means to defend freedom and liberty: Not with a gun, but with a stand. On principle.

Always.

My first interview - and it's about Skipper Smith!

2015-02-28T00:00:00+00:00

I got interviewed about our upcoming movie, Skipper Smith! If you want to contribute, please visit our Indiegogo campaign!

Q: Tell us a little about your acting background. What are some of the other films you have been in?

I started acting about 7 or 8 years ago: My daughter and I auditioned for the local community theatre’s annual Christmas pantomime. It was a great way to spend time together but I wasn’t expecting to get hooked the way I did. I did another play, part of their regular season, and wanted more. But you marry a play, it becomes your life for weeks and months. Tough when you’ve got a job, a spouse, kids, etc.

My first movie was an action film shot locally. I started with a small part at the end, but got “promoted” when someone didn’t show (I just don’t get that - you say you’re going to be there, you be there, you do what you said you would). I ended up with the first line and the first death. Kinda cool, really. Since then I’ve been eaten by a shark, stabbed to death by a spirit from beyond the grave, shot then hung by a treacherous deputy, puked on by a zombie, left for dead by the man who stole my life and my wife, been a comic goon, and, on smaller screens, I’ve sold cheese! Sometimes the lead, sometimes in the background, more often in between.

Q: How is being in a comedy like Skipper different from other genres?

When I first started, I didn’t know how things worked behind the scenes, how much waiting there is, how to manage and maintain my energy all day - to not burn it all off during the wait, to keep the right amount of stoke so that I could be “on” when my turn came - and how, uhm, I’m not sure what the right word is, not forgiving, maybe “flexible”? Yeah, how flexible movies are: You shoot a master, usually wide or half wide, then maybe come closer, maybe do a different angle, then do closeups, turn the camera around to get the other person, usually do several takes. That gives the director and the editor a lot to work with: The actor said his line perfectly, but his eye twitched? Show the other guy listening.

Comedy ain’t like that. You cannot fix comedy in post the way you often can fix action or even drama. Sometimes the comedy depends upon the audience being able to see the whole scene in one shot, one take. On My Fair Zombie the male lead and I had a pretty important interaction toward the end of the movie, it was only going to work if we could do it in a single shot, and there was one bit of dialog I just could not spit out. We tried several takes and finally the director changed a couple of words. It worked, but it was frustrating for me - especially when I figured out how to accent the line about 5 minutes after we’d moved on.

You just don’t know sometimes, until you are there under the lights with the camera rolling.

Q: What is your favorite Indiana Jones movie and why?

I think my favourite Indiana Jones is the first one. It flows nicely, the comedy is good, the bits that are supposed to be threatening are threatening, the music complements rather than overwhelms, the emotions are real. The second, well, it’s a prequel, and prequels lack threat somehow. I really liked the third, but sometimes they forced the emotion between Ford and Connery, though sometimes it worked really, really well. The fourth? I’d have to see it again….

Q: What are you most looking forward to about the shoot?

What am I looking forward to the most? Oh, that’s easy. The script. I’ve only seen about half so far, and there are so many L-O-L moments, so many really good jokes, some subtle, some in your face, some groaners, some witty urbane, some sophomoric. And it all works.

I have to admit, though, that I am also sometimes totally freaked out. I mean, it’s Indiana-frikkin-Jones! An iconic character of our age, iconically Harrison Ford. When Brett offered me the part, I was just too enthralled to think of that, but as it sunk in I would get really, really nervous from time to time.

But Brett starting sending me script updates as they became available, he and Trevor working from the beginning, so it gave me chunks to play with, to imagine. I got used to it one scene, sometimes one joke, at a time. Skipper is almost the vehicle for the comedy, it happens to him and around him in a lot of ways. In some ways, I just have to play it cool, play the adventurous archaeologist, and let the comedy happen. I’ve got some good comic lines, but it’s almost like the straighter I play it, the funnier it will be.

Q: Anything you want to tell fans and potential Skipper supporters?

Absolutely! The cast of this movie is awesome! I’ve worked with a lot of the folks involved before, but never all at once! It’s just, like, wow, I get to be on set with Candice and Christine and Ian and Lawrence and Ray and so many others, people whom I admire for the acting chops, their comic abilities (you gotta see Spyfall, Ray and Trev just steal their bits!). Brett has made two great comedies in a row, Zombie and Spyfall, both of which have won awards at various festivals, and Skipper Smith should be as good. I laughed when I read both of those scripts, but not as much as Skipper. If we pull it off, it will be a romp, a hoot (I think I just dated myself, eh?).

We need help to pull it off, though, to pay for sets, and costumes, and locations, and cast and crew. Brett almost seems like a magician, making such good movies with small budgets, but he isn’t pulling things from the air. It’s hard work, and figuring out how to deliver high quality on low dollars is a real slog. The more money we have to take care of things that money is best at, the more time and energy we have for what the people are best at, which is the comedy, the creativity.

We want to put our time and creative energy into the comedy, into the movie, not into the logistics behind the movie. That’s what we need your help for: Help us make the movie you want to see.

Avoid frustration with new gmail accounts, keep these things in mind....

2015-02-13T00:00:00+00:00

tl;dr:Trying to set up "send as" as a new email address in an existing gmail or GAE account? Remember that Google disables both IMAP and POP3 by default in new accounts. If you read and retain nothing else, read and retain that. It will save you considerable grief.

I’m doing some work that requires a dedicated email address, one that contains neither my company name nor any personal identifier. The why’s and wherefore’s don’t matter much, the important thing is simply that the client wants me to use such an address, the requirement is sufficiently reasonable and not worth arguing (bigger fish to fry and all that).

Since this is going to be used for work purposes I want to keep all email sent to and from this address with the rest of my work emails. You know, for convenience of organization and reference, satisfying my inner neat freak, etc.

I use Google’s email services exclusively: My personal email address is at gmail.com, my company email is hosted by Google App Engine (GAE), and I use the gmail web interface and the Android gmail app exclusively; I don’t use any other mail clients whatsoever.

So it was pretty much a no-brainer that the new email address would also be at gmail.com (yes, this was acceptable, and I know that seems a bit strange, given the other restrictions imposed by the client, but just accept and move on, as I did :->): Google makes it very easy to create a new gmail address and to have multiple gmail sessions in a single browser (at any given time I have multiple tabs open to my gmail.com and GAE addresses, and I keep personal and work emails strictly separate).

OK, Google, +1 for making it easy to set up the new address. So far so good.

I wanted to be able to send as this new address using my GAE business account, and, ideally, I wanted all email sent to this new email account to end up in my GAE business account.

Simple, clear requirement, right? You would think it would be easy to do, right?

Well, it is. As long as you remember that GMAIL DISABLES IMAP BY DEFAULT!!! Sorry to shout (well, not really), but this caused for much frustration and lost time until I finally stumbled across this this morning.

(Before you retort that this is obvious: No, it’s not. In fact, it makes only limited sense. It’s yet another example of Google assuming a) that they are the world and b) that they understand every use case or at least c) that use cases they do not understand don’t matter. Just think of how limited Inbox is, e.g. (completely unsupported with GAE, etc.).)

I’ve enabled “send as” before, I knew where it was in “settings”. I found it, entered the required information, and caught what was reported as an authentication error. WTF? I won’t bore you with the many and varied steps I tried, including logging out of all gmail/GAE accounts, restarting the browser, logging back in, etc., etc., all of which worked…

…and yet the “reported authentication error” persisted. I write “reported authentication error” because that’s what Google claimed: words to the effect of “Cannot establish connection, check username and/or password”.

“Check username and/or password”. Not “connection refused, check host, check that IMAP is enabled, etc., etc.”

Bang bang bang my head. Give up. Send email using new account, Bcc old account, leave problem for another time.

After supper, decide to resurrect imapsync, which I’d used to migrate a previous job’s email from GoDaddy to GAE (worked like a charm).

Cannot connect. WTF?

(Note: As I write this I realize that I was blocked on the authentication error idea, and wasn’t getting enough distance from the problem to seek other causes. I was anchored, but good.)

Bang bang bang my head. Give up. Go to bed.

Try again this morning.

Bang bang bang my head. AARGH!

Poke, poke, poke. Well, it appears I won’t be able to set up sending as the new email from the work GAE.

Well, at least I’ll configure retrieval of new email so that they end up in the GAE account. I’ve done this before…

…but I click on the IMAP tab instead of the Accounts and Forwarding tab (don’t know why, don’t ask, lucky accident, really)…

…and it’s there that the penny drops: In the GAE, IMAP is enabled (I had a reason for this, never disabled it). Drop, drop, click, click.

Go to the new gmail account. Settings, IMAP. Enable IMAP.

Return to GAE account. Try setting up send as again. “Oh, I made a bingo!” (Sorry, watched The Zero Theorem, have Waltz on the brain.)

Finally. I can “send as”. Yay.

Whimper….

Return to new gmail. Enable POP3. Return to GAE, enable fetching.

Everything works.

The bottom line?

GOOGLE, WTF? THEY’RE YOUR OWN ACCOUNTS AND YOU CANNOT REPORT A MORE USEFUL MESSAGE, LIKE, MAYBE, CHECK THAT IMAP IS ENABLED OR CONNECTION REFUSED, NOT AN AUTHENTICATION ERROR OR SOMETHING!?!??!

Nope, not sorry for the shouting. My second major “Google, WTF?” this week (see a recent G+ post of mine if you’re curious).

Another step closer to saying goodbye to Google. Most stuff just works, pretty well, but they do a completely shitty job with stuff people do rarely but need to have just work. Sigh.

If you are reading this paragraph, thanks. We must be siblings-in-frustration, we feel each other’s pain.

Deeper sigh.

The Canadian Government's archaic and embarrassing security screening management application

2015-02-07T00:00:00+00:00

tl;dr: The Canadian Government's online security screening application supports pretty much all browsers on all platforms for 99% of the process - but that last one percent is Windows only, IE only, and Adobe only. Many of us have neither Windows nor IE, let alone Adobe, and even on Windows many of us use neither IE nor Adobe. The solution is simple: Upgrade the server side of the app - which hasn't been touched in 15 years or more - so that it generates the PDF instead of offloading PDF generation to the client. Done. Immediate support for most platforms, most browsers. And no need for the slightly inaccurate 28 step process supplied by government support agents if you know enough to question failure and are patient enough to wait to get through to a human being. Deep sigh. Proud to be Canadian, proud of most of what we do, not so proud of this made-in-Canada application.

Many governments require that employees and contractors be screened prior to being granted access to certain types of information. Canada is no exception, with two distinct but related regimes, one for private information (information about people, business, etc.), one for sensitive government information (cabinet secrets, military information, intelligence, information, etc.). To access the former, you need a reliability status (the often heard “enhanced reliability” no longer exists); to access the latter, you need a security clearance at an appropriate level.

So far so good.

In an effort to simplify and streamline the administrivia required to obtain either a reliability status or a security clearance, Canada had provided a web application that allows a request to be initiated by a company security officer, assigned to either the applicant or to a security officer for completion (that is, for providing the pages and pages of data required), assigned to a security officer for submission to the government, etc.

So far so good.

Up until this point, the process can be successfully navigated with pretty much any browser on pretty much any platform. I’ve worked through this process with Firefox on both Linux and OS X, colleagues and friend have used Chrome or Firefox on Windows, OS X, and Linux, Safari, and within the government it’s all IE, all the time.

So far so good.

For reasons that will be neither discussed nor explained herein, the government requires a wet signature on the forms: electronic signatures are insufficient, and secure electronic signatures, as defined in Canadian law (PIPEDA and its regulations; perhaps I’ll write about this in a future post) are not supported at all.

OK. Probably not so bad, right? Probably so far so good? Print the form, sign, scan and submit via email, or fax, or send the paper in, right?

Almost. And not so good.

While all the data exists on the server side, the government’s process offloads generation of a printable form to Adobe products tied into IE.

And you have to submit their form of the form, not just all of the data as available in the form visible in your browser as you work through the process. Only a paper copy formatted exactly like the paper form originally designed almost two decades ago will suffice. And the only way to get it is with IE and Adobe.

In other words, one can 99% of the way through the process with pretty much any browser on pretty much any platform, but can complete the process only with Windows, only with IE, and only with Adobe.

When I last used Windows, I had IE, one has to, after all, but I didn’t use it; it was there, but I didn’t use it. And I didn’t have Adobe at all: I find their PDF readers to be slow, unstable, and insecure, and I much preferred FoxIt and others of that ilk: Apps that could do one thing only - read PDF - and could do it really well.

I don’t even have Windows anymore: My family and my company are all OS X (I was the last to convert, and then from Linux, having abandoned Windows long ago).

Punchline? To finish the process, I have to seek friends or colleagues with Windows and IE and Adobe, and beg the use of their machines and printers for a short while.

OK, you say, not so bad, probably pretty good.

Except that it doesn’t work. At least not the first time you try it. To quote a friend,

Is that the problem where you spend ages filling in forms, then click on PDF and all you get is the blank form? After a while you give up and get the security officer to print them for you?

That’s right. Blank forms. Because nowhere do they bother to tell you that there is a 28-step (I’m serious) process that you have to follow before this will work.

I didn’t know that any of the other times I’d tried this, I would visit friends and colleagues and, if I had an on-site contract, government offices, logging into a supposedly secure system and trying again and again until it worked somewhere.

This was the situation Thursday night: I tried, it failed. Friday morning, after being on hold for 40 or more minutes, I got the scoop: It was an issue with Adobe’s security settings, and there was a process.

Offered without comment, here is the process - which I have to execute on a friend’s machine, hoping that they are either ignorant or sufficiently trustworthy of my skillage to know that I won’t harm their machine - and which I cannot execute at all on locked-down corporate or government machines:

Click the Start menu
Click “All Programs”
In “All Programs”, Locate Adobe Reader and Click to Start
A blank Adobe Reader page will open
From the top Adobe menu bar, Click “Edit”
From the Edit menu, Click “Preferences…”
In response to the Preferences menu selection, you will be presented with a new window called Preferences
On the left side of the Preferences, there is a Categories list box
From the Categories list box, Click “Trust Manager”
From “Internet Access from PDF outside the web browser”
Click “Change Settings…”
A new window called Manage Internet Access will open
From the Manage Internet Access window, Click “let me specify a list of allowed and blocked websites”
In the Allow/Block web Sites field, Type https://sedsi-oliss.tpsgc-pwgsc.gc.ca
Click “allow”
Then Click “OK”
From the Preferences window, inside the Categories list box, Click “Security (Enhanced)”.
From the Enhanced Security, Click “Add Host”
A new window will open called Add Privileged Host
In Add Privileged Host, Type: https://sedsi-oliss.tpsgc-pwgsc.gc.ca
Click “Secure connections only https” check box
Click “OK”
The user will return back to the Preferences window
From the Preferences window, inside the Categories List box, Click “Internet”
From within the “Web Browser Option” (top right) ensure that “Display PDF in Browser” has a check in the box
Click “OK” at the bottom of the Preferences window
Adobe Reader configuration is complete
Close the Adobe Reader

These instructions were followed by the caveat

Now go back to OLISS with your user id and password and try to print a request that was completed. When selecting the “print” button, if the security screening form comes up and it’s still not populated with the individual’s information, try to clear/delete the browser Cookies and Temporary Files and restart your computer.

Well, offered mostly without comment. Here are two (which I sent to the government support agent this morning):

With certain (more recent?) versions of Adobe, step 23 closes preferences entirely and the user must re-enter.
Once the process is complete, there is still a popup in Adobe that must be recognized and agreed-to; again, this could be an effect of a more recent version of Adobe.

Wonderful, eh? 2015 and the government’s online security screening system requires this level of futzing about just to get things to work…

…to get to the point where I can print the paper, collect signatures, and get the signed documents back, whether by scan and email, by fax, or by snail mail. Anything works, as long as they have them.

They don’t tell you about this 28 step process anywhere on this site…

…because they haven’t touched it in years. Other than to change the stuff and fluff around the app.

This is how it should really work (this is was I sent them this morning as well):

The current system favours only Windows users, and excludes users of both OSX (a growing community: I regularly attend technical fora and meetups and it is more and common for private sector teams to be 100% Apple) and Linux (arguably growing, but who knows for sure). The retort “just use Windows under a VM” is empty and meaningless: Why would we, users of other platforms, spend money to license Windows just (and I must emphasize just) for this application?

Furthermore, even within Windows, the current system favours only users of IE and of Adobe: The last few times I had Windows machines, I didn’t even have Adobe installed, opting instead for lighterweight, more performant, more secure, and more stable PDF readers such as FoxIt - and there are many others.

This is particularly vexing when you consider that many of use get 99% of the way through the OLISS processes online using neither Windows nor IE, e.g., using Firefox or Chrome under OSX, only to have to scramble to find an appropriately configured Windows machine for that last 1%. The level of frustration in the community is considerable. Large organizations will not complain, as they tend to use older, more archaic technology, but smaller, nimbler players are considerably impacted by the government’s lethargy in this area.

There is, however, a simple solution: The technology for server-side dynamic PDF generation is cheap and easy to use; in fact, there are numerous viable open source alternatives.

Such technology could be integrated relatively easily and simply into the server-side of the OLISS process: OLISS would generate the PDF on request.

The end-result? All users of all modern browsers on all modern platforms would be supported 100% of the way through ALL OLISS online processes.

Please urge this upon your superiors and technical support teams. This would be fantastic step forward, and relatively inexpensive and straightforward one for your IT group.

Please feel free to share this with government support agents, decision makers, etc. Frankly, I find it embarrassing that my government’s systems are this archaic.

Oh, did I fail to mention that this system hasn’t changed in, uh, 15 years or more? Sigh.

How I use GitHub Issues, labels, and milestones, to manage this site

2015-01-17T00:00:00+00:00

Update summary, 2015-02-14: Working as described, with one additional milestone, Unsure. More towards the end.

A major reason for having this blog here and not elsewhere was to learn git and GitHub (cf the Colophon). Part of that is using GitHub Issues to manage things. Over the last ten or so years, I’ve become a huge fan of issue-based development management, or, perhaps more accurately, ticket-based change management (which subsumes both changes to code, i.e., development, and changes to operational systems): Having tickets means being able to track what is being done, by whom, when, and by what date; a little more process adds support for change approval mechanisms, rollback, escalation, etc.

It’s more complicated than that, of course, but that’s the basic idea. Pretty much from the beginning of my work on this site, I started tracking site features and development using the issues associated with the site’s repository - but being a command line kinda person, first I found an amazing, clean, simple, and effective, command line tool for opening, updating, closing, and otherwise managing GitHub Issues, ghi. The install was pretty much a breeze:

brew install ghi
ghi config --auth $YourGitHubID

The only hiccup was that I had to run the ghi config command a few times to get the authorization to take: An interaction between keychain and the GitHub API, I suppose, never did figure it out, just tried it a couple of times until it worked… …and since it was working, well….

So my basic workflow is

/* Have an idea */
ghi open
/* describe the idea in one line */
/* force myself to add a body, just in case I cannot later interpret that one line */
vi $theAppropriateFile
git add !$
git commit -a -m "A useful description of what I did; closes #$IdeaIssueNumber"

Sometimes /* Have an idea */ is inspired by other sites (e.g., issue 47 and issue 23), other times it’s from running ghi list, which shows me all current issues (assuming I am in the site’s top-level source folder).

According to ghi list|wc -l, I have 28 open issues right now. That’s not a lot, which means I don’t really need to put in much effort to manage them, but it’s also a nice, manageable, workable number if I want to start experimenting with labels and milestones…

…which was what inspired this particular post: After poking through a few articles on how to use GitHub Issues (your google fu is as good as mine), both from GitHub itself and from GitHub users describing their processes, I realized that one of the great strengths of GitHub’s minimalist, nay, Spartan, approach to issues is that it forces absolutely no process on anyone.

The flipside of this, of course, is that should one wish to go beyond having a flat list of issues, somehow using labels and milestones to manage them, one has to have a process in mind, a business problem to solve.

The notional idea that I had prior to googling was managing dependencies. After reading, it wasn’t at all obvious how to do this: GitHub Issues has neither hierarchy nor dependencies, just labels and milestones.

After some mental hemming and hawing, I decided on the following:

Use labels to denote “clusters”, that is, related issues, and
Use milestones to denote sequence.

I’ve created the following milestones:

Anytime
Next
NextA
NextB
NextC
Roadmap

Anytime basically means quick little things I can or should do whenever the fancy strikes me. Next is for things important enough and of the right scope to be priorities (right scope means both that I understand the scope and that they will fit in to the windows made available by the union of {my day job, my home life, my hobbies}). Roadmap basically means later: Things I think I should do but I’ve neither scoped nor considered carefully enough to know whether I really should do them.

That leaves Next[ABC]. Basically, A comes before B comes before C. That’s it. Or almost.

Next[ABC] are meaningless without clusters, which are denoted by labels, which will appear, change, and disappear, as work is done. I don’t have any good examples yet (I’m writing this post to consolidate the idea in my mind before I sift through the issues to apply it) but the basic idea is this:

I have a cool idea, but it depends upon something I haven’t done yet.
Create an issue for the cool thing (call it issue 2)
Create an issue for the thing I haven’t done yet (call it issue 3)
Create a label that captures the essence of why I am going to do those two things. In this case, it might be “CoolThing”. Give that label a colour not currently in use.
Apply the label to the two issues.
Associate issue 3 (the one that has to come first) with milestone “NextA”.
Associate issue 2 (the cool thing) with milestone “NextB”.

Then, when “Anytime” is either empty or boring and “Next” is empty, look for things in Next[ABC] to move to Next. There is only one rule: Never move anything from NextB or NextC to Next if there something with the same label in NextA.

(In my mind, that rule is simpler than having to always shift items from NextC to NextB to NextA as items are popped from NextA and NextB.)

Why Next[ABC] and not Next[A-F], e.g.? Well, anything more than three levels of dependency likely means I don’t understand the scope and magnitude and complexity of what I am doing. Having said that, it is quite possible that NextA for label Fred might be a simple thing, NextB for Fred a little more complex, and NextC a lot more. That might mean than when I close the issue for Fred in NextA, I might then break the NextB issue into two, and put one of them in NextA.

Or at least that’s what I’m thinking right now. I don’t want to overdesign nor do I want to overproscribe (though it does feel overdescribed by now, eh?).

Enough for now. I’m going to start categorizing things. I’ll report back on this after I’ve played with it for a bit.

Update, 2015-02-14

I've been meaning to add this update for a while now: I proceeded as described above and found it worked pretty darned well. I get a nice view of how issues relate using ghi to list issues, labels, etc., at the command line, while the view in GitHub Issues is workable, at least for a small number of milestones.

Important point: If you want to work this way, figure out what your milestones are first, then figure out the order in which you want to display them, then create them in that order: GitHub (or maybe it is just ghi) will assign each one a number, and, by default, present them in numerical order, even if the number is not displayed (it is in ghi, it is not in GitHub Issues, at least not as far as I can see).

One change from the original post: As I began working with milestones as buckets for when things should happen, I realized I needed another bucket, Unsure, which is for things that aren't in the roadmap (and hence shouldn't be brought forward) but which should not yet be dismissed. Think of it as a "check again later" sort of holding pen. Fortunately, given that I thought of this bucket last (cf important point, above), it is fortunate that it should indeed appear last. Serendipity FTW.

Why I act

2015-01-15T00:00:00+00:00

(Another Facebook-inspired post, this one because of a comment “you should always get paid, never act for free”.)

Why is everyone assuming that people act because they want to get paid for it?

The likelihood of me making as much acting as I do on my day job is, well, practically infinitesimal.

I act because I thrive on it. I act because I crave the art. Payment is lagniappe. My cheese commercial has been playing for almost two years, I still get comments about it. It paid better than all of my other acting gigs combined (and it paid, by hour or by day, better than my day job, and that is saying something).

And since that commercial, I have been shot and hung by a treacherous deputy, swallowed by a shark, had a gunfight with Jesse James, been puked on by a zombie, and am in pre-production on a few other things, to name but a few.

If I get as much as lunch money from my gigs, I’m happy. Hey, I’d be even happier (maybe) if I could do this full time. And yes, I know there are people who succeeded starting as late as I did - but that’s not why I’m do it.

I act for the sheer joy of it. I’m good at it, as it turns out, but that’s only important because it allows me to work over and over again with some of the best people I’ve ever met.

That’s why I act.

Charlie Hebdo, I can manage. Boko Haram, not so much

2015-01-13T00:00:00+00:00

(From a comment on yet another Facebook conversation….)

When I first read of the 2000, I was stunned. Then I realized that that was all I would ever be: Stunned. I simply do not have the mental or emotional capacity to perceive 2000 sudden brutal deaths. That’s twice our entire high school population, 6 times our graduating class, killed in a matter of hours.

Call it a human failing. I fancy that I can wrap my head around a dozen journalists killed for promoting one of my highest values, self-expression and free thought. I cannot wrap my head around the destruction by a madmen of a small village and its heterogeneity and promise.

So I focus on those I can deal with, those I can frame in my mind.

One tragedy is manageable, the other overwhelming.

We recoil from horror writ large, ignore it, forbid it from entering our minds, touching our hearts, sapping our souls.

Sony Vs North Korea (really?)

2014-12-19T00:00:00+00:00

This afternoon a Facebook friend wrote to me in a comment,

I’m sure you’ve been fascinated by this entire ordeal [The Sony compromise, accusations against North Korea, etc.] too! I mean really, an unencrypted folder called “Passwords”? sigh

Fascinated? Not so much fascinated as jaded. Most large corporations simply fail to take security seriously. In 25 or so years in the field, I’ve seen far too many people who are experts in other areas believe they can simply master security (or software development) with very little time and training. “How hard can it be?”

Anyone who has ever worked in IT or ITS has received tremendous amounts of helpful advice from experts in HR, accounting, finance, etc., all of whom think their experience is portable, transferable, translatable. But how many of those experts ask the IT or ITS folks for advice on their work? NONE. And why is this?

Oh, it’s obvious? If it’s so obvious one way, why isn’t it obvious the other?

Witness RSA itself, for a long time the world leader in secure authentication systems. Until their entire network was compromised over a periods of months in a successful effort to break their authentication systems in order to attack one of their largest clients.

And these are folks who should have known better. As for Sony, they have a long history of bad IT, bad ideas, and worse responses. Witness the root kit episode a few years back.

Witness CurrentC: When Apple introduced iWallet, retailers supporting CurrentC disabled NFC for iOS devices to hamstring their future competitor, a move widely seen (rightly IMHO) as anti-consumer, anti-choice.

Within days, CurrentC had been compromised by hackers offended at their position. They simply were not ready to play in that league, but somehow were arrogant enough to think that because they could several other things well they could ITS well.

Wrong, thanks for playing.

As for blaming NK, well, sigh. Holding up a credible (or better yet, incredible?) strawman simply diverts attention from those who are truly responsible: Sony.

Sony didn’t attack itself, that’s not what I mean. What I mean is that IMHO Sony was negligent. At this point, wilfully (they could not not have known otherwise). Whether criminally, well, IANAL and all that.

I am so looking forward to the day when a Home Depot or Target or Sony or other is found criminally negligent for the exposure of customer records following a compromise. I’m not really sure whether I’m interested in anything happening to the attackers. I just think it’s about time that companies that treat and truck with so much employee and customer data start treating it properly.

The thing is, good security is actually cheaper than the opposite: Design security in from the beginning, small incremental cost.

Get compromised, huge cost. Simple economics.

Big company with large Internet presence? Visible target, will be attacked. Guaranteed. So pay a little now or a lot later. Simple economics.

(A few years ago I prepared a short presentation to justify that last statement for a client. I’m going to dig it out and turn it into a blog post sometime soon. The graphic on relative costs at various stages ranging from design to operations is really quite striking. Stay tuned.)

OK, now what?

2014-12-18T00:00:00+00:00

tl;dr: What do you want me to write about first/next? The experience of setting up this blog? IT Security? Acting? Something else altogether? Let me know via the various social/feedback links available herein.

I’d been thinking of setting up a blog for long time. Actually, I’ve been thinking of setting up two, one personal (right here!) and one for EdgeKeep, my consulting company. But there were the questions: Where to host? What software to use? Yada etc. and so on.

I’d also been thinking about learning git, GitHub, etc. I’ve been exposed to them a few times, but never enough to really sink in my teeth, and having a working knowledge of these tools is practically de rigueur for some tech groups and meetups I enjoy.

Long story short, I combined the ideas and built this, the site you are visiting to read this post. May not seem like much, and for an experienced git user and web developer, it isn’t. But for someone who had practically zero hands-on experience with git, web development, templating, and especially the interplay between CSS and HTML, it was sumthin’. I learned a lot. Made fewer mistakes than I expected, but did have some gut-clench moments when I thought I’d made mistakes (git rebase is very, very scary and will remain so until I understand it better).

I started banging on this site in late November. It took me until yesterday to get it where I wanted, to the point I could write this post and publish it in just a few minutes, with either just a few clicks (ew) or just a few commands (vim and git push FTW!).

Pretty much from the beginning I kept an eventual blog post about the experience, one intended to help other technically savvy but git-and-web inexperienced people do what I did, a complete tutorial and reference in one as-brief-as-possible, as-comprehensive-as-necessary article.

The draft - and it’s decently high quality - is over 6000 words. So far. I reckon the complete article will push 9,000. That’s a lot of content yet to come, plus the TLC to turn a draft of any quality into a readable article.

And I’ve got the high-and-doldrums that come with completing anything, the I did it! What’s next… feeling.

So I ask you: What’s next? Or, more to the point, what’s first? Do you want me to:

Complete the “how I got here” article? It’ll be a while….
Write about IT security?
Discuss consulting?
Talk about acting?
Share an exciting and potentially wordy anecdote about my life?
Something else altogether?

I’ve yet to add commenting to the blog (it’s on my list), but elsewhere on this site there are social links and a link for creating a new issue for site problems. Feel free to use one or more of those to let me know what you would like to read next.

Thanks!

Peter Whittaker

About time: Setting up ctags for vim and git

Building universal-ctags

My ~/.vimrc

Adopting tpope’s git hooks

Obligatory "What I wish I knew about git" post

So, Pete, you don’t use branches, then?

What about remotes, then?

References and acknowledgements

Paradox-free Time Travel! Yay! Wait, not so fast....

Python Best Practice re methods Vs external functions

I am nothing ⊃ ¬Patient

Doing Python development under Mac OS

UGH.

Just The Recipe

Buying a used Wrangler JK/JKU: Rubi or Sport?

Well, I'm back

Five years is a long time. Why stop? Why start again? And why was restarting so (technically) painful?

The backstory - getting started, back then, and stopping, and starting again….

A digression - how to manage dot files across multiple machines

A digression within a digression - WSL is awesome

We now return to our regularly scheduled inception, er, digression

We will now pause the first digression

Another digression - keeping a Mac OS development environment up-to-date

We will now pause the second digression

Merging digressions the first and second

You have reached the end of the first and second digressions

Enter the Makefile

Resurrecting digressions the first and the second, temporarily

The digressions are dead. Long live the main post.

Getting ready to build a Jekyll site on Mac OS

A note on installing the XCode command line tools

Command-line précis

Finally! We can correct GitHub Pages build errors!

Was it worth it?

OK, what about stopping six years ago? And all the foreshadowing?

Is that it?

A final note

Why people think software is so much easier than it really is

Scott Aaronson, expertise, engineering, and the Bell inequality

The right time to refuse a search is always

My first interview - and it's about Skipper Smith!

Avoid frustration with new gmail accounts, keep these things in mind....

The Canadian Government's archaic and embarrassing security screening management application

How I use GitHub Issues, labels, and milestones, to manage this site

Why I act

Charlie Hebdo, I can manage. Boko Haram, not so much

Sony Vs North Korea (really?)

OK, now what?

Building `universal-ctags`

My `~/.vimrc`

A digression - how to manage `dot files` across multiple machines