jonathan tsai

Building Mission Control for My AI Workforce: Introducing OpenClaw Command Center

2026-02-12T00:00:00+00:00

The Backstory

I’m not a “script kiddie” or weekend hobbyist. I’m a UC Berkeley-trained Computer Scientist with over two decades of professional experience in Silicon Valley. I joined Iterable and EasyPost after their Series A rounds — both are now unicorns. At EasyPost, I managed 4 teams totaling ~20 engineers and delivered 8 figures of revenue.

I know what production systems look like at scale.

A few months ago, I read Peter Steinberger’s seminal post about shipping at inference speed. steipete is possibly one of the greatest programmers of this generation, and ClawdBot (now OpenClaw) was immediately on my radar. I was already racing to build my own AI Agent Swarm orchestrator — but I thought, “He’s good, but can I trust him?”

Then, three weeks ago, OpenClaw went viral. I went all-in. I’ve been hacking until 4am, 5am every night building out what I call the OpenClaw Command Center.

This year alone: After switching to Claude Code, I got a ~20x productivity boost. After adding OpenClaw, I got another 50x on top of that.

The math: 1000x productivity multiplier. That’s not hyperbole. That’s my lived experience.

What I’m Running Right Now

5 OpenClaw master instances — one for each domain of my life
10 satellite agents — specialized workers
1 “Godfather” orchestrator — coordinates everything
20+ scheduled tasks per instance — running 24/7
Hardware: Mac Studio M2 Ultra + Mac Minis + MacBook Pro + VirtualBox VMs on top of old Windows host

Each OpenClaw instance is a “GM” (General Manager) that oversees one aspect of my personal or professional life. They advance my goals and keep me locked in — even when I’m sleeping.

I’m literally coding at the gym on my phone… via Slack… in between bench pressing 315 lbs.

The possibilities are endless. AGI is here.

See It In Action

The Vision: Bring the Work to Where Humans Are

I’ve seen the mockups and prototypes online — “the future of work” dashboards, agent orchestration UIs, yet-another-SaaS-tool. That’s the wrong direction.

Here’s the thing: humans are already in Slack.

I’ve worked at companies with dozens, hundreds, even thousands of Slack channels. That’s where work happens. That’s where context lives. That’s where people communicate.

So instead of building another tool that forces context-switching, I asked: what if I brought the visibility to where I already am?

The agents live in Slack threads — that’s their native habitat. Command Center doesn’t replace Slack; it gives you the bird’s-eye view you’re missing. It’s the air traffic control tower for your AI workforce.

Think of it like a Starcraft command center (yes, I’m dating myself):

High APMs (actions per minute)
Lots of AI workers running in parallel
Ensure all agents are unblocked
No idle workers sitting around

You need to see everything at once to coordinate effectively.

What I Built

Real-Time Visibility

The dashboard shows everything that matters:

Session monitoring — Every active AI session, with model, tokens, cost, and context
LLM Fuel Gauges — Never get surprised by quota limits (we’ve all been there)
System Vitals — CPU, memory, disk — is your machine the bottleneck?
Cost Intelligence — Know exactly what your AI workforce costs

Topic Tracking (Cerebro)

One of the most powerful features is automatic conversation organization. I call it Cerebro — inspired by the machine that augments Professor X’s innate telepathic abilities.

My setup: multiple Slack channels, one per project. Within each channel, one thread per feature. Cerebro auto-detects topics from threads and organizes them.

Each thread becomes a trackable unit of work:

All topics across your workspace
Thread counts per topic
Jump directly into any conversation

This is possible because OpenClaw integrates deeply with Slack threading. Every message goes into the right thread, every thread has a topic, every topic is visible in the dashboard.

I worked really hard to allow OpenClaw to “stay focused” on topic. That discipline pays dividends.

Scheduled Tasks (Cron Jobs)

AI agents shouldn’t just react — they should proactively check on things, generate reports, clean up stale work. The cron dashboard shows:

All scheduled tasks
Run history
Manual triggers
Configuration at a glance

Privacy Controls

When demoing or taking screenshots, you can hide sensitive topics with one click. Learned this the hard way — you don’t want to accidentally share internal project names in a public post.

The Technical Details

Zero Dependencies, Instant Startup

Command Center is deliberately minimal:

~200KB total — dashboard + server
No build step — runs immediately
No React/Vue/Angular — vanilla JS, ES modules
Single unified API endpoint — one call gets all dashboard data

Why this approach:

AI agents can understand and modify it easily
No waiting for webpack/vite compilation
Works in any environment with Node.js

Security-First

Since this gives visibility into your AI operations, security was non-negotiable:

Localhost by default — not exposed to network
No external calls — zero telemetry, no CDNs
Multiple auth modes — token, Tailscale, Cloudflare Access
No secrets in UI — API keys never displayed

Real-Time Updates

The dashboard uses Server-Sent Events (SSE) for live updates. No polling, no websocket complexity. State refreshes every 2 seconds, cached on the backend to stay responsive under load.

The Philosophy: Use AI to Use AI

Here’s the key insight that changed everything:

Recursion is the most powerful idea in computer science.

Not loops. Not conditionals. Recursion — the ability for something to operate on itself. And the same principle applies to AI:

Use AI to use AI.

Think about it: Why are you manually configuring your AI agents? Why are you manually scheduling their work? Why are you manually routing tasks to the right model?

The agents should be doing that. The meta-work of managing AI should itself be done by AI.

This is how I gain an edge — not just over people still coding manually, but over vanilla OpenClaw users. I built the infrastructure for AI to optimize its own operations.

Advanced Job Scheduling (What’s Already Working)

After years of production experience with Spark, Airflow, Dagster, Celery, and Beanstalk — each with their own strengths and painful limitations — I had strong opinions about what an AI-native scheduler should look like.

I pulled concepts straight from CS162 (Operating Systems): multi-threading primitives, semaphores, mutex locks, process scheduling algorithms. These aren’t academic exercises — they’re exactly what you need when orchestrating dozens of AI agents competing for limited resources.

The scheduling primitives I’ve built:

run-if-idle — Execute only when system has spare capacity (no resource contention)
run-if-not-run-since — Guarantee freshness: “hasn’t run in 4 hours? run now”
run-at-least-X-times-per-period — SLA enforcement: “must run 3x per day minimum”
skip-if-last-run-within — Debouncing: “don’t spam if we just ran 10 min ago”
conflict-avoidance — Greedy algorithm prevents overlapping heavy jobs
priority-queue — Critical tasks preempt background work

This isn’t theoretical. It’s running in production right now across my 5 master instances and 10 satellite agents.

Intelligent Quota Management

I’m on the $200/month Claude Code Max plan. Without optimization, I’d blow through my weekly quota by Wednesday and be paying overage the rest of the week.

Instead, I’ve never paid a cent of Extra Usage. Conservatively, this system saves me at least $10,000/month in what would otherwise be API costs and overage charges.

How? The scheduling system is quota-aware:

It knows when my weekly quota resets (Saturday night 10pm)
It tracks current usage percentage via the API
It batches low-priority work for off-peak hours

Real example: It’s Thursday night. I’ve used 75% of my weekly quota. The scheduler sees this and thinks: “We have 25% left, 2.5 days until reset, user is asleep. Time to burn cycles on background work.”

So it wakes up my agents and has them iterate on unit tests — grinding my monorepo toward 100% code coverage while I sleep. Work that needs to get done, but doesn’t need me present.

By the time quota resets Saturday, I’ve maximized value from every token. Then Sunday morning I have a full fresh quota for the real creative work.

LLM Routing: Right Model for the Job

Not every task needs Claude Opus 4.6.

I built a routing layer that matches tasks to models:

Task Type	Model	Why
Code review, complex reasoning	Claude Opus 4.6	Worth the tokens
Boilerplate, formatting, tests	Local models (Qwen, Llama)	Fast, free, good enough
RAG retrieval, embeddings	Local	Zero API cost
Documentation	Claude Sonnet	Sweet spot

The router examines the task, estimates complexity, and picks the appropriate model. Heavy thinking goes to the heavy model. Routine work stays local.

This is “Use AI to Use AI” in action — I didn’t manually tag every task. The routing agent figures it out.

What’s Next

Multi-Agent Orchestration

The real power unlocks when agents work together:

Swarm coordination patterns
Structured handoff protocols
Specialized agent routing (SQL tasks → SQL agent)
Cross-session context sharing

Voice Harness

Next, I’m working on STT/TTS integration so I can orchestrate my agents with just my voice — while I’m out walking my dogs, playing basketball, lifting weights. The keyboard becomes optional.

Try It Yourself

Command Center is open source and free:

# Via ClawHub
clawhub install jontsai/command-center

# Or git clone
git clone https://github.com/jontsai/openclaw-command-center
cd openclaw-command-center
node lib/server.js

Critical setup: Enable Slack threading in your OpenClaw config:

slack:
  capabilities:
    threading: all

This is what enables proper topic tracking.

The Bigger Picture

We’re at an inflection point. AI agents aren’t just tools anymore — they’re becoming teammates. And like any team, you need visibility, coordination, and management.

Command Center is my answer to: “How do I actually manage an AI-native life?”

It’s not the final answer. It’s the foundation I’m building on. And I’m excited to share it with the community.

OpenClaw Command Center is MIT licensed. Star it on GitHub, try it out, and let me know what you think.

Links:

SSH Commit Signing Part 2: Automation and Multi-Machine Setup

2025-10-17T00:00:00+00:00

In my previous post, I covered the basics of signing Git commits with SSH keys instead of GPG. This post covers the automation and multi-machine setup I built to make SSH signing seamless across 12+ machines.

The Challenge

Managing SSH commit signing across multiple machines introduces several challenges:

Multiple keys - Each machine has its own SSH key
Multiple emails - Different projects use different commit emails
Verification - Git needs to verify signatures from any of your keys
GitHub - All keys need to be registered as signing keys
Consistency - Configuration should be identical across machines

The Solution: Centralized Configuration

I created three interconnected repositories:

dotfiles (public) - Git configuration and aliases
bash-ftw (public) - Bash utilities and installation helpers
pubkeys (private) - SSH public keys and automation scripts

Key Components

1. Dynamic Key Selection

Instead of hardcoding a specific key per machine, use ssh-agent to automatically select the first available key:

# In ~/.gitconfig or ~/code/dotfiles/.gitconfig
[gpg]
    format = ssh

[gpg "ssh"]
    allowedSignersFile = ~/.ssh/allowed_signers
    defaultKeyCommand = ssh-add -L | head -n1

[commit]
    gpgsign = true

Benefits:

No per-machine configuration needed
Works with any key loaded in ssh-agent
Portable across all your machines

2. The allowed_signers File

Git’s allowed_signers file verifies commit signatures. The format is:

email key-type key-data comment

The key insight: Create a cross-product of all your emails × all your keys:

hello@jontsai.com ssh-ed25519 AAAAC3... laptop-key
hello@jontsai.com ssh-rsa AAAAB3... desktop-key
hello@jontsai.com ssh-ed25519 AAAAC3... server-key
user@example.com ssh-ed25519 AAAAC3... laptop-key
user@example.com ssh-rsa AAAAB3... desktop-key
user@example.com ssh-ed25519 AAAAC3... server-key

This allows Git to verify commits signed by any of your keys with any of your email addresses.

3. Automated Generation Script

Create scripts/generate_allowed_signers.sh:

#!/bin/bash
# Generate allowed_signers file for Git SSH commit signing

set -e

EMAILS_FILE="${EMAILS_FILE:-emails.txt}"
OUTPUT="${OUTPUT:-allowed_signers}"

# Read emails (filter out comments and empty lines)
emails=$(grep -v '^#' "$EMAILS_FILE" | grep -v '^[[:space:]]*$' || true)

# Clear output file
> "$OUTPUT"

# Enable nullglob for Mac compatibility
shopt -s nullglob

# Process all .pub files
for pubkey in *.pub delegates/*.pub; do
    if [ -f "$pubkey" ]; then
        key_content=$(cat "$pubkey")

        # For each key, add entry for each email
        echo "$emails" | while IFS= read -r email; do
            echo "$email $key_content" >> "$OUTPUT"
        done
    fi
done

shopt -u nullglob

echo "Generated $OUTPUT with $(wc -l < "$OUTPUT") keys"

Create an emails.txt file:

# Email addresses used for git commits
hello@jontsai.com
jontsai@users.noreply.github.com

4. Makefile for Easy Management

Create a Makefile to orchestrate everything:

.PHONY: help install install-authorized_keys install-allowed_signers github-signing-keys

## help - Display available targets
help:
	@cat Makefile | grep '^## ' --color=never | cut -c4- | \
	  sed -e "`printf 's/ - /\t- /;'`" | column -s "`printf '\t'`" -t

## authorized_keys - Generate authorized_keys file
authorized_keys: $(wildcard *.pub) $(wildcard delegates/*.pub)
	cat *.pub delegates/*.pub > authorized_keys
	chmod 600 authorized_keys

## allowed_signers - Generate allowed_signers file
allowed_signers: emails.txt scripts/generate_allowed_signers.sh $(wildcard *.pub)
	scripts/generate_allowed_signers.sh
	chmod 600 allowed_signers

## install - Install authorized_keys and allowed_signers to ~/.ssh
install: authorized_keys allowed_signers
	cp -v authorized_keys ~/.ssh/authorized_keys
	cp -v allowed_signers ~/.ssh/allowed_signers
	chmod 600 ~/.ssh/authorized_keys ~/.ssh/allowed_signers

## github-signing-keys - Add all keys to GitHub as signing keys
github-signing-keys:
	scripts/add_github_signing_keys.sh

5. Automated GitHub Key Upload

Create scripts/add_github_signing_keys.sh:

#!/bin/bash
# Add all public keys to GitHub as signing keys using gh CLI

set -e

# Check if gh is installed
if ! command -v gh &> /dev/null; then
    echo "ERROR: gh CLI is not installed"
    echo "Install from: https://cli.github.com/"
    exit 1
fi

# Check authentication
if ! gh auth status &> /dev/null; then
    echo "ERROR: Not authenticated with GitHub"
    echo "Run: gh auth login"
    exit 1
fi

# Check for required permissions
echo "Checking for required permissions..."
if ! gh ssh-key list &> /dev/null; then
    echo "ERROR: Missing required permission scope"
    echo ""
    echo "To grant this permission, run:"
    echo "    gh auth refresh -h github.com -s admin:ssh_signing_key"
    exit 1
fi

echo "Adding all public keys to GitHub as signing keys..."

success_count=0
skip_count=0
error_count=0

for pubkey in *.pub delegates/*.pub; do
    if [ -f "$pubkey" ]; then
        title=$(basename "$pubkey" .pub)
        echo -n "Adding $title... "

        output=$(gh ssh-key add --type signing "$pubkey" --title "$title" 2>&1)
        exit_code=$?

        if [ $exit_code -eq 0 ]; then
            echo "done"
            success_count=$((success_count + 1))
        elif echo "$output" | grep -q "already exists"; then
            echo "already exists (skipped)"
            skip_count=$((skip_count + 1))
        else
            echo "FAILED"
            echo "  Error: $output"
            error_count=$((error_count + 1))
        fi
    fi
done

echo ""
echo "Summary: $success_count added, $skip_count skipped, $error_count errors"

6. Git Aliases for Viewing Signatures

Add to your ~/.gitconfig:

[alias]
    # Compact log with signature status
    slog = log --pretty=format:\"%C(auto)%h %G? %C(blue)%an%C(reset) %s %C(dim)(%ar)%C(reset)\"

    # Full signature details
    logs = log --show-signature

Signature status codes:

G = Good signature
B = Bad signature
U = Good signature, unknown validity
N = No signature

7. Bash Installation Helper

Add to your ~/.bashrc or bash-ftw:

# GitHub CLI installation with OS detection
function install-gh {
    KERNEL=$(uname -s)

    if [[ $KERNEL == 'Darwin' ]]; then
        echo "Installing GitHub CLI via Homebrew..."
        brew install gh
    elif [[ $KERNEL == 'Linux' ]]; then
        echo "Installing GitHub CLI via apt..."
        curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | \
          sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg && \
        sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg && \
        echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | \
          sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null && \
        sudo apt update && sudo apt install gh -y
    else
        echo "Visit https://cli.github.com for installation instructions"
        return 1
    fi

    echo "GitHub CLI installed! Run 'gh auth login' to authenticate"
}

Complete Setup Workflow

Initial Setup (One Time)

Clone your dotfiles:

cd ~/code
git clone https://github.com/yourusername/dotfiles

Create your pubkeys repository structure:

mkdir -p ~/code/pubkeys/{scripts,delegates,obsolete}
cd ~/code/pubkeys

# Copy all your .pub files here
cp ~/.ssh/*.pub .

# Create emails.txt
cat > emails.txt <<EOF
# Your git commit emails
you@example.com
you@users.noreply.github.com
EOF

Copy the scripts (from examples above) into scripts/

Install GitHub CLI:

install-gh  # or manually from https://cli.github.com
gh auth login
gh auth refresh -h github.com -s admin:ssh_signing_key

Install configuration:

cd ~/code/pubkeys
make install

cd ~/code/dotfiles
cp .gitconfig ~/.gitconfig

Upload keys to GitHub:

cd ~/code/pubkeys
make github-signing-keys

Per-Machine Setup

On each new machine:

# 1. Clone repos
cd ~/code
git clone https://github.com/yourusername/dotfiles
git clone your-pubkeys-repo  # if you made it a git repo

# 2. Install
cd ~/code/pubkeys && make install
cd ~/code/dotfiles && cp .gitconfig ~/.gitconfig

# 3. Configure ssh-agent (if needed)
ssh-add ~/.ssh/id_ed25519

# 4. Test it
cd some-repo
git commit -m "test signed commit"
git log --show-signature -1

Benefits

Zero per-machine configuration - Same setup everywhere
Automatic key selection - Works with any key in ssh-agent
Multi-email support - All your commit emails are verified
One-command GitHub sync - make github-signing-keys
Easy verification - git slog shows signature status inline
Makefile dependencies - Auto-regenerates when keys/emails change

Lessons Learned

1. Mac Compatibility

Mac’s bash 3.2 doesn’t support <<< heredoc syntax. Use pipe instead:

# Don't do this (fails on Mac)
while read line; do ...; done <<< "$var"

# Do this (works everywhere)
echo "$var" | while read line; do ...; done

2. Makefile Dependencies

Use $(wildcard *.pub) to track file dependencies:

allowed_signers: emails.txt scripts/generate.sh $(wildcard *.pub)

3. Error Handling in Scripts

Always check exit codes and provide remediation:

output=$(command 2>&1)
exit_code=$?

if [ $exit_code -ne 0 ]; then
    echo "ERROR: $output"
    echo "To fix: <remedy steps>"
    exit 1
fi

4. GitHub CLI Permissions

The admin:ssh_signing_key scope is required for managing signing keys:

gh auth refresh -h github.com -s admin:ssh_signing_key

5. Verification is Separate from Signing

user.signingkey or gpg.ssh.defaultKeyCommand - Which key to sign with
gpg.ssh.allowedSignersFile - Which keys are trusted for verification

Verification

Check if commits are signed:

# Quick check
git slog -10

# Full details
git log --show-signature -1

# Specific commit
git verify-commit abc123

Public Resources

My dotfiles - Git configuration and aliases
bash-ftw - Bash utilities and helpers

Feel free to adapt these scripts and configurations for your own setup!

Conclusion

SSH-based commit signing is simpler than GPG, but managing it across multiple machines requires automation. With centralized configuration, automated scripts, and proper dependency tracking, you can maintain a seamless signing setup across all your machines.

The key principles:

Automate everything - Scripts eliminate manual steps and errors
Centralize configuration - Dotfiles repos ensure consistency
Use cross-products - All emails × all keys for maximum flexibility
Make it idempotent - Safe to run commands multiple times
Provide clear errors - Always show how to fix issues

Now all my commits are automatically signed, verified, and visible on GitHub with that coveted “Verified” badge. 🎉

Signing Git commits using SSH instead of GPG

2024-11-21T00:00:00+00:00

TIL you can sign Git commits using SSH instead of GPG

This tip is 🏆, learned from my really smart colleague.

tl;dr;

To configure Git to use your key:

Configure Git to use SSH for commit signing:
git config --global gpg.format ssh
Specify which public SSH key to use as the signing key and change the filename (~/.ssh/examplekey.pub) to the location of your key. The filename might differ, depending on how you generated your key:
git config --global user.signingkey ~/.ssh/examplekey.pub

To sign a commit:

Use the -S flag when signing your commits:
git commit -S -m "My commit msg"
Optional. If you don’t want to type the -S flag every time you commit, tell Git to sign your commits automatically:
git config --global commit.gpgsign true

Source: https://docs.gitlab.com/ee/user/project/repository/signed_commits/ssh.html

Embrace the power of Regex

2023-07-13T00:00:00+00:00

Too often, while reviewing code, I’ll see examples like:

def extract_id_and_env(key: str) -> dict:
    """Extracts the object ID from `key`

    `key` is a string like 'namespace_prefix_12345'
    In some cases, `key` could also look like `namespace_prefix_12345_environment`

    Returns a dict with the object ID, an integer
    """
    parts = key.split('_')

    parsed = {
        'id': int(parts[2]),
        'environment': parts[3] if len(parts) == 4 else None
    }
    return parsed

When I see this, I ask, “Why?”

Instead, this is my preferred way of handling this is to use a regex with named capture groups:

import re

KEY_PATTERN = re.compile(r'(?<namespace>[a-z]+)_(?<prefix>[a-z]+)_(?<object_id>\d+)(?:_(?P<environment>[a-z]+))?

def extract_key_components(key: str):
    m = KEY_PATTERN.match(str)
    parts = ['namespace', 'prefix', 'object_id', 'environment', ]
    values = [m.group(part) for part in parts]
    return values

In another example (contrived, but modified from a real world application), from a Django which serves both students and educators, and displays two different landing pages depending on the intent:

def login_view(request):
    url = request.GET.get('next')
    last_word = url.split("/")[-1]
    is_student = True if last_word == 'scholarship' else False

    template = 'login/student.html' if is_student else 'login/educator.html'

    response = render_to_response(request, template)
    return response

The problem with this code is not immediately apparent. It works. However, this code lacks robustness.

An arguably better approach:

import re

STUDENT_LOGIN_INTENT_PATTERNS = [
    re.compile(r'^/path/to/(?P<some_id>\d+)/scholarship$'),
]

def is_login_intent_student(request):
    is_student = False
    next = request.GET.get('next')
    for pattern in STUDENT_LOGIN_INTENT_PATTERNS:
        if pattern.match(next):
            is_student = True
            break
    return is_student
    

def login_view(request):
    is_student = is_login_intent_student(request)
    template = 'login/student.html' if is_student else 'login/educator.html'

    response = render_to_response(request, template)
    return response

In addition to the readability and maintainability of the regex approach, it is overall more robust, allowing the programmer to extract multiple components from the string all at once! This mitigates the need for updating the function in the future, if other parts of the string are needed later on (and it’s quite often that it would be the case!).

My preference for Regex over Split stems from:

Somewhat related to the principle of https://www.joelonsoftware.com/2005/05/11/making-wrong-code-look-wrong/
If code is wrong, it should fail catastrophically and loudly, not subtly or obscurely
It’s hard to make a regex that looks maybe right? Either a regex is right, or obviously wrong. (It could also be that I have lots of experience using regexes, and can write them without lookup up references)
OTOH, while split is conceptually easier to learn, for me, it’s hard or nearly impossible to see at a glance whether the code is write or wrong. For example, if you look at a block of code using split and various indexes, how would you instantly detect a possible OB1 (aka off-by-one error; https://en.wikipedia.org/wiki/Off-by-one_error)? Not possible. OB1s bugs are prevalent in software because the learning curve, and therefore barrier to entry, is low, so bugs are more likely to be introduced.
Regexes, OTOH, have a slightly higher learning curve, slightly higher barrier to entry, so those who use it tend not to make trivial mistakes
If the code never has to update ever again, then, great! split is sufficient. If the next engineer has to update it, they would not necessarily benefit from the existing code, and would have to re-evaluate all of the code in their head to make sure indexes are right.
Maintaining a list of patterns, or regexes, encourages a Solve for N mentality, whereas using split encourages a “solve it quick and dirty mindset”

Use Fully Qualified datetime in Python

2022-11-02T00:00:00+00:00

Whenever using the datetime module in Python, a highly recommended practice is to just import datetime at the top of the file, and use the fully-qualified module name in the code, as much as possible:

datetime.datetime
datetime.timedelta
datetime.date

If one does from datetime import datetime, it’s hard to figure out at-a-glance what datetime is referring to in the middle of a several-hundred-lines-of-code file.

For similar reasons, another common best practice in Python when using the typing module (https://docs.python.org/3/library/typing.html) is to import is as import typing as T or import typing as t (e.g. https://github.com/pallets/flask/blob/cc66213e579d6b35d9951c21b685d0078f373c44/src/flask/app.py#L7; https://github.com/pallets/werkzeug/blob/3115aa6a6276939f5fd6efa46282e0256ff21f1a/src/werkzeug/wrappers/request.py#L4)

Get Is the Worst Function Prefix Ever

2022-07-14T00:00:00+00:00

tl;dr;

Unless the function you are writing is a getter (which complements a setter), please avoid naming methods get_.

get_ lacks descriptiveness, precision, and is boring.

Rationale

Software engineers are supposed to creative, and get is possible the least creative function name possible.

Too often, I see codebases cluttered with get_ methods throughout, when the implementations of those methods do things far more complex than simply reading or getting a value from an object.

When half of a file with hundreds of lines of code are all named get_*, this makes code difficult to read, scan, and reason about. (Future blog posts will address code that is easy vs difficult to reason about.)

Alternatives

Since much of the world’s software (historically) has been produced from within English-speaking countries and by English-speaking programmers and software engineering teams, please allow me introduce to you the robustness of the English language.

While my hope and expectation is not for every software engineer to have a Shakespearean command over English vocabulary, I do think that it is quite tenable to learn a few prefixes to help codebases become more manageable and pleasing to read.

Without further ado, these are my suggestions:

build_
calculate_
extract_
fetch_
look_up_ / retrieve_
format_ / transform_

Below are examples and sample code, in Python (my language of choice).

`build_`

When to use it: Use this prefix for a method which takes in some data, and builds a more complex structure as a result.

Analogy: You have multiple loose LEGO bricks, and want to assemble those pieces to build a structure out of that.

Bad

def get_response(color, food, location):
    response = {
        'color': color,
        'food': food,
        'location': location,
    }
    return response

Good

def build_response(color, food, location):
    response = {
        'color': color,
        'food': food,
        'location': Location(location),
    }
    return response

Usage

response = build_response('green', 'eggs and ham', 'in a car')

# Result:
# {
#     'color': 'green',
#     'food': 'eggs and ham',
#     'location': {
#         'name': 'in a car'
#     },
# }

`calculate_`

When to use it: When you have some data, and some formula is applied to calculate a result.

Analogy: If it’s not doable via “mental math,” and needs to be calculated.

Setup

items = [
    {
        'color': 'green',
        'food': 'eggs and ham',
        'location': {
            'name': 'in a car'
        },
        'price_cents': 1525,
    },
    {
        'color': 'red',
        'food': 'hot chili peppers',
        'location': {
            'name': 'with a mouse'
        },
        'price_cents': 299,
    },
    {
        'color': 'orange',
        'food': 'carrots',
        'location': {
            'name': 'here or there'
        },
        'price_cents': 399,
    },
]

Bad:

def get_total(items, unit='cents'):
    total_cents = sum([item['price_cents'] for item in items])

    if unit == 'cents':
        total = total_cents
    elif unit == 'dollars':
        total = float((Decimal(total_cents) / Decimal(100)).quantize(Decimal('1.00')))
    
    return total

Good:

def calculate_total(items, unit='cents'):
    total_cents = sum([item['price_cents'] for item in items])

    if unit == 'cents':
        total = total_cents
    elif unit == 'dollars':
        total = float((Decimal(total_cents) / Decimal(100)).quantize(Decimal('1.00')))
    
    return total

A method named calculate_ will mentally prepare the engineer to be extra careful and meticulous when maintaining this code, because the goal is to be accurate and precise.

`extract_`

When to use it: When you need one, or a few pieces of information, from a more complex structure.

Analogy: You have a plain rock (diamond ore, gold ore) which is relatively useless on the surface, and want to extract the valuable contents (diamonds, gold).

Setup

response = {
    'color': 'green',
    'food': 'eggs and ham',
    'location': {
        'name': 'in a car'
    }
}

Bad

def get_color(response):
    return response['color']


def get_location_name(response):
    return response['location']['name']

# Usage

color = get_color(response)
location_name = get_location_name(response)

Better

response = {
    'color': 'green',
    'food': 'eggs and ham',
    'location': {
        'name': 'in a car'
    }
}

def extract_color(response):
    return response['color']


def extract_location_name(response):
    return response['location']['name']

# Usage

color = extract_color(response)
location_name = extract_location_name(response)

Great

Consider using object-oriented programming:

class Response:
    def __init__(self, raw_response):
        self.raw_response = raw_response

    @property
    def color(self):
        return self.raw_response['color']

    @property
    def location_name(self):
        return self.raw_response['location']['name']

# Usage

r = Response(response)

color = r.color
location_name = r.location_name

`fetch_`

When to use it: Use this prefix when the method makes an HTTP call or other API call. Inspired by fetch from JavaScript (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch)

Analogy: If you have a warm and cuddly friendly dog, and you’re at the park playing the good ol’ game of fetch; the object you’re intending to retrieve is accessible to you, but not immediately within reach.

Bad

def get_story_details(story_id):
    response = requests.get(f'https://api.example.com/story/{story_id}/details')
    return response

If the method is named get_, there’s no way to distinguish at a glance whether this function calls out to another server/service.

Whenever the flow of your code leaves your control (like making an API call), there is inherent risk and potential for errors to occur (e.g. “What if the remote API goes down?”)

Good

def fetch_story_details(story_id):
    response = requests.get(f'https://api.example.com/story/{story_id}/details')
    return response

By naming methods that make API calls to remote resources withfetch_, you allow engineers to quickly identify risky sections of code at a glance, without requiring them to read the details of a function – and this saves time – allowing a flywheel effect of: by writing code faster, teams produce more code / fix bugs more quickly, delivering more business value, enabling these teams to hire more team members, to build more products….

So, if I see a method named fetch_, I can immediately make mental note to make these sections of code more resilient (such as with try and except error handling, retry logic with exponential backoffs, etc).

`look_up_`

When to use it: Use this prefix when the goal of the method is to retrieve data that was previous stored in a local storage, like a database.

Analogy: There is a single piece of information you want to retrieve from among a larger collection, like *looking up** the defintion of a word in a glossary.

Setup

# MySQL

| id | item              | price_cents |
----------------------------------------
|  1 | eggs and ham      |        1525 |
|  2 | hot chili peppers |         299 |
|  3 | eggs and ham      |         399 |

Bad

def get_price(item):
    sql = connect()
    q = sql.query('items').where(item=item)
    price = q.execute()['price_cents']
    return price

Better

def look_up_price(item):
    sql = connect()
    q = sql.query('items').where(item=item)
    price = q.execute()['price_cents']
    return price

By naming a method look_up, you mentally prepare the next engineer who reviews this code that this method involves some form of database retrieval, and they can also keep in mind the performance characteristics of database retrieval.

Best

Use the database repository design pattern.

# repos/items.py

class ItemRepo:
    def get(self, item: str) -> Record:
        sql = connect()
        q = sql.query('items').where(item=item)
        record = q.execute()
        return record

    def look_up_price(self, item: str) -> float:
        record = self.get(item)
        price = record['price_cents']
        return price

`format_` / `transform_`

When to use it: When the desire output is a derivative of the inputs, or a metamorphosis such that output is not recognizable from the original form, but only to a connoiseur.

Analogy: When the input is uncooked potatoes and the output is mashed potatoes, you are transforming the raw ingredients into a useful end-product.

Bad

def get_mashed_potato(raw_potato):
    boiled_potato = boil(raw_potato)
    mashed_potato = mash(boiled_potato)
    return mashed_potato

Good

def transform_potato(raw_potato, form_factor):
    result = raw_potato

    if form_factor == 'raw':
        result = raw_potato
    elif form_factor == 'baked':
        result = bake(raw_potato)
    elif form_factor == 'boiled':
        result = boil(raw_potato)
    elif form_factor == 'mashed':
        result = mash(transform_potato(raw_potato, 'boiled'))

    return result

Alternatively, format_potato with the same function body above would work.

Conclusion

Please, for the love of all things proper, think twice before creating another method with the prefix name get_, and use one of the better alternatives: build_, extract_, fetch_, look_up_, retrieve_, transform_.

I promise you – your future self and your teammates will thank you!

Feedback

Agree? Disagree? Love it? Hate it?

Please leave comments or drop me a line!

14 Rules for Being a Godly Employee

2022-06-06T00:00:00+00:00

Back in 2016, I came across this article: https://thecripplegate.com/14-rules-for-being-a-godly-employee/

Recently, I’ve resolved to start my work week by reviewing the 14 rules:

Rule #1 – Eagerly start the day’s main work
Rule #2 – Do not murmur at your busyness or the shortness of time but buy up the time all around.
Rule #3 – Never murmur when correspondence is brought in.
Rule #4 – Never exaggerate duties by seeming to suffer under the load, but treat all responsibilities as liberty and gladness.
Rule #5 – Never call attention to crowded work or trivial experiences.
Rule #6 – Before confrontation or censure, obtain from God a real love for the one at fault. Know the facts; be generous in your judgment. Otherwise, how ineffective, how unintelligible or perhaps provocative your well-intentioned censure may be.
Rule #7 – Do not believe everything you hear; do not spread gossip.
Rule #8 – Do not seek praise, gratitude, respect, or regard for past service.
Rule #9 – Avoid complaining when your advice or opinion is not consulted, or having been consulted, set aside.
Rule #10 – Never allow yourself to be placed in favorable contrast with anyone.
Rule #11 – Do not press conversation to your own needs and concerns.
Rule #12 – Seek no favors, nor sympathies, do not ask for tenderness, but receive what comes.
Rule #13 – Bear the blame; do not share or transfer it.
Rule #14 – Give thanks when credit for your own work or idea is given to another.

Use Python iSort to Automagically Organize Imports within your Favorite Editor

2020-05-07T00:00:00+00:00

tl;dr;

isort (PyPI; GitHub) is a wonderful tool that will sort imports in Python automagically, so that you no longer have to either a) ignore eye-sores during code reviews, or b) sound like an angry grandparent asking people to sort/organize imports.

Quick Setup

If you know your away around the *nix environment, these are the abridged instructions:

Make the isort binary available somewhere in your path.
Install the isort plugin for the editor of your choice.
Profit

NOTE: (Optional but recommended) Add a .isort.cfg file to your HOME directory, so that even you are working on a random script or project that doesn’t have one, the powers of isort are still available to you.

Drudgerous Line-by-Line Instructions (or my setup)

If you don’t have one already, create a new system-wide Python virtualenv.
1. The way I’d do that is to do: /path/to/bin/python/virtualenv ~/.venv
Install isort.
1. ~/.venv/bin/pip install isort
2. (Generic command: /path/to/venv/pip install isort)
Add the bin directory of your system-wide virtualenv to your path, or just the select binaries that you want.
1. I have already added ~/bin/ to my path via bash-ftw, so my preference is to just symlink the specific binaries that I need.
2. For convenience, I’ve symlinked the following:
  1. ln -s ~/.venv/bin/isort ~/bin/isort
  2. ln -s ~/.venv/bin/python3 ~/bin/python3
  3. ln -s ~/.venv/bin/pip ~/bin/pip
Install an isort plugin for your editor (in my case, emacs, The best text editor in the world™).
1. For emacs only:
  1. For EZMODE™, my dotemacs setup is on GitHub (just git clone, make install, and you’re set!)
  2. Add two lines to your dotemacs (typically ~/.emacs.el or ~/.emacs.elc, or somewhere in your Emacs load path):
    1. (require 'py-isort)
    2. (add-hook 'before-save-hook 'py-isort-before-save)
2. No longer have to manually organize your Python imports anymore! The isort plugin will do it for you automatically whenever you save your file.

Thank You!

Thanks for reading; now go forth and write some awesome Python code!

Questions, comments, suggestions? Leave a comment or subscribe to the blog for future helpful tips!

What, How, and Why?

2019-12-12T00:00:00+00:00

In order to learn, there are three kinds of questions to ask: What?, How?, and Why?

Of these, Why? is the best question to ask, and What? is the worst; How? is in the middle but right near the bottom near What?.

The reason Why? is the best question for learning is that it conditions you to go to first principles and learn how to learn.

What? is the absolute worst type of question to ask, because usually the answer to What? is a reference lookup away. A better question to ask is: Where? can I find my own answer to my What? question?

Likewise, How? is rarely a good question to ask. Knowing how to do something is different from being able to do it. The answers to How? questions are best answered by not asking it directly, but rather by finding an expert who already possesses the knowledge of How?, and simply watch that master at work. To achieve this level of mastery takes years upon years of honing one’s craft, and the answer to How? is rarely succinctly communicable, therefore frustrating both the mentor and the apprentice. Instead, just Watch and learn.

Hopefully, at whatever institution you are in, you are surrounded by individuals who have mastered the How? and you can find opportunities to learn from them. If not, there are plentiful and abundant (not to mention, free) resources online to learn How?, such as YouTube. So, really, How? is a pretty bad question to ask since it is actually a series of What? questions in disguise. Instead of asking How?, just observe those who already know how and do, and as you observe, find opportunities to ask them, Why?. And, if they are not available, a still better question is, Where? can I find the resources to learn How?

Debugging Address already in use errors

2019-07-31T00:00:00+00:00

All too frequent an occurrence in the development lifecycle is doing some work, closing up your laptop/suspending the machine, and coming back to your work hours or days later.

As you try to start up the local webserver or API server, you get cryptic error messages like the following:

Address already in use
Another process is already listening on port
Port xyzw is currently used by another application
OSError: [Errno 98] Address already in use
self.socket.bind(self.server_address) blah blah blah blah

And after checking all of your open terminal windows, that you have not in fact any running or killable-processes…

At this point, the ~~n00b~~ naive way to fix this problem is to simply restart your entire machine. To be fair, this technique works almost all of the time, but kills productivity, and forces you to save work-in-progress that’s not at a good stopping point, or worse, accidentally restart without saving your progress.

But, there is a better way.

Let me introduce you to a command:

netstat -tulpn

This command will print out the bound network ports on your machine, and which processes and process ids are running them. To free up the port to be used by your development server once again, simply kill PID, where PID is the process ID.

Now, how to remember this command? I haven’t figured that out yet, nor have I thought of an alias I want to save it to, that is just as memorable. The arguments almost spell out “tulip”–like the flower, except missing the i, and you just add an n to it. Maybe a mnemonic like, “If you fix your network address port in use issue, you will smell the essence of n tulips”?

Whatever you do, netstat -tulpn is now a friend and welcome companion in my software toolbox.

Debugging like a Boss with Slack

2019-07-11T00:00:00+00:00

Edit: This article also got published on the GoodAudience Blog.

I’ve been using println debugging since forever. It’s the best! It’s minimal, is the least surprising form of debugging, and allows you to set-it-and-forget it. I’ve also used interactive debuggers before, but when println debugging techniques are used effectively, I’d argue that step-through interactive debuggers are not necessary at all, and actually slow you down.

For years now, I’ve been using an evolved form of println debugging, which I affectionately call “Slack debugging,” and I’ve written various manifestations of utility/helper functions called slack_debug over the years.

This has been a close-kept secret for myself and select other teammates and colleagues who were curious to know what exact wizardry I was doing.

And now, for the first time, I’ve decided to clean up the solution, open-source it, and share it with the world.

Behold, the Power of “Slack Debugging”

In [1]: from htk import slack_debug

In [2]: from htk import slack_debug_json

In [3]: slack_debug('This is seriously awesome!')
Out[3]: <Response [200]>

In [4]: slack_debug('Yeah, no kidding.')
Out[4]: <Response [200]>

In [5]: slack_debug_json({'A':1,'B':2,'C':3,'X':['foo','bar','baz'],'Z':{'nested_key':'nested_val
   ...: ue'}}),
Out[5]: (None,)

And without further ado, Slack debugging is available here: https://github.com/hacktoolkit/python-htk and https://github.com/hacktoolkit/pyhtk-lite. (And for Ruby: https://github.com/hacktoolkit/htk-rb).

Love it? Hate it? Please share your thoughts and comments, or even better yet, submit pull requests to make it better!

Kill Netskope Client on Mac

2018-10-23T00:00:00+00:00

Netskope is a corporate security tool installed on corporate-owned devices that will introduce and override the default SSL certificate authorities by injecting its own local server.

For developers, this often poses an inconvenience especially if they need to develop applications that make API calls and HTTP requests to other web services, if the developers are hitting a web service that has not previously been white-listed by the IT department. The IT department may not be in close communication with the engineering teams, and require additional information from the developers in order to configure a new whitelist rule, impacting the ability of engineers to meet deadlines.

tl;dr; How to ~~kill~~ temporarily disable Netskope

sudo launchctl unload /Library/LaunchDaemons/com.netskope.stagentsvc.plist

TIL: pbcopy

2018-10-03T00:00:00+00:00

TIL: Discovered a super game changing technique. From command line, you can copy contents of any file to Mac OS clipboard by running this command:

cat yourfilename.txt | pbcopy
Open your other application, press Command+V to paste as normal
Profit. Immensely.

When Slack links don't work, Restart Google Chrome

2018-03-01T00:00:00+00:00

Periodically, web hyperlinks in Slack would stop working.

The fix is surprisingly quite simple: Restart Google Chrome.

The best way to restart Google Chrome safely and preserve your current open tabs is to type into the address bar:

chrome://restart

Is there an engineering talent shortage in Silicon Valley?

2017-10-16T00:00:00+00:00

Yes, there is a huge engineering talent shortage in Silicon Valley. There are more companies trying to build disruptive technology that have greater hiring needs than the available supply of engineering talent.

I currently work at Iterable, and we are as of present a Series B company with around $30M raised total. We get hundreds of applicants a week, but few of them are qualified enough. I joined as the 3rd engineer of the company and now we are around 20 engineers; virtually every single engineering hire on our team has been an internal referral.

A company like Iterable is able to reward any qualified engineer quite handsomely; but for these top candidates, they are often like the proverbial kid-in-a-candy-shop and are highly sought after by multiple companies working on the most exciting, cutting edge technologies.

Iterable is poised to be a unicorn or deca-corn some day, as we are quickly becoming one of the essential, best-in-class tools for omni-channel growth marketing automation.

I love to reason and explain concepts using analogies. If you look at the world of professional sports, such as basketball or baseball, you will know that it seems every year, only a few teams consistently make it to the playoffs, semi-finals, and then finals. The teams that make it to the post-season are a small fraction of the total number of teams in the league; an even smaller fraction are serious championship contenders.

If you equate working at a software company in Silicon Valley to being an athlete that makes it to the MLB or NBA, then you will have a very good idea of what people are talking about when it comes to shortage of engineering talent. Many players are simply happy and consider it to be a life achievement to make it to the pro-leagues; only some of the players who are the hungriest and want to achieve greater success are willing to put in the hard work and dedication to reach the upper echelons of the game—whether it be basketball, or baseball, or software engineering.

Furthermore, just as you have specialist positions in basketball, the same also applies to software companies. To be a championship contender (e.g. IPO, $1B+ exit, etc), you need the absolute best, top-notch talent in each position — backend engineers, frontend engineers, infrastructure engineers, security engineers, mobile engineers, etc, and finding top notch talent to fill each of these positions is quite challenging and difficult. It takes years and years to hone and finely tune the skills in any particular area, such that for any position, there are only a few candidates who possess the skills, and then even fewer who are available and not already committed to another team, or working on their own startup.

Hence, it is quite safe to say that there is a huge shortage of engineering talent in Silicon Valley.

Edit:

Initially I did not think to make this a “recruiting” pitch for Iterable (seriously), but just to give an idea of how good we have it, here are some of our perks:

Free lunches every day, catered by Zesty, from a variety of top eateries in San Francisco. Fridays we can eat out anywhere, with the team, and just Expensify it. On several occasions we’ve eaten at places where it cost more than $30 per head.
Quarterly stipend of $300 for personal / professional development — I use it to buy books, audio books, and an Audible subscription — for any employee, not just engineers.
Monthly stipend of $160 for gym membership. I use it for gym membership and workout sessions with a personal trainer — for all employees.
Quarterly team off-sites — this past week we did an escape room as an engineering team, for team bonding
Semi-annual (twice a year) company outings — an annual sailing trip on the San Francisco Bay, and an all-expenses-paid company retreat (we went to Bodega Bay this past year)
Monthly all-hands and office happy hour with lots of beer and delicious food. I don’t really drink, and there are lots of non-alcoholic options
Fully stocked kitchen, refrigerators, and snacks with fresh fruit, kombucha, Stumptown coffee, Quest protein bars, Krave beef jerky, Tcho chocolate, etc luxurious snacks
Expense ANY computer hardware, monitors, or desktop equipment. Some of my teammates have dual 34″ ultra widescreen curved monitors and $300 mechanical ergonomic keyboards and vertical mice
Expense ANY resources needed (software, IDE, books)
Flexible vacation, flexible hours, and flexible location (remote)
Interesting guest speakers and world-class learning opportunities — the former CEO of Twitter, Dick Costolo, came in for a fireside chat

Here are some photos from the above mentioned team outings: Iterable

Even with all of these amazing perks, not to mention being able to work with some of the top minds and high achievers in the industry, we have been having such a hard time finding engineers that meet our bar for good talent and good culture and team players (you see, just like in sports, sometimes you have superstar athletes who are not team players, and these would be simply toxic to your organization).

Iterable is currently ranked at #1 on the SaaS1000.

If you’re out there, would you please DM me?

Edited Oct 27, 2017:

Someone just shared this article with me that describes the housing crisis not just in Silicon Valley, but in all of California: https://www.bisnow.com/national/…

There is a severe engineering talent shortage in Silicon Valley, and that goes hand-in-hand with the housing shortage in California.

View original question on Quora

Communicating with Engineers: How to File Good JIRA Tickets

2017-10-12T00:00:00+00:00

By Jonathan Tsai (@jontsai | https://linkedin.com/in/jontsai)

Original Permalink (working Quip doc) : Communicating with Engineers: How to File Good JIRA Tickets
License: CC BY 3.0

This article was originally written as part of a mini-training for the customer success team at Iterable in July 2017. For the most updated version of this article, view the Quip version at the above permalink.

Edit: This article also got published on the GoodAudience Blog.

Purpose

“Those who fail to learn from history are doomed to repeat it.” - George Santayana

Engineers build things, and also fix stuff when things break. Everyone’s time is valuable, so how can we effectively capture and document issues that come up, so that we make the product better, and learn as a team rather than “repeat the mistakes of history.”

Common Pitfalls to Avoid

Conflating issues (that shouldn’t be conflated)
- Symptoms:
  - Combining multiple issues into one JIRA ticket.
  - This is happening for me, too!
- Why it happens: Upon initial inspection or cursory examination, two or more issues can appear to have the same cause. However
- How this hurts: If all reported under one issue, we may solve one problem but unintentionally overlook others that got masked.
- Recent examples: Emails being sent out slowly for Spotify
- Actual root causes: Three separate issues, actually, with 3 different solutions
  - CloudAMQP memory failure
  - Unhandled exception in our messaging dequeuing code causes processor
  - Inadvertently downgraded date-timezone library dependency, which failed to parse a particular timezone
- Recommended alternative courses of action:
  - Do some research to pinpoint when (time), where (context: app? mobile? API?), and how (what the user was doing)
  - Capture the major themes
Prematurely filing a ticket / fragmenting tickets (opposite of conflating issues)
- Symptoms:
  - You’re filing too many tickets
  - The rate at which you open tickets far exceeds the rate at which they get closed or completed (optimal ratio is hard to specify, but maybe 3:1 or 5:1, but not definitely 10:1 or 20:1)
  - The ticket has already previously been filed
- Why this happens:
  - Want to avoid conflating issues
  - Did not research if an existing known issue
  - Did not consult or check with other teammates
- How this hurts:
  - Too many issues get logged, but never worked on.
  - Generates a lot of noise and distraction
- Recommended alternative courses of action:
  - Talk to teammates and if at least 1 or 2 other CS persons agree it should be a ticket, or 1 other engineer, then file it
  - Do some research whether an existing ticket already exists. This may take more than a basic search or two. Scan through past 2 weeks at least, and search a couple of most likely specific keywords.
  - When it does happen, find the other ticket, enhance the ticket with more detail, and resolve one as duplicate.
Providing too much detail.
- Symptoms: Parroting the exact request from the customer
- Why it happens: You feel inundated or overwhelmed with customer requests, so you report them in JIRA or to an engineering Slack channel for assistance
- How this hurts: CS is the first line of defense for engineering team, otherwise engineers wouldn’t be able to focus on making stuff better and spend most of the time troubleshooting. Engineers aren’t necessarily better problem solvers than CS, so they would start to feel inundated or overwhelmed, too. The engineer would have to
- Recommended alternative courses of action:
  - Imagine you are a 911 operator/dispatcher. How would you field calls?
  - Do some translation or summarization of the actual issues at hand. No need to provide a diagnosis
  - Perform some high-level filtering, triaging, and pattern matching before filing a JIRA ticket or hitting up Slack channel
  - Ask a teammate or someone in person (counter-intuitive, yes, but a poorly researched JIRA ticket or Slack message disrupts more people, since N individuals need to read and process that message); an alternate could be using Direct Messages in Slack over blasting a channel.
Providing too little detail
- Symptoms:
  - JIRA ticket is really short, or took you less than 10 minutes to create.
  - Engineer follows up with more questions. You answer some questions, and more questions are asked.
- How this hurts:
  - And endless cycle of questions with no further clarity being established.
  - Or, the JIRA ticket never gets addressed at all, because it wasn’t actionable.
- Recommended alternative courses of action: A good JIRA ticket:
  - Should take at least 5, maybe 10-15 minutes to create, if not longer
  - Is comprehensible by any engineer
  - Actionable. The problem is succinctly described, and it is clear what the next steps are.
  - Tracks only one issue or actionable item. Only has “one user story.”
Not filing a ticket
- Symptoms:
  - You have lots of conversations about an issue.
  - You’ve already spent 20-30 minutes thinking or talking about an issue
  - You’ve already spent 20-60 minutes trying to address and issue on your own, and it is still not resolved
- Why this happens:
  - You get caught up in the moment or simply forget
  - Not wanting to file too many tickets
- Recommended alternative courses of action:
  - File a ticket!
  - Include a good title. A good title describes a course of action, rather than the symptom of the problem. Focus on solution, rather than the problem
  - Include a good description.
    - One sentence summary that answers several of “Who? What? When? Where? Why? How?”
    - A short 2-4 sentence paragraph of the expanded problem and proposed solution
    - Any supporting details in a bulleted list
      - Links to the actual issue in the product or app
      - Links to customer conversations, Slack conversations
      - Links to supporting documentation
    - Additional attachments, key snippets of conversations or key pieces of “evidence” gathered or research performed, or troubleshooting steps already taken

Choosing Good Titles

This is one of the MOST IMPORTANT things you can do. You may not think it, but software engineers spend a lot of time reading every day. Reading code, reading tickets, reading Slack, etc. Therefore, choosing a good title for the ticket will have significant effect on comprehension and speed of delivery.

A good title:

Describes a course of action, rather than the symptom of the problem
Describes the MVP. Some people like to write user stories, but I’m not a fan.
Usually starts with a verb, stated in the form of an imperative, as if authoritative:
- Simply, “Do this” is much preferred to “Can you do this because So and So needs this or that?”
- This is not rude; it’s succinct and to the point
- The reasoning and background should go in the description.
- Engineers can push back or discourse on it in the comments, with their own reasoning
Important: No verb implies no action, and therefore, none will be taken.
JIRA ticket titles should not be nouns, generally.
- For new features:
  - Bad: Facebook Custom Audiences
  - Good: Implement Facebook Custom Audiences
- For improvement or requests:
  - Bad: IP Whitelisting
  - Verbose: Customer X is asking for a set of IPs or an IP range that they can whitelist when it comes to receiving webhook posts
  - Good: Provide Fixed IPs for Webhook Servers for Customers to Whitelist

How to pretty print JSON from CLI

2017-10-09T00:00:00+00:00

Software engineers frequently have to deal with JSON and inspect or manipulate it.

There are two easy ways of pretty-printing JSON from command-line to aid in visual inspection. The first requires no installation, and the second requires a minimal installation but also provides syntax highlighting and manipulation capabilities.

Method 1: Using Python - Nothing to Install

Optional: Add alias to .bashrc
alias json='python -mjson.tool'

Example:

$ echo '{"cool": { "story" : { "bro" : [1, 2, 3] } } }' | json
{
    "cool": {
        "story": {
            "bro": [
                1,
                2,
                3,
            ]
        }
    }
}

Reference: http://www.restlessprogrammer.com/2013/03/how-to-pretty-print-json-from-command.html

Method 2: Using JQ - Minimal Install

Mac install via Homebrew: brew install jq
Ubuntu install apt: sudo apt-get install jq

$ echo '{"cool": { "story" : { "bro" : [1, 2, 3] } } }' | jq .
{
    "cool": {
        "story": {
            "bro": [
                1,
                2,
                3
            ]
        }
    }
}

Reference: https://stedolan.github.io/jq/

If you start learning to code in your mid 20s, can you make a good living off it?

2017-07-06T00:00:00+00:00

I know of several people who start learning to code in their mid 20’s, and are doing fine now.

However, it’s very subjective what a “good living” is.

Compared to the rest of society, programmers (anyone who writes code) and software engineers (a subset of programmers, with more emphasis on “engineer”) might make a relatively higher salary.

However, just as with any field, there are different grade levels, and how well you do depends on what level you are able to attain, and that has a correlation to how much time you spend working on your craft.

If you work hard and dedicate yourself to software engineering now, you can probably make a decent living in 2–3 years. Whether that will exceed your earning capacity in whatever other endeavors you may try, or are currently in, varies widely depending on your individual background, circumstances, opportunities, and experiences (and ability to grow in all of those).

I think it may take 10–20 years of smart and hard work to get to the top tier of any field, software engineering included.

If you’re able to keep your head down and stay focused, go for it. If you’re easily distracted by others around you and are discouraged by others who seemingly effortlessly have more success than you, this may not be the path for you.

I have one friend who was working in the field of tech and executive recruiting, who was considering a switch to software engineering when he was already in his early thirties. I quoted to him a proverb, “Do you see a man skilled in his work? He will stand before kings; He will not stand before obscure men,” and he decided to stay the course and excel in what he was already good at. Fast-forward two years and he’s now crushing it as an executive recruiter, making much more impact than he would have, compared to if he had switched his career to software engineering.

View original question on Quora

IntelliJ ProTips™: Start the IntelliJ JVM process with additional memory

2017-05-25T00:00:00+00:00

Assorted tips and tricks for IntelliJ

Start the IntelliJ JVM process with additional memory

Open the application once, normally, after a new install before editing this file.
a. Otherwise, the signatures won’t match and the OS will think that the package has been tampered with, and security settings will prevent launching of the application. You will then have to delete the entire application and reinstall it.
Quit IntelliJ (Cmd+Q)
Edit /Applications/IntelliJ IDEA.app/Contents/bin/idea.vmoptions
a. emacs "/Applications/IntelliJ IDEA.app/Contents/bin/idea.vmoptions"
or
vi "/Applications/IntelliJ IDEA.app/Contents/bin/idea.vmoptions"
Increase the following settings from their defaults (note: only works if your machine has sufficient RAM):
a. -Xms2048m
b. -Xmx4096m
c. -XX:ReservedCodeCacheSize=1024m
Start up and enjoy your upgraded IntelliJ on steroids.
Note: These steps must be repeated after every upgrade.

Other options listed here: http://tomaszdziurko.pl/2015/11/1-and-the-only-one-to-customize-intellij-idea-memory-settings/

Enriched objects

2016-09-21T00:00:00+00:00

random: my new favorite coding adjective recently is “enriched”

val enrichedObj = obj ++ someAdditionalStuffs

reminiscent of / inspired by https://en.wikipedia.org/wiki/Enriched_uranium

Universal Directives for Shell Scripts

2015-10-12T00:00:00+00:00

When writing shell scripts or other types of executables, you often put a directive at the top of the file to indicate which kind of interpreter to use.

#!/bin/python

Oh wait, that didn’t work?

#!/usr/bin/python

Okay, that worked on my Mac. But why all the guesswork? There is another command that is better.

Introducing env. Use it as a directive like so:

#!/usr/bin/env python

And if it’s a bash script:

#!/usr/bin/env bash

env is kinda like which. It takes the first argument and figures out the full path of that, and uses it as the directive

man env to read more about it.

Basically using #!/usr/bin/env INTERPRETER makes your script more portable

portability (adj.)
the ability to run your script anywhere, on any OS/platform

The holy grail goal of programming/CS = write once, run everywhere

It is somewhat circular in that it depends on env living inside of /usr/bin/, but I believe it is a convention that is much more rarely broken than where python or bash lives.

convention (n.)
another CS vocab word.
The quality of being predictable, doing expected things.

e.g you walk into a dark room, you expect the light switch to be near the doorway opposite of the door hinge, not a mousetrap clamping down on your fingers when you reach along the wall.

The Most Valuable Programming Languages to Know for the Future

2015-06-01T00:00:00+00:00

Originally posted on Quora: Jonathan Tsai’s answer to What is the most valuable programming language to know for the future and why?

I will have to agree with Michael Hanson’s answer as well as some of the other ones that suggest learning multiple programming languages is more beneficial than picking just one.

I myself am a multi-linguist of both human and programming languages. If you randomly survey a large body of people living within various civilizations in the world on what the most valuable human languages are, the answers you’ll get back are likely to be English, Chinese, French, Spanish (not necessarily, but roughly, in that order).

When you know multiple languages, learning additional ones become much easier, because through immersion/osmosis, you are made aware of the different types of parts of speech, recognize the cognates and influences of one language to another, etc. That’s why you will frequently come across Europeans who speak at least 3 languages (English, French, and the language of their country), and Indians who speak 3-5 languages or more (English, Hindi, the language of their state/region, + neighboring state/region).

Programming languages actually have a much simpler syntax than human languages. The more you know, the easier it is to learn more.

I would rephrase the question as plural instead of asking for a singular response: What are the most valuable programming languages to know for the future and why?

If you like lists, this section is for you, but not in any particular order:

Python - frequently used by scientific and statistics communities and data science; it’s a fairly easy language to learn and remember with libraries for just about everything you would need, and a very active development community who can whip up a library if one doesn’t exist already. Personally, Python is really easy to me and reads and writes just like pseudocode. I’ve written as many as 100-200 lines on a whiteboard or a text editor in one sitting and was able to execute the code in the interpreter without any syntax errors.
Ruby - I am not a regular practitioner, but Ruby is also very powerful in that it is an interpreted language like Python, which means that it doesn’t need to be compiled before it’s run. A lot of famous websites are built on, and many of my friends who are paid handsomely, use Ruby, due to the popularity of the Rails framework (Ruby on Rails)
Java - This one is nice because it’s a compiled language and the compiler can optimize the heck out of the code, and it runs really fast. There are lots of Java practitioners, and the majority of financial websites (banks, stock trading platforms, etc), will use Java because of its outstanding speed and performance characteristics. Incidentally, Java is also the language used by the Android operating systems, so if you want to learn to write for something like 79-80% of the world’s mobile users, this is the way to go. (Source: 2014: The iPhone 6 Had Better Be Amazing And Cheap, Because Apple Is Losing The War To Android; 2015: Apple iOS And Google Android Smartphone Market Share Flattening: IDC)
JavaScript - Rich web applications / the entire frontend / look-and-feel of all modern websites. Enough said. JavaScript is a language that is write once, runs everywhere. Browsers going all the way back to 1998 (?) and even earlier have been using JavaScript, and the language along with its frameworks and ecosystem have improved tremendously over time, as well as its performance characteristics due to progressive developments of JavaScript runtimes (e.g. V8).
Shell Scripting / Bash - Unix utilities are awesome. They are lightweight, fast, and often times, you can just use an existing Unix utility for doing common but advanced tasks like searching and sorting on a single machine, and additional various things.
C++ / C - it still powers lots of high performance systems, but is a bit more cumbersome to write than Java, hence the reason that Java is the language of choice when building such systems. However, this is a very low-level language, and the thing about programming languages is that their interpreters and compilers are quite meta–if you chase it up/down the stack far enough, you’ll find that some/most of them are inevitably implemented in C. Not much daily use, but if you like learning the nitty gritty or really knowing what’s under the hood (analogous to learning Greek / Latin to having a better grasp on English), then this is something you have to learn.
Swift - this is a new language that Apple developed over the past few years and just launched last year (2014). I won’t go digging up the links because you can easily find them, but the language designers designed Swift to be a modern language that can be compiled but also has several characteristics of interpreted languages and IDE tools (like quick turn-around of write code-and-execute), and admittedly it borrows a lot of ideas and nice features of other languages like Ruby, Python, JavaScript, as well as Java and C. And it’s also inter-operable with Objective C. Look up the WWDC videos of the Swift language being unveiled (https://developer.apple.com/vide…), and you’ll hear loud cheering and clapping and whooping over small features. I cheered along with the rest when I watched from home.

I would be hard-pressed to give you a ranked list since I don’t know the statistics off the top of my head, but my impression is that the above are your programming language lingua francas, and most likely the highest value in terms of popularity/ubiquity, economic value, and utility.

View original question on Quora

Hack GitHub Streaks

2015-03-25T00:00:00+00:00

Tonight, I was doing some research and maintenance on a couple of my open-source projects. In particular, I had recently learned about EditorConfig (which BTW, is a great idea! and a standard EVERYONE should adopt), and wanted to add it to most of my projects.

That brought me to my GitHub homepage, and I caught a glimpse of my GitHub streak. Only 2 days! I was on a roll for a few days, and then a few days of working on private projects reset my streak.

Not that it matters, anyways, but I was slightly bothered, and being the fan of automation that I am, I quickly thought up how to automate my GitHub contributions to keep my streak alive, artificially.

30 minutes later, github-streak is born and released to the world. I call that being diligent at slacking off.

It’s a fairly straightforward project that had a few requirements/constraints as I designed it:

Had to be portable, modular
Had to be cross-platform, as much as possible
Had to be really simple technology and minimal dependencies–don’t overthink it

The result:

Written in BASH/shell script (my favorite shell)
Even uses the conscientious #!/usr/bin/env bash shebang directive
Creates a symlink in /etc/cron.daily – probably won’t work for every system, but at least was available on my webserver and others that I’ve used in the past
The script just appends a date string to a .streak file (hey, a new convention!) and git commits/git pushes
Added in the .editorconfig and .travis.yml goodies
MIT licensed (my favorite open-source license)

So, here you go internet, have at it: github-streak

Show some star/fork love. Plz thx kbye.

Edit: Here are some other GitHub projects worth checking out that manipulate commit history as art:

Equality and Inequalities in JS and PHP

2015-02-26T00:00:00+00:00

It’s 2015, and from time to time I still see a lot of == peppered throughout codebases that I work with, and just wanted to share a few posts that might change your mind =)

tl;dr;

=== and its complement !== and should be used like 99% of the time because it is faster and more accurate.

> 0 == "0"
< true
> 0 === "0"
< false
> 0 == false
< true
> "0" == false // really?
< true
> !!"0"
< true
> !"0"
< false
> "0" === false
< false

There are more perverse and mind-bending examples that are out there, but I’ll leave you with those for now.

As you can see, the former == does type coercion, which is lazy way of doing comparisons without casting data explicitly, and can find yourself in hot water down the line. The latter === does type checking first, and then checks equality of the values.

Protip #1: If you want to test a JS expression or figure out some syntax really quickly, just open your JS console in the browser inspect tool.

Protip #2: If testing for something truthy or falsy, a nice trick is to just do use the ! operator twice (!!). As can be seen above, JS treats non-empty strings as a truthy (yet, strangely, "0" == false)

WordPress' New JetPacks All-Sites Dashboard

2014-12-16T00:00:00+00:00

The JetPacks 3.3 upgrade that allows you to manage and administer multiple WordPress sites from one WordPress.com account is a breeze to use!

I’m not quite used to it yet and still getting acclimated, but I think it’s a great time-saver and will grow on me more over time.

I can see it as a great feature to use once a site for daily workflows in updating content once a site’s major structural and design work has been done.

Cheers to the WordPress team for this great feature!

Read all about it in the original post.

Upgrading to Ubuntu 14.04.1 LTS

2014-09-06T00:00:00+00:00

tl;dr; When upgarding from Ubuntu 12.04 to 14.04 LTS, the Apache also gets upgraded from 2.2 to 2.4. There are lots of backwards incompatible changes.

I just spent the last 3 hours fixing my server after impulsively upgrading to Ubuntu 14.04.1 LTS from Ubuntu 12.04.

Well, not exactly that impulsively. I had seen this pestering MOTD-style message upon login for weeks now, and I figured that since 14.04.1 is a point release nearly 6 months after the initial release, it should be relatively issue-free.

I had already done 3 upgrades from 12.04 LTS to 14.04 LTS… on desktops. Two had gone successfully, and one on my parents’ computer unfortunately messed up pretty badly in the middle so that some of the ubuntu-desktop stuff is just not working properly, but I digress.

I figured that since it was a Friday night in addition to the above–“What the heck, why not?”–and dove right in. The OS upgrade was pretty much issue free. I let out a sigh of relief.

And then my luck ran out. All of my websites didn’t load. Read: Nothing worked. I started getting Pingdom pages and Nagios alerts non-stop.

Thankfully, the Internet is a great resource and I was able to identify the problem pretty quickly. Applying the fix is what took most of the time.

Most useful of all the resources that I came across was Upgrading Apache 2.2 to 2.4. I’d recommend reading through it thoroughly if you’re going through the upgrade yourself, but here are some highlights:

Don’t be alarmed if all you’re seeing for any of your websites is just the contents of /var/www/ or /var/www/html
All separate virtual host (VirtualHost) files and configurations need to end in *.conf due to the command in /etc/apache2/apache2.conf IncludeOptional sites-enabled/*.conf
It still wasn’t pulling in my VirtualHost
Got rid of NameVirtualHost *:80 as the first line in all of my VirtualHost config files. It wasn’t necessary. Good riddance.
Directory permissions now need to be explicitly granted. Require all granted (the old style was Order Deny, Allow; Allow from all)
I noticed that my websites were loading slowly, so needed to set EnableSendfile On in the main config file (apache2.conf)
For my Django sites, I needed to change the Directory permissions for the static files directory
For my custom WordPress sites, I needed to set the Directory permissions for the DocumentRoot as well as explicitly set AllowOverride all to allow picking up the .htaccess
/etc/php5/apache2/php.ini has short_open_tag disabled. I decided to leave it off and change my limited number of PHP applications that were using short open tags, because it’s better practice.

Other helpful resources:

Use Intel Hardware Acceleration for Faster Android Emulation

2013-09-20T00:00:00+00:00

tl;dr;

If this is you:

Android developer with an Intel-based computer
Sick of dealing with a slow emulator

Then:

Install Intel HAXM (Intel Hardware Accelerated Execution Manager)

I never knew this, and thought that I was doomed to a slow emulator or had to debug and deploy over a constantly plugged-in device, which could at times be inconvenient.

Thanks to the guys at CodePath (@thecodepath) for this useful tip! If you’re interested in learning iOS or Android, definitely check them out.

Finally! A mobile school for engineers

2013-09-12T00:00:00+00:00

Got an email today about a new Android training program starting up in SF for engineers. Best of all, it’s free!

I’m definitely going to check them out and try to sign up.

Update (2013-09-15): If you’re unable to attend the class, they may have a “course observer option where you will have access to all the materials for the class, but you just won’t participate in the live classes, discussion forums, or the group projects.”

–

Jonathan,

I’m a co-founder of CodePath, a startup that runs iOS and Android training programs. We’ve trained hundreds of engineers at Yahoo who are moving from web development to mobile development.

We’re offering the same training program to engineers in the bay area for free. It’s a project-based course, meets twice a week for 6 weeks, and the next class begins on October 2nd (another one starts in January). It’s an intensive program, but we’ve seen great results with it.

We’re currently accepting engineers with a technical degree OR at least two years of object-oriented programming experience. If you’re interested, sign up here:

Please forward to any engineers or mailing lists that might be appropriate, additional details below.

Tim

–

Additional Program Details

This is an evening bootcamp meaning that all sessions are held after work hours and all of the bootcamp projects can be done after hours and on the weekends. Here’s a summary of what you need to know:

Free of charge for engineers
We will select up to 30 applicants for each cohort
6-week evening bootcamp starting Oct 2nd
2 on-site sessions each week in the evening (instruction and lab, 7-9 pm)
Sessions held at Zynga SF office at 8th and Brannan
A mini-app is built each week (5 total projects)
Group project designed and developed over 6 weeks

Here are the requirements for the course:

Physically present in San Francisco and available Mon+Wed from 7-9 pm during sessions
Technical (CS, EECS) Degree and/or 2 years of professional software development experience
8-10 additional hours every week to dedicate to projects and collaborative learning
Existing development experience with object-oriented languages
Passionate about learning and collaborating on mobile projects
Laptop with Mac OS X (iOS, Android) or Windows (Android)

Read more details about this program and us here and on our website at http://thecodepath.com. Also, you can follow us on twitter at @thecodepath for future updates!

Pinterest wasn't hacked

2013-05-12T00:00:00+00:00

Seriously though, I messed up in my previous blog post. Pinterest wasn’t hacked, and I’m sorry that I jumped to conclusions when I actually stumbled upon a spammer’s C&C. I still feel that the actions I took at the time were correct according to the knowledge that I had, but based on what I know now, I would have approached it different (contacted Pinterest first, now that I actually have their contact info, before writing a blog post).

Thankfully, Jon Jenkins from Pinterest did contact me:

Hi Jon,

I lead the engineering team at Pinterest. I saw your blog post you wrote about Pinterest security.

http://www.jontsai.com/2013/05/11/aws-ec2-security-vulnerability-and-pinterest-hacked/

For the record the site you found is not one that is associated with Pinterest in any way.

It appears that you most likely found a command interface for a Pinterest spam net. We have systems that detect and mitigate the impact of these spam nets. It would be convenient if you could pass along the addresses and passwords you obtained so that we can validate our spam detection systems are picking up these accounts.

Thanks,

Jon Jenkins

And my response:

Hi Jon,

Thanks for reaching out to me. I later realized based on the discussions and Hacker News comments that this wasn’t Pinterest (although, there was still a small possibility that it was an internal tool written by a small team within Pinterest, or by an individual employee for a Hack Day contest, etc). I’m sorry for jumping to conclusions and accusing Pinterest of security vulnerabilities, but I had thought that just as if my friend felt he was having a heart attack, I would call 911 first and then wait for diagnosis later, rather than let the window of urgency elapse. I’m planning on writing a follow-up blog post on that matter.

Anyway, regarding the collected addresses and passwords, I forwarded them to engineering@pinterest.com, not sure if that was the correct email address or you picked it up. I just forwarded it to you again.

The majority of the spam accounts came from 3 domains, omitted, although there were two accounts that weren’t from those domains (omitted)

Also, not sure if it would be worth your time and effort, but maybe Pinterest could issue a subpoena to AWS for the information of the accounts/human users behind that spam net, or simply work with the Spam/Abuse teams at AWS to make sure these guys get shut down. Building systems to detect and prevent spam is a never ending cat and mouse chase, but if we could shut down some of the human elements, that might stir a greater effect?

The original discovery happened in the window of about 30 minutes to an hour before I posted this tweet:

http://twitter.com/jontsai/status/333090050231373825

The Elastic IP address of the C&C during that window:

ec2-omitted.compute-omitted.amazonaws.com

Thanks,

jontsai

Jon’s response to my response:

Thanks for passing along the email addresses. It’s always useful to get a known bad set of accounts to verify our classifiers are working as intended.

I was planning to write a blog post pointing out some of the problems in your original post. However, if you are going to correct things that might be good enough.

Regarding the specific claims you made in the original post: 1) We salt and encrypt all the passwords in our systems; 2) Our admin servers are protected in a number of ways in addition to those you suggest in your second point, 3) Regarding your third point we use more than just passwords to restrict access to our admin applications.

To everyone who made comments (constructive and otherwise) on the original post and on Hacker News, thanks. To the critics and the snide commenters, thanks for making me laugh at my own mistakes. I’m interested in learning about what others would have done differently if they were in my situation. Should I have spent more time collecting data about the spam C&C? What kinds of data?

What kinds of action can be taken against spammers like these? Is it worth having lawyers issue cease & decist letters and takedowns to the owners of those domains, knowing they can just as easily pop up in more places?

I certainly don’t have the time and resources (and motivation) to proactively fight against spammers, but I was hoping that Pinterest and AWS might. Also, this was a good reminder (similar to things I’ve heard in a security talk a few years ago by Billy Rios) that all the spammers and malicious hackers out there are just regular programmers like you and me. They aren’t necessarily super geniuses, they use pretty much the same languages and tools we do, and as all humans do, they make mistakes (like letting anyone with the IP address access the C&C).

It would also be interesting to see Amazon improve the way they are currently giving out Elastic IPs. AWS probably shouldn’t reassign recently released Elastic IP addresses and delay recycling for as long as possible, like how telecom companies don’t reassign discarded phone numbers until 3-6 months afterward. They could use an LRU list for assigning Elastic IPs out of regions–unless Elastic IPs really are that scarce a resource in high demand that even an LRU implementation would cause an Elastic IP to effectively get reassigned right away?

AWS EC2 Security Vulnerability and Pinterest Hacked

2013-05-11T00:00:00+00:00

Update 2013-05-12: Pinterest wasn’t really hacked. See follow up to this blog post.

–

Well, almost hacked. This is rather embarassing (for Pinterest, and maybe AWS?), in that I was able to access what seemed to be their admin page. Furthermore, I discovered through this interface that it seems they do not store passwords encrypted or salted. If I’m not mistaken, I saw usernames, emails, and passwords in plaintext. Let me describe what happened.

I was migrating a Nagios monitoring server, which involved decommissioning the existing EC2 instance and releasing the Elastic IP. Because I didn’t delete my Route 53 DNS entry, and left the Nagios URL open in my browser, I soon noticed that some other content had taken the place in my browser window.

Basically, the apparent Pinterest admin server had requested an Elastic IP from the useast region, and had associated itself to the IP address that I had just released.

A few lessons can be learned from this:

Pinterest should encrypt and salt their passwords, shame on them for not doing so.
Pinterest should put their admin server so that it is only accessible from behind a firewall. Ever heard of EC2 security groups? I set up all my servers to expose only certain ports to certain other security groups. Only the external-facing webapp servers have port 80 open to the world.
Pinterest should put a password on their admin site. Even a shared password internally on the HTTP server is better than having nothing at all.

A good hacking strategy would be to allocate

associate

disassociate

release Elastic IPs, and to squat on/poll that IP address via several common ports. I got lucky and found one immediately through a browser.

Want proof that I actually did this? How about a screenshot:

–Edit–

My friend tells me that I should also mention that I was able to log in with the emails/passwords leaked. I saved around 37 of them, and tested successful logins for 2 of 2 that I tried. For discretionary purposes, those credentials will not be posted here. However, I will say that those accounts I saw in the admin interface seem to be fake hacker accounts, with the emails being mostly from the same domains.

I have contacted Pinterest through whatever means I could find, via a Twitter @Pinterest msg and #pinterest hash tag, and also emailing engineering@pinterest.com.

–Edit #2–

Discussion on Hacker News

Sticky Footer Redux

2013-02-23T00:00:00+00:00

I call myself a generalist, but feel that I’m more adept at backend development than web frontend, so I’m always amazed by the simplicity and cleverness of really cool HTML/CSS/JS techniques out there.

Responsive Sticky Footer by Timothy Long [demo] [code]

This was very informative. I have yet to use this solution (maybe when I need a dynamically-sized footer), but it seems to be very promising–not to mention that the solution is very clever.

CSSSitckyFooter [code]

I’ve been using this exact technique for several (dozens) websites, and it’s a tried-and-true method.

Another CSS Sticky Footer by Ryan Fait [code]

I haven’t used this one, but it’s neat to see the code in action and how few lines it requires.

On a separate note, I recently started using LESS, a dynamic stylesheet language which allows you to write less CSS by using variables and functions. I can’t emphasize enough that it’s a HUGE timesaver. It’s a must-have for any project.

Domino's ShopRunner Flub

2013-02-06T00:00:00+00:00

On Sunday, February 3, 2013 (Super Bowl Sunday), Domino’s Pizza disabled ShopRunner free delivery on their site. This screenshot was taken from the Domino’s Order Page on Tuesday, February 5, 2013, when they added it back in.

Coincidence? Sneaky Domino’s. I didn’t think to take a screenshot at the time, though. Too bad!

Let’s see what sharing to Twitter yields…

How to Clone a Car

2013-01-24T00:00:00+00:00

Today, I was looking up biking directions to my new office. I wanted to take VTA Express from Fremont to Sunnyvale, and then bike the rest of the way.

So, I looked up biking directions on Google Maps, and, to have a better idea of the actual road conditions, looked at street view. That’s when I noticed something interesting–all the cars on the road look the same!

Am I seeing multiples?

The Google Streetview car was slightly ahead and to the right of the car in the picture driving down Java Drive, and therefore, that same car was pictured in every frame of map tiles along that stretch of road.

What appears to be an optical illusion is… an optical illusion created by stitching together several separate frames together.

Dropbox on NTFS Mount in Ubuntu

2012-11-07T00:00:00+00:00

After I switched to Ubuntu as my primary OS (dual-boot alongside Windows XP), I had a problem syncing some Dropbox files to an NTFS mount.

While some files synced without much trouble, the Dropbox icon would consistently be spinning and several files and folders would not sync.

I got around to searching for the issue today “dropbox ntfs ubuntu” and found this helpful blog post that answered my problems.

tl;dr; – the fix was to simply add uid=1000 to the /etc/fstab entry for the NTFS mount, because Dropbox, running as your user, tries to change permissions on the file (owned by root in the absence of the uid=1000 option).

Don’t have Dropbox yet? Why not use my referral link to sign up to get a free bonus and start off with 2.25GB? Dropbox is one of the best cloud-based file-sync programs for the average user, allowing access from Windows, Linux, Mac OSX, iOS, and Android!

Apache on OSX 10.8 Mountain Lion

2012-11-07T00:00:00+00:00

I upgraded to Mountain Lion (10.8.x) from Lion (10.7.x) a few months ago, and only discovered today that Apache wasn’t working–going to http://localhost showed the default “It works!” page.

A quick search for “apache mountain lion” found this helpful guide: AMP Guide for Mac OSX 10.8.

tl;dr; for those who know what they are doing:

There is no longer a Web Sharing section in System Preferences > Sharing.
Use sudo httpd start|stop|restart to control the Apache process
Main configuration file: /etc/apache2/httpd.conf
Default DocumentRoot is /Library/WebServer/Documents
Enable PHP by uncommenting the PHP module include in httpd.conf
User configuration file: create one at etc/apache2/users/YOURUSERNAME.conf

Sample user conf contents:

<Directory "/Users/YOURUSERNAME/Sites/">
Options Indexes MultiViews FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all
</Directory>

Cool Video of Strange Chinese Song

2012-10-05T00:00:00+00:00

Here’s a cool video.

The song, named Tan Te, is sung by Gong LinNa. It is considered both modern and traditional Chinese music.

And in case you’re wondering–no, the words don’t make any sense.

Ubuntu'd out

2012-10-04T00:00:00+00:00

Last night, I made the best decision of my computing life, ever.

Old Setup:

Two desktop machines under my desk, two 24” LCD monitors, controlled by one keyboard and mouse via Synergy
Main computer was Windows XP 32-bit Core2 Quad 8200 4GB RAM (2.8GB usable after OS and video card reserved)
- Primary uses were general browsing, word processing, gaming (Starcraft 2, WoW, Diablo 3), organizing and uploading photos (Picasa + Flickr), IE-only testing, some coding
Secondary computer was Ubuntu 12.04 64-bit Core2 Quad 6600 6GB RAM
- Ran 4 Ubuntu VMs in VirtualBox for various development and ghetto dynamic DNS hosting during the days before I rented my current VPS from Linode
MacBook Air that I would use when working outside of my home office or lounging in the living room

Transition to New Setup:

Transferred the VMs onto my main computer–turns out that I really only need 2 of the 4.
- The problem I had before was that the Core2 Quad 8200 doesn’t have Virtualization Technology enabled, so it can’t emulate 64-bit VMs even if the host machine is 64-bit, so I had to run it on the Core2 Quad 6600. Now, I only need 2 32-bit VMs that will run fine on the C2Q 8200.
Decommissioned the Ubuntu 12.04 64-bit, pulled out 2x2GB and stuck it in the main computer for 8GB RAM total
Install Ubuntu 12.04 LTS 64-bit on the main computer, can still dual-boot into Windows if I want to
Connect MacBook Air with USB hub and video dongle to secondary monitor

Thoughts So Far:

It feels good. It feels REALLY good.

Since I’ve been using Ubuntu for over 6 years now, it feels really comfortable. In the past, I had run Ubuntu exclusively on several machines, including netbooks like Dell Inspiron Mini 9 and Samsung Series 5 Chromebook.

All Pros, No Cons So Far:

Lots of good FOSS replacements for Windows-only software, like VALO-CD replacing a bunch of stuff and Frogr replacing Flickr
I already feel a productivity increase since the terminal is right there
Removed temptation to waste time playing games.

Brogrammer

2012-10-04T00:00:00+00:00

See original blog post from Twilio Con: https://www.twilio.com/blog/2011/09/twilio-presents-the-definitive-brogramming-primer.html

Code Intro

2012-04-03T00:00:00+00:00

I’m newer to open-sourcing projects of my own on GitHub, so it’ll take me some time to “wrap” things properly and provide some documentation and examples so that these projects are more useful to others. I think sharing is viral, and I want to thank the open-source community for making it what it is today. I love what Jade Dominguez has done with Jekyll Bootstrap. Through him, I also learned about Derek Siver’s post on the benefits of sharing.

So, check out the codes!

jonathan tsai

Building Mission Control for My AI Workforce: Introducing OpenClaw Command Center

The Backstory

What I’m Running Right Now

See It In Action

The Vision: Bring the Work to Where Humans Are

What I Built

Real-Time Visibility

Topic Tracking (Cerebro)

Scheduled Tasks (Cron Jobs)

Privacy Controls

The Technical Details

Zero Dependencies, Instant Startup

Security-First

Real-Time Updates

The Philosophy: Use AI to Use AI

Advanced Job Scheduling (What’s Already Working)

Intelligent Quota Management

LLM Routing: Right Model for the Job

What’s Next

Multi-Agent Orchestration

Voice Harness

Try It Yourself

The Bigger Picture

SSH Commit Signing Part 2: Automation and Multi-Machine Setup

The Challenge

The Solution: Centralized Configuration

Key Components

1. Dynamic Key Selection

2. The allowed_signers File

3. Automated Generation Script

4. Makefile for Easy Management

5. Automated GitHub Key Upload

6. Git Aliases for Viewing Signatures

7. Bash Installation Helper

Complete Setup Workflow

Initial Setup (One Time)

Per-Machine Setup

Benefits

Lessons Learned

1. Mac Compatibility

2. Makefile Dependencies

3. Error Handling in Scripts

4. GitHub CLI Permissions

5. Verification is Separate from Signing

Verification

Public Resources

Conclusion

Signing Git commits using SSH instead of GPG

Embrace the power of Regex

Use Fully Qualified datetime in Python

Get Is the Worst Function Prefix Ever

tl;dr;

Rationale

Alternatives

build_

calculate_

extract_

fetch_

look_up_

format_ / transform_

Conclusion

Feedback

14 Rules for Being a Godly Employee

Use Python iSort to Automagically Organize Imports within your Favorite Editor

tl;dr;

Quick Setup

Drudgerous Line-by-Line Instructions (or my setup)

Thank You!

What, How, and Why?

Debugging Address already in use errors

Debugging like a Boss with Slack

Behold, the Power of “Slack Debugging”

Kill Netskope Client on Mac

TIL: pbcopy

When Slack links don't work, Restart Google Chrome

Is there an engineering talent shortage in Silicon Valley?

Communicating with Engineers: How to File Good JIRA Tickets

Purpose

Common Pitfalls to Avoid

`build_`

`calculate_`

`extract_`

`fetch_`

`look_up_`

`format_` / `transform_`