pl patterns

How to Convert a Local Branch in Git to a Remote SVN Branch

Thu, 05 Nov 2009 18:24:00 -0500

This is a little more obscure git-svn recipe I derived today. Thought I would share it.

Basically, I was prototyping a new feature on a local branch in git. Later, I decided I wanted to push that local branch into an svn branch so that my svn comrades could see it. So I started with a local branch following svn trunk with purely fast-forward commits. And the following did the trick.

Create the svn branch.

svn cp /trunk /branches/final-remote-name
git fetch        # Sync w/ remote svn repo.
git branch -r    # This should list final-remote-name.

Make a new local branch with your feature that follows the new svn branch.

git checkout local-feature-branch   # This is the local branch where your prototype feature is committed.
git checkout -b final-local-name    # This makes the local branch derived from your local feature branch that will eventually become the branch that tracks the remote svn branch.
git svn info     # This will say you are following trunk.
git rebase remotes/final-remote-name

Verify.

git log         # All of your feature commits should be on top, followed by the commit that created the svn branch, followed by trunk commits.
git svn info    # This should now say you are following the svn branch, final-remote-name.

If everything looks good, push all your feature commits to the remote svn branch.

git svn dcommit

You can optionally delete the local feature branch that is following trunk, so you don’t use it by accident.

git branch -D local-feature-branch

Basically all it amounts to is a rebase, with some svn trickery.

So again, all together now…

svn cp /trunk /branches/final-remote-name
git fetch
git checkout local-feature-branch
git checkout -b final-local-name
git rebase remotes/final-remote-name
git log         # Check your commits.
git svn info    # Check you are following the svn branch.
git svn dcommit
git branch -D local-feature-branch    # Delete old local branch.

I hope someone finds this useful.

Blub by Convention (In Defense of Arc)

Tue, 03 Nov 2009 08:25:00 -0500

The entire Arc language is just a few Scheme macros. …Or so the criticism says.

I can’t use Common Lisp. mapcar, lambda, destructuring-bind, etc. These are things I use all the time, yet they have some of the longest, most unnecessary names.

Scheme is the Java of Lisps. define, begin, list-ref, etc. It was seemingly designed to be as descriptive as possible, and in doing so, also became as verbose as possible.

Yes, it’s true that I can redefine all those things. But at that point I’ve created Arc. So if you see the pain of using mapcar and list-ref, then in a nutshell, you see the need for Arc. Period.

In the same way that Perl has functional programming features, but the community and conventions don’t use it that way, which means the libraries aren’t designed for functional code, hindering your use of it, so too are Common Lisp and Scheme hindering. The conventions hinder the use of its full potential.

Lisp is potential. Untapped potential. Arc is an attempt at tapping it.

The blub paradox implies that once you truly get the features of Lisp — first-class continuations, condition-signals and restarts, macros, serialization of code to syntactically valid code — all the imitation features like function objects, exceptions, Ruby’s catch/throw, the C-preprocessor, and serialization to XML are… disappointing at best. Atrociously unusable at worst. This is why Lisp is more powerful than non-Lisps.

But Arc is better than other Lisps because Lisp by definition is flexible. And flexibility is a double-edged sword. You can define anything to be anything — both in a good way and a bad way. Arc was specifically designed to pull out common patterns and be as concise as possible. If it’s true that Arc is just a bunch of macros, then these macros that I would have had to define myself in Common Lisp or Scheme are part of the core language or conventions of Arc. So instead of only me knowing about those macros, all Arc programmers know about them.

Code that does not follow a convention is more difficult to integrate with code that does, so much so, that the fact that libraries exist using a convention is a huge incentive to use the same convention. Additionally, if I write code that does not follow the convention, people who maintain my code inevitably re-write it in their own image. It happens slowly over time perhaps, but inevitably. Every one of those names (mapcar, lambda, list-ref), and dozens more in the core language and standard libraries, is a kludge. But if I redefined them, the first thing someone else touching my code would do is look at it, say “wtf!”, and delete my redefinitions and change everything back to the canonical use. In Arc, on the other hand, instead of my code degrading to the conventions of Common Lisp over time as I move on and others take over my code, it holds the high caliber of the conventions of Arc.

However, if you actually succeed at changing all those kludge names in Common Lisp, you will have improved the language by an order of magnitude. And when you improve something by an order of magnitude, you haven’t made something better, you’ve made something new. (Hence, Arc is genuinely new.)

Basically, non-Lisps are blub compared to Lisp because they limit what you can abstract. However, once you have a dialect of Lisp, there are so few visible barriers to abstraction. Instead, non-Arc Lisps are blub compared to Arc because the conventions adopted by the community — and lack of a ability to change those conventions — create barriers to flexibility, removing one of the very features that makes Lisp so great.

Put another way, non-Arc Lisps are blub by convention. The macros that make Arc Arc are a big deal because they are an attempt to change the convention.

What I am getting at is:

Many languages could be better than they are if they used different conventions.
It’s damn-near impossible to go against the conventions.
Conventions are difficult to change, more of a way of thinking of the community.
It’s worthwhile to push good conventions into the core language/libraries, and if you do that, you’re essentially making something new.

This has little to do with Arc, except that Arc is an example where someone (Paul Graham) tried to do #4 because of 1 through 3.

Photo by rachell.
Thanks to Kyle Burton for helping develop the idea for this post.

Alternative to XML Validation

Sun, 18 Oct 2009 15:44:00 -0400

The desire to validate the structure of a document has come up a number of times recently. Say you have a public service where one of its operations takes in a complicated tree-structured document. You want a way to:

use a programming-language-agnostic data format for your document;
formally specify the expected structure of the doc;
validate that input to calls to your service actually follow the structure;
and hopefully, allow clients to do this validation themselves, w/o hitting your service.

A well-known language-agnostic way of doing all this is with XML as your document format, an XML schema as your specification of the expected structure, and XML validation as the publicly known algorithm for validating.

Having done some type-theory, I would much rather say that my data format is a data structure in a programming language, the spec of the structure is a type declaration, and validation is type-checking.

So I’ve wondered, is there a solution to my problem 1 through 4 — without using XML?

A few well-known programming-language-agnostic data-formats are: s-expressions, JSON, YAML. I have found Kwalify for YAML, but not much else.

This is a shame because people who believe they might ever need 2 through 4, choose XML for 1 (pl-agnostic data format) simply because they don’t know of any alternatives. This sucks because if all you are doing is 1, then you’ve made the worst choice for the job.

XML sucks to read as a human b/c there is so much extra noise. It gets to the point where the majority of your effort spent reading XML is filtering. It also sucks to use in programming b/c the API is terrible. XML doesn’t map perfectly to the data-structures in most languages, even Java, its biggest fan.

So again, if all you’re using it for is 1 — a programming-language-agnostic data format — XML is one of the worst choices. Yet, the fact that XML dominates in 2 through 4, it remains the default.

The unspoken truth about managing geeks

Thu, 10 Sep 2009 10:59:00 -0400

The unspoken truth about managing geeks:

Articulates the mindset of many developers.

It’s not about being right for the sake of being right but being right for the sake of saving a lot of time, effort, money and credibility. IT is a team sport, so being right or wrong impacts other members of the group in non-trivial ways.

Truly insightful.

IT pros would prefer to make a good decision than to get credit for it. What will make them seek credit is the danger that a member of the group or management who is dangerous to the process might receive the credit for the work instead.

Thank you for stating this. It’s the truth.

Unlike in many industries, the fight in most IT groups is in how to get things done, not how to avoid work. IT pros will self-organize, disrupt and subvert in the name of accomplishing work.

… in fact, getting things done faster is the reason I started programming in the first place.

We can all learn from articles like this.

Sending an IM via Gtalk in Clojure

Sat, 22 Aug 2009 23:39:00 -0400

Email has its place, but when it comes to short notifications — of the status of a job, for example — IMs work great.

If you think of your software as your co-worker — someone you work with who helps you do your job — all kinds of functionality just make sense. Would you rather your co-worker emailed you or IMed you? Then the software should do that.

This code snippet sends an IM using Smack 3.1.0, a Jabber client library (Apache 2.0 license).
Download the Smack jar.
Browse Smack source.

Note: you must invite the username to chat and confirm in Gmail before messages will get sent.

user> (def conf (org.jivesoftware.smack.ConnectionConfiguration. "talk.google.com" 5222 "gmail.com"))
#'user/conf
user> (def con (org.jivesoftware.smack.XMPPConnection. conf))
#'user/con
user> (.connect con)
nil
user> (org.jivesoftware.smack.SASLAuthentication/supportSASLMechanism "PLAIN" 0)
nil
user> (.login con "my.username@gmail.com" "mypassword" "resource")
nil
user> (.isAuthenticated con)  ;; check that we are authenticated
true
user> (def chat (.createChat (.getChatManager con) "send.to.username@gmail.com" nil))  ;; this nil is the listener -- put something here to receive IMs!
#'user/chat
user> ;; Invite to chat in Gmail -- sending messages won't work w/o doing this!
nil
user> (.sendMessage chat "it works!")
nil
user> (.disconnect con)
nil

Update: I created a project that uses this along with log4j, letting you pull a logger out of nowhere and send out status IMs.

If you would be ...

Mon, 10 Aug 2009 00:56:48 -0400

Here’s something I’ve noticed recently: if you would be Xed, be Xable.

A few concrete examples…

If you would be taught, be teachable.
If you would be understood, be understandable.
If your software would be used, make it be usable.

And last but not least, the quote of Ovid that inspired it all: If you would be loved, be loveable.

Git Config

Thu, 09 Jul 2009 14:43:00 -0400

Here’s the .gitconfig file I’ve been using. The aliases let you type “git ci” and execute “git commit -v” for example. And color is essential. Going back to svn is like going back to black & white TV.

[user]
        email = you@blah.com
        name = Your Name
[color]
        ui = auto
        diff = auto
        status = auto
        interactive = auto
[alias]
        st = status
        au = add -u
        pt = add --patch
        ci = commit -v
        cia = commit -av
        br = branch
        co = checkout
        df = diff
        dfc = diff --cached
        dfw = diff --color-words
        lg = log -p
        spull = !git-svn fetch && git-svn rebase --local
        spush = !git-svn dcommit

Setting Up Clojure

Mon, 22 Jun 2009 11:37:00 -0400

Notes I made back in April for setting up Clojure on my MacBook. Kyle is planning a workshop on this soon.

# get Clojure and clojure-contrib library
cd ~/projects
svn checkout http://clojure.googlecode.com/svn/trunk/ clojure
svn checkout http://clojure-contrib.googlecode.com/svn/trunk/ clojure-contrib

# build Clojure
cd clojure
ant

# make a clojure executable on your path
echo 'java -cp ~/projects/clojure/clojure.jar clojure.lang.Repl' > ~/local/bin/clojure
chmod a+x ~/local/bin/clojure

# setup env
cd ~/projects
git clone git://github.com/mdelaurentis/env.git
cd env
git submodule update --init
cd
ln -s projects/ src
echo '(load-file (expand-file-name "~/projects/env/emacs/dot-emacs.el"))' >> .emacs

# slime and swank
cd projects/
git clone git://github.com/jochu/swank-clojure.git
git clone git://git.boinkor.net/slime.git

Knowledge Representation

Sat, 16 May 2009 14:50:00 -0400

When I write a program, what I’m essentially doing is writing down my best understanding of how to solve a problem or accomplish a task. My best understanding.

For the sake of this discussion, let’s say the problem I’m trying to solve is how to best peel an orange for eating.

An interesting thing happens, though, when I spend hours and hours talking about the problem, thinking about it, drawing diagrams of it on the whiteboard, relating it to other problems I’m familiar with, and coding it: I begin to understand the solution better. But not only that, I begin to understand the problem and its entire domain better.

With this deeper understanding of the domain, comes more precise terminology. For example, in my study of orange peeling, I discover that when there is a breach in the skin, the juice squirts out when I put pressure on it. But when the skin is perfectly intact, the juice doesn’t squirt out; it kind of oozes out from all over. So I might call these spots on the orange “squirt points” and “ooze points” from now on. But as soon as someone new to the domain of orange peeling hears the phrase “squirt point”, that person probably won’t know what I mean.

To me, the same statement using the term “squirt point” is much clearer than the one that says “the points on the orange where juice tends to squirt out when pressure is applied to the skin, usually implying there is a breach in the skin”, even though they technically mean the same thing. Someone unfamiliar with the term, on the other hand, will probably think the exact opposite.

Yes, the term “squirt point” is a reference to this longer, more descriptive definition, and it can just about always be replaced by its definition without changing the meaning of a sentence. But the fact that I’m using the term has more information. It means that when I say “squirt point” over here, and “squirt point” over there, I’m referring to the same exact thing. What if over here I use the full definition inline, but over there, I use the full definition except I say “when pressure is applied to the orange” instead of “when pressure is applied to the skin”. Is that the same thing? Is it a mistake? Did the person who wrote it (whether that be another physical person, or me years ago) actually mean to write “skin”? On top of that, even if the full definition is used exactly alike in both places, it takes significantly more work for the human mind’s limited pattern matching ability to recognize that similarity, compared with the short term.

We have to be careful though. It’s not as simple as making a rule that says: everyone should always use the term.

What if there are thousands of terms specific to this domain? Does everyone seriously have to learn them all before writing a single thing about orange peeling? What if there is a similar concept that overlaps 90%? Should a new term be coined? Or should the original term be modified? Say I want to talk about the points on the orange that juice squirts out when pressure is applied after the skin is already peeled. It’s essentially the same as “squirt points”, where the orange’s skin is not present. If I know the term “squirt point”, then I will be tempted to use that term, slightly modified. But if I overload that term, essentially creating a new definition for it, I’ve just changed the meaning of every other thing that refers to it, whether that be pieces of writing, learning material, documentation, code references, project plans, issues, or even other people’s mental references to the term.

In the worst case, I could have just broken something, making what made sense before no longer make sense. Regardless, making the definition of a term more general implicitly makes everything that refers to it more complex, because in the more general version of “squirt point”, the fact that pressure is being apply to the skin is no longer contained within the term. So when relevant (which itself may not always be clear), things referring to “squirt point” now must also say whether the skin is present or not.

For those that haven’t already been doing this, I encourage you to imagine what happens with this discussion when you replace the word “term” with “function”, because this problem constantly arises when more than one person are involved in a software project. Because developers are literally writing down their understanding of the solution to the problem in code, the resulting software can only be as good as the developers’ understanding of the problem domain.

This means that if you are a developer, you should do everything in your power to understand everything about your problem domain. Not just the specific task at hand, but the big picture where it all fits in. Not just how to fulfill the requirements given to you, but what requirements were probably missed completely because the people writing the requirements have less understanding of certain domains related to software.

But this also means that if you are running a company, and you are not a developer, you should do everything in your power to get the developers to understand everything about your problem domain — because the software that they produce, can only be as good as their understanding. If you simply give them requirements, they will do their best to interpret those requirements in the context of everything they understand (just like anyone else). That isn’t enough. There are 53.2 million ways to skin a cat. You won’t get the way that you want unless in addition to saying what you want, you also say why you want it. This is essential for a developer. Because every developer is a designer, whether they like it or not. They are constantly making decisions that affect the business as a whole. If developers don’t understand why end-users want a feature, for example, it’s likely they will misjudge the importance of the feature and make an unacceptable trade-off in the design process.

Good Design = Balance

Sun, 12 Apr 2009 20:00:00 -0400

Good design is all about balance.

Aesthetics makes your product attractive and approachable. It makes people want to try it.
Usability makes your product pleasant to use to get a job done. It makes people want to continue to use it.
Economy makes your product practical. It makes people willing to pay for, and thus, support you making it.

Without any one of these things, your product will struggle. But a lot of the time, increasing one can only be done by trading another.

For example, say you were designing a door. Yes, a door in a building that people walk through. You could make it aesthetically pleasing by making it symmetrical and blending into the wall around it. But the symmetry hides which side the hinges are on, so people end up trying to push or pull the wrong side. Blending into the wall may make people not even be able to find the door at all. These both decrease usability. So you decide to install weight sensors under the floor to detect when someone approaches, and open the door automatically. Now you need motors to open and close the door and wiring to link it to the sensor, increasing cost, in materials, the design of the mechanism itself, and the installation. So you could use a motion sensor above the door instead, reducing installation costs. But now the sensor is visible, decreasing the aesthetic appeal. You could use a smaller invisible sensor, but that costs more. So say you figure out the right combination of sensors and motors. You just decreased usability of the door. What happens when the power goes out? Suddenly your door doesn’t do the one thing it was designed to do! Open. You could fall back to a manual push or pull, but this makes it inconsistent for users. In a building with many windows, it may not even be clear to people trying to walk through your door whether there is power. You can fix this usability problem with some kind of battery backup. This not only increases the initial cost, but maintenance cost too. And where do you put that battery? Unless the exact building you’re putting this door in is conveniently set up with big access panels, you’ve done a number on the usability for the maintenence staff. The motors also introduce noise pollution and the possibility of opening a door and hitting someone. Unless, of course, we spend more on sensors and the intricacy of the mechanism. And on and on and on and on…

Design is not purely opinion. Good design is about finding the right balance. Like anything, this is something you can get better at with practice.

Learnability is a combination of other aspects that work together to help turn new users into experts. Experts spend less time to do the same tasks, have the ability to help others, and can fix things when they go wrong. Some things that increase learnability are a simple approachable appearance, context-sensitive tips, and giving users the ability to explore when they choose. Big, oversized things feel simpler and more approachable, even though they’re really the same. Game aspects built into your most boring app can make people want to use it. And the more people use it, the more of an expert they become.

Providing help on how to fix a problem when something goes wrong, without the user having to know how the thing works, is essential; because without this, you’re basically forcing users to be dependent on experts when something goes wrong. And something will go wrong. Many times it’s not the user’s fault. How many times has your boss told you to do one thing, only to tell you to do the exact opposite the next day? Every action should be undoable.

Emacs is a good example of a program that is usable and cheap, but not pretty or learnable. Some developers swear by it and use nothing else. But to do anything besides basic typing and saving, you literally have to stop and learn it. Even accessing the help requires a significant amount of knowledge. Ditto for Linux and git.

Eclipse, on the other hand, is more aesthetically pleasing and approachable. Instead of inventing all of its own keyboard shortcuts that have to be learned, it uses the convention of other editors. It’s usable, but usable for a different audience than Emacs.

Everything is a trade-off. Balance is everything.

Extensibility is another important aspect that allows users to add on to or extend your product in ways that you either don’t have the resources to do yourself or are very specialized for a small group of users. Extensibility is important because no product exists in a vacuum. Users will use your product alongside all the other things they already use now. Having your product play nicely with all those other apps is something that users often can’t live without. Empower your users, even if it’s just the experts. Allow the forces of nature to take your product where you alone couldn’t have imagined.

The only reason Emacs still exists is because of its extensibility. In Eclipse, the barrier to extension is higher. True, both editors require you to learn a programming language, but in addition, Eclipse requires you to learn and use another language — the XML configuration language that allows the editor to actually use the extension. In Emacs, it’s just two keystrokes away. And since Emacs’s interface is only text, it’s easier for programmers (the ones making the extensions) to fit their extensions in with the rest of the editor’s conventions. One-off scripts can be born, which have the ability to evolve over time. But with Eclipse, you have to stop, decide you’re going to make an Eclipse plug-in, and learn how to do that.

When it comes to extension, Emacs and Eclipse are exactly the opposite of what they are for learnability of their regular interface. The choices each made for one aspect, traded something, and thus determined the other aspect.

Balance is everything.1

1. On a side note, the “good design = balance” equation goes the other way too. When something is perfectly balanced, there’s almost no question that it was designed. This applies to products, but also the Universe. It’s funny how people argue over intelligent design vs. evolution as if they were mutually exclusive. If you think like a designer, there is as much evidence for one as there is for the other. One is the cause, and the other is the mechanism.

Important Points from Stumbling on Happiness

Wed, 01 Apr 2009 02:54:00 -0400

These are my notes from reading Stumbling on Happiness by Daniel Gilbert. I’ve recounted mostly just the conclusions, so if you’ve never read the book, I urge you to do that before reading this; it’s kind of a spoiler. Plus, I find that hearing conclusions before I hear the evidence or argument sometimes increases my defensiveness, thus decreasing my ability to extract value from something.

Regardless, it’s a great book, and although there are a few things I disagree with, it is an amazingly insightful look at the human condition and the mind. If you have any interest in hacking your own mind (or life), knowing its shortcomings is essential, and you’ll find this book invaluable. Daniel Gilbert also has a great sense of humor; I found myself laughing out loud while reading this more than any other book I’ve ever read.

Notes and Excerpts

263 pages + 35 pages of references and an index

Chapter 1 talks about the imagination and how we are different from apes.

Chapter 2 p31-59 first says what he means by happiness and breaks down different kinds of happiness (emotional, moral, and judgmental). It then discusses issues of subjectivity and making inferences about happiness based on the reports of happiness by others.

Chapter 3 p60-79 Gilbert basically motivates his assumption that people can and do accurately report how happy they feel in the present, and uses this as a basis for the rest of the book.

Realism

p94 “the psychologist Jean Piaget noticed that the young child often fails to distinguish between her perception of an object and the object’s actual properties, hence she tends to believe that things really are as they appear to be—and that others must therefore see them as she does.”

p97 “we do not outgrow realism so much as we learn to outfox it, and that even as adults our perceptions are characterized by an initial moment of realism.” … “we automatically assume that our subjective experience of a thing is a faithful representation of the thing’s properties. Only later—if we have the time, energy, and ability—do we rapidly repudiate that assumption and consider the possibility that the real world may not actually be as it appears to us.” … “We believe what we see, and then we unbelieve it when we have to.”

p109 “when ordinary people want to know whether two things are causally related, they routinely search for, attend to, consider, and remember information about what did happen and fail to search for, attend to, consider, and remember information about what did not.

Presentism
The Future Is Now

p125 “when brains plug holes in their conceptualizations of yesterday and tomorrow, they tend to use material called today.” p126 “Memory uses the filling-in trick, but imagination is the filling-in trick, and if the present lightly colors our remembered pasts, it thoroughly infuses our imagined futures. More simply said, most of us have a tough time imagining a tomorrow that is terribly different from today, and we find it particularly difficult to imagine that we will ever think, want, or feel differently than we do now.”

Comparing: p148 “we tend to imagine how we would feel if those things happened now, and then we make some allowance for the fact that now and later are not exactly the same thing.” p149 “The problem with this method of making judgments is that start points have a profound impact on ending points.”

Comparing: p158 “(a) value is determined by the comparison of one thing with another; (b) there is more than one kind of comparison we can make in any given instance; and (c) we may value something more highly when we make one kind of comparison than when we make a different kind of comparison.” … “because we make comparisons without even thinking about them (‘Man, that coffee has gotten expensive!’ or ‘I’m not paying double to see this concert [after losing my ticket]’), we rarely consider the fact that the comparisons we are making now may not be the ones we will be making later.” p159 “the person you might have been making love with is largely irrelevant when you are in the middle of making love with someone else” even though beforehand, you imagine that it will matter a great deal.

Disambiguating Experience: p165 People claim that seemingly terrible experiences were the best experiences of their life. p170 “… we respond to the meanings of such stimuli and not to the stimuli themselves.” p175 Like a Necker cube which is fundamentally ambiguous, the subconscious part of the mind actively finds ways of appreciating our experience which is also inherently ambiguous.

Finding Facts: p180 After buying a Honda instead of a Toyota, you find yourself pouring over all the Honda ads which talk up Hondas, reinforcing your decision. p181 You spend a lot of time finding friends who have similar beliefs as you.

Challenging Facts: p187 “We ask whether facts allow us to believe our favored conclusions and whether they compel us to believe our disfavored conclusions.”

Psychological Immune System

Looking Forward Vs. Looking Backward: p195 “Why didn’t volunteers realize that they would have more success blaming a judge than a jury? Because when volunteers were asked to predict their emotional reactions to rejection, they imagined its sharp sting. Period. They did not go on to imagine how their brains might try to relieve that sting.”

Intensity Trigger: Like a security system that must balance between responding to threats and not being overkill, p199 “… it is sometimes more difficult to achieve a positive view of a bad experience than of a very bad experience.”

Inescapability Trigger: p202 “We just can’t make the best of a fate until it is inescapably, inevitably, and irrevocably ours.” p204 “We have no trouble anticipating the advantages that freedom may provide, but we seem blind to the joys it can undermine.”

Improving

p218 “Are there more four-letter words in the English language that begin with k (k-1’s) or that have k as their third letter (k-3’s)? … You probably answered this question by briefly checking your memory (‘Hmmm, there’s kite, kilt, kale…’), and because you found it easier to recall k-1’s than k-3’s, you assumed there must be more of the former than the latter. This would normally be a very fine deduction … [but] our mental dictionaries are organized more or less alphabetically, like Webster’s itself, hence we can’t easily ‘look up’ a word in our memories by any letter except the first one.” p219 “The k-word puzzle works because we naturally (but incorrectly) assume that things that come easily to mind are things we have frequently encountered.” “the frequency with which we’ve had an experience is not the only determinant of the ease with which we remember it.” “infrequent or unusual experiences are often among the most memorable”

p232 “Our memory for emotional episodes is overly influenced by unusual instances, closing moments, and theories about how we must have felt way back then, all of which gravely compromise our ability to learn from our own experience.”

Reporting Live from Tomorrow

p233 “Yes, of course the future is hard to see. But we’re all heading that way anyhow, and as difficult as it may be to envision, we have to make some decisions about which futures to aim for and which to avoid.”

p236-243 Beliefs, like genes, spread when having the belief leads to more spreading of the belief. (Daniel Gilbert calls an instance of these a super-replicator.) This is why so many people believe that having children is a joy when, in actual studies, people’s happiness is inversely correlated with having children at home.

p239 “Some of our cultural wisdom about happiness looks suspiciously like a super-replicating false belief. Consider money. … Economists and psychologists have spent decades studying the relation between wealth and happiness, and they have generally concluded that wealth increases human happiness when it lifts people out of abject poverty and into the middle class but that it does little to increase happiness thereafter.” … “Economists explain that wealth has ‘declining marginal utility,’ which is a fancy way of saying that it hurts to be hungry, cold, sick, tired, and scared, but once you’ve bought your way out of these burdens, the rest of your money is an increasingly useless pile of paper.”

p241 “the production of wealth does not necessarily make individuals happy, but it does serve the needs of an economy, which serves the needs of a stable society, which serves as a network for the propagation of delusional beliefs about happiness and wealth. Economies thrive when individuals strive, but because individuals will only strive for their own happiness, it is essential that they mistakenly believe that producing and consuming are routes to personal well-being.”

p247-250 “Imagination has three shortcomings….” 1) “its tendency to fill in and leave out without telling us”, 2) “its tendency to project the present onto the future” (like shopping on an empty stomach vs. full stomach) and 3) “its failure to recognize that things will look differently once they happen—in particular, that bad things will look a whole lot better”.

p251 “the best way to predict our feelings tomorrow is to see how others are feeling today.”

Rejecting the Solution

p251 “If I offered to pay for your dinner at a restaurant if you could accurately predict how much you were going to enjoy it, would you want to see the restaurant’s menu or some randomly selected diner’s review? If you are like most people, you would prefer to see the menu, and if you are like most people, you would end up buying your own dinner. Why?”

“Because if you are like most people, then like most people, you don’t know you’re like most people.”

p252 “the average person doesn’t see herself as average.”

“This tendency to think of ourselves as better than others … may be an instance of a more general tendency to think of ourselves as different from others—often for better but sometimes for worse. When people are asked about generosity, they claim to perform a greater number of generous acts than others do; but when they are asked about selfishness, they claim to perform a greater number of selfish acts than others do.”

p253 “Even when we do precisely what others do, we tend to think that we’re doing it for unique reasons.”

“What makes us think we’re so darned special?” 1) “even if we aren’t special, the way we know ourselves is.” 2) “we enjoy thinking of ourselves as special. Most of us want to fit in well with our peers, but we don’t want to fit in too well.” Showing up to a party and finding someone else wearing the same shirt “temporarily diminishes your sense of individuality”. 3) “we tend to overestimate everyone’s uniqueness—that is, we tend to think of people as more different from one another than they actually are.” “Individual similarities are vast, but we don’t care much about them because they don’t help us do what we are here on earth to do, namely, distinguish Jack from Jill and Jill from Jennifer.”

p256 “surrogation is a cheap and effective way to predict one’s future emotions, but because we don’t realize just how similar we all are, we reject this reliable method and rely instead on our imaginations, as flawed and fallible as they may be.” p257 “Alas, we think of ourselves as unique entities—minds unlike any others—and thus we often reject the lessons that the emotional experience of others has to teach us.”

Afterword

p259 “Most of us make at lease three important decisions in our lives: where to live, what to do, and with whom to do it. … Making these decisions is such a natural part of adulthood that it is easy to forget that we are among the first human beings to make them. For most of recorded history, people lived where they were born, did what their parents had done, and associated with those who were doing the same. … But the agricultural, industrial, and technological revolutions changed all that, and the resulting explosion of personal liberty has created a bewildering array of options, alternatives, choices, and decisions that our ancestors never faced. For the very first time, our happiness is in our hands.”

p262 “There are many things other than the size of a person’s bank account that influence how much utility they derive from the next dollar. For instance, people often value things more after they own them than before, they often value things more when they are imminent than distant, they are often hurt more by small losses than by large ones, they often imagine that the pain of losing something is greater than the pleasure of getting it, and so on—and on and on and on.”

p262 “The sad fact is that converting wealth to utility—that is predicting how we will feel from knowledge of what we will get—isn’t very much like converting meters to yards or German to Japanese.”

p263 “There is no simple formula for finding happiness. But if our great big brains do not allow us to go surefootedly into our futures, they at least allow us to understand what makes us stumble.”

Absolute Subjective Truth

Sun, 15 Mar 2009 12:44:00 -0400

When someone I know has a baby…

I say: Congratulations! :-D
I think: I’m sorry… :-/

Yes, I was lying to all of you. The funny thing is that people with kids reading this are probably thinking “I’m sorry…” towards me. After all, they get so much joy and life fulfillment from their kids.

Truth depends on your perspective. Depending on what fundamental beliefs you have, you draw different conclusions from the very same facts. You can literally change what must be true by changing your most-basic beliefs. Like the axioms and inference rules of your life, your most-basic beliefs define the system that you live in.

Truth is also subjective because you can’t even have a perspective without a subject and object, viewer and view. And something, even the truth, can not be known without a knower, so there’s no such thing as truth independent of all perspectives. Quantum mechanics is starting to see this. The computer screen isn’t there (or is in multiple places) until you look at it.

That doesn’t mean that truth is relative, though. There could be an Absolute Truth — knowable only from an Absolute perspective. Douglas Hofstadter in Gödel, Escher, Bach talks about how the mind is software running on the brain (the mind’s hardware). But a mind could run on different hardware such as ants in an ant colony. Throughout the ages, people have written about cosmic consciousness. My take is that this experience is the mind transcending the brain — jumping off its hardware! — and running on the combined hardware of all humanity.

The Absolute perspective, though, is one in which the viewer and view become one, what people call the experience of no-self. As in Taoism, opposites define each other. It’s so difficult to imagine death because the mind thinks in opposites (or binary). The mind can’t remove one side (itself, the viewer) without removing the other (the world, the view).

As for having kids… in the grand scheme of things, it probably doesn’t matter. But now I’m starting to sound like Dr. Manhattan, so I’ll stop right there.

What Web Frameworks Are Missing

Wed, 11 Mar 2009 11:40:00 -0400

The single biggest feature common web frameworks like Rails are missing out of the box is credit card payments.

Right now, no one bothers implementing credit card payments until a site has become “production”. Single-person projects or small startups don’t bother making this at first b/c they would much rather spend their time running up hill, prototyping new, hard, or interesting features. Credit card payments, on the other hand, are a simple yet difficult-to-get-right chunk of software. So unless it’s already done, no startup wants to waste its time doing it.

This is bad in the long-run for everyone, including end-users, b/c the default is to offer (often innovative and really great) services for free, which means only already-funded teams or people with lots of extra time can even get to the point of making something.

If this feature existed, were packaged up neatly, and were as dead-simple as Rails scaffolding, small projects could default to whatever payment scheme actually made sense — like pay-per-use, donations, trial periods, first 5 free, etc. — making it possible to fund the really great products out there, instead of forcing them to rely on ads (which require high traffic) or outside funding (which often ruins the product, takes ownership away from the creator, and has a much higher barrier to entry).

In a real physical store, people expect to pay. If the person on the other side of the counter gives me something for free (w/o buying anything at all), I say “Are you serious??” and walk away bewildered. But online, I almost expect it. This is bad! Because now everyone expects it, and people trying to make great things can’t focus on actually doing that b/c they are too busy trying to figure out how to make it great and free.

Oftentimes, there are free things out there online that I like so much that I would like to pay for, but there’s simply no easy way to do it. And I’m not alone. I use tipjoy and have donated to things that have a PayPal donation setup like Auditorium, but I don’t see these as adequate.

The best thing out there that I’ve seen is RailsKits. But as far as I understand, they aren’t packaged properly so that I can add them to my site whenever I please like a plugin or gem.

Is there something out there I just don’t know about? What is it? If not, please, someone make this! (And yes, if it’s good, I might even pay for it.)

Macro Patterns

Sat, 31 Jan 2009 17:42:00 -0500

I was talking about Lisp macros with Kyle and we decided to write down the patterns of different macros we’ve seen and what they allow you to do, with the intent of teaching others.

Interesting things you can do with macros:

resource management (e.g. with-open-file)
behavior injection à la AOP (e.g. memoization, debug printing)
alternate control-flow constructs
alternate looping constructs
alternate binding constructs (e.g. destructuring-bind, cut)
lazy evaluation
DSLs
code generation

Things that macros actually give you that you can’t do otherwise:

programmatic binding
non-hygienic binding
controlled evaluation
reifying the (calling) source code

If you’ve seen other types of macros or know of good examples of the above, please comment. I will try to update this list as we come up with more.

(Non-)Total Functional Programming

Thu, 25 Dec 2008 06:47:11 -0500

I’ve been getting back into language design again. (I haven’t written anything about it since August!)

Not too long ago I was introduced to Total Functional Programming. The paper is very interesting, and at first, the concept sounds great. Increase the ease of proofs of properties of your code at the expense of Turing-completeness. Who needs Turing-completeness anyway.

But after reading some comments on Lambda the Ultimate, something occurred to me. Maybe removing infinite looping, in an attempt to remove a certain kind of partial function — those that never return — is acceptable. But removing partial functions completely is not acceptable (to me).

To implement division, for example, you’d have to modify the definition of it, or change the type to make it impossible to type-check with a zero divisor. But in general, the compiler can’t prove that a runtime value is non-zero, so this forces the programmer to explicitly handle the case all the time. Maybe that’s desirable in some places, but is it always? I’m not sure.

What I take away from this is that, (1) partial functions won’t go away completely and (2) we have to accept some kind of undefined exceptional value.

Both could be eliminated by making your types über-complicated. But I think that’s the wrong way to go. An intuitive type for division is int -> int -> int or perhaps Num n => n -> n -> n. But not something artificial just to make the function total like Num n, NonZero nz => n -> nz -> n.

The Abstraction Machine

Mon, 17 Nov 2008 16:00:00 -0500

The Lord of the Rings is not fantasy; it’s abstraction. J. R. R. Tolkien strived for applicability, the idea that symbols do not stand for just one thing, like say, Middle-earth standing for England in World War II, but that the reader can instead apply the story to many many different things.

Abstraction and application are antitheses. One builds, the other breaks down. The mind is an abstraction/application machine.

Abstraction and Beliefs

Let’s play a game. I will list a few things, and you guess what the pattern is. Try this: tomato, fire hydrant, stop sign. What’s the pattern? … Red. What you just did was abstraction. Here’s another one: sky, blue jay, sapphire. Pattern? Blue. What you just did again was abstraction. Here’s another one: canary, owl, eagle. No, the pattern is not yellow. What you just did was application — application of a pattern that was abstracted away, probably without you even being aware of it. The fact that the patterns of my first two lists were colors made you abstract away that fact, and you tried to apply that to the third list. Seeing canary made your mind jump to yellow. (If you’re not used to introspecting your own mind, you may have missed that, because it all happens within fractions of a second.) But the pattern was actually birds.

This is why magicians will repeat a hand motion, like sliding a coin up to the edge of a table to pick it up, many times without doing anything special. The repeated observation of nothing special happening makes your mind abstract away the pattern. So the next time when the magician misdirects your attention and pulls a fast one, dropping the coin on his lap, your mind simply applies the pattern to what it wasn’t paying much attention to, and you believe you saw the same ordinary hand motion with the same ordinary result of the magician picking up the coin.

Specific experiences are constantly bombarding the mind. These experiences are remembered, and patterns are abstracted away from similar or related experiences. Then, when a new situation arises, the mind applies the patterns it has to the current specifics, attempting to predict the outcome based on those patterns.

These patterns are hypotheses of how the world works. When our predictions come true, these hypotheses are reinforced, and they become beliefs.

Many beliefs become so engrained that we latch onto them as truth. This is where the Problem arises. Instead of our experiences determining our beliefs, our beliefs determine our experiences.

The Belief Feedback Loop

For the skeptical readers, I’m not going to bother with the claim you’ve probably all heard and quickly discounted: that what you believe to be reality is. Instead, look at how beliefs are formed in yourself.

When I introspect my own mind, I see that there is a feedback loop. My beliefs cause my thoughts, and my thoughts cause my actions. My actions, along with more-external factors, determine my experiences. Experiences, in turn, lead to insights of patterns and eventually determine my beliefs.

[TODO: Feedback Loop diagram — beliefs → thoughts → actions → experiences → beliefs]

If this loop were observed unaltered, I might get a good idea of what the Universe is like. My beliefs would align with reality. But my mind filters experiences based on its beliefs, and only sees what it wants to see. A common mistake is to only see what aligns with patterns I already know, and ignore what doesn’t.

For example, if I believed that stopping at traffic lights while driving were very painful because they increased wear and tear on my car, the amount of time it takes me to get places, and pollution, I would be likely to notice every time that I got caught at a red light. I would also be more likely to pay attention to times when I had to stop at an intersection when there were no other cars going the other way, making the stop even more pointless, and thus, more painful. Since the pain of stopping at traffic lights is always on my mind while driving, I’m likely to express my opinion about the terrible design of traffic lights to other people. And other people will inevitably disagree with me. To prove my point, I will likely consciously look out for even more examples of my theory, my belief, swaying my bias even further. If someone gave evidence to the contrary, by say, counting the number of red lights vs. green lights on a trip, my ego would immediately look for ways to discount the evidence. It was only sampled from this one trip, after all.

The truth is, I don’t know if most traffic lights are well designed. But having such a belief leads to bias — a preprocessing phase in the mind filtering all experiences — that is a slippery slope, and it reduces your ability to see the truth.

When our predictions do not come true and we actually realize it, we try to form different hypotheses. If this happens often enough, we feel confused. How often do you find yourself feeling confused?

If you said “not often”, this could be a bad sign. It could be a sign that your mind is doing such a great job at filtering, that you can’t see that the earth is revolving around the sun and not the other way around, even after experiencing the evidence time and time again.

The Mind and Programming

It’s no coincidence that abstraction and application are the foundation of the λ calculus, rather, it is a purposeful coinciding of events.

λ x. e — abstraction
e1 e2 — application

Using memory of past experiences, our mind abstracts away patterns to predict future events. The mind is doing this all the time on a micro-scale, like when you contract your bicep, your mind predicts that your arm will bend. But it’s the same process we use to predict larger events. When the light bulb goes on in your mind, you’re feeling the integration of many specific examples into a generalized pattern that explains them all. To write these patterns down, it makes sense to have a language for expressing generalized patterns — abstractions — and their use in specific cases — application.1

Functional programming anyone?

Now, I know what you’re thinking. Silly author. You’re falling into the bias trap you just talked about since you are obviously already in love with functional programming. Ah, that may be true. But when I introspect my own mind, abstraction and application is the closest thing in the physical world that I can relate to what I see.

Can you find a better analogy? In other words, a pattern that more closely aligns with truth? Maybe the abstraction/application model is blub.

1. In the same way that map is a very useful, general function or abstraction that almost every single functional programmer uses or applies, The Lord of the Rings is a very general story that almost anyone who knows it can apply it to their life. The story is a function, an abstraction, that can be applied to many different situations.

Techies Vs. The Business

Mon, 20 Oct 2008 09:01:00 -0400

I love making software; I’m a techie geek. I’ve also always been interested in running and owning a business — an aspiring businessman. Right now, though, I’m employed. And I’ve learned there is always an infuriating tension between the technical people and the business people — infuriating to everyone on both sides of the divide.

If you’ve ever worked for a company that didn’t consist of only one of the two mindsets — technical or business — all throughout, from worker bee to head honcho, then you know exactly what I’m talking about. Like-minded people gather at happy hours or private meetings spewing all their complaints and frustrations caused by the other side. The business insists we keep adding new features without refactoring. Arrrgh! … or … Software dev just won’t hire enough people. Why?! Others nod their heads in agreement. It’s eventually followed by a few seconds of silence before someone finally says, “well, enough venting…” Everyone pushes on, now with slightly more frustration, slightly more dissatisfaction, and slightly more apathy than before.

This is just the start. These kinds of disagreements go on for years without ever getting resolved. They inevitably increase the amount of politics that come into play when decisions get made, which is detrimental to the organization as a whole. What starts out as a decision about whether or not to start a new project becomes a power struggle between egos. Not to mention the underlying dissatisfaction and eventual misery of the people working there who feel powerless to change anything for the better.

The divide is so bad that two people on either side can’t see eye-to-eye on anything. They even refer to the other group as a separate entity — “the business” or “the board” or “software dev” — as opposed to “Brett and Diane”, the actual team members. Not only is this dehumanizing, promoting the us-against-them mentality, but it doesn’t hold the actual people responsible accountable for their actions. It’s the fault of “the people upstairs” or “the dark side of the building”. Instead of working towards fixing the problem, it blames entire departments, and most of them are just along for the ride when a manager makes a bad decision.

Even though both technical and business people alike are (usually) striving for the same goal, they are divided on their belief of method that will achieve that goal. And this, over anything else, is the reason the divide exists.

This deep-rooted difference in belief is how to achieve passive income.

The Passive Income Mecca

Active income is income based on a per hour or per year rate. Anyone employed to work at an hourly wage or on salary is making active income. Once that person stops working, he or she stops getting paid. Period. In geek terms, I = wh, where I is income, w is income per hour, and h is the number of hours worked. When h is 0, so is income.

Passive income, on the other hand, is income that does not depend on the amount of time you work. A writer like Stephen King can write a book once (a fixed cost), sell it, and collect royalties for each copy sold. The more copies sold, the more income he gets (variable income). Although the amount of income might depend on how good the book is, it does not depend on how much time Stephen King spent writing the book — how much time he actually spent working, h. That’s passive income.

In the world of money-making, everyone is striving for sustainable passive income. If you suddenly become ill or get into an accident that makes you unable to work, and your only income is active, you’re screwed. How will you pay for dinner? How will you buy those 2-hundred-dollar designer-jeans for your teenager? Not to mention the health-care bills accumulating as a result.

The solution: passive income. With passive income, it doesn’t matter if you’re lying in bed, surfing in Hawaii, site-seeing in Europe, or getting a lap-dance in Tokyo. The money keeps rolling in, thanks to the hard work of tens, hundreds, or even thousands of others dependent on active income. (Those suckers!)

Now, the divide between technical-minded and business-minded people is in what they believe is the best way to obtain passive income.

I’m Right

For technical people, they know computers. They know software. Given the right resources, they can make a computer do anything — anywhere, anytime. Their deep-rooted belief is that passive income can be achieved by writing software once (a fixed cost) and distributing it to millions who each pay a fee (variable income).

For business people, they know cashflow. They know the symbiotic relationship between employees and business owners. And in this day and age, there will always be people looking for jobs. Given the right resources, they can employ people to do anything — anywhere, anytime. Their deep-rooted belief is that passive income can be achieved by creating a repeatable business process once (a fixed cost) and teaching it to thousands who each execute the process (bringing in variable income).

What technical-minded and business-minded people are doing is essentially the same. What differs is their belief in what scales.

The differing belief is due to their experiences. Technical people believe they can make computers do anything because there has never been a time when this wasn’t the case for them. Every computer mindlessly executes instructions, so if you can obtain a computer and specify those instructions, it will do whatever you want. If it was a problem with software outside of a technical person’s control, there was always the possibility of re-writing the software in-house, so those situations don’t count.

Business people, on the other hand, believe they can make people do anything because there has never been a time when this wasn’t the case for them. People need money. So if you pay them, the market dictates a fair trade, and they will do whatever you want. If there was ever a problem with an employee not being able to do a job, there was always the possibility of replacing that employee with another more skilled, more teachable person, so those situations don’t count.

Each group grew up with its own viewpoint and believes its method is superior. So neither group bothers to learn the other side’s perspective. The dispute rages on.

You’re Wrong

Likewise, due to differing experiences, each group believes the other group is mistaken.

In the technical sphere, the amount of overhead (variable cost) of hiring a person — not to mention laying off or firing — is seen as huge compared to the overhead of distributing another copy of software or adding another server. And this alone is reason enough.

But on top of that, 80% of all problems are human error. Someone typed in the wrong command, someone clicked the wrong button, or someone loaded the wrong file. The software was not to blame; a person told the computer to do the wrong thing. And so, technical people — as a direct result of their vast amount of experiences — believe that people do not scale. It doesn’t matter how detailed your process is — someone, at some time, is going to forget a step. They’re going to forget to make a backup copy. They’re going to forget to update 1 of the 13 places to change the current date. They’re human; it’s expected. Computers, on the other hand, if there’s one thing they’re good at, it’s being consistent. Developers can test and debug software ahead of time to make sure it follows the process before going live in a production environment. It doesn’t matter how long a person has been doing something, though. To not expect a person to make mistakes is naive. So people do not scale.

However, in the business realm, not only is development of technology extremely costly, it is extremely costly to maintain. Although the technology may improve productivity in certain use cases, there are inevitably “edge cases” that the technology doesn’t handle. I use quotes because the edge cases actually happen all the time when you’re doing it as often as a big business does (kind of like those race-conditions in multi-threaded software). So you need people to know the business process anyway to handle the cases that the software doesn’t handle. By introducing technology, you’re increasing the number of things a person has to learn before he or she can be productive, thus increasing costs.1

Business people — as a direct result of their vast amount of experiences — believe that software does not scale. Technology systems inevitably become so complex trying to handle all the edge cases that fewer and fewer people are able to effectively maintain them, and that maintenance gets slower and slower as time goes on, experts leave the company, etc., etc. So any software written in-house is a liability. It’s on the business to shell out resources regularly just to stay afloat. On top of all that, there’s no such thing as bug-free software. To expect to write software once and forget about it is naive. So software does not scale.

People-Farming

Employees are effectively a business’s server farm.

A job is shipped out to the server farm, the job owner queries the farm for status from time to time, and the results get shipped back to the job owner. The farm is compensated for the amount of CPU time the job took (via wages), but any and all profits resulting from the computation (the fruit of the actual labor) are owned by the job owner.

That’s right. Business people treat employees the same way technical people treat hardware. The process is written in English or German or Chinese instead of C or Java or Lisp so that the number of people who can interpret/execute the process is maximized. This is also the reason why big businesses will always prefer languages designed for the masses like Java over languages designed for smart people like Lisp. Java coders are more interchangeable and come significantly closer to the theoretical limit of a code-monkey — basically a slave that outputs working software.

Ironically, each group wishes for more of what the other wishes they had less of in terms of smart people who think outside the box versus those who put their head down and follow a process as they are told.

In the technology sphere and with software especially, it’s completely up to your imagination — practically anything that can be coded goes. Non-technical people tend to have a huge misconception that programming is purely methodical and mechanical. This couldn’t be further from the truth. Programming takes a huge amount of creativity, arguably more creativity than what people normally think of when they think of art. Ask any programmer with a passion for the field and he or she will tell you that programming is an art. As a result, the best of the best — giants like Google and Microsoft — are notorious for hiring based on puzzles and brainteasers. People who don’t think outside the box, don’t come up with creative solutions, don’t think abstractly — not only do they make sub-par programmers, but they pull the rest of the team down drastically.

It’s like if the Pittsburgh Steelers have an amazing defense, but they pickup a lousy cornerback. Not only will this position suffer due to his lower skill, but the rest of the team will perform worse because they will constantly have to worry about covering what the cornerback can’t and chasing down the offense that breaks through the cornerback’s defense. The rest of the team tires more quickly, and their attention is spread thin over a larger area, all to make up for the below-average draft. In software, it’s the same. Lower skilled developers introduce bugs that have to be fixed by higher-skilled team members. Time is now spent fixing bugs that could have been spent making new bells and whistles. But it’s actually much much worse than that.

Even poorly-written code that does not have bugs, per sé, can hurt the team in the long-run. Highly skilled developers make development look easy because of foresight. They know what problems will eventually arise, and so, they write code that is easier to extend, easier to change, easier to test, and easier to fix. On the other hand, less skilled developers don’t have the foresight to do such things, and their code becomes a detriment to the entire team, invisibly incurring technical debt. It won’t show, perhaps, until years down the road after they have long quit the company, as other developers must spend valuable time learning and adapting the code. And this is why technical people would rather not hire anyone than hire someone below their standard — in direct opposition to the business-minded idea of scaling.

In the business realm, they see things differently. Although you may need extremely bright people to do R&D, only the richest companies have enough capital to spend on R&D. Your typical company has little need for brilliant (and expensive) people. On top of this, people who are hired to execute the business process are, more times than not, better for the organization when they don’t think too far outside the box. If they do, they end up by-passing the well-thought-out process which has been proven over the years to work (like code that’s been tested), and instead, fly by the seat of their pants, taking unnecessary risk. If the process doesn’t handle something, it should be brought to the attention of upper-management who can modify the process to fit the business needs. Bringing these issues to the attention of management should also be part of the process, so nothing ever strays outside the process.

If one brilliant employee is doing something outside the process, then when he or she gets hit by a bus (or more likely, leaves the company for a startup), the company will fall apart because no one else knows what the brilliant employee was doing, let alone how he or she was doing it. Single points of failure are an absolute no-no when it comes to scaling. And let’s face it, creativity is hard. Not everyone can be creative. An organization is much more likely to survive having a simple, repeatable process that can be taught to anyone, not just your Sergey Brins. If something happens to any one person, the organization as a whole can move on, the same way a biological organism losing a few skin-cells scraping its knee on the sidewalk can just get right up again.

It’s like if one server started ignoring the code a programmer wrote because it discovered a “better” way of doing things. If that way were proven to be correct, like compiler optimizations, fine. But as far as we’re concerned, it hasn’t been proven at all and it can’t be copied to other servers, so we’re better off without the dependency.

So yes, there is actually a reason for those TPS reports, and I wonder what Office Space would look like if it were told from the business people’s perspective.

Basically what this comes down to is that all technical people are creating new processes all the time, usually in the form of software, so those people have to be smart and creative. When technical people gather requirements, what they’re doing is market research, aka marketing. They then architect a process that computers will eventually execute.

On the other hand, only business people in upper-management ever create processes, and they have an entire department to do marketing for them. Those at the “bottom” of the hierarchy are only supposed to execute processes like a machine.

An Organization’s Combined Hierarchy

Put another way, software developers in the technical world are doing the same thing that upper-management is in the business world. The only difference is what executes the processes they create — either computers or people. Middle-managers, on the other hand, are only hired to execute processes, and yet, they’re telling software developers who create processes what to do.

It’s as if technical people and business people are at the top of two completely different hierarchies that get connected by middle-managers, allowing each group to abstract away part of a complete sustainable organization. But instead of the two hierarchies being equal peers, the technical hierarchy gets pushed to the bottom because of its lack of understanding of (social) networking and cashflow. They get lumped in with the crops, when they’re really running their own farm.

So when it comes to scaling, which group is really right?

Resolve

Actually, both groups are right. And at the same time, both groups are wrong.

Both perspectives have really good points that the other group simply doesn’t get, even after recurring time and time again in the industry. At the same time, both groups have huge misconceptions about how fool-proof their method to passive income is.

Technology like software can be extremely costly to maintain. Thus, making technology easy to maintain — i.e. making variable costs as close to zero as possible — is one of the most important jobs of developers creating it. People will always be needed to maintain it. On the flip side, business people need to place more emphasis on hiring and keeping highly-skilled technical people who reduce variable costs, as they are essentially as vital as upper-management. Servers are simply different kinds of employees that don’t ask for benefits.

In the end, I think this is yet another example of one of the re-occurring themes of the Universe: neither one extreme nor its polar opposite is the answer. Instead, the best solution is some kind of hybrid that takes advantage of strengths of both sides.

Just look at the greats. Bill Gates started out on the technical side — like me, an interpreter-writer — but eventually leveraged the business side to build one of the most successful software companies in the world, not to mention one of the richest. Those who can master both sides will rise above the rest.

As for easing the tension between technical and business types… If I had one piece of advice for technical people to work better with business people, I would say, talk to business people in terms they can relate to. They don’t care about how the NullPointerException is triggering deadlocks in parallel at runtime between cache-misses. They care about resources spent or saved, features being shipped, and how fast they can be shipped. At a company whose output isn’t technology, when it comes down to it, you are just a service-provider to them. Instead of thinking of your software as the end product, think of the end product as your service of writing the software.

When you order pizza from Papa John’s and ask them how long it’s going to take, you don’t care about problems they have cooking pizzas in parallel because one oven is broken. You want to know when your pizza will be ready because you’re hungry god damn it! And when it comes to software, it’s no different. If you’re striving to create the best pizza man has ever seen, either keep it to yourself or work at a place that shares your dream.2

Last but not least, read some books on social interaction and sales, and practice what you learn. Non-technical people don’t understand what you do, so it’s up to you to convince them of your value.

If I had one piece of advice for business people to work better with technical people, I would say, ask technical people to explain things in terms you can relate to. They think that once the NullPointerException triggering the deadlocks in parallel at runtime between cache-misses is fixed, everything will be fine. To them, what that mouthful is is obvious, and they can’t imagine it not being obvious. They also think that once all the details are specified, the big picture is immediately clear, and you can’t blame them considering that’s how computers work. They simply don’t know what you don’t know.

Swallow your pride and ask them to explain things in terms you actually care about, and the sensible technical people gladly will. It’s not really a matter of having them come “down” to your level as “up” to the big picture.

Last but not least, learn about technical debt and accept the fact that revisiting working technology is necessary to paying it off. Building software is different than building a car, and it requires different approaches. Technical people only respect merit and ability. If you’re not interested in learning what they do, at least learn how to recognize value when you see it and not fight against them in trying to approach it.

Questions

Not everyone is cut out to run a business. It’s a lot of work, and not everyone wants to do that. The majority of people are the worker-bee type who enjoy having a stable income with a regular 9 to 5 schedule to get up and go to the office. From the business perspective, business owners are doing these people a huge favor by providing this environment for them. In return, employees are effectively the business’s server farm.

As I described before, businesses treat people as hardware. They ship employees a job, the employees do the job, and the results get shipped back. Java coders, as opposed to Lisp coders, are more desirable because they are more expendable and come significantly closer to the theoretical limit of a code-monkey — a slave that outputs software. All profit and assets like intellectual property, regardless of who did the work, by law, are owned by the business owner. The employees themselves sign all the necessary papers when they take the job.

This begs the question: is employment slavery?

I think the answer is not so clear. It’s like asking, is it wrong to take something from someone if that person wants to give it to you? Usually we call that a gift. Employees seek out jobs and complain when there are few available; so they must want to be employed, right? Additionally, employees always have the option to quit; they’re not being held captive in any way, and they’re never physically forced to do anything.

But somehow, this doesn’t seem right. If you ask an employee why he gets up so early to commute to work during rush hour, he probably won’t say that he does it because he likes it. He will probably say: because he has to. It is hardly a “gift” to spend all day, every day, working hard, spending time away from family and friends just so that a business owner, who he probably never even met, can get rich. It’s more like exploitation with no hope of ever breaking free. 9 to 5 till 65, as they say.

And as for not being held captive, most employees will tell you how they’ve felt trapped at one point in their career or another. Others will tell you how they’ve feared losing their job. A few describe work life as torture, being confined to a cube where if they fall asleep, they lose their job and thus their home. Even though they’re not being physically forced into labor, it’s effectively the same through indirect blackmail. For most, they depend on the income from their job so much that their employer might as well be their master and they the slave.

It’s obvious to fresh graduates joining the workforce for the first time. But as years go by and responsibilities like supporting a family and paying a mortgage build up, increasing their dependence on their employer, any aspirations of freedom are slowly forgotten.

In a society where 90% of all people are employees who inherit debt from their kin, not assets, the entire concept of financial freedom is alien. All this — in the land of the free.

I’m honestly not blaming business owners for attempting to be slave drivers. In societies where slavery is acceptable — this one — it’s much more desirable to be a master than a slave. We are all at fault for allowing it at all. Employment being comparable to slavery is a much larger issue with society not educating the poor- and middle-classes about cashflow, which, by the way, will eventually lead to a greater and greater divide between the haves and have-nots, until the pressure builds up enough for revolution. You mean history repeat itself? Nah, that couldn’t happen…

Regardless of whether that happens, though, I believe exploitation is wrong. Each worker should own a piece of the fruit of his or her labor and profit from it. Arguably, having small companies like startups where every worker is a partial owner is the only right way of doing things. But this doesn’t seem to scale.

If only there were a way to accurately measure the value a worker was really contributing… Is there a way? What is it?

Investors reading this, looking for a bargain? I’m interested in funding. jtran@alumni.cmu.edu

1. Of course, if your technology is well-designed, it can actually decrease the amount of things a person has to learn before being productive. But like any good thing, good design is hard, and most businesses aren’t going to spend much on the design of internal software. It’s an unfortunate decision given that designing things to be easy to learn and use is a fixed cost, while having people learn to use them is a variable cost.

2. If that place doesn’t exist, create it! Think of the universe as being as malleable as ideas, and you will go far.

Hacking FriendFeed with Metafeeds

Tue, 23 Sep 2008 09:04:00 -0400

I did a search in FriendFeed and noticed that at the bottom it had a feed icon. This gave me the idea of creating an imaginary friend out of the search results so that I could get everything about “meta”, for example, in my FriendFeed home page. This could be really useful; kind of like Google Alerts but streamed into my FriendFeed.

I tried it and it works great! Trying to learn about CouchDB or some underground tool that has little documentation? No problem. Just add the search for it as an imaginary friend, and anyone on FriendFeed that posts anything about it will show up in your feed. It’s like a customized news feed of the entire web streaming directly from the hackers using and creating stuff, even when you don’t know who they are.

This gave me another thought though… What if I added “http://friendfeed.com/search?q=meta&public=1” or even “http://friendfeed.com/public” as one of my own feeds. Every single FriendFeed post would theoretically show up in my feed, which should also show up in http://friendfeed.com/public, creating a self-referential loop of importing posts of imported posts of imported posts, etc., etc.

I tried it, and it really works. This could potentially lead to an exponential blowup of posts, and possibly even a DoS vulnerability, depending on how FriendFeed is implemented. I tried to contact FriendFeed about this, but I got no response. As smart as they are, I trust they’ve taken this into account. If not, I’m sure they will as more people discover it.

Code Maintenance Tool Wish-List

Sun, 17 Aug 2008 02:35:00 -0400

At my day-job, I am forced to maintain code that I did not write, that I am not familiar with, and whose authors no longer work at the company. Did you say “documentation”? Hah! No, my friends… it is just me and the editor.

I am a designer. A prototyping programmer. I do my best work when given vague requirements, lots of freedom, and a blank editor. And in my experience, unlike conventional wisdom, people do not know what they want… That’s a whole post in itself, so I’ll leave it at that.

Not only do I dislike maintenance, I am actually not that great at being a maintenance programmer whose job is to modify or add to an existing (usually unfamiliar) code-base. However, as much as I dislike it, I’m reluctant to say that I’ve actually learned a lot from the experience. I hate it so much that my mind is constantly looking for ways to make it easier. Here is what I wish I had.

Specifying and seeing scope more precisely. For example, I should be able to say these 2 functions can call this, or just the class Foo and this class’s testing module, or this is a project entry-point. Actually, what would be better is to have intended scope, and then have your tool tell you what actually refers to it. (To this end, it would help if dynamic lookup were restrictable. Perhaps whether a function can be looked up dynamically should be a metadata tag on the function.) Additionally, I don’t want to have to search every time I want to see the references to a code object. I want to be able to see this in real-time as I move my cursor, click, or even mouseover something in my editor. The way it is now, finding references feels expensive because you have to search and it takes a long time. As a result, I only do it when I feel like I really have to, which is not that often. But if it were cheap, I would think of it differently. Perhaps I would use it to learn how things are related; I’m really not sure. But I know I would use it differently.

Comments and function names should be indistinguishable from log statements, or at least toggle-able. My editor should allow me to toggle logging of function arguments whenever the function is called. I should never have to type log("in myFunction x=" + x + ", y=" + y), yet somehow I always do. It should be as simple as log(x, y), and this could be solved with something as simple as macros that output the arguments that were evaluated, in addition to the values of those arguments.

Inline, implicit functions instead of code “paragraphs” which are merely lines of code separated by blank lines. These implicit functions would show which variables the block is actually touching as parameters to the function (which can be inferred). These new kinds of function-paragraphs could easily be pulled out into actual functions, with the function application automatically generated in that spot of the code.

Viewing code fully inlined, highlighting patterns. Long functions, as opposed to many short functions, are actually easier to read if you’re not familiar with the code. For one, it is fewer levels of indirection. But also, you can think of functions in a module as a mini domain-specific language. For domain experts, it is infinitely easier to use this terminology. However, when you’re new to the field, you can’t understand anyone because they’re all using what seems like cryptic lingo.

Before the year 2000, who in the world knew what a hanging chad was. But the few experts who coined the term must have found it useful. The first time you heard “hanging chad”, you probably asked, “What the hell is a hanging chad? Pregnant chad??” Likewise, a maintenance programmer should be able to view any function calls he pleases inlined, basically saying, “Instead of using this term, tell me what you mean in the standard terminology that I already know.” Of course, the editor won’t literally inline the code by copying and pasting; it only displays it that way. After all, code is just data. We can view the code as if it were inlined without throwing away the fact that there is a pattern of shared code. The editor could use something akin to the implicit function-paragraphs I described to keep the patterns already created by the original programmer, and at the same time, slowly teach the maintenance programmer the domain lingo.

Oftentimes when a few classes have commonality, they are refactored into an abstract base class containing the common code and several subclasses which implement or re-implement dynamically-dispatched methods. When you understand the flow of all the classes, this is a great way to factor out patterns and reuse code. However, when you’re unfamiliar with the code, it’s damn-near impossible to look at the flow to try to modify it. A method in the base class is possibly used by the derived classes, but not necessarily. Some might use it, but some might not. Others could use it but also add to it by overriding and explicitly calling the overridden method. There is no adequate tool that I know of that will show me the collapsed view of code for a derived class, to see inherited and overridden methods and all — the final result of all the abstraction and reuse. Firebug does this for inspecting CSS.

Firebug shows you the rules that cascade together to style each element. Rules are sorted in the order of precedence, and properties that have been overridden are stricken out. Each rule has a link back to the file where it came from which you can click to jump to the line.

Any web developer will tell you this is the most useful thing in the world; why not do this for methods in an OO programming language…? With a function, at least you can apply it to get its result in specific cases. With a macro, you can expand it. But with abstract classes — abstractions over classes — you’re stuck having to imagine it all yourself. Sure, you can instantiate the class and then call the methods, but you’d have to do that for each method. Depending on your language, sometimes it’s not even clear what all the methods of a given object are, let alone the source code that created them.

Currently, in order to reuse a piece of code, you have to pull it out (of context) and name it so that the 2 places you want to use it can refer to it (whether it’s a class, a function, a variable, etc.). But pulling something out and having 2 things be semantically linked are separate things. The fact that our tools do not allow us to do one without the other (when sometimes I really do only want one) is an indication of inadequate tools.

There is a general pattern of solving programming problems with an extra level of indirection. However, too much indirection creates more problems, especially when working with code you aren’t familiar with. Or put another way, when someone else works with code you wrote. It is a limitation of the human mind that we must take into consideration.

Programs are written by people, and they must also be read by people. For practical reasons, simplicity and clarity — which amount to human readability — are the first things to be sacrificed when the only benchmark for code being shipped is whether it executes correctly. Like rushing the composition of an essay, which results in prose that can be interpreted by a reader but is not necessarily well-written, code that is rushed is often similarly poorly-written.

It has been said that programs should be written primarily for people, and secondarily for computers to execute. The more I learn and the more I experience, the more I agree with this. And our tools should help this cause.

Story Lessons, Continued

Tue, 29 Jul 2008 18:42:00 -0400

I’ve written about Story before and gave it its own post. But as I learn about people, I’ve learned some new things.

First of all, most people don’t realize they’re characters in a story — their life story. They don’t step outside the system, as Hofstadter says. Alfred, for example, doesn’t realize he is a character playing a role in the movie The Dark Knight. He is a butler, and he does his butler thing. Sure, he’s so much more than that: a surrogate father to Bruce, an uncle, etc., etc. His character is deep with an entire lifetime of backstory. But he is not aware that he is merely a character in a story. If he were, he might speak to the audience, for example, as they watch the movie.

Most people are in Alfred’s position. There’s nothing wrong with that. We need people like that to keep the Story going. To be the supporting cast.

Secondly, if you start talking to people about the Story, only those who see they are characters will understand you. The majority will just think you are weird (at best).1

The way to communicate with people is to gently prod (often referred to as small talk) until you find out what they’re interested in. Then talk to them in terms of that.2 Not only will you be able to get your point across infinitely easier, but they will think very highly of you and genuinely enjoy your company. This is one of the reasons analogy is so important. If you can make an analogy that someone can relate to, you’re opening the doors of their mind.

Thirdly, your Story depends on other people. Without Sam, Frodo would have given up long before he ever even reached Mordor, let alone conquered the Ring. But if you expect to have an amazing sidekick, you have to be an amazing sidekick for someone else in their Story. It doesn’t make sustainable sense to behave otherwise.

Two people together can do more than twice one person. If you have a project, find people who are interested in the same thing, and work with them. On the days you can’t inspire yourself, inspire someone else. Be the Sam in someone else’s Story.

The Gods lift those who lift each other.

The best fiction I know of related to this is The Neverending Story by Michael Ende. It is an amazing mix of fantasy, adventure, Zen symbolism, and metaness that any geek would enjoy. The hardcover edition (well-worth the investment) is printed with two different colors corresponding to which level of the story is currently being told (i.e. the story, or the story within the story), which merge about half-way into the book.

If you’ve seen the movie adaptation of The Neverending Story made back in the ’80s and were turned off, promptly forget it and read the book! There were numerous times reading the book when the metaness got to me. For example, in the book, the main character reads this book (called “The Neverending Story”), and he gets the feeling that the book is written about himself. Being aware of my life story, I got the same feeling reading The Neverending Story in real life! — a mind-blowing realization of metaness — which you can’t get from the movie.3

It is often found in the young adult section of bookstores, and this is a slap in the face. That’s like saying Star Wars is for kids because it’s rated PG. No! It is simply that universal that its audience is everyone; “adult content” is not needed. And the same goes for The Neverending Story.

1. There are two ways to react to someone doing something you don’t understand. One way is to avoid it. You don’t see value in something, so why bother? Why waste your time when you know of other things that are valuable. The other way of reacting is to simply be curious. Wonder, what value does that person see that I don’t? If you figure it out, then, and only then, can you make a conscious judgment.

Which category do you fall under?

2. In other words, be a compiler!

3. One movie you can see some good metaness in is Adaptation. Interestingly, Robert McKee, author of Story , is a character in the movie.