Mental Hopscotch

Moving this blog

2021-03-01T08:41:00.001-08:00

I'm moving off this platform and over to 👉 https://stigmergist.github.io/thinking-whilst-typing/

Software : Tradesmen not Craftsmen

2013-08-04T06:30:00.001-07:00

I've always felt uncomfortable with the Software Craftsmanship term. It smacks of elitism, doing something for its own sake, boastfulness, idealism, disconnection from the fuller reality of the rest of the world. I have never understood why we would want code to be built by craftsmen, it makes me think of finely wrought things built in economically unjustifiable ways, invested with value beyond the material or utilitarian. Craftsmen seem to be isolated specialists honing the individual item for a high-margin market of collectors, or the hobbyist hand-building an item of legacy.

Craftsmanship feels like therapeutic self-absorption, a kind of narcissistic activity - one which seeks self-affirmation and views everything as an extension of the practitioner's will. The product of craftsmanship is an item that does not have independent value - it is cherished by the craftsman and he is proud of what he has made, it is cherished by the owner as an extension of the craftsman. Craftsmen are possessive and tied to the incarnation of their labours.

This does not sit well with my feelings about software and good software practitioners - who in my experience are more tradesmen than craftsmen.

Tradesmen operate in a way that is aimed at being extrinsically useful to the rest of the world. The objects they deal with are of utility outside of the tradesman to people who will never know them. Tradesmen operate in a hands-on manner to build or fix systems, working with what they find, self-correcting through doing, in the service of others and take satisfaction in the manifestation of concrete work - which somehow removes the need to vindicate their worth through explaining the value to others. They work in a manner which shows solidarity with other tradesmen : working the code into understandable, recognized patterns that make it easy for the next tradesman to build or fix, they often work together as peers, using their better judgement when the rules run out, preferring face to face interactions that support how they take responsibility for their own work. Tradesmen gain many of the same satisfactions of the craftsman but provide economic utility to a broader market, plus the work is meaningful to them because it is useful to others.

I think the Software Craftsmanship movement is badly named and in its naming is setting the wrong objective. It is not the software itself where the craftsmanship needs to lie but in the practitioners themselves. To be a tradesman you need to understand the craft of your work, but this does not mean you are delivering craftsmanship pieces.

To be clear, I think many of the activities undertaken by the Software Craftsmanship community are exactly what need to be done to generate an effective software-as-a-trade approach - but a good deal of renaming and refactoring needs to be done to clarify why it is of value to the general population of developers out there. The fact is software development is a trade, practiced to various degrees by a large number of people who have no interest in being elitist coding craftsman, and in fact are turned-off from what might be valuable insights for them by the very notion of "craftsmanship". Scientists, engineers, accountants, (god-forbid) management consultants and many more all routinely write software, but have no engagement with the craft of software development. As a literal comparison they are the bodgers to the Software Craftsmen cabinet makers - and it does a disservice to both camps that this separation exists.

What can be done ?

Firstly, let's just admit it: software development is manual labour undertaken by a variety of people with different aims in mind. There will always be room for everyone from craftsmen to architects to bodgers - but as with many other industries, the general economic progress begins with the recognition that there is a trade, which is not elitist and exclusive, but inclusive and aimed at adding value to society.

Secondly: embrace being a Software Tradesman

I make my livelihood from creating and fixing software. I get a kick from the use of my software by people who have never met me, nor would care about the code itself. I use other peoples' code and systems and contribute my own. I work closely with other tradesmen improving both my skills and theirs through hands-on co-operation. I have solidarity with other tradesmen. And I do it because I am skilled at it and it is useful to others.

I am proud to be a Software Tradesman, not a Craftsman.

Modelling Windfarms with SketchUp and Google Earth

2013-06-01T06:42:00.003-07:00

The area where I live in Scotland is under threat from 130 industrial wind turbines. These 125m+ pieces of machinery are being proposed by five different organizations in groups of 17-35 turbines. Each organization turns up with their presentations, rhetoric and photo-renderings of the turbines. Some do show 3D models but they are quite bland and you can't really get to play with them. But the major issue is that you can never see the impact on the landscape of all five proposals TOGETHER.

Now I think this is an unethical practice : they are all aware of the other proposals and if they had true concern/care for the community would they not group together to provide the community a full model of the combined proposals? It feels underhand to only show their own proposal in full knowledge that there are other proposals along side.

So I decided to build a model myself of the full picture, combining the information provided so far by the developers. If they won't do it, I would.

Using Google SketchUp I found and scaled a wind turbine models according to the data (thanks "Peter" whoever you are). Using the SketchUp Geo-Location feature I was able to place these models with tolerable accuracy in the landscape, according to the data provided by the developers so far. I then could export this into a format that Google Earth could render (KMZ). And: Hey Presto! A fully interactive model of all five proposals TOGETHER.

It took me an afternoon. So I now have a metric of how much the developers value giving good information to the community : an afternoon's work.

There are some huge advantages to this Google Earth model :

full interactive and can be viewed from various vantage points in comfort of own home
works on laptops, PCs and tablets
can see how different light levels effect wind turbines in landscape : in particular I never thought about how the morning/evening light will be incident on the turbines when it is already dark in the valleys. They shall be towering shafts of light above the dark earth - how is this not destroying the landscape ?
contains colour landscape based on satellite photographs (developers' ones tend to be topology only and monotone/wireframe)
can be easily shared for others to explore (see below)

I have to say I'm very impressed with the SketchUp/GoogleEarth combination. Thank you , Google.

There are a few limitations :

it's tricky to get elevation right between SketchUp and Google Earth. What appears correct in SketchUp is sometimes off in Google earth. (Unless I can't figure it out - if you know how to do it, please let me know)
need a network connection for Google Earth to work

But, hey, that's not a huge list or set of problems !

Now, if you're a developer and want to challenge the accuracy/validity of this model, please do get in touch. I am happy to work with you to get this right - in fact, I am happy to hand the model over to you so you can keep it up to date. It's not 100% correct but it's good enough to show the full picture of the combined impact.

On a "protecting the landscape" note: please do download and have a look at this model. Try going to various places and looking at the aspects (even compare StreetView with the rendering at ground level). The model gives a really good feel of how these skyscraper sized turbines are entirely inappropriate in this landscape.

If you want to look at the model:

download and install Google Earth
download this KMZ File
if it does not automatically open in Google Earth, save it and open it directly from Google Earth

Your Help Is Needed To Protect A Community

2013-03-17T03:13:00.000-07:00

I am privileged to live in a beautiful place. Its physical landscape is breathtaking, but more than that the people and the community have created a wonderful place - both to live and for tourists to visit.

But it is under threat of destruction by the building of 130 giant, industrial wind turbines. They threaten not just the physical landscape, but the people and the community, too.

It is entirely inappropriate for these wind-farms to be built in this location.

Before you dismiss this blog entry as another anti-wind-farm, NIMBY rant, let me be clear: I am neither pro- or anti-wind-farm, I am arguing for the protection of beauty. Beauty which is as much yours as it is mine. Energy is far more abundant in the world than beauty, so we must fight to preserve beauty, in landscape and community, where we find it.

The Assault On Beauty

Adapted from Yoni Alter's Shapes Of Cities

There are 5 proposals by separate companies in this area. They are all for wind-turbines with tip-heights of 115-146 metres. In total the proposals are for 130.

To give you some idea of scale, they are 1.2-1.5x times the height of Big Ben, or just under half the height of The Shard. Or for non-Brits, 1.2-1.5x times the height of the Statue of Liberty, one third the height of the Empire State Building.

130 of these in any human landscape is unacceptable. Consider London with 130 of these scattered from Hyde Park to the City. Or New York with 130 turbines of this scale from Central Park to Midtown.

So why is this acceptable where we live ?

In this landscape and community, the acceptable number is zero.

For more information see the Planning Overview

And it's not just somewhere on the hills near Straiton, its effectively a "ring of steel" around the village.

Furthermore, there are no clear plans for the power lines to take the energy away from these wind-farms to the supply grid (there is no existing infrastructure here of that scale). Amazingly, it is not a necessary part of the planning applications.

The siting of wind farms near any habitation - that's any house not just near towns or villages - is at best irresponsible if not unethical. The long-term effects of exposure to the audio and visual pollution they generate is unclear. There is much speculation and rumour out there on both sides of the argument, but have we not learnt from other technologies we have rushed to benefit from that the ignorance of the long-term effects can have catastrophic results? Radium, thalidomide, cigarettes, internal combustion engines, oil/coal fired power generation… If nothing else we should be guided by these histories and look to place an uncertain technology well away from communities and people - if anything, for wind-power, is offshore not the most responsible place ? (The irony of pro-wind-farm lobby ignoring the lessons of the oil/coal power effects would be funny if it were not so irresponsible)

There is further outrage with this particular assault on beauty: Straiton is sited on the edge of the UK's first Dark Sky Park, is on the edge of a UNESCO Biosphere Reserve and contains a number of recognized points of natural and scientific interest (inc. SSSIs, rare animals, birds, etc.). As my wife said, "not only the earth beneath our feet but the heavens above us have been recognized for their beauty by the world". Yet it is possible that this internationally recognized place could be scarred with 130 giant turbines. Not only does it make me angry, it makes me ashamed that this is possible. How is it that our local government, the Scottish government and the UK government would see this as permissible ? And that the defense of this area comes down to volunteers spending their own money and raising funds against the onslaught of well-funded developers ?

So this is one reason why we need your help.

Another reason is because this community is small and relies on tourists for much of its economy. And with 130 industrial turbines, would you stay in such a place on holiday ? There is much to be argued against in the claims of "economic benefits" in these proposals - but that is for another blog entry - but just think of the most basic fact: a small community will lose one of its primary incomes. A community which is too small by itself to hold sway in Edinburgh or London or the World - but a place where people from Edinburgh, London and the World currently come to visit, for its peace, beauty and community. A peace and beauty that will be lost, and so the community will be lost. We need your help.

How Can You Help ?

Firstly: become informed : about this particular situation, and others just like it elsewhere - maybe even on your doorstep. Think about the "positive benefits" advertising for cigarettes, and the subsequent consequences of ill-informed adoption: what might you do to to not be ill-informed?

Have a look at Save Straiton For Scotland for more information and up-to-date stories about this situation. Follow @savestraiton on twitter.
Be skeptical about the default "green is good; wind-farms are good" attitude pushed by the PR and marketing departments of many companies and political bodies. Read up on the subject, and make up your own mind

read informed, balanced journalism like The Telegraph's or The Guardian's pieces on wind-farms
think about the economics of wind-farms and take some time to investigate where the money goes - and how value in property and the loss of primary tourist based income might effect a community. Do the economic claims of wind-farm proponents really add up? How many jobs, and of what type, do they create ? Permanent or temporary ? Local, domestic or foreign ?
find out about how much energy it takes to build, transport, and install a wind-turbine (or 130!). How does this balance out against the efficiency of the power they generate and the long-term energy requirements to then dismantle and remove them after 20 years ?
what are they alternatives to siting large scale, onshore wind-farms near human habitation ? Offshore ? Hydro ? Wave ? Personal scale wind/water/solar ?

Secondly, provide support to SaveStraitonForScotland :

obviously there are costs involved in challenging the plans and applications by the deep-pocketed wind-farm companies. Please donate if you can.
if you have any relevant skills that you could provide, please contact the committee (or I can put you in touch). Particularly: planning, lobbying, legal, fund-raising, PR or awareness.
please retweet/link/blog/write/text/talk/shout about this to people you know. Highlight the assault on beauty that is happen in Straiton, and support the efforts of the group.
please contribute comment on SaveStraitonForScotland and provide links in to it from any web-properties you have (like blogs or websites)

Thirdly, take action :

please help lobby the relevant politicians and decision makers. Write to them, call them, tweet them. Especially if you are not a local - tell them what you think of these plans, tell them if it would effect your willingness to visit Straiton and South Ayrshire on vacation.
lobby your own representatives about the issues and bring pressure to bear. The people, the environment and the community are inseparable, and need to be preserved.
if you are an investor in funds, please investigate whether any of your management companies are undertaking any such proposals; and if they are and you find objection to the concept, please intervene with them and withdraw the funding. Let me be clear: it is not green-energy/wind-farms that is the issue, it is the destruction of beauty and undermining of communities which is the problem. Investing your funds in alternates like offshore or wave power is possibly more socially responsible.
come visit Straiton ! Showing your physical support and coming to see and understand what an asset this area is to the country helps our cause and supports the morale of the team. Plus your tweets/blogs about your visit can increase awareness of the fight. If you do come, please let me know so I can at least give you a wee dram in thanks !

Finally, no matter what: save energy. Switch off lights when you leave a room, don't leave items on stand-by, buy something like a Watson and monitor your power usage. By doing this, you help reduce the amount of energy we need - which ultimately reduces the need for wind-farms, power stations and other generation technologies on industrial scales.

The Value is the X-ing not the X

2013-01-28T04:07:00.000-08:00

A Twitter interaction over the weekend reminded me of some other exchanges I've had in recent years about Agile software development. A number of times I have encountered people and groups who vehemently argue that X is a problem, where X is any of Tests, Estimates, Pairs, Plans, Backlogs, etc. The rhetoric often revolves around X being imposed by management and therefore bureaucratic, or X being a waste of time and so by Lean rules should be eliminated (...let's not go down that rabbit hole, eh?) , or one of a dozen other positional, somewhat questionable arguments. When I take the bait and fall into the discussion I find my understanding of the purpose of X and their understanding of the purpose of X is vastly different. I see the interaction with other people in X-ing as valuable and the X is just the driving goal, whereas they see X as the valuable thing in itself no matter how you get there.

Upon reflection, it seems to me that the adoption of Agile has concentrated on the artifacts (X) and not the interactions between people (X-ing). It seems to have skipped the first value line of the Agile Manifesto : "Individuals and interactions over processes and tools".

In my model of Agile software development, the artifacts X are really just aide-memoires for the people involved in the interactions. Both the people and the Xs are involved in multiple interactions and move together. The Xs are not isolated, complete, self-describing artifacts - they are pointers into conversational contexts that are held within the people involved. They are extensions of group memories. Without the interactions of X-ing (or subsequent related interactions, Y-ing) they have NO VALUE.

Yes, you heard me right: Tests, Estimates, Pairs, Backlogs, etc. HAVE NO VALUE in themselves.

It is the interactions of Testing, Estimating, Pairing, Backlog-ing, etc'ing which has value in an Agile software development process. The artifacts are a way of dealing with volume and time. If you had a small amount to do in a short time, would you need the artifacts? I think not. Would you need the interactions? I think so.

This has somehow been lost during the adoption of Agile. You can see it in the arguments about the best format for story cards, or whether JIRA is good for storing tasks, or which mock testing framework is best. It's all about the artifacts, rarely the interaction.

In my more cynical moments, I blame it on the cargo-cult approach to Agile adoption. A prophet (read Agile Evangelist/Coach/Guru) joins a team, bringing the message that Agile is good; by adopting the artifacts and rituals of Agile the gods of software delivery shall grant peace, harmony and profitability to all. The team undertake the rituals of parade drilling, Saturday dances, etc. (read stand-ups, planning, etc.) and create the bamboo airfield, bamboo rifles, etc. (read estimates, test, etc.). At first things seem good as everyone is co-operating on this new venture, but somehow...somehow...over time...it doesn't work. The gods don't deliver. What went wrong ? Obviously it was either the artifacts weren't good enough or the rituals weren't observed correctly. The team members are at fault - try again. Nope, still not working. It must be the artefacts and rituals that are wrong; reject them and adopt another cult...or better: invent your own!

What cargo-cult adoption fails to install is the behaviours required for successful interaction. To me, that was the big difference with Agile. The nature of the interactions changed to real conversation about what we are doing, how we will succeed. The artifacts and practices were supportive of the conversations, but the value was the different nature of conversations.

I think one of the reasons early adopters were successful with Agile (and I think this may be true of most new technologies) is that by the very nature of trying to learn and discover this new technique and what it meant to the team in a vacuum of definitive description (i.e. no manual), the behaviour was naturally more reflective, more listening to each other, more experimental with feedback. Once, it "crosses the chasm" and prophets bring the news to the masses the nature of adoption changes, and that form of conversation and interaction has been lost.

So next time you see me rising to an artifact argument, please remind me: the value is the X-ing, not the X.

Debugger-First VM Development

2013-01-25T07:43:00.001-08:00

Virtual Machines (VMs) are a popular mechanism for language developers to implement their ideas. VM development enables a clean separation between the High-Level Language (HLL) application code and the model and services necessary to execute that code. But in my experience, many of these efforts fail to satisfy as a usable technology because they treat debugging of the application as an after thought that can be improved later.

I believe you get a better technology if you start with the Debugger.

Allow me to explain...

Back in the late '80s I discovered the power of the truly interactive debugging in Smalltalk. It was a revelation to go from the command-line/printf approach to something where you could inspect everything, even change the values on the stack or return fake values just to see what happened. With "self halt" you could leave markers in so they system would throw up the debugger where you were unsure what to do next; you could then write the code when you needed it based on the real context (true JIT development!). We even used this latter feature in production systems, allowing us to be able to diagnose and fix edge cases when they occurred (we could fix the code and/or data in place and continue). You could develop inside the operating machine of your system.

But what the Smalltalk debugger also gave me was an ability to learn and understand object-oriented programming. It was a new paradigm to me, but through the debugger I could explore and interact with the total system. It helped me understand classes, meta-classes, message sending, instances, and all the other object paraphernalia through direct interaction with the operating machine. When an error occurred, it allowed me to explore the context of failure and attempt to correct and continue the scenario - interactively.

A powerful debugger enabled me to learn a new paradigm quickly and effectively. After I "got it", a powerful debugger enabled me to develop the system rapidly and continuously learn from others.

This is a lesson I would wish people developing there own languages would appreciate. It is extremely frustrating to get poor or obscure error messages out of a system, especially when you cannot then explore the meaning within the context of the code. Moreover, "clever" uses of, say, Java VMs as a short-cut to supporting your language which give meaningless Java Exceptions which need to be interpreted in the context of the language compilation into Java byte-codes is frankly the point I will stop using a technology (which also applies to compilers that spout huge warning stacks).

If you choose to implement a VM for your language, start with thinking about the debugging experience. VMs are ideal for building rich debugging experiences as you can expose the model of the language and the VM services directly from the operating machine through an orthogonal interface to the execution model. Further, the ability to manipulate the state of the truly virtual machine makes interactive debugging far easier to implement.

Key to a successful debugger is of course the visualization of the system state. You need to sometimes think well outside the GDB or JDWP models of debugging. What exactly does a reactive functional programming model look like in your debugger ? How are event emitters and receivers represented ? These are the elements which not only help people to learn your new language/paradigm/system but ultimately help them develop and build bigger systems.

You also need to decide the capabilities of your debugger, and this is where it gets interesting. I have implemented a few personal research VMs taking this Debugger-First approach, and have implemented the following unusual (and IMHO valuable) capabilities:

Execution Rewind : the ability to run the VM backwards to a given point in time, seeing at each step how the data in the VM changed. Also made it that you could change the code and re-start the execution from that point forward (got this idea from Bruce Anderson - someone did this years ago)
Sequence Breakpoints : if this sequence of commands or changes to data occurs, halt
Interaction Breakpoint : halt when this object sends any message to this object
Return From Frame : under defined set of circumstances return these values (think interactive mock injection)
A/B Algorithm Testing : having more than one implementation of a function and keeping separate memory spaces for effected data, so you can see which is more efficient/effective and what is touched by the algorithms
Dynamic Loading of Byte-Code Implementation: the ability to introduce a new byte-code to the VM during its execution (e.g.: add a subtract byte-code to a system that never had one)
Dynamic Load of Debug Capability: the raw VM has no debug code - it is loaded on demand at run time. It can be as small and light as a single break-point or as large as a memory monitor.

Doing this in a Test-Driven manner has produced some interesting effects in the VMs (plural) code:

VM without select-loop - event driven and can be loaded as a library into other OS processes, or on embedded system driven by interrupts
dynamic byte-code selection - only the code for the necessary byte-codes for the app is loaded/compiled in. You can produce very "large" function instructions (load/load/sub/load/mult) resulting in smaller app code (creates interesting challenge of compiling app code and selecting/generating smallest byte-code set). The core VM becomes very small.
state-machine heavy VM implementation - very observable model, also made snapshot persistence extremely easy. Also enabled a form of distributed/multi-core-processing - although interesting not useful (yet!)
replaceable memory models - easy to substitute different models of memory management
VMs written in VMs - the old Squeak/Squawk trick. Haven't yet achieved the ultimate minimal set of byte-codes that can implement other byte-codes to build a self-defining VMs

More importantly, having a debuggable, observable VM has allowed me to understand the system/languages I am trying to implement better. Usually, in a way that stops me wasting my time on dead-end ideas, I have to admit, but that is as valuable in research as success.

Thinking Debugger-First about your language/system is user driven, rather than technology driven. That can only make a better system, no?

Patently Outraged

2012-12-14T04:31:00.002-08:00

If you have a few moments go to the Google Patents search engine and look for things you do every day, may have invented or at least helped originate.

Here's a few I found that a relevant to me:

US8225270 : Intuit gaining patent on using a computer system for managing a project using software

Applicants: Thomas A. Frasher, Todd M. Fitch
Year: 2007
ISSUED July 2012
By my reading of this you can't create a spreadsheet to estimate a project without their permission

US20070168918 : Siemens trying to patent online management of software development tasks

Applicants: Jeanne Metherall, Philip DiJoseph, Nicholas Conti, Catherine Britton
Year: 2007

US20110289475 : SalesForce.com trying to patent online prioritisation lists

Applicants: Felix Sukhenko, Steve Greene
Year: 2011

US20120240097 : Ralph Saunders trying to patent providing software development services via a website

Year: 2012

US7797708 : SAP trying to patent Mock Object Testing technique

Applicants: Walter Weber, Pragnesh B. Mistry, Carsten Ziegler, Stefan Baeuerle, Rolf Schulte-Rebbelmund, Abhay A. Tiple
Year: 2006

What I particularly like about this last one is they explicitly cite a paper from Connextra (a company of which I was a founder) which contains the idea ! In fact, if you search for "mock objects" in the patents it is incredible how many patent applications there are on the subject. They are all registered 5-10 years after the idea started and became common practice.

Is this really a reasonable thing to do? Why are you trying to patent these things ?

No, really. I'd like to know. If you are one of the people in the list above (or have a 'mock object' patent application). Please leave a comment, tweet me or email me. I genuinely do not understand your motivations. And frankly it outrages me.

Now, I should declare that I have had patents awarded (US7761906 and US7669186). I've been through the process a few times. I even have registered applications from 1999 that I could follow up to challenge Google's AdWords (US20030110210).

What can I say ? At the time I believed patents were a good thing and would help protect my ideas.

But frankly, I now think I mainly did it for vanity reasons. In my defense, they were original ideas not the plagiarism of common practices developed by a community of peers.

I've talked about the morals and ethics of software developers before. I think this issue lies squarely within that argument. I'm not talking about whether patents themselves are moral/ethical (there are lots of people out there discussing that subject!) - what I'm talking about is whether it is moral/ethical to plagiarize a common practice and try to patent it?

I think not. I think it is immoral and unprofessional.

"What about 'prior art' ?", I hear some of you cry. Well, as far as I understand it, prior art can be used to show a patent application is not original. However, it has to be shown while the patent is being applied for, otherwise the patent can be granted. Once granted, any use of the embodiments thereafter requires the permission of the patentee.

Think about that: the emphasis is on us (the practitioners) to monitor who's applying for patents and show if prior art disqualifies the application as unoriginal. If we don't spot it we then might be subject to the legal powers of the patentees to impose their right of ownership upon our activities. Who has the time for that ?

And let's not get into the subtleties of what constitutes 'prior art'... just doing it and talking about it is not counted as 'prior art'.

I know there are laws in most countries embodying a "Duty of Disclosure" - that is you have a duty to declare prior art you know about when applying for a patent. If it is found you failed to properly disclose prior art the patent can be found invalid for "inequitable conduct" - but really? Who of us has the time and money to attack a granted patent ? Isn't it just a huge waste of time and money ?

I have an idea: don't apply for patents on common practices in the first place. Have some integrity.

And if you're doing it for vanity reasons, write a blog.

PS. If you find any other outrageous patents in your travels - please put links them in comments to this blog

Jack and the #Lean Talk : A Pantomime

2012-12-13T07:00:00.001-08:00

...there was a poor guy with an idea who worked alone with his developer, Jack. All they had in the world was the idea and a little bit of time to develop it.

One day the guy with the idea got frustrated with progress and sent Jack away to take the idea to market.

On his way to find a market Jack met a man. The man was lonely and wanted to be rich and famous. He promised to help Jack get to market so long as Jack would be his friend and introduce him to all Jack's other friends. All Jack had to do was listen to the man give a talk. Jack knew the guy with idea would be very angry if he spent his time with the man.

"Its a very special talk", said the man. "It's a Lean Start-Up talk! - it will help you find your niche so you will make a fortune!"

Jack thought that he and the guy with the idea could do with a niche and a fortune, so he spent time listening to the man, reading his blog and bought his book.

As you can imagine, the guy with the idea was furious with Jack when he returned with no market and just a few ideas. He threw the book out the window and sent Jack to bed without any supper. Poor Jack! He really thought that the ideas were magic. Now he just felt silly.

Next day he woke up and checked his social networks. On looking around he found there was a huge "Lean Start-Up In The Cloud" three day conference going on right outside his door. There were lots and lots of other start-ups going. So they were magic ideas after all!

Without stopping to think he climbed out of his window and went straight to the conference.

In no time at all he was sitting in the auditorium. On stage was a huge, over-valued company that had once been a start-up. They were re-writing their history to show how they were a Lean company. Suddenly there was a terrible pounding sound of a huge round of applause that hurt Jack's ears. The spotlight shone as they introduced the biggest man he'd ever seen. It was an INVESTOR!

Jack shivered and shrank into his seat. Luckily, the investor was so important and Jack so insignificant, he didn't see Jack. BUT - he seemed to smell him!

The investor roared in a voice as loud as thunder...

"Fee Fi Fo Fup, I smell the blood of a new start-up!
Be it real or be it a pitch, I'll buy its shares to make me rich!"

At that point Jack was terrified!

He rushed out the back of the auditorium and bumped into a beautiful woman. She was an Start-Up Accelerator. She quickly told Jack to hide in her booth and before he could argue, she pushed him into the booth and gave him a coffee.

Inside the booth Jack founds lot of leaflets describing how the Accelerator could help Jack find a market full of gold. He popped one into his pocket, thinking it would please the guy with the idea.

When the session in the auditorium had ended, the beautiful woman let Jack out of the booth and gave him a business card. She told him, however, that if he wasn't careful the investor would "eat Jack for breakfast!"

Jack thanked the beautiful woman and ran away from the conference as fast as he could.

The guy with the idea was so excited when he saw the leaflet he kissed Jack and told him how clever and brave he was. However, when she heard about the investor he made Jack promise never to go there again.

He promised, but somehow as the hours passed, he kept thinking about his adventure. He really had enjoyed all the danger and he kept thinking about all the investor's gold.

"We could be rich!", he thought.

The next day, when the guy with idea wasn't looking, he sneaked out of the office and went to the conference again.

When he got there he started to head towards the booth. But, oh dear ! The investor could always smell little start-ups and suddenly Jack heard the roar...

"Fee Fi Fo Fup, I smell the blood of a new start-up!
Be it real or be it a pitch, I'll buy its shares to make me rich!"

Jack ran into a side room and hid in it, trembling. The investor walked around and networked for a while but did not come across Jack.

While hiding in the side room, Jack found there was a session in progress entitled, "How To Make Money through Growing Object Oriented Software". The "Golden GOOS" technique was beautiful. At the end of the session, without stopping to think, he grabbed the book and ran from the conference.

The guy with the idea could not be angry with Jack for long when she saw the quality of the software produced by the Golden GOOS technique. They had no market, or money, but were misguidedly happier. But that's not the end of the story...

The next day, Jack became bored. He thought of the investor's kingdom of start-ups in the Cloud and wanted so much to be part of them.

"Surely it wouldn't hurt to go to the conference again?", he thought. So that's what he did!

This time he managed to stand near the investor and listen in to the conversations. He was amazed at what he saw and heard.

The investor was listening to start-up guys harping on about their ideas with vacuous rhetoric. It seemed to go on and on, lulling the investor into a false sense of security.

"The guy with the idea would love me to harp on like that", Jack thought to himself. So seeing his chance, Jack started to practice on other people at the conference.

He hadn't got very far, however, before he heard those terrible words again...

"Fee Fi Fo Fup, I smell the blood of a new start-up!
Be it real or be it a pitch, I'll buy its shares to make me rich!"

The investor had noticed him and heard his rhetoric. The investor was coming after him!

Jack was terrified and fled the conference. He ran back across the road to the office as fast as he could, but to his horror, the investor was following him ! Closely shadowed by the beautiful Accelerator woman.

Jack was very quick and as soon as he got to the office, got the demo running. The investor came crashing through the door demanding to see the the business plan. The guy with the idea and Jack gave it their best shot and, lo and behold, the investor wanted in as did the Accelerator. They all agreed how wonderfully they could work together and help each other out.

Jack and the guy with the idea had all the money they needed to start the company properly. And thanks the the Golden GOOS, Jack was never bored.

Unfortunately, they didn't understand Lean or GOOS and never applied them properly. Nor did they ever get any real help from their investor or the Accelerator.

It all came crashing down 6 months later.

The End

Dear Start-Up Looking For A CTO,

2012-12-10T01:13:00.000-08:00

It was nice to hear from you. Good to know you are out in the world, strutting your stuff and looking to grow. It's an exciting time isn't it ? I remember it well: the feeling of potential; trying to make an impact on the world; the frisson of having your destiny firmly in your own hands. Enjoy the ride wherever it might take you.

You've said you want a CTO but I wonder if you know what you're looking for? You've essentially asked for "a fast programmer to churn out features in technology X" and have hung the title "CTO" on it. This is like advertising for "a good samba dancer" and expecting him to be the father of your children. Not that samba dancing skills are undesirable in a father - we've all seen "dad dancing" - but they're not the skills or traits that should be top of the list for a long term relationship. I can understand how they are attractive at this stage of your life - sexy, alluring, exciting, giving you exactly the tingly feeling you want right now - but have you thought about the relationship beyond the next few months? Once you get what you want and have scratched this particular itch? Then where will you be ?

Being the CTO of a start-up is more than just writing feature code. Its about dealing with the hundreds of non-functionals that can make or break your business: choosing a technology stack that matches your business; creating an MVP that helps you find your niche and won't cost you an arm and a leg to then fill it; creating technology that means you can operate as a business; ensuring low operating costs; dealing with privacy and security; availability; source code control; back-ups; builds; testing and all those other technical issues.

But its also about the people, process and practice: finding techies; hiring techies; firing techies; creating a repeatable process for defining changes and getting them implemented in a timely manner; ensuring your business processes are supported by the product; helping you moderate your enthusiasm to chase every feature opportunity; sorting any patents; ingraining quality; getting the tech incentives right; discouraging cowboy-ism; taking risks; not chasing trends because everyone else is; saying 'no'; making sure you can deliver on your promises and your potential. I could go on.

None of this is covered by "a fast programmer to churn out features in technology X".

Now, I'm not saying you're asking for the wrong person at this point in time. You're still young as an organisation and maybe having fun and being edgy and risky with a technical "samba dancer" is part of all that. But don't call the role CTO. You're undermining the term, possibly fooling yourself and maybe even risking your investor's money in order for someone to learn through making mistakes with your business.

Enjoy your youthful fling with the "samba dancer" but do not convince yourself that the outcome is guaranteed to be a foundation for a long term, fruitful business. In fact you might try a few different hip-cool "dancers" to try ideas with the UI, platform, etc. Usually you'll get a Minimum Plausible Product (MPP) - one that shows your ideas can have value - but not one that is Viable as the basis of an operating business. Its often a simple question that tells you this: could I sell this product as it is and, without changing it, operate and grow a business from it?

When you are ready to become more established and settled that's the point to look for a CTO. It's around the point you start looking for real investment or when you want to change your MPP into an MVP. If you do take this route, know that there will be a period of re-engineering which will pause your feature rush to get your MVP out to the market. It could be a few months. Its just the price for this journey.

On the other hand if you really are looking for a CTO and think hiring "a fast programmer to churn out features in technology X" is a cheaper way of gaining one, might I suggest an alternative ? Rent a CTO and get them to help you hire some decent "samba dancers" (not the really ace ones who claim they can do everything).

You'll go to an investor for their experience with finance and advice on growing a business but you don't hire them - you sort of rent them as a CFO. So why not do the same thing with a CTO? Find someone with the experience and capabilities to help you grow your technology but don't hire them - rent them.

There are experienced people out there (like me) who are happy to consult to start-ups a few days a week to help guide a team through the tricky initial stages of setting up the technical side of a start-up in such a way that takes all those nasty non-functionals into account. If they're any good they'll even help mentor your team and possibly grow one of your "Samba dancers" into your full-time CTO. I'm not talking "coaches", consultancies or advisers here, I mean hands-on people who have been (or even currently are) CTOs of start-ups. Ask around - you'll find them.

Of course, there is always the third alternative: invest in a CTO. This of course means knowing what you need. The list above is just a thin slice of what's on a start-up CTOs list, but I think you get the idea? It's all those things you never thought of, nor really ever want to worry about, but expect to happen with the technical side of the business. Finding the person who is capable of taking this on can be hard. You don't always need to get an experienced CTO. Even though it is less risky it can be more expensive and they come come with their own challenges. Sometimes finding a person with less experience that has the capabilities to step-up to the challenge is better. But I would suggest getting some expert help from a "rentable" CTO to interview them : it really does take one to know one in this case.

Sorry if this sounds a bit preachy. It's just that by asking for an uber-geek and calling it CTO makes me think you don't know what you really need. It is difficult when you're growing to know what that is sometimes.

Take time to understand where you are. Figure out what you need rather than desire. When you "dance" with the techies ask yourself "what's the attraction?" and state your expectations accordingly.
And only say CTO when you mean it.

I wish you all the best and hope you succeed. Don't be a stranger. Call whenever you like.

And call your mother. She worries.

Your curmudgeonly uncle,

John

What Software Might Learn From Leveson

2012-11-30T07:05:00.003-08:00

The Leveson Inquiry into the Culture, Practices and Ethics of the Press has just concluded in the UK. Whilst it does not have anything to do with software development, I thought some of Lord Leveson's comments, and the nature of relationship between the press and public, had some bearing on software development.

In his statement, Levenson said:

The press, operating freely and in the public interest, is one of the true safeguards of our democracy. As a result, it holds a privileged and powerful place in our society.

But this power and influence carries with it responsibilities to the public interest in whose name it exercises these privileges. Unfortunately, as the evidence has shown beyond doubt, on too many occasions, those responsibilities (along with the Editors’ Code which the press wrote and promoted) have simply been ignored. This has damaged the public interest, caused real hardship and, also on occasion, wreaked havoc in the lives of innocent people.

Software development has an attitude that it also operates "freely" and "the public interest", yet (as I previously blogged about) the developers don't seem to practice an ethically responsible approach to their work. Codes of Ethics written by associations or movements are often paid lip-service, if they are even considered at all. It is my belief that the same outcome, of damage to the public interest, creation of hardship and havoc in the lives of individuals not only is possible, but inevitable unless something changes.

Leveson also commented:

I know of no organised profession, industry or trade in which the serious failings of the few are overlooked or ignored because of the good done by the many.

Clearly, he has few dealings with software development. Or maybe he does: the phrase "organised" is not one which software development can be accused of, nor is it a true profession by any definition. Are we guilty of overlooking serious failings in our rose-tinted view of the "good" we do?

This inquiry and others like it (e.g. the World Medical Association 1948 leading to the Declaration of Geneva) are often reactions to the public revulsion at the practices and outcomes of certain occupations - usually ones which are on the front-line of the developing society. There is always a sense of outrage at the betrayal of public interest and a cry to unearth "why something was not done about it".

Maybe software developers should listen and learn from this. We are operating at the fore-front of societal development in a manner which appears to be consistent with inevitable catastrophe and sanction. What can we do ?

There are two options: precipitate a catastrophe that will bring about public outrage and leads to high-level scrutiny, or learn from other domains and try to get our house in order. The first of these is clearly difficult to achieve and possibly illegal; leaving the second to consider...

Leveson's suggestion to manage the press issues is clear:

What is needed therefore is a genuinely independent and effective system of self regulation of standards, with obligations to the public interest.

[....]

The press needs to establish a new regulatory body which is truly independent of industry leaders and of Government and politicians. It must promote high standards of journalism, and protect both the public interest and the rights and liberties of individuals. It should set and enforce standards, hear individual complaints against its members and provide a fair, quick and inexpensive arbitration service to deal with civil law claims.

The Chair and the other members of the body must be independent and appointed by a fair and open process. It must comprise a majority of members who are independent of the press. It should not include any serving editor or politician. That can be readily achieved by an appointments panel which could itself include a current editor but with a substantial majority demonstrably independent of the press and of politicians.

He is suggesting an independent public self-regulation body which can hold members of the profession accountable. The standards code should come from the industry but be done through a process of public consultation. This is similar to other professions such as medicine and accounting.

He also says:

I am proposing [new legislation] only for the narrow purpose of recognising a new independent self-regulatory system. It is important to be clear what this legislation would not do; it would not establish a body to regulate the press; that is for the press itself to do.

In fact, the law he suggests recognizes the independence of this regulatory body and essentially empowers it.

If software development were to undertake such a move, what would it take?

By its very definition it is not possible for this to be constituted from within the industry. Maybe this is at the very heart of why the software industry fails to create true professional bodies to which the practitioners subscribe : it cannot be done from within or in the privacy of our own conversations.

It would seem we need to get this debate into a more public sphere - talk to the press and the politicians. Talk to our clients and neighbours. Get a more public debate going. Of course, making them care is going to be hard - usually it requires a catastrophe for this to be relevant - but maybe we can use the Leveson Inquiry as a point of leverage?

It would seem we need to try and constitute a body of representatives from outside the industry to help us regulate our behaviour. And try to get it recognised in law as the appropriate body for complaints and concerns.

Relatively simple, then?

"Why bother?", I hear some of you ask. I am concerned at the potential scale of the catastrophe we might make through software development. Not only in an instantaneous sense of a break-down of vital systems but also the creeping changes to our society the introduction of some technologies might have. Changes, which after they are established, we may look back upon with regret and yet another cry of "why something was not done about it". But, more selfishly, I am concerned that the backlash against software developers when such a catastrophe occurs will be over-zealous, onerous and inappropriate - fueled by the howling of politicians and journalists who will easily forget the events that precipitated Leveson Inquiry.

Software: The Amoral, Unethical Occupation

2012-11-27T08:49:00.001-08:00

As part of Harvard Law Professor Michael Sandel's course he presents a classic Trolley Car problem during his Moral Side of Murder lecture:

Suppose you're the driver of a trolley-car and your trolley-car is hurtling down the track at 60 miles an hour. At the end of the track you notice 5 workers working on the track. You try to stop but you can't. Your brakes don't work. You feel desperate because you know if you crash into these 5 workers they will all die. (Let's assume you know that for sure).

And so you feel helpless until you notice there is, off to the right, a side track. And at the end of that track there is 1 worker working on the track. Your steering wheel works. So you can turn the trolley-car, if you want to, on to the side track killing the 1 but sparing the 5.

What's the right thing to do?

This is a beautifully simple dilemma has been used for many years as a tool for discussing the difference between morals/ethics, right/wrong and the conundrums of law and living.

There is a variation of this I use to explore the amoral and unethical habits of software development (see my "Is It Just Me Or Is Everything $#!t" rant at GotoCon2012 in Copenhagen) :

To protect their trolley-car drivers, the company has decided to automate the switch-track, removing the responsibility from the driver. Using the web or mobile apps, subscribers can see personal details about the workers and give them a rating. The system in real-time, based on the collective score, figures out which direction the trolley-car should go to minimize the impact of the impact.

Suppose you are a software developer who has been asked to design and implement this system. Do you?

More than 75% of the hundred or so software developers I have asked readily admitted they would do it.

75%

Think about that for a moment.

When pushed on the moral and ethical aspects of creating such as system, a number of responses were typical:

"If I don't do it, someone else will"
"It's not the creation of it which is the problem, it is the use of it"
"The responsibility lies with the company for wanting it and using it. Its my responsibility to create what the company needs"

All of the responses from those who would implement the system can be summarized as an abdication of responsibility. The last of the responses above is tantamount to the Nuremberg Defense.

This data supports my experience that there is little or no consideration taken by software developers of ethical, let alone moral issues surrounding the technology they develop. There is little or no responsibility taken for the consequences of technology by those who have the greatest power to control it.

The trolley-car scenario may be hypothetical, but consider :

the implementation of trading systems which allow the traders to expose the institutions to billions of risky/fake trades
the implementation of strong encryption between BlackBerries (BBM) being impenetrable by police or other authorities
the development of the mobile phone intruding on the lives of everyone - even if you do not own a phone ("hello, I'm on the train..."). Let alone the greater ability to track individuals
the deployment of websites with poor security measures allowing the theft of personal financial data

Ignoring whether you think these are positive or negative things: In how many of these cases was there consideration taken by the software developers that made the technology about the moral or ethical consequences of enabling these technologies to exist.

We create technologies of great potential value but also potentially perilous cost. We don't account for those costs - I'm not taking £/$/€ here, I mean the societal and personal costs. For example, what's the cost of having a mobile phone as opposed to fixed line ? Who questioned this and the impact on society it might have?

For any technology we might ask, "Who would have the right/power to say 'no'?", but I ask , "Who has the right/power to say 'yes'?" - and the answer would appear to be "any fool with a computer".

We rummage through Panadora's Toy Box like 3 year-olds, shouting "Look what I've got" and hurling technology over our shoulders into society without consideration.

We have a duty as members of a society and software developers to ensure proper care and respect is had for the morals and ethics of the societies to which we subscribe. We cannot nor should not abdicate this responsibility, yet we do.

I offer no solution to this problem. It is beyond my power or understanding to do so. All I would ask is that you (if you are a developer) consider the moral and ethical impacts of the technology you are working upon. Do not abdicate this or expect someone else to have thought about it. Ask, "What is the cost of this?" and "Can I be responsible for this?". Listen for yourself using the Nuremberg Defense. Listen for yourself arguing Moral Neutrality.

Eigenharp Tau Whisky Dispenser

2012-01-25T05:12:00.000-08:00

So the Eigenharp Developer Conference was a complete blast. Three days of discussion, coding, playing and drinking. But the product innovation was the realization of the "Eigenharp Tau Whisky Dispenser" accessory.

Now this innovation had been discussed on the forum but it took a perfect storm of pieces, people and madness to achieve...all of which came together Saturday night. During our tour of the Eigenharp manufacturing area, John Lambert had handed be some elastic bands and Tygon tubing with a particularly impish gleam in his eye....

Parts List

1 x Whisky Miniature (I'd supplied some Eigenlabs branded Bladnoch for the attendees)
1 x Eigenharp Tau (WARNING: DON'T TRY THIS WITH ALPHAS)
2 x Elastic bands
1 x 75cm length Tygon tubing

Instructions

Slide elastic bands up Tau and attach Tygon tubing to 'spit port' on bottom of Tau.

Slide bottle (with small amount of tissue to protect Tau!) behind the elastic bands.

Insert Tygon tubing into top of bottle.

Position optimally - somewhere around the percussion keys seemed to work best.

Suck like mad (actually it's not too bad)

Share with friends
(this is Duncan Foster - whose photos these are cut from)

Of course, the discussion almost immediately turned to how to engineer it so you could play at the same time - requiring a bypass system for blowing (unless you wanted the sound of bubbling). John Lambert's enthusiasm was such that I am expecting to see an engineered design within the next few weeks - and hopefully availability as an accessory on www.eigenlabs.com accessories soon thereafter.

Adding Type Overloading Methods to Python

2011-07-18T04:15:00.000-07:00

Python does not have type-based method signatures. One consequence of this IMHO is that a lot of 'dirty code' is being written that would be much improved with the presence of type-signatures. Now I started to write this with a whinge about the liberal use of if isinstance() or if type() statements within Python methods/functions, poor implementation habits, lack of understanding of clean code design and testing, blah, blah, blah... But I think that's just a waste of time for me to write and you to read. So I'm just going to cut to the chase:

I've implemented a Metaclass that enables type-dispatch on method signatures. What's that mean? Well, here's an example:

from typed_calls import *

class Printer:
    __metaclass__ = ClassWithTypedCalls
 
    def show(self,an_object):
        print str(an_object)
  
    @when('show',list)
    def show_list(self,a_list):
        print 'A list:'+str(a_list)
  
    @when('show',int)
    def show_int(self,an_int):
        print 'An integer:'+str(an_int)
  
if __name__=='__main__' :
    p = Printer()
    p.show('Just a String')
    p.show([1,2,3])
    p.show(67)
    p.show(p)

When this runs you get the following output:

Just a String
A list: [1, 2, 3]
An integer:67
<__main__.Printer object at 0x100497d90>

This syntax/mechanism allows you to override methods based on the type of the first argument. In this case there are overrides in place for list and int but everything else uses the standard method.

Why is this useful?

The code is clean (both in the implementer and the user)
The unit testing is simple
You don't have to add code to other classes (which might be out of your control)
It makes writing Visitor patterns much cleaner

An example of Visitors using this code would be:

from typed_calls import *
from copy import copy

class A(object):pass
class B(object):pass

class NodeWithChildren(object):
    def __init__(self,*children):
        self.children=children

class CopyingVisitor():
    __metaclass__ = ClassWithTypedCalls
 
    @when('do',object)
    def make_copy(self,an_object):
        return copy(an_object)
 
    @when('do',NodeWithChildren)
    def do_children(self,a_node_with_children):
        new_node = self.make_copy(a_node_with_children)
        new_node.children = [self.do(c) for c in a_node_with_children.children]
        return new_node
 
class TreePrinter():
    __metaclass__ = ClassWithTypedCalls
  
    def __call__(self,an_object):
        self.depth=0
        self.buffer=''
        self._print(an_object)
        print self.buffer
 
    @when('_print',object)
    def default_print(self,an_object):
        self.buffer += '  '*self.depth
        self.buffer += str(an_object)
        self.buffer += '\n'
  
    @when('_print',NodeWithChildren)
    def node_print(self,a_node):
        self.default_print(a_node)
        self.depth+=1
        for c in a_node.children : self._print(c)
        self.depth-=1
  
if __name__=='__main__' :
    child3 = A()
    child4 = B()
    child1 = NodeWithChildren(child3,child4)
    child2 = A()
    top = NodeWithChildren(child1,child2)
 
    print_out = TreePrinter()
    copier = CopyingVisitor()
 
    print 'Original'
    print_out(top)
    copy_of_top = copier.do(top)
    print
    print 'Copy'
    print_out(copy_of_top)

There are two Visitors in here the CopyVisitor ad the TreePrinter. Note the TreePrinter is implemented as a Functor, too (using the standard Python __call__ as the initiator). I like Visitor Functors as they allow you to implement clean and simply testable approaches to execution-state-based behaviour (e.g. then indentation depth of the tree) without compromising the implementation of the data-structure being walked or having to pass long lists of parameters.

Now you might think, "Isn't this just multi-methods?", and in a way it is except this implementation allows for inheritance - both in the argument types and in the implementers. That is to say, if you implement an override with @when and an object is a subclass it will be recognized. Secondly, the mechanism will look back up the implementation tree to find an appropriate method. Essentially the rule goes: look for a typed method the specific typeof the argument in this class, then in its parent classes, then look for a method for the next abstract type of the argument in this class and its parent classes, [repeat up the argument abstract types....]. An example is easier (and check out the tests):

from typed_calls import *

class A(object):pass
class B(A):pass
class C(B):pass

class BaseClass:
    __metaclass__ = ClassWithTypedCalls
 
    @when('method_call',B)
    def call_in_base(self,argument):
        return 'call_in_base'
 
class TestClass1(BaseClass):
 
    @when('method_call',A)
    def call_in_test_class_1(self,argument):
        return 'call_in_test_class_1'
 
class TestClass2(TestClass1):
 
    @when('method_call',B)
    def call_in_test_class_2(self,argument):
        return 'call_in_test_class_2'
 
test_object1 = TestClass1()
assert test_object1.method_call(A()) == 'call_in_test_class_1'
assert test_object1.method_call(B()) == 'call_in_base'
assert test_object1.method_call(C()) == 'call_in_base'

test_object2 = TestClass2()
assert test_object2.method_call(A()) == 'call_in_test_class_1'
assert test_object2.method_call(B()) == 'call_in_test_class_2'
assert test_object2.method_call(C()) == 'call_in_test_class_2'

Caveats:

this only dispatches on the type of the first argument. You can pass more arguments through but they will not effect the dispatching (exercise for reader to extend...)
mechanism is relatively efficient, but will not be as fast as doing a hand-written double-dispatch/Visitor pattern
when debugging you will see and extra level of stack per call - can be off-putting to some
all instances of a class share the same typed-methods. Cannot (currently) override on a per instance basis like you can with normal Python functions

Now as much as I've written this and am sharing it, I would say that this is done as a 'patch' to provide a better/cleaner way for developers who cannot/do not want to write better code. If you think you have need of this, please consider changing your code to be better by:

implementing double-dispatch on your classes. Makes thing much easier to test and allows greater extendability to other programmers without having to change your code
not using base types (int, list, dict, etc.) as primary types in your model. Wrap basic types and use trivial sub-classing. In concert with double dispatch makes your code more testable, understandable and extendable

If you do want to use/try this download typed_calls.py and the accompanying tests _test_typed_calls.py

Objects, eXtreme Programming, Eigenharps : How Innovation Turns To Infection

2011-04-27T02:20:00.000-07:00

Its happening to me again. That feeling. I've had it a few times during my life. It's a combination of excitement, confusion, giddiness, occasional nausea and mind-expanding joy. Of course there is a downside: long hours in dark rooms alone, staring into space and the raving. If it sounds like I'm on drugs, it's not far off - and I'm addicted to it.

What am I talking about ? New technologies that undermine the current thinking and enable a whole new way of doing something but are not yet fully formed and accessible. I'm not talking about those "new technologies" that marketing people push at you as "the next best thing", nor the latest tech-fashion received via Twitter, I'm talking about those ones way over before the "innovator" section of the technology adoption curve. The ones that creep up on you and lurk about on the edge of your perception until you notice them - at which point they come to possess you and the world is never the same again.

They're relatively easy to recognize: They're quirky and weird. Often referred to as "academic" or "cool but not practical". There's usually little or no effective marketing involved. They have interesting demos or descriptions that make me go "hmmm" but I'm often left with the feeling that the demo doesn't quite get across the whole point. People using them often say, "I've learned a lot..." but have not actually achieved anything practical. The documentation and tutorials are written in some magic incantation jargon which feels like it involves the use of a pointy hat and a stick. Now many ultimately poor technologies fit these parameters - but always, always these other ones have a "something extra" that nags at the front of my brain and possesses me, and usually it starts with "But what if...".

This has only happened to me twice in the last 25 years: Objects/Smalltalk and eXtreme Programming (XP). And its happening to me again with Eigenharps. This has caused me to reflect on the process where these weird, lurking things turn into acceptable and (dare I say) mainstream ideas.

The current, broadly accepted model of technology adoption is known as the Diffusion of Innovations model - popularized in the book and zeitgeist of Crossing The Chasm (always nice to see an old idea profited upon by others 40 years later). But I have always felt this model was weak on how ideas become infectious in the first place - how do the Innovators get the "bug" and how does that get translated into a palatable form for the Early Adopters?

In practical terms, these are the practices I have seen happen that enable these ground-shaking ideas to become infectious and practical:

Small Interest Discussion Groups - personally, Objects had the BCS OOPS group (by extension the OT crowd) and XP had the eXtreme Tuesday Club (XTC). Small groups of similarly obsessed people who got together to talk and share thinking. Much talk involving, "What does it mean?", "How do I...", "What I've found...". Now to be clear these were not like-minded people - in fact the very variety of minds produced a synergy where the fundamental idea was converted to the more palatable, accessible and understandable form. This is highly interactive and face-to-face. Email and web supports it but the real grist is in the eye-to-eye, blow-by-blow, hands-on collaboration. Often this involves a lot of critical thinking (and criticism) of the original idea and how it is portrayed, this means that the originators need to have thick skins and....
Let The Idea Go And Accept What Happens - originators need to let the idea grow-up by itself. You can't control it. You can't necessarily predict what its impact or meaning will mean to the people down the line. Not so much "build it and they will come", more "build it and let it find them". Listening to what is discussed and how its described helps you re-work the way you talk about it
Find the Discipline for the Flexibility - most of these ideas contain huge amounts of flexibility which is always vaunted as a good thing, but its like a box of very sharp knives - extremely dangerous in the wrong hands. Success seems to come when people find the right discipline of practices to apply when using the flexible tools (however, in my observation, discipline is the thing most adopters struggle with)
Let Others Talk / Translators & Ambassadors - originators can often be detrimental to their own ideas. Letting a "secondary" set of people - let's call them the aunts and uncles of the idea as opposed to the parents - do the talking seems to produce the most effective route to adoption. Aunts and uncles have enthusiasm having struggled and achieved with the technology and can talk to others like them. Originators can never reach and talk directly to all the sub-interest groups effectively, the aunts and uncles can - and usually for free!
Use It As A Teaching Tool (for Adults not Kids) - new ideas often are pushed as 'great/new ways of learning X' but I think a mistake often occurs in this thinking by then targeting children. What I have seen as a more effective tool is to use it to educate adults - essentially giving them access to a domain or discipline they feel excluded from currently. These technologies often get you more quickly to the 'value' than other approaches which usually require more training than learning. I saw Objects and XP draw in a number of people outside of what would be considered programmers/technologists through accessible education approaches. These people then succeed and become aunts and uncles of the idea - in fact, they are often the greatest proponents

I experienced these five practices occuring with Objects and XP, and have talked to people in other disciplines who have observed similar early-early-adopter practices. Ultimately, the outcome of these is a more understandable, applicable version of the original idea which can then be taken up by the Diffusion process. Its sort of the 'infectious idea' petri-dish.

So, as an open letter to the Eigenlabs guys, here's my thoughts on how this applies to the Eigenharp:

Small Interest Groups - need to start these in various places. Get Eigenharpists together regularly to just talk/jam. Can't be done by the company, needs to be a grass-root movement. This is really an action for me - need to start one in London. Anyone up for a London-based Eigenharp jam once a month?
Let The Idea Go And Accept What Happens - think Eigenlabs doing this well enough - maybe need to listen more to the non-musicians who are adopting your product? Opportunity is huge in that market IMHO
Find the Discipline for the Flexibility - there's definitely a whole missing section of "how to..." documentation and "my way of playing...". Capability is there but its so flexible that its difficult to adopt quickly. Documenting "How to..." discipline helps with this.
Let Others Talk / Translators & Ambassadors - think this is happening with people like Geert - but need more and more out there
Use It As A Teaching Tool (for Adults not Kids) - think this is missing completely. I've handed the Pico to some non-musicians and within 10 minutes got them "sounding good" without any trauma. In fact, they've all said something along the line of, "Wow! I could imagine getting one of those and learning to play" - its an inversion of the usual music learning model: sound good then learn why.

Playing the #Eigenharp Tau like a Guitar with AAS Strum GS-1

2011-04-18T07:31:00.000-07:00

Geert Bevin put a video on YouTube of him playing his Alpha with the Orange Tree Samples Evolution Acoustic Guitar. Having just received my Tau and having a copy of Applied Acoustic Systems Strum Acoustic GS-1 I thought it was a good exercise to learn about the Tau and get an alternate playing style/sound which was more guitar-like. I also had an idea about using the Arranger to do strum/pick patterns....

You can see from the picture on the left (that's not me, it's Dave(?) from EigenLabs playing at NAMM 2010) that the playing stance of the Tau is generally around the big keyboard on the right in the picture. The keys on the lower-left are usually set-up as percussion keys with drum sounds, but being an Eigenharp they are entirely configurable. So why not set the percussion keys up to strum/pick the instrument like it was a guitar? (a la Geert)

Play It Like A Geertah
Strum, like some other Guitar VST/AUs, allows you to use a section of the notes (C5-C6) to control the strumming/picking action of the synthesized guitar. For example, holding down a G chord does nothing (if you turn Auto-strum off!) until you hit C5 which gives you a down-stroke. D5 gives you an up stroke. E5-C6 give you string picks of the 6 strings (well, kind of E5/F5 gives 'bass' and 'alternate bass' which is not the same as strings 1/2). Black notes give you other things like palmed strings, muffled strokes, etc. So using Strum with the Eigenharp percussion keys was a matter of configuring them to be C5-C6.

Now there is a little Eigenharp subtlety here: You can configure the Eigenharp to split the big keyboard into sections, or keysets. I wanted to have the Tau configured so I could play chords with top 5 rows, strum/pick with the percussion keys and use the remainder of the large keyboard with a different voice. So that means using keysplit2 (for the Tau players out there) and configuring the voice for the top keyset and the percussion keys to be the same. This meant I needed to use Audio Unit 3 (AU3) as it is the only one as standard shared between a keyset and the percussion keys. Handy.

Next I needed to configure the percussion keys to start at C5 and be in the major scale (so I'd get the white notes for control). This meant I needed to do some Belcanto (the configuration script for EigenHarps). Its not the most accessible language/tool I've ever come across and I'm a geek! But I persevered. Eventually, I ended up with the following script which configures the percussion keys:

description
    Set up percussion keys to work for AAS Strum GS-1 as strum/pick keys
script
    percussion kgroup hey tonic to notec set
    percussion kgroup hey octave to 5 set
    percussion kgroup hey scale to major set

This can be saved into ~/Library/EigenLabs/Scripts and accessed through User Scripts in the EigenBrowser. Or alternatively, you can type the bits below 'script' into the command line of the EigenCommander. (Or if you're really adept play it via the Eigenharp itself - not figured this out yet!)

Once run, configure AU3 to use AAS Strum via the EigenBrowser (switch Auto-strum off in the Strum UI). Select keysplit2 and assigning AU3 to the voice of keyset2. Then set the voice of the percussion keys to AU3. Then play!

This set-up provides for up/down strum on the first two keys of the percussion section and picking on the next 6. You get a much more guitar-like stance and style to your playing (plus proper picking opportunities with your right hand) and a great selection of guitar sounds. I believe this should work with the Electric Guitar version, too.

(NB: careful when changing scales not to do a "global" scale change or the Strum set-up on the percussion keys may not work as you think)

So what's it like to play? Really good! You get a very natural, expressive guitar playing experience. Because you have such control with the Tau in combination with Strum it possible to not only get nice bends, slides and hammers but also you can control the effects with lateral movements of the keys. I have one set-up where I control the WahWah this way - fascinating to play!!

One thing I wasn't sure of, but turned out just to work, was the recording of strum/picks with the built in Eigenharp scheduler. But it works beautifully. You can lay down a nice guitar strum/pick sequence to a drummer track then swap to an alternate keysplit where you can play two different parts with your now free two hands.

Making Arrangements
The other idea I had was to use on of the Tau's Arrangers to define a strumming/picking sequence. The Arrangers are step-sequencers which you configure by using the large set of keys to define a pattern - usually of drum or rhythm samples. So why not use it to fire the C5-C6 in an appropriate pattern? This way I could play chords with left hand, get a fancy custom strum/pick sound and play melody/lead with right hand in another keyset? (think arpeggiator)

Strum does have a built in arpeggiator which uses MIDI loops, so I could just use that. But that would mean having to create/load a MIDI loop any time I wanted to do an arpeggio. One of the main things I like about the Eigenharp is its 'liveness' and not having to go back to the computer keyboard to endlessly edit loops/sequences. No - I wanted to use the Arranger!

But I hit a speed bump: Arranger 1 on the Tau can only use AU4, but you can't access AU4 via any of the keysets !?? This means I can't have the Arranger firing C5-C6 and a keyset playing the notes for the chord. This is surprising to me given the flexibility of everything else on the Eigenharp. I have asked a question about how to change this (I presume some clever Belcanto script might do it) but with a standard set-up it just not possible :-(

But...

Maybe I can send MIDI from Arranger 1 and from the keyset to Strum running outside EigenD ? I'd loose some expression but I'd get my combination of arranger and keyset.

Hmm...not obvious. Nothing in the config seems to let me send MIDI from the Arranger....arrrrgh

So I ask the Twitterverse for inspiration - and hey, presto!! @MikeMilton sends me a reference to a snippet of Belcanto he got in a reply to a similar question.

Again after some twiddling with EigenCommander I got it working. Here's the script:

description
    Set up Arranger 1 to work with AAS Strum GS-1 as strum/pick control
script
    arranger 1 listen
    clear
    midi rig 1 recorder listen
    note 15 with velocity 1 when 1 play
    note 16 with velocity 1 when 2 play
    note 17 with velocity 1 when 3 play
    note 18 with velocity 1 when 4 play
    note 19 with velocity 1 when 5 play
    note 20 with velocity 1 when 6 play
    note 21 with velocity 1 when 7 play
    note 22 with velocity 1 when 8 play
    all join

This sets up Arranger 1 to send out C5-C6 to the MIDI. (as before save as a User Script or use EigenCommander). Select MIDI in your keyset. Run up Strum as an application. Set MIDI to Omni (simplest approach). Swap to Arranger 1, set-up a strum/pick pattern (course 1 is down stroke, 2 up-stroke, etc in chromatic order). Swap back to keyset. Start metronome and hold a chord pattern down. Voila!

Haven't quite got used to playing it 'live'. Still not got start/stop smooth. I've only had the Tau 2 days, though! There's definite value in the approach. I've found myself being quite funkadelic with it. Hopefully some EigenharpBelcantoMeister will help me out so I don't need to do this via MIDI.

The combination of Guitar synth and Eigenharp is incredible. The ease of playing and expressiveness is stunning. It leaves you in that zone of playing where you listen to the instrument rather than concentrate on the mechanics. You can play manually or you can get some CAP (computer aided playing) - your choice. But one way, or another, you get to play expressively, sound great and enjoy.

I love the Eigenharp (sigh)

#Eigenharp, My Ignorance and @david_harvey

2011-04-08T02:49:00.000-07:00

I freely admit I am pretty ignorant when it comes to being a real musician. In fact, I was quite happy being ignorant and playing stuff by ear/instinct for the last 30 odd years. But playing my beloved Eigenharp and writing EigenTab (a helper app for mapping chords to key/scales) it has been quite an education. Nothing like having to write an application from scratch to make you question your understanding of a model!

I enjoy discovering and understanding models through writing software. I'm not good at memorizing things but need to understand how things are constructed (bad at history, great at maths, you know the kind). Software is great for this, especially when you take a test-driven approach. So with EigenTab I got so far and realized that I couldn't move forward without getting some answers to basic axiomatic and WTF kind of questions. For all the information out on the Internet, like most subjects you get to a point where the jargon over-powers your ability to understand it. Now I'm no dullard and am capable of understanding this stuff but I am not an idiot - why would I spend ages trying to decode this mass of jargon when I can ask someone who knows it? (Nolan's Rule #4: Shared Seat-Of-Pant's Rule : if you can't do it by the seat-of-your-pants, use the seat of someone else's more experienced pants).

Enter Mr. David Harvey, stage right (well the back of The Water Poet, really)

I've known David a long time through our shared profession of software development, however, David is a real musician. I mean he was actually educated in it, has degrees in it, still performs, he was even CTO of Sibelius . So if there was a seat-of-pants to consult, it was his. (And he owns an Eigenharp Tau)

So I had, what I thought, was a simple question: In EigenTab it now shows the note names on keys for the configured scale. If the note is a 'black' note do you care if it shows sharp or flat? Aren't they the same? Isn't it just musician prissiness?

Half an hour later (that's how much time we had) I'd gotten a full low-down on scale construction, chord theory, modes and even how Jazz musicians improvise. I think about 35% stuck. But I thought it worth writing down what-I-learned-from-David:

sharps and flats are not the same! For example F# and Gb are not the same. Now I'd come across this when overhearing muso's (mainly rock like guitarists) talking, and after thinking about it I remembered the guy who taught me basic trumpet saying something similar. Apparently, F# is lower than Gb
the main reason there's a good/bad way to refer to notes in a piece of music in a given scale is because scales are constructed from a pattern of rules about what to do to generates the notes in the scale. For example, a minor scale contains a minor third - that is to say it contains a note that is three notes up the stave (literally counted from the root or tonic note) and then dropped back down a semi-tone (making it flat if it's black). Whereas you might see a similar 'black' note in a different scale containing an augmented second, which is constructed by going two notes up the stave and adding a semitone (making it sharp if its black) (see previous point that they are not actually the same note). So scales are actually defined by a set of rules not just notes.

this means I need to change the EigenTab implementation to be correct

I was always a little perturbed by all that Ionian, Dorian, etc. talk. But apparently to figure out the note intervals in semi-tone terms you can do it by thinking about the white keys on a piano. It seems that if you start at C and play the white keys to the next C its an Ionian scale, and if you start at D and play to the next D its the Dorian ! In fact there is a scale starting on each key:

C = Ionian
D =Dorian
E = Phrygian
F = Lydian
G = Mixolydian
A = Aeolian
B = Locrian

It's spelt loCRian not loRCian - for years I've thought it was loRCian. duh.

Now in the 65% of stuff I didn't retain was something very interesting about Jazz improvisation where Jazz guys improvise by playing say a minor scale above a triad chord played in a different scale (??? I think that's what he said...). The really interesting point for me, though, is that with the Eigenharp (well Tau and Alpha) because you can split the keyboard and have different scales, it might be possible to make Jazz-like improvisation easier without needing to have full chromatic scales on the keys. David did say the choice of scale is dynamic depending on the chord played, but that's a solvable problem in software (id the chord, pick a 'good' scale). Of course to do this I need a Tau or Alpha and the imminent release of the open-source version of EigenD.

There were two other outcomes of this meeting:

I now know what it must be like for normal people to ask me a question about technology (c.f. me/music/david with them/tech/me).
I need to talk to David more often about music and try to translate it into understandable, useable knowledge

Thanks, David. Next time I'll buy the wine.

Adventures in #Eigenharp Land

2011-04-03T03:30:00.000-07:00

So I was given an Eigenharp Pico for my birthday (a month early - result!). It is, quite simply, the best musical instrument I've ever played. And I have played a lot - I can play 8 different types of instrument not including drums (none of them brilliantly, though).

If you are a techy and in anyway musical YOU SHOULD BUY ONE OF THESE NOW. I'm not going to give you a list of reasons - JUST DO IT.

For all its brilliant-ness, it is challenging to learn the Eigenharp. Unlike conventional musical instruments the Eigenharp does not have all the notes available on the keyboard - you configure which key and scale you wish to play so you end up only having those notes. This is extremely useful for improvising and general noodling - but if you are an untrained, picked-it-up-as-you-went along musician like me, trying to play written music can be very frustrating. I can't look at a set of chords or sheet music and say, "oh, of course, F# dorian" - especially those chord sequences scribbled on the back of a beer mat in bad light at a dive during a folk-funk-punk-fusion session. Also, this key/scale configuration means there is no standard way to play "Am". The same chord shape is "Am" in C/major but "Dsus2" in E/blues.

So being a programmer I wrote myself an application to help me with this: EigenTab. You provide a set of chords and the app tells you what your key/scale options are. It also shows you the chord shapes. It has helped me get to grips better with the Eigenharp.

Of course, I set myself unreasonable goals in writing the app: write once, test-driven so that it will work on major platforms (including windoze, osx, linux, iOS). And no conditional code that says anything like 'if windoze...'. Also, had to work offline without a server.

After a couple of false starts I ended up back with a most basic HTML/Javascript approach. Had to strip the UI of all graphics and 'fancy' interactions (like hover or other event behaviour). But in the end developed a 20K application that does work under all main OSes/browsers and even can be cached as an 'app' on the iPad. Have to say I'm pretty pleased with it and it makes my Eigenharping much easier.

Lessons in Managing Self-Organizing Systems: My Father, Squirrels & Birds

2010-11-16T05:44:00.000-08:00

My father had a running battle with squirrels. The age old story of escalating hostilities between man and sciurine ingenuity: bird table, nut-feeder, wire, so called squirrel-proof feeders, mesh, string...you probably know the story. All to no avail. Mr.S.Nutkin was not to be deterred; birds were going hungry; my father's tactics were heading towards firearms.

This problem was a classic self-organizing system with an issue: each element was autonomous and independent (father, squirrel, bird, table), each operated within the same environment, communication was indirect via the environment, no direct control could be exerted between the elements and the emergent results were sub-optimal for all involved.

Amongst the things I have learned whilst studying and building self-organizing systems (and running S.O.teams) are

often the solution to a problem like this is counter-intuitive and based on inverting an assumption or goal
by trying to fix the problem you become and element in the environment yourself, and all the rules apply to you too - especially the no direct control rule
the only effective way to alter a self-organizing system is to alter the environment through adding/removing elements and allowing new behaviour to emerge
you cannot predict the total outcome of changing the environment and must be prepared to accept consequential behaviour

So applying Lesson 1, I decided to address the basic problem: How to stop the squirrels eating the birds' nuts? Inverting this I came up with the simple question: How to stop the birds eating the squirrels' nuts?

Lesson 2 told me I couldn't do anything to change the birds, the squirrels or my father directly. I could spend quite a bit of time on this but in the end it would just come down to an argument about philosophical positions. And my father probably wouldn't listen either.

Lesson 3 told me I had to add or remove something from the environment to affect a change. There was no way to remove all the birds or all the squirrels, nor stop my father feeding the birds. So it meant I had to add an element.

So I bought my father a squirrel feeder that the birds couldn't use.

Within 2 days the birds and squirrels were sticking to their own feeders. The birds were happier. The squirrels were happier. My father no longer had that look of murderous intent.

In fact, this simple act of management was so successful that some of the shy Red Squirrels from the forest behind us started frequenting the squirrel feeder (who knew it was the birds they were afraid of?) That's the upside of Lesson 4. The down side was that there are now twice as many birds and squirrels in the garden and my father is having to buy twice as much food. There is also some emergent behaviour in the pheasants hanging out under the squirrel feeder for the cast-offs which is causing the grass to be worn away. Oh, well. That's self-organizing systems for you: never done.

So why do I offer this parable?

The term self-organization is thrown about very freely when people talk of groups, teams, management and organization. When it comes to looking at the actions, rhetoric and advice given I feel that much of it falls into the "pop-sci" or "pop-psych" category, and often misses the point that the manager/facilitator/coach is an element of the environment and subject to the same rules. Moreover, self-organization is referred to as if it were an understood thing - which it is not by a long chalk.

I offer this parable and 4 lessons as items from my experience. I place them in the environment. And awaiting to see if there is any emergent behaviour.

Contempt, not for software developers, but for all they do

2010-11-13T11:16:00.001-08:00

I have recently spent quite a lot of time reading tweets, blogs and current publications on Software Development. I feel despondent, despairing, bitter and isolated. I feel the "agile movement" has lost it's way, got diluted, compromised it's fundamentals, become a banner of convenience. I feel contempt for this movement of which I was an early practitioner, proponent, zealot. Though, for the individuals involved, I still have respect and admiration - they are bright, articulate, rounded people.

And for all this, I felt ashamed. My arrogance, my hostility, my mourning for a passion lost to the betrayal of others.

But these feelings were familiar. They resonated with a recent empathy. Then I remembered a passage from Seven Pillars of Wisdom (T.E.Lawrence's account of his part in the Arab Revolt during WWI). Talking of how he felt some years after having left Arabia, he wrote he had

"a resultant feeling of intense loneliness in life, and a contempt, not for men, but for all they do" (ch.1,p32)

This is the feeling I have.

Other passages have congruence with how I see the current state of the 'agile movement':

"As time went by our need to fight for the ideal increased to an unquestioning possession, riding with spur and rein over our doubts. Willy-nilly it became a faith. We had sold ourselves into it's slavery, manacled ourselves together in it's chain-gang, bowed ourselves to serve it's holiness with all our good and ill content. The mentality of ordinary human slaves is terrible - they have lost the world - and we had surrendered, not body alone, but soul to the overmastering greed of victory. By our own act we were drained of morality, of volition, of responsibility, like dead leaves in the wind." (ch.1,p29)

Others speak to how I feel about my history with agile:

"The moral freshness of the world-to-be intoxicated us. We were wrought up in the ideas inexpressible and vaporous, but to be fought for. We lived many lives in those whirling campaigns, never sparing ourselves: yet when we achieved and the new world dawned, the old men came out again and took our victory to remake in the likeness of the former world they knew. Youth could win, but had not learned to keep: and was pitiably weak against age. We stammered that we had worked for a new heaven and a new earth, and they thanked us kindly and made their peace." (introduction to later editions)

In these words I find solace. Men far greater than I, more central to more important histories, have fallen prey to these feelings. In the pursuit of what you believe to be a better way of the world, maybe there is an inevitability in the course of human history for the vanguard to become dissolute with the world they help forge. Yet in these new worlds there is value and benefit for others - it is a better world for them. For the vanguard it was about the pursuit of the ideal, the journey. For the rest it is about the destination of being somewhere better. These two things can never sit well together.

So where does this leave me?

In looking for new employment I discovered that I unwittingly undertook a similar path to Lawrence. Although I am being offer Head of Development/CTO roles, I find myself drawn to being a programmer at the bottom of the pile. I even went as far as to draft an alternative version of my CV removing my management and agile experience (but have not used it). Lawrence re-enlisted in the RAF under an assumed name, but was discovered, at which point he enlisted under another name in the RTC where he was unhappy. Maybe this is something I should take heed of?

Further, I have come to recognize that I am a 'vanguard' kind of guy. I suppose I always have been. And in that there is an acceptance. I should feel no shame but know it is time to move on and let the followers settle. Those who make the war cannot make the peace.

- Posted using BlogPress from my iPad

To Make Money Selling Shovels: First, Buy All The Shovels Then Create The Demand

2010-11-02T08:35:00.000-07:00

The adage "The people who make the money in a gold-rush are those who sell the shovels" cropped up again as I was perusing the blogo's fear. It is an analogy I have used myself but I have no idea if there is any truth to it, or like many urban-truths it is merely short-hand for a presumed truth? Or, if there is truth to it, what more might we learn from the fuller, less sound-bitten story?

Thankfully, there is truth to the statement and I have not been guilty of repeating an urban-truth. Reading historical research of census data (like this difficult to read paper) there is statistically significant differences in the relative benefits to merchants (including shovel sellers), hotel proprietors, doctors, etc. (i.e. service providers) relative to the miners themselves. In fact, although the miners were better off in absolute terms (earning on average 3-4x what they would have elsewhere) they were worst off in relative terms because the merchants and other service providers were charging them at extortionate rates.

What is not clear from the analogy, but is from the census data, is that on the whole it was the service providers who were already there and held the distribution points and land that made the money. Many who came along later suffered a similar economic fate to the miners themselves, although they were still more assured of moderate, relative wealth than the miners. I could find no examples of anyone who had come along later with "a better shovel" who became extremely wealthy as a result. Being there first seems to be key.

The man who became the richest guy in California at the time was Samuel Brannan , who is the focus of the adage about shovels. In 1848, Brannan found out about the gold discovered at Sutter's Mill through people buying goods from his store using the gold they'd panned during their spare time (he also collect Mormon tithes in gold). Brannan was no 'poor store keeper' - he had established a couple of newspapers and had the store in Sutter's Mill and was the Mission President of the Californian Mormon Mission. Using his capital, he bought up as much prospecting equipment (including shovels) as he could in San Francisco and then, brandishing a bottle full of gold dust, ran through the streets shouting, "Gold! Gold! From the American River!". He created such a sensation that one of his papers couldn't publish the news as all the staff had left to look for gold. Brannan used his income to buy and sell land during the ensuing Gold Rush.

So what can we learn from this story?

Be observant and find something considered valuable being done by other people
Monopolize required, necessary resources before you....
Generate unreasonable demand through sensational promotion of the thing and then...
Charge over-the-top margins whilst you...
Re-invest the profits in further monopolisation

I feel so much better about this analogy, now.

Leadership: Listening, Understanding and Disagreeing At The Same Time

2010-11-01T05:32:00.000-07:00

Reading the contents and comments on Paul Dyson's recent blog about "the Whole Team" and eXtreme Business I was reminded of a piece of dialogue from the West Wing: Josh and Toby are walking down the street arguing, and the conversation goes something like,

Josh: You're not listening to me
Toby: I am listening
Josh: But you're not understanding me
Toby: I do understand
Josh: But you're disagreeing with me
Toby: I am doing all three: I am listening, understanding and disagreeing simultaneously

Conversations of this form often seem to crop up in the context of running a business in an eXtreme manner and dealing with the "whole team". I think there is often a misunderstanding of the practice of "whole team" , confusing participation with democracy. If you think "whole team" means 'democracy' you're ignoring the factors of responsibility and accountability.

Running any business or team requires both leadership and management skills. Leadership provides the vision of the direction and formulates the goals for the group, which they can then work on together (so becoming a team). Management defines the constraints (like budgets and resources, choice of technologies, etc.) and ensures that there is consistency and achievement of the agreed goals over time. Both of these require a lot of listening and understanding of what the group has to say. However, the decision about what the direction, goals and constraints for the group must be made by those responsible and accountable for the outcome of the group's efforts. This is often where the disagreeing comes in, usually because participants think their opinions (which have been solicited by good leadership practice) have been ignored rather than appreciating there are often a huge number of factors they have not taken into account that outweigh their position. To me, this is one of the big differences between a team and a group - in a team, members understand that the have been listened to and appreciated even when the decisions seem to be against their thinking, moreover, they continue with the same amount of effort and don't throw down their teddy-bear.

My attitude towards leadership and management is strongly influenced by John Kotter's early work. Part of his position was that it is hard to find individuals who have both the leadership and management skill-sets sufficiently developed and balanced to provide that mythical "charismatic business leader" whose presence guarantees success. From my experience I think this is true - individuals are usually more one or the other - however, I think the Kotter model overlooks the practice of having a pair, or a trio, running a team or business.

Having a true pair working together means that one can be undertaking leadership practices while the other balances with management practices - often these role swap backwards and forwards between the two on the scale of minutes, hours and days. With businesses or large groups, I think three is a better "responsible team" size - this provides the extra mediator practices sometimes required to balance the conflicts that can occur between leadership and management goals. I have worked with this model both at Connextra and e2x and found it to be a most effective model. Though I don't ever say it explicitly, I also try to influence any teams I have created or work with to form this same triumvirate structure for the responsible group. Needless to say, with three it is often the case that one is listening, one is disagreeing and the other is trying to understand how to resolve and synthesise the situation (allusions of classical thesis, anti-thesis and synthesis). However, I find it works better than other forms of organization.

Domain Specific Language Snobbery

2010-10-07T01:58:00.001-07:00

At JAOO, James Gosling gave a talk before mine on his recent experience with the Autonomous Audi TT at Stanford. Within this presentation he talked about the use of Matlab and Simulink as Domain Specific Languages (DSL) for mechanical engineers. The tone of this, and even more so talking with people afterwards, was that somehow DSLs like these are not 'real languages' and are to be sneered at for their lack of scoping or inability have closures. Even the fact that when the translated code (from Matlab to Java) was shown to the engineers and they're said it didn't make sense to them was seen as an amusing anecdote about how misguided mechanical engineers are about software, the poor dears.

What unmitigated, bourgeois snobbery and arrogance.

All programming languages are DSLs, you dummies. Its just a different domain - that of current models of software. You are no better with computers than these people you sneer at. There is more of their code controlling the world and making it a better place for the majority of humanity than there is for the systems delivered by so called software engineers. We should be listening to them and helping them, not sneering at them and congratulating ourselves on our superiority. Need to destroy the 1v0ry towers we're inhabiting and learn a little humility.

Here Be Algorithms

2010-10-07T01:33:00.001-07:00

I gave my "need for speed" presentation a JAOO in Denmark on Monday (4 Oct 2010). It's an overview of using "exotic hardware" like FPGAs, NFPs, SIMD processors and GPUs for accelerating applications. Must admit didn't feel I gave my best performance (sorry to attendees for that) (Update: got my audience feedback and I got a "great presentation", top marks! Shows what a judge I am) - it was a bit of a formal set up with banked seating and bright lights pointing at the presenter so it was difficult to see the audience and engage them.

Having said that, the questions and discussions afterwards did re-emphasize a belief (prejudice?) I have had for some time : algorithms are a weakness in current software developers. In fact, I told a story of using Linear Programming to solve an optimisation problem rather than using the brute force approach used by the original programmers. Of the 9 people listening, all software developers, only 2 were aware of the technique and neither of them knew how to even approach solving a problem with it. I found this shocking. Although I shouldn't be: in my experience I find software developers are unaware of many techniques to solve certain types of problem effectively and efficiently.

I wonder whether this is a co-evolutionary outcome of computers becoming more computationally powerful and programming tools becoming more accessible and productive. It has become possible over the last 10 years to solve bigger problems brute-force within the cycle-time programmers are comfortable operating within (you know: the code/compile/debug/coffee cycle). As a consequence, it seems to me the emphasis on good algorithms has faded. Additionally, the removal of technical requirements "solved" by things like garbage collection and the misinterpretation of techniques like "do the simplest thing" seems to have lead to a place where algorithmics are "Terra incognita".

I think their are visible signs of this when you start investigating systems with performance issues. Most commonly these days I find the problems are due to the algorithm used not taking the computational constraints into account. I see everything from creation of unnecessary amounts of garbage to disregard for the effect of network traffic. There seems to be a belief that the system should "solve" these problems for you - and more often than not when I give this "exotic hardware" talk the hope is that these solutions will magically solve performance without the need for the developer to make any changes to their simplistic approach to algorithms. Unfortunately, this is not true, and furthermore I see no chance of current development tools being able to hide the impacts of these newer hardware architectures from the developer.

Some of the people I talked with were students who told me that they have never had any kind of education about the practical implementation of algorithms. They have done classic computer science algorithms (bubble sort, etc) and have been taught programming languages, but they had never had to join these two things up! In fact, when asked, I could think of no reference apart from the greet 'numerical recipes' books - and they don't cover the use of higher level algorithms like Linear Programming or Map-Reduce or Data Parallelism (I.e. How to implement create a model to utilise these approaches for efficiency purposes).

So it seems to me there is a big hole in software development currently: algorithms, using them and implementing them. I feel that we are all the poorer for it and fear it may actually be limiting us. In fact, I think that we are going to hit a technological speed-bump when shortly we need to start using hardware that incorporates on chip SIMDs and FPGAs for performance reasons. The compilers will not be able to save us. We need to go back to a more basic algorithmic approach to get the advantage, and it seems to me current software developers have lost or have never gained this skill.

- Posted using BlogPress from my iPad

Thoughts on John von Neumann's "The Computer and the Brain"

2010-08-02T03:41:00.000-07:00

I like to read old computer texts to get a different perspective on the technological context we find ourselves in today. Given my current obsession with how the brain works (or doesn't, rather) it struck me that I had never read John von Neumann's "The Computer and The Brain".

In 1955, von Neumann was invited by Yale university to give the Sillimann Lectures in the spring of 1956. For his theme he chose, "The Computer and the Brain", a subject he'd been interested in for a number of years. Sadly, von Neumann became too sick to give the lecture series - in fact, he never finished the manuscript before his death in February 1957. Being only 82 pages long, it took me less than an hour to read. Yet for all its brevity it was extremely thought provoking...

why are we emulating neurons in software when their may be a hardware equivalent? This is not a unique thought - Carver Mead did some work in the 1990s which has lead to the Neurogrid project a Stanford. (Mead is the guy credited with coining the term "Moore's Law"). The Neurogrid chip emulates ion-flow across a neuron's membrane using electron-flow through a transistor (its the same physical forces after all!). Seemingly, they can emulate neurons with as few as 8 transistors. The chip therefore is populated with these hardware neurons instead of the usual logic gate configurations of a standard CPU. Seemingly, they get a hugely more efficient emulation of neuronal activity.
much emphasis is put on the neuron, leaving the brain as a collection of neurons rather than considering it as a whole complex system. It is a continuously running system which has a number of physical and chemical processes being applied to the neurons. Von Neumann uses the analogy of neurons being like AND/OR gates for nerve impulses from 'upstream' neurons (with a bit of non-linear sigmoid functions thrown in). This is a view I've never been comfortable with, but in my experience it is still the prevalent belief amongst technologists.What about the effects of neurotransmitters? What about the physical effects of increased blood flow when areas of the brain are active? (surely the physical movement of neurons will effect the excitation patterns?). We seem to think of the neuron as a node which has some consistent, simple behaviour but when in reality it is a complex item in its own right and is a member of a complex system where it is behaving in a stigmergic manner. (NB: stigmergy = where individual behaviour is effected by and affects the environment, producing larger scale emergent behaviour amongst independent agents)
ignoring the power issue (brain ~=20W, modern single CPU ~=80W) how close are we to emulating major neuronal activity with conventional hardware? In 1989, TJ Sejnowski wrote a paper revisiting "The Computer and the Brain" which predicted that by 2010 we should have computer that could emulate the "minimal processing capabilities of the brain", which he put at 10¹⁵ operations per second. Let's be generous and just call these petaFlops (I know they're not floating point ops, but this is only a blog...) . Currently, the Top500 list has 6 supercomputers peaking over 1 petaFlop (in fact the first petaFlop machine was around 2006) - these of course are using 10s of thousands of cores and drawing megaWatts of power!! There is work going on with these, and there was the controversial "big as a cat's brain" paper last year. But what about more realistic hardware, say something I could run on my desktop? An NVidia Tesla M2070 can produce about 1 teraFlop (and its SIMD so suitable for neuronal simulation) so surely it could be used to simulate the neuronal activity of a simpler organism such as an ant (~10,000 neurons) ? (During this same investigation I came across Steve Furber's (the ARM chip guy) involvement in the SpiNNaker Project - using arrays of ARMs to simulate neuronal activity. Looks interesting , too)

101: (Lack of) Progress Report

2010-08-01T04:01:00.000-07:00

As the engine isn't moving on the starting handle any more and working on the maxim "try everything else before stripping the engine" I began removing and checking...

As its likely the clutch is stuck and for some reason the gears were engaging (even though it seemed to be in neutral in both gearboxes) I decided to address the known problem of the hydraulics not working for the clutch. Stripped out the master cylinder, which amusingly requires the removal of the pedal housing which is bolted to the front panel of the 101. This made me think of just how close to death I am when driving this vehicle - is there a negative star rating for crashing testing? (see diagram)

Anyhow...turned out the master cylinder was gunked to high-heaven, so with some penetrating oil and some huile de coude it freed up. Reassembled the master cylinder and the pedal housing in to the vehicle then went to bleed the clutch system at which point I discovered I had no tubing in my workshop or house suitable. Now given the amount of stuff I keep hold of "just in case its useful" this is quite amazing. After a 40 mile round trip to my nearest motor factor (yes, 40 miles), I bled the system and got to the point where I now know that the clutch is stuck - hydraulics are working but the plate won't move. No surprise here, but the 'standard' way to free up a Land Rover clutch that's stuck is to start the vehicle in gear and bounce it until the clutch frees up. Small problem there...

So I then decide that the starter motor may have jammed when trying to start it on the ignition. So following the instructions in the Repair Manual, I dutifully remove the panel in the right-hand front wheel arch, "to facilitate access to the area". This gives me a nice view of the exhaust manifold but no access to the starter motor! Top tip: do not bother opening panel. Removal is best done from underneath the vehicle. So after removing the starter, I tried turning it on the handle again. No joy. Sigh - starter motor back in...

Cautionary note: if you have long hair, as I do, and are using a wheeled crawler board, ensure that your hair is fully secured in a pony tail before rolling under the vehicle and getting a bunch of stray hairs entwined in the wheel of the crawler. No amount of swearing releases you.

As a final prevarication I decide that there is a possibility the gearbox is not going into neutral and that the "stuck" feel of the engine is from the transmission being fully engaged. So I jack up one wheel of the 101 using a high-lift jack (can't find the handle to my trolley jack) and try to turn it by hand, both in and out of gear. Sadly, it moves when in neutral but not when in gear (although its clear I'm going to have to completely strip and redo the brakes).

Arse.

This means that its really likely to be something in the engine and I'm going to have to strip it.