Events like Kevin's code retreats, Black Girls Code, and other initiatives are important to improving our community and software development in South Africa, in general.  Yet, we are still facing glaring disparities in our sector and in our country that I find difficult to ignore.  My fear is if we all take defensive positions, we may never ever discuss such matters openly and respectfully.  Like I said in my twitter conversation to Kevin, this is not about him and not about me too.  It is about us and our future.

The fight for equality is so much tougher when we have freedom.

Update: Kevin Trethewey has extended this conversation on his blog.

I commented on a photograph that Kevin Trethewey shared of the code retreat that was hosted in Johannesburg, South Africa recently.  His response was very defensive and it clearly showed that he was taking my observation personally.  That was something that I didn't anticipate, as I have not seen Kevin carry prejudices of other race groups.

What followed was quite an interesting conversation on the disproportionate representation of non-whites in the South African software community.  I'm going to comment on parts of this conversation.  For the full conversation, go here.

A point to note here is how quickly people focused on racial composition as opposed to gender composition of the group.  That is a problem in itself, even for the most gender-aware man in a software community.

This is a long, long, long post, mostly because it is a brain dump of the themes that were flowing in this conversation.

There  is adequate opportunity and access for everyone

Kevin believes that there is fair access and opportunity for all of us to attend an event such as the code retreat.  I disagree. For example, I know a few black software developers that don't live in a conveniently accessible middle class suburb.  For them, getting to work is a challenge.  To get to a community event outside of the peak public transport service hours is a huge issue.  The same applies to people attending university.  A black kid living in a township with minimal to zero essential services loses at least 3 hours of study time a day when compared to her more privileged peers.  Accumulate that over a month and we find that she loses about 6 days of work a month.  Our lives are not equal in far more fundamental ways than we wish to acknowledge.

It is also a matter of survival, and not that people lack ambition to be better developers.  When we have less than enough of everything that is essential for living, then a meagre salary is a big deal.  Many black developers I know are supporting their entire family, if not an extended family.  When we're in survival mode, ambition is lower priority.

I would love to see events hosted closer to the people that have greater logistical challenges.  That's why Braamfontein was a good venue for Agile Africa.  Maybe in 2014, we will have a software conference in Soweto for the first time.  When I want to help someone out of a dark place, I prefer to go into the dark and walk out with them, taking the bumps together, than to just stand outside and shine a light in the distance to where they should go.

People are apathetic

Simon Stewart was of the opinion that it is a lack of caring - an apathy.  I don't know which group of people Simon refers to here so I will steer clear of making assumptions.  What I have noticed is that the majority of software developers don't care much for self-learning.  The code retreat that sparked this discussion is a case in point.  Twenty seven developers pitch up to practice their TDD skills.  I don't know how many developers there are in Johannesburg, but I guess that this is less than 1% of the total.  Regardless, I don't have a problem with this apathy.  Some people "give a shit" about different things, like earning a salary first.  Because they care less for self-learning, does not mean they don't care at all.  It just means that their value system is stacked differently from others.

I don't believe it is about software craftsmanship either.  I think software craftsmanship as a community appeals to people that share that value system and the way that community engages with others.  I'm not one for the craftsmanship movement for reasons that are irrelevant here.  However, I do respect that this community has a right to exist alongside all the others.  Not caring about software craftsmanship does not mean that a developer does not care about growing.  Some of the most accomplished, self-taught developers I know care far less for craftsmanship than I do.  We need to help people understand their potential, and provide diverse ways in which they can fulfil their potential.  Sometimes people will break through that ceiling, setting up a new potential to fulfil.

Lack of diversity is commonplace

Simon rightfully points out that the lack of diversity is common.  I concur.  I've spoken at a few conferences outside our continent and it is the same in most places, as much as it is within ours.  Again, that it is so does not mean that we should accept this.  Make up your own mind, but I cannot accept that.  I've been talking about diversity publicly for several years now.  I've been on projects where the direct result of us not working with our diversity resulted in a tragic cost of the human spirit.  I cannot accept that we are prepared to allow that to happen again and again and again.

I try to be a lot more aware, a lot more empathetic.  I've been fighting my own unsettling biases to appreciate our diversity, to celebrate our differences and embrace our uniqueness.  The scary part is this very process of appreciation demands that we drop all our prejudices.  To do that requires self-reflection and clarity to face the truths of our deeply internalised, and camouflaged stereotypes.  Be prepared to cry for ourselves.  Be prepared to feel lighter on the other side of that.

Quotas and Meritocracy

Steven asked what he thought was a silly question.  But it is not silly at all - "How should a code retreat represent the demographics of the country?"  For me it comes down to us building a significant software development sector in South Africa.  The white minority cannot satisfy that agenda alone because there aren't enough of them.  We need black software developers too.  Parallel to this, we need fair and equal gender representation too.  South Africa has one of the most humane constitutions in the world.  Yet, I see many software developers adopt the non-inclusive behaviour of frat boys, country clubs and colonial style thinking of the west and north.  That is a learned behaviour that I refuse subscribe to.  Given our history of the inhumanity of apartheid, I find this behaviour equally abhorrent.  Incidentally, I was part of a very diverse group of developers that were reflecting on what we don't like of the software development community, locally and globally.  What emerged were distinct pointers to this imported, inhumane subculture: elitism, arrogance, fear, bullying, sexism, racism, monopolies, nepotism and a lot more.

Steven also dangled the option of quotas.  South Africans are masters of quotas.  We've tried that in almost everything from sports to business.  It turns out that Steven is not offering quotas as an option.  What is relevant for me is the other side of the quota argument - inclusion by merit.  Meritocracy is a subtle form of exclusion.  In Africa, where we have significant disparity on multiple levels.  Meritocracy just widens the gap between those that have excelled and those that may excel but don't have opportunity to do so.

I've been guilty of exercising meritocracy to the extent of building a company based on inclusion by merit.  Being elitist is one of the easiest ways of being exclusive, and oh so easy to cover up whilst staring in the face of everyone.  What changed for me was a change in my attitude.  I now carry an attitude where I respect the potential of each individual.  I realised that we need people to contribute in their own little way, acknowledge that contribution and then help that person raise their own ceiling.

We must challenge the bias of minorities

Here is Steven's alternative to quotas.  I ask that we not flame him or attack his position.  This is his position and I want us to respect that.  If I want to change Steven's mind, then I need to offer him alternatives, not antagonise him.

I agree that tech events are open to everyone.  I don't accept that non-whites and women - the "minorities" - don't attend because they hold prejudices against the pale males.  I think the problem is to do with accessibility, opportunity and a necessity for survival prioritised higher than self-learning, as discussed above.  By the way, Steven, as a white male, you are the minority.

Yet, there is a strong element of intimidation.  This, we cannot ignore.  I've been on the receiving side of intimidation.  It is subtle and, strangely, more powerful in its exclusionary effect.  It exhibits itself as quiet arrogance, and other times as open boorishness.  Tech conferences are testosterone events where geeks flex their muscles using absurd measurements of superiority like "my big data is bigger than your big data" or "I've got more pull requests accepted on blah-blah-blah repo than you".  All it takes is one moment of acknowledgement, a moment where there is a little light shining on us and people smile and say that we are awesome.  It is that moment when we pack the soap box in our suitcase and take it everywhere.  From that moment, the superior position is taken and others struggle to fit in, white or not, male or not.  Tech events are open for those of us with soap boxes. Oh, and you must make sure your soap  box is bigger than other soap boxes.

This is why I promised myself to always step off the podium and be accessible to everyone in the room and beyond.  This topic of soapboxes, self appointed popes, cardinals and priests of industry, blogging, tweeting, etc deserves deeper discussion more than this already far too long blog post.  And, yes, I know that my irony is that I'm blogging and tweeting too.

Tell me when you're done talking

I was uncertain by what Kevin meant when he asked me to let him know when I'm done talking.  It would have been easy for me to fire a flame attack along the lines of "WTF makes you think I'm all talk and no action?".  That's not useful because twitter is a horrible medium that draws assumptions out of people.  So, I just asked for clarity first.  This is important; a behaviour that I try to instil in myself all the time - pause, listen, think and ask for clarification if I have even the slightest doubt.  That double check saves me a lot of unnecessary flaming.  I also found that asking for clarification in the face of cynicism does defuse the situation. Then again, rushing around with a burning flamethrower, defending every single one my opinions is a horrible way to spend my life.  It doesn't take me forward in the least bit.  Worse, still, is that it causes me to stagnate.

I do try my best to not draw public attention to the work that I do.  I respect the privacy of being in a group and have learnt a long time ago that it is self-serving to tweet about such work.  As a mentor to a few people for many years, I respect that unspoken code of confidentiality.  Being a mentor, for me, is a two way street.  I need to be equally open and honest as I expect of the other.  Sometimes, that means being quite personal and deeply private.  So, I don't crow about my work.  But I blog and open conversations based on observations which I've extracted from my time with others.

Those that work closely with me know the extents to which I go to change the status quo.  Then there are those that are even closer that know the extent to which it pains me when I can't change the status quo.  At a deeply philosophical level, is it possible to give unconditionally - without even the tiniest drop of self-serving satisfaction?

Celebrate the success

Mark Pearl, on a separate conversation stream, reminded me that that progress was being made.  That is true, there is progress being made.  I mentioned that at Agile Africa that the demographics are changing, slowly for the better.  I think we are moving forward and I am not so concerned about the pace either.  There is a part of me, though, that is cautious of grand celebrations for small steps.  That is just me.  I prefer constant acknowledgment of progress for each tiny step.  It keeps me humble, knowing that the big picture is still not fully painted.

Can I plug in, please?

Hearing that I am doing something, Kevin invited me to breakfast to hear more so that he can plug in.  I'm happy to share with most people, but this work isn't about plugging in.  It is not a thing at all.  It is about my individual behaviour and attitude.  There are no recipes that I can offer to make this better.  It requires me to find a connection with myself.  It is about holding a heightened sense of self-awareness, of each thought, each action and each consequence.  It demands a level of honesty with myself that is, at times, extremely hard to face.  This is a journey that I've been trying to walk.  It is painful to look in the mirror and see beyond the reflection.  I am motivated by a belief that we have a primal encoding for decency and an inclusive humane culture.  Everything else is an invention that is designed to obscure that basic encoding.

For more than 50% of the time, I get it wrong - my behavior and attitude.  So, I cannot stop here and I must persevere with this.  This is why demographic representation is important.  It is a measure of our collective shedding of invented biases and a statement of us including each other without prejudice.

To Kevin and the others, thanks for opening a powerful, multi-faceted conversation.  And if you want to plug-in to this inclusive, humane culture, then you will need to plug-in to yourself first.  This is not something I can help others plug in to.  I can only behave in a way that I want our software community to behave.  I believe that others will change in step.

Excerpt from Circular 9/2013. Source: dkeats.com

That Delphi is old is irrelevant.  For me, the issue is that Delphi is not current.  Old and current is not the same.  For example, Python is old.  Python v1.0 was released in 1994, but it is still current.  It is still under very active development and a popular choice for building software in today's world.  Delphi is old and has tried desperately to keep up with the times.  Delphi is no longer current.  Please do not bother pointing me to  Embarcadero's Delphi XE5.  Do your own homework on its market share and developer mind share.

Currency makes the things that we wish to achieve today and tomorrow possible.  Currency of software languages and tools instills wonder and amazement for the current generation and next.  It was C that did it for me when I was starting out.  That I could make LEDs flash sequentially when connected to a parallel printer port, left me breathless.  Delphi excited some people in 1996.  It excites nobody anymore.  It has lost its currency decades ago.  It no longer inspires people that imagine possibilities in the world of cloud, mobile, touch and the Internet of things.  

Certainly, we can teach kids programming using Delphi and Pascal.  We've taught programming with Pascal for years.  And, yes, design is the hardest aspect of programming and we can argue that language is an implementation detail.  This argument may stand up in the generality of teaching programming.  But it falls flat when it comes to specifics of teaching programming in South Africa.

The reality is that South Africa does not have a significant software development sector.  As one of the BRICS we have an expectation that our software development sector should be comparable, in relative size, with our "developing" sisters.  Get real. We just don't have enough developers in South Africa to build a significant software development sector.  What we do have is a marginal sector where the majority seats are occupied by a minority group.  I can only foresee a significant software development sector when the majority of seats are occupied by the majority group in South Africa.  Let me cut through the euphemisms.  The majority group are not white.  The minority group is predominantly white.  But this is not a race issue, it is much deeper than that.

We are in a software development crisis in South Africa.  We do not have enough software developers.  Forget talent, simply on quantity, the minority group cannot fix this situation.  We need black software developers to fill the seats, first with numbers.  The black talent drought is a different problem, but let's just stop the black software developer drought.  We need the numbers first.  The majority of our kids go through state schools will be left uninspired by Delphi, and turn to other careers. Software development is not even close to being a common career option for all South African kids, and the Department slams the door shut. 

Kids in private, independent and previously advantaged schools will be ok.  I know this because my son attends a private, independent school.  These kids will be inspired.  They will feel that sense of wonder and will reach for a career in software development.  And we will continue fooling ourselves building a software development sector with a minority group.

I wonder whether the Department of Basic Education knows that it is responsible for the under-development of South Africa.  Yes, Mr. SG Padayachee, you and whoever is advising you are killing an entire generation of software developers.  It is a genocide of software developers.

I've been taking a poke or few at SUGSA on twitter in the last few weeks around the2013 South African Scrum Gathering.  It's time to put it into perspective.  My proposal to present a session at the conference been accepted and I have declined this acceptance.

This is not news to the SUGSA committee.  I have communicated my decision and my reasons to the SUGSA committee.  I have also had one-on-one discussions with a few committee members.  I am satisfied that the committee has received and digested my reasons.  Individuals on the committee have assured me that my feedback is in line with their values and that they aim to address it in the future. 

My poking is just me pressuring SUGSA to recognize a few issues which I believe are important to our community, and to force the committee to address us on their position.  Tell all of us, not just me, of your position and we shall listen.  That is all, no more, no less.

These are the issues that I raised with SUGSA.

• African conferences need to be representative of our community.  One way is to nominate an African keynote.  It will take a courageous decision but it can still happen.   Agile Africa responded symbolically very late and it was fantastic.
  
• We must embrace our diversity and find ways of breaking the polarised community that exists in Cape Town.  I think the people from Cape Town that attended Agile Africa in Johannesburg were surprised by the diversity of speakers and audience at that conference.
• Be transparent with the review process.  The committee needs to maintain its independence and having committee members occupying several speaking slots sends the wrong message.  I've seen that in other communities where the committee became the soapbox for individuals.  If I was on the committee, I would withdraw my proposal on principal.  I know of one committee member that did this.  Thank you for taking a courageous step.
  
• I also questioned the expertise within the committee, especially for technical submissions.  That is my perception and there is every chance that I might be wrong.  I welcome the correction. 

This is not a boycott.   I will still attend as a paying conference attendee.  I appreciate the learning I receive from this community and look forward to more of the same.

I believe my issues have been heard and my actions, seemingly harsh to some, was necessary.  I still want SUGSA to take a position publicly and I leave it them to figure out when and how.  The SUGSA committee have invited me to discuss this further and  I look forward to that.

Thanks go out to Austin Fagan, Sam Laing and Karen Greaves who gave me perspective to resolve a few internal conflicts, and shape my thoughts on why I am doing this, and what I expect out of it.  Had they not given input, this could have been unnecessarily ugly.

The actual problem is that I was irresponsible.  The situation called for precision in my design and I was being casual.  I focused on the plumbing (i.e. HTTP requests) and not the underlying questions that needed answers.  For example,

• How big a buffer do I need for the data set?
• How frequently must I poll for new data?
• How frequently can I push data out? 
• How can I reduce the push rate if the receiver cannot accept data faster than I can give it?
  

One very precise model comes ready made and is known as the leaky bucket problem.  The leaky bucket solution is applied very often in low level networking solutions, but is just as applicable in my problem space.  Now before you roll your eyes and fake a yawn at the maths behind this, hear me out.

The moment we go into higher order control systems, we need higher order maths to build precise mathematical models of these systems. That is not the maths that I'm suggesting we chase.  Instead, with a little bit of creativity and some strategic design constraints, we may be able to reduce many classes of systems to first and second order systems. In other words, introduce constants until you have one or two variables that you can tweak.

This is exactly the situation that I had.  Sure, there were more than two buffers to be managed, but I was in a position to fix the size of some buffers or the flow rates.  Then, the buffers and flow rates that were critical to me could be modeled with straight forward linear equations.

Had I done this, then I would have been in a position to have built a system that could have adjusted itself to a steady state, without frequent reconfiguration.  The fact that I was casual about my design led directly to a case of the pipeline being in a non-deterministic state.  This problem was highlighted early when the users started asking for different kinds of "status reports" for data flowing through the pipeline.  Of course, being casual about it, I treated it as a feature request and implemented a few reports.

This is when maths and science make a difference in software development.  Unsurprisingly, mathematical models are generally deterministic, self contained (i.e. highly cohesive), and at the right level of abstraction for the problem.  And all of those characteristics lead to highly testable models that you can do in wee little steps, test first.

That's why it matters to have a maths background.  And if you came into software development through some other route, then do some layman studying of algorithms, control systems and simple higher order maths.  It will serve you well forever. It will certainly give you a design advantage when you need it most.  Right now, I'll take any advantage because design is just so darn difficult.

Before I go off bleating about how this bothers me, I thought I'd understand this thing called a keynote.  It originates from music, a capella in particular.  At the start of a performance, someone sang the key note and others followed, calibrating to that note.  In a similar vein, a conference keynote is meant to set the tone for the conference.  It lays the foundation for the other speakers to build upon.  It calibrates the conference for the speakers and attendees.  If the keynote inspires, all the better.

What should we expect of anyone that is giving a keynote?   The first criteria is that the person must have good insights into the subject of the conference.  The person does not need to be an expert, but should have a good understanding the subject and the context thereof for the audience.  Second is to have the ability to deliver a decent talk, one that shapes the conference.  Being a great speaker is either a talent or a learned skill. Either way, a person gets better the more they speak.  Lastly, the person must speak with passion, from the heart and create an energy that inspires, even if it only lasts for the duration of the conference.

What, then, does it take for someone to give an African keynote?  I think it is the most important to understand Africa, her people, her history, her challenges and her future.  Without this, an African software conference lacks context for our the challenges and constraints.

It's great to hear someone from Amazon or Facebook about their grand ultra-scalable, cloud infrastructure.  Sure, now try that in Africa when a break in an undersea cable is the equivalent of severing your carotid artery.  Then try that in Africa where the minimum living wage is equivalent or less to the bandwidth costs for a month for an average middle class home.  How do you build a scalable cloud infrastructure that reaches into the tiniest village in, literally, dark Africa.

Forget technical challenges, let's take agile conferences where trust, collaboration and self-organising teams are common themes.  And magic and transformation too.  If you grew up in Africa, then you will understand that trust is not a commodity item.  When an entire continent of people have been exploited as slaves for centuries, cultures systematically eradicated for the wealth of privileged few, establishing trust and collaboration is a different ball game.  Try transforming a team with distinct divisions, each with deeply entrenched prejudices for each other.  This is Africa. She is not your average, homogeneous nanny state that tells you what to think.

I believe that we do have people in Africa that easily meet all these criteria, and we are certain to have more in coming years.  A couple or years ago, ICSE was held in Cape Town and Archbishop Tutu gave the keynote address.  Here is a person of Africa addressing the academic elite of the software world, and he is not even a geek.  We don't need to have instantly recognisable names.  From the top of my head, here are three names that you may or may not have heard that are more than qualified to keynote African software development conferences.

• Enyo Kumahor
• Herman Chinery-Hess
• Jonathan Jansen

Some technical topics transcend culture.  For example, object orientated or functional programming can be learnt regardless of who we are and where we come from.  And in a localised way, a few refactoring steps to clean out a piece of smelly code doesn't impact on our culture or being.  That I can accept.   What I cannot accept is when this reasoning is abused to deflate the situation of under-representation of our people. Let technical things be technical, but we also need Africans to be technical leaders. That is not the topic under discussion here.

This is important. Whether they like it or not, every African keynote speaker creates the path for many to walk.  Even if it is not a keynote, an African that shares a room with a few people gathered around, listening and engaging with what is being shared is cutting down brush and laying a new path.  These will become well worn pathways.  And each traveler walking on it will see virgin brush that needs to be cut, and a new path will be laid.  An African keynote can have this effect, if we allow it to happen.

I have a challenge.  Next up is Agile Africa which will happen in August 2013.  This is the keynote lineup:

• Martin Fowler
• Mitch Lacy
• David Hussman
• Amr Noaman
• Ivar Jacobsen

Amr, from Egypt, is the one African in the keynote lineup.  That is fantastic. I challenge the JCSE to rather reduce the number of keynotes to just two or three and nominate other Africans to stand alongside Amr Noaman. 

In October is the Scrum Gathering in Cape Town.  Here's the keynote lineup.

• Geoff Watts
• Dave Snowden
• Alexander Kjerulf

I challenge the SUGSA committee to offer these speakers a regular slot and give our African community African keynotes.  I am not asking for the current of the keynotes to be rejected.  I want these experts at our conferences.  I want to attend their sessions and learn.  I also want us to realign them to an African context, that broadens their perspective.  We share, we learn, we teach, we grow - it's all the same.

But  I cannot accept that there are no African keynote speakers to be found.

In the years that followed, my career always surfaced a blatant fact.  I was always the minority in a white, male dominated industry.  In my first job I was hired by people that also despised apartheid.  For that I am grateful because it gave me a chance to learn without fighting for place.  But the places in which I plied my trade were dominated by white, male developers and managers with an attitude of superiority by entitlement, birth, law and any other decree.

And today I am still the minority in a white, male dominated industry.  I don't need hard stats and precise surveys when it is patently obvious from the demographics at community events, conferences, start-ups and cubicle farms that black software development has not gained a foothold in our industry.  It is especially so in Cape Town, and only marginally better in Johannesburg.  I can't speak for Durban because I haven't worked there since 2005 but it was not any better then.  It is not just South Africa.  It is the same in Europe and North America.

 There are at least two sides to any story. Here, the one side is with corporate South African software development as a conduit for opportunity or suppression thereof, and the other side lies with the black software developer with messed up priorities.  By the way, when I say "black" I mean "not white", and I everything I say is a generalization because I believe it is in the majority.

In my experience corporate South African software development perpetuates apartheid.  There are official policies and frameworks for employment of people and for furthering their careers.  There is nothing wrong with these frameworks. But like all rules, how we use them matters.

A few weeks ago I had another black developer share his entrance into a prominent consulting company.  He was offered an internship whilst a white male of equal qualification was offered a full contract.  When his internship was over, he was not offered a contract on the grounds that his line manager did not like his "work ethic".  Really? Who's work ethic needs to be questioned?  Today, this developer is employed as an equal in more progressive organisation with a highly respected brand that is determined to breakdown apartheid in South African software.

I am yet to meet a black software developer that is leading a team, making the call on design decisions.  Along the way, I have met many blacks in leadership positions. Many are tokens and lack the leadership and technical skill to justify their position.   This is also the fault of these individuals, exploiting the rules for self gain.

The majority of black software developers are either doing "maintenance" or being instructed on how to write to specification or requirement with a full blown design being given to them.  I have met many black developers who I've seen argue their design ideas only to be later classified as not being "team players".  This also occurs in the sugar-sweet agile "we-love-transformation-i-love-my-job" work places.

What gives credence to the "inferior" black South African software developer is that black software developers are apathetic.   I find it difficult to identify with someone who voluntarily accepts being treated as a second class citizen.  I can understand the risks of 20 or more years ago, but life today is different.  We have a voice and we have a deliberate choice of the battles that we can fight.  This is mental slavery all over again.  And to some extent this mental slavery is self imposed.

I can appreciate that someone with a history of poverty that is offered a monthly salary that their parents could only ever earn over several years, is driven by a more basic need and deep empathy.  It goes horribly wrong when that is all there is to it.  You earn your dough, but you don't earn your stripes.  I know several black developers that have doubled or tripled salaries in a few hops inside of two years.  These developers do nothing but give credence to the ingrained belief that black developers are not skilled.

I am fully aware of black developers working from a weaker maths and science base.  But I am also aware that there are white software developers that also don't have a strong maths and science base.  I have wondered about this difference: two people with equally weak programming prerequisites yet the black developer struggles and the white developer excels.  I don't think spoken language is a significant variable here.  The one thing that I have observed is that the white developer receives more opportunity and attention than the black developer by his white leadership.  The black developer in turn believes that they are inferior, and the lack of opportunity and attention furthers erodes self-belief.  Self-doubt is by the far the greatest obstacle to learning.

If you have read this far, there are several among you that have already formulated counter arguments, specific cases that justify your position too.  I am open to hear your side.  The problem with this blog post is that everything until now is just background to create context to put forward my belief.

I believe that there is a wealth of talented software developers in Africa that understands our history and our future. I believe that as a Pan African software community we are capable of designing frugal solutions for our people and for the world.

To achieve this, we must break free of our own mental slavery.  We need to build our own solid foundation that is unshakable at our core.  We must equip ourselves as masters of our craft.  We must let our voices be heard in Africa and the rest of the world as voices that speak substance.  And we must return to be with our people as no more than just people.  Our first step is to rid ourselves of apartheid in software development.  Like Marcus Garvey said and Bob Marley sang "none but ourselves can free our mind".

I dream that the next thought leaders of software development will be African.

I remember Dan North explaining BDD stories to me when it was just a thought in his own mind.  That was around 2005 or 2006 and I remember being so inspired by the simplicity of the BDD grammar. So, seven or eight years later, let me share how my use has changed.  And, Kevin, I hope you enjoy your own journey too. It's a lot of fun.

Who is the best judge of a story?  The customer is unlikely the best person to articulate these stories, nor judge the quality of the story. I have to guide them and extract that. I now ask the following questions.

• Who are they?
• What do they need?
• What do they think they need?
• What do they really want?

What does the story describe? Of the above questions, the last is the most powerful for me.  It balances my perspective. It stimulates creativity and moves me from the problem space to solution space. The story then exists in the solution space; i.e. it is now reflects a design intention, not a requirement statement in the problem space.

BDD stories are great conversation artifacts. It's like a book on a coffee table. It stimulates conversation.  It is of same value as using a metaphor. In conversation with the customer, the story is mostly about things in the problem space. In other words, it is an analysis and clarifying tool. I found that direct, literal and very early use of this analysis statement as an executable specification results can result in brittle tests.

On the use of technical references. When I'm working in the design space and writing design stories, then I don't mind if there is reference to a technical implementation such as a login screen.  At some time, I have to get concrete. I like getting concrete very early and abstract from that. It's just how my mind works. So, if there is alternative authentication mechanism (say, LDAP or ActiveDirectory), then it is just another concrete implementation. If the authentication choice is an exclusive one, then, the abstraction of proprietary authentication and ActiveDirectory authentication doesn't offer any benefit. So, I'll just go for one of those and the story on task board will make reference to the technical aspects directly. It's a great reminder of a design choice that is explicit to everyone.

Most stories start out as bad stories. My early stories in an unfamiliar domain are awful.  Like code, the story should exhibit single responsibility. That takes a lot of domain insight and discipline. Unfortunately, refactoring stories towards single responsibility is not trivial. It's not as simple as extract class/method/variable. The result is that my story based test suite is in constant turmoil longer than it is calm with a few small ripples. For this reason, I use the story grammar as a conversation piece, but not as code artifact.

BDD Stories on the backlog. To avoid confusion about when the story is in the problem space or solution space, I don't use BDD stories on the backlog. I prefer the XP style of a short phrase as a reminder of something to discuss.

On the use of outside-in style testing. I like outside-in to analyse the problem space, but I often find it equally valuable to evolve the design from the assertions inside-out.  I oscillate between the two perspectives rapidly and quite often. I think I'm searching for that harmony in perspective.  I then make it a choice to use the BDD story as an executable test for the outside in perspective. Often, though I find it unnecessary because I already have tests that reflect that perspective; it's just not using the BDD grammar. Yet, the BDD grammar was a starting point. I just am not fixated on the BDD grammar being the ending point.

On the BDD grammar. From a language perspective, the BDD template is a general grammar that can be used to express any domain.  Just like we can use a general purpose language to solve any problem,  the BDD grammar can be used similarly. Yet, we have learned that domain specific languages can be more expressive of a particular domain.  Equivalently, I keep my mind wide open, looking for a story grammar that is domain specific.  For example, in a time sensitive domain such as investment portfolios, I might extract a grammar that expresses time as a first class citizen. There won't be a "When" clause. I might have something like "At time t(0), the portfolio capital amount is..., at t(n) the portfolio has ..., at t(n+1) the surrender value should be ...".

Remember, these are just reflections and observations about myself. Please don't treat it as a gospel. You have your own journey that takes different pathways. Just enjoy those excursions.

Below are just three techniques that are quite commonly used by Scrum teams.  I steer away from them at all costs.

• CRUD.  I find that thinking in terms of these database operations removes a lot of the richness in the domain.  Instead, I think about the life cycle of things.  For example, there is no CRUD for an invoice.  Instead a customer buys something which means that a sales person issues an invoice. The customer pays the invoice. Perhaps, a debtors clerk requests payment for an overdue invoice.  These are all different things that can be done with an invoice at different times in its life.  Note also that "creation" is a very special case in any life cycle, and to bring something into existence that maintains integrity is quite an effort.

• Dependent Stories.  I try to break all dependencies between stories.  I've found that looking to create "stand-alone" stories results in some very deep and powerful analysis of the domain.  Inadvertently, you will crack open a crucial part of the domain.  Often the concept which holds several stories together in sequence turns out to be orthogonal to the original stories.  For example, there is a workflow for invoices (issue, authorise, pay, remind, resend, etc) that can result in several dependent stories.  Alternatively, we can model the invoice state (and operations allowed for each state) independent of the sequence(s) of states.  Now we can build software that deals with specific sequences, independently of the operations for each state.  This separation can lead to such powerful discussions with the domain expert.
  

• Job Functions: I've never found job functions to yield useful modules.  Extending the invoice example above, a job function breakdown could be on sales (create, authorise), debtors (record payment, payment reminders), customer service (credit notes), marketing (cross sales campaigns).  Now each of those job functions work with invoices in some way, but the conceptual integrity and cohesion is stronger around the invoice and its states.  Designing good modules is by far, the hardest part of any software design effort.  Get it wrong and it will hurt.  More often than not, it is just too costly to try to create new cohesive modules from code that is already laid down along job functions (or any other weak boundary criteria).

There are significant consequences to splitting stories in the solution space.

• The product owner just needs a simple phrase or sentence that describes the problem space, but remains part of the feedback loop for the solution space stories.
  
• Backlog grooming becomes an exercise in understanding the problem space. 
• Sprint planning blitzes (one day or less) is not sufficient.
  
• To be effective, sprint planning becomes continuous; i.e. design is continuous
  
• Each story can (potentially) be released on completion
• Sprint boundaries (time boxes) become less important moments in time
• ... and you will tend towards continuous flow.
  

So, I'm quite thrilled to be a contributor and collaborator for their Advanced Agile Developer course.  The inauguration course happens on 19 November. That's not much time, so book a spot quickly.  I don't want to give too much away, but I can tell you that it's going to be fun, intense, and inspiring - and seriously code centric.  There will be times when you will feel like you just don't know how to code anymore, and then feel like you can conquer the world.

I think this is a course with a difference.  We want to bring together developers that are already competent at writing code and want to become proficient at being agile developers.  We made a big decision to go deep, and not skim the surface of lots of topics.  The result is a course that is very code centric, working at quite an intensity that passionate developers will find inspiring.

Working with Lorraine Steyn and team KRS has the same warmth, openness, and security from 20 years ago. This is something that I know they will bring to this course in a way that I can only hope to, someday, emulate.

I ended up in a twitter conversation last weekend about estimation and velocity.  It started when Annu Augustine asked about what qualities to look for in a lead developer, other than technical skills.  One of the qualities I put forward was accurate estimations. This went around a bit and ended up, not surprisingly for me, at velocity.  There are a couple of points that I need to offer my opinion on:

Accurate estimation is not an oxymoron.  Let's just get over some technicalities first: "accurate" is not the same as "exact".  Estimation in software is never exact, but the magnitude of accuracy is very significant.  If I say that something will take approximately 2 weeks to achieve, then I need to qualify "approximately".  What is the magnitude hidden behind "approximately"?  Is it 10% accurate, 50% accurate, 80% accurate?  Let's say it is 50% accurate.  This means that I can finish this work at best in 1 week or worst 3 weeks.  That's a big swing.  As a good developer, it has to be better than 50%.  If it's 50% or less, then it is an indicator that you are lacking understanding (i.e. knowledge) in a key part of the problem and/or solution domain.  Then you have to estimate on what it will take to overcome that ignorance, before committing to the original request.

An estimate is not a commitment.  If you mix these two concepts together, then you most likely will be in trouble sooner than later.  The estimate is based on something that you understand and that you can foresee with good accuracy.  In other words, an estimate is a prediction of the future.  The commitment is a promise that you make on part of the estimate in order to make that prediction come true.  If I predict that a problem can be solved in 3 days, then I may make a promise for some part of it by the end of the first day. This distinction may surprise some who have been using the poker game, or other estimation technique in Scrum. Scrum teams use estimates in planning make commitments for the entire sprint, then track and calibrate velocity via burndowns, which leads me to the next point.

Team velocity or relative estimation does not make for better estimation.  Knowing what a team's velocity is based on tracking historical trends is only a measure of the energy the team has expended to date.  This expended energy (the area under the curve) is used to predict the potential energy of a team.  That's all, nothing more nor less.  I will stick my neck out and go so far as to say that relative estimation in points is not at all about estimating, but a way to accurately predict the potential energy of a team.  I'll go even further: relative sizing does not work because software development is about crunching new knowledge, and every new piece of work should be crunching more knowledge.  If it's the same thing over again, then the design is flawed or at least at the wrong level of abstraction.  Jumping levels of abstraction, and re-shaping designs takes quite a lot of knowledge crunching and is not relative to anything else but your existing knowledge.  So, relative estimation does not make for better estimations and velocity just tells you the potential energy of the team.

Where does this leave me?  I've given up on relative sizing and estimation.  Knowing my own velocity has not added any value to my ability to estimate because every problem is unique in some way.  I estimate based on the knowledge that I have at that point in time, and force myself to be aware of my accuracy.  All of this, and more, has made me appreciate the value of one piece flow a lot more than time boxed flow.

• SD Times' Leaders of Agile Webinar: This is the fourth or fifth in this series hosted by Kent Beck.  It's always very engaging for me, and Kent normally asks far too many difficult questions without warning me in advance.  It's also the third time that I will be working with Henrik Kniberg.  This time we will be talking about Kanban, and should be another really fun 3 hours.  What I enjoy the most is the balance between practical experience, challenging the norm, and always the realisation that I know far too little.  It's free and you can register at SD Times.
• Oredev 2011:  I've been invited back to Oredev in Malmo, Sweden.  This is always a fun conference and I get some face time with my factor10 colleagues in Sweden again.  On this occassion, I will be giving two talks.  In the first I will explore the messes we make over and over again, yet we are missing some old teachings and learnings consistently.  The second is on the Java track and I will share my experiences on experimenting with Scala and Clojure, more from design and mental mindshifts, instead of the usual language and syntax changes.  Check it out at Oredev 2011.
• rubyfuza 2012: This is the second edition of the only South African Ruby conference that I know of.  The 2011 edition was well put together with great local and overseas speakers.  Although the program is not finalised, I will be talking about using JavaScript and CoffeeScript to build parsers and grammars.  This is something that I've spent a lot of time on recently and it's changed a lot of my thinking on domain driven design implementation options.  Check out the details at rubyfuza.
• AgileIndia2012:  This is my first trip to India, and one that I am absolutely excited about.  I will be running two sessions.  One on product ownership which is a slight improvement on the sessions held for SD Times and the S.African Scrum Safari.  The other is a culmination of years of introspection on cultural differences that I face.  I explored this quite emotionally at Oredev last year and will run a more in-depth session for AgileIndia.  Check out the abstracts for my sessions Practical Product Ownership: balancing strategy and development and Collaboration lessons from the Rainbow Nation the early bird program.

Scrum Master fail... Improvement action: Bigger stories where only few members can participate should be scheduled to run over 2 sprints.

After a quick twitter conversation, Karen explained.

It removes the ability to measure progress via working software at the end of the sprint

My response was

Working software is not the only way to measure progress in a sprint. And what if it works? I think it can.

It may well not be the ScrumMaster that failed when it was decided to have a bulkier story span two sprints.  I appreciate that it is a whole lot better when a story is in one sprint, and that should be our default objective.  However, it's often not so trivial.  Given that I don't have a lot of context, I will be assuming a lot.

In this case, it is more likely a failure of the team that they (a) accept a story to develop over 2 sprints, or (b) was unable to do enough analysis to consider what can be done in one sprint.  It can also be a failure on the product owner for not entering into a meaningful conversation on the details of the story, and then, again, a team failure on not engaging the PO significantly to take the problem forward.

If I encounter a story that spans two sprints, and that is more often than you think (often discovered mid-sprint), then I'm not interested in working software but seek clarity of understanding in that sprint.  The outcome at the end of the immediate sprint is an unambiguous story (or maybe several stories) which is a statement of the problem domain, or even better, the solution domain.  That is what I mean by working software is not the only measure of progress in a story.  It is more important to "measure" increased understanding in each sprint, and good statements in the solution domain is at the heart of knowledge crunching.

The most neglected aspect of working software is a measure of understanding of the solution domain.  In my experience, many teams are great at expressing the problem domain in software and their code reflects the analysis of the problem.  Consequently, the code does not reflect the understanding of the solution.  The end result is a weak design.  Over many sprints that require work in the vicinity of the weak design, there will be a degradation in velocity because the code is just baking in the problem statement, and not a well crafted solution to the problem.

Next, let me deal with the case of just a few team members participate and not the entire team.  I think that is completely feasible approach.  I've done it many times to great effect and with great efficiency because the conversation is a lot more direct and contained.  It is later that the distilled knowledge is shared with the team.  This is largely crunching in the problem domain with some rough models in the solution domain.  When I've included the entire team in the analysis, then the effect is one of dilution and inefficiency - too many people over a longer period.

Lastly, I asked "What if it works?".  While this may seem to be brash or provocative  question, it is meant literally.  What if it really does work? Hey, then what we've achieved as an odd case of delivery over two sprints instead of one.  If it happens often enough, then we need to adapt accordingly: increase sprint duration, have people dedicated to analysis (oops, bite me 'cos I'm creating a silo) and maybe more if we just think a bit about it.

So, in my opinion, it is absolutely OK to attempt the proposed solution because it is an admittance of ignorance, but if the team does not understand the actual situation they're in, then it is the failure of the team not the ScrumMaster.  I also think that we should understand the rules of the game a lot more deeply.  Like I've said in the past:

It's not the rules that matter, it's what we do with the rules that counts.

To each their own, and life goes on.

If you are planning to come along, bring your laptop with your development environment.  I will likely have code samples in Java, C#, Ruby, Javascript, and even, yup, Basic (well, maybe).  All the samples should be very readable and you could easily translate them to something equivalent in your language pretty easily.  Better still, bring some of your own code along, that you want to share.

In reality, this is stuff that Scrum does not teach you, but need to know to avoid Scrum burnout.  Looking back, I should have done something like this sooner.

#5 Look, we can synchronize our databases with our ESB

You have two application databases and you want the changes in one to be propagated to the other.  The ESB seemed like a perfect way to spray data around.  Hey, just give me an end point and I'd pump some data through it.  Well, syncing data is a replication problem with different challenges such as change detection, conflict resolution, retry or abort strategies, and bulk materialization on both ends when things get horribly out of sync.

#4 You can call my stored proc from the other side of my ESB

Somewhere in your app you had something that called a stored procedure. The ESB seemed like a really easy way to wrap that SProc with a service and hand out a new end point.  Cool, now everyone can call your stored proc.  Well, perhaps that original thing that was calling your stored proc was a wrong architecture decision in the first place.  If it was the write decision, suddenly opening it up to multiple callers that you have no control over means you got to be certain about whether your SProc is re-entrant, can handle the concurrency, and a whole lot more. 

#3 My ESB now manages my transactions

Enough said.

#2 My authentication runs as a centralized service on my ESB

Login seems like an easy enough "hello world" service to get up and running.  Hey, since our ESB has all of these apps hanging off it, we can get our Login Service to do single sign on (SSO).  We just need to call all the login services for each app and front it with our single sign on service and then this service will dish out authentication tokens.  Oh, we might as well put in a centralised authorisation service too.  Well, firstly, login is not a service.  You can take that thought further from here.

#1 I replaced my call stack with my ESB

To be fair, most developers don't know that they did this.  Let me explain.  You had some app that called a method that calls a few more methods and so on.  We all know that this builds up a call stack that gets popped on the return path.  When you take those classes that have those methods and dump them as services on your ESB, the call stack has not changed, but now there is an ESB in between.  Well, you can't just shift classes wrapped as services to your ESB.  You actually have to get service oriented in your architecture.  Only then will that call stack change.

It's not surprising that we see this now.  It takes a long time for an architectural style to be understood.  Unfortunately, there are some costly mistakes along the way.  We also can't blame the ESB, but, beware, the sardine run happens every year.