Agile - it's people!

Refactoring, functional and market requirements changes from the material product development world

noreply@blogger.com (CyberusFaustus) — Sat, 27 Dec 2014 16:11:00 +0000

https://medium.com/dome-kit/slow-flexible-cheap-5598ca91fb38

Exercises in Agile Discipline

noreply@blogger.com (CyberusFaustus) — Tue, 28 Oct 2014 04:38:00 +0000

In my previous post I wrote about how one of the primary things lacking in teams floundering with their attempts at Agility is a lack of discipline. I also linked to articles on two particular disciplines: Timeboxing and Feedback loops.

In reality, every process part of Agile Delivery can be thought of as applying a timebox, setting up and using feedback loops, or a combination of both. The timebox and the feedback loops are the mechanics by which we traverse problem domains in an Agile manner. Problem domains can be delivery of a product (software or other), organizational transformation, team or self improvement, holding a party, or any other problem complex enough to be broken into steps, but also vague enough that the understanding of the constraints around successfully solving it will change over time until the time of the completion of the solution.

Every timebox should be defined as the shortest amount of time in which to produce a meaningful result. By meaningful result we want to focus on:

Delivering requested value,
Coming to a better understanding of what value to deliver next,
Coming to a better understanding of what value remains to be delivered, or
Coming to an understanding on how to better do the first three.

Feedback loops are essentially a series of timeboxes in which one of points 1-3, as well as some information about point 4 is gleaned from each timebox and fed into the next in the series.

In another prior post I compared using these disciplines to the use of footwork in a martial art. In the fencing salle, footwork is so important, so foundational, that we have had new students spend at least a month doing nothing but footwork before they were even allowed to touch a weapon.

If timeboxing and feedback loops are such foundational disciplines in Agility, we should all be rigorously exercising in using them. To that end, I propose the following challenge exercises. These are simply suggestions of where to apply the disciplines, not a complete exercise regimen. You can choose to do one, some or all of them and in whatever order you deem appropriate. Better yet, come up with your own regimen from these or other exercises and post it here for others to try!

Pick some chore or home project you are having difficulty finishing and set aside a timeboxed amount to work on it each day for a week (notice, the week is itself a timebox). Assess your progress after each session and at the end of the week.
Pick some chore or project your kids are having trouble finishing. Have them determine the timebox for each incremental progress. You facilitate the assessment at the end of each increment and the end of the week.
Try using the pomodoro technique at work or at home.
Take one of your agile ceremonies or meetings on your team and try to make it more effective using timeboxing.
Apply timeboxing to breaking down large stories.
(This can be done individually or even better, as a team exercise) Make a list of all the time-boxes your team is employing - whether consciously or not. Now, from this list, make a list of all the time-boxes where feedback could be collected but is not. Pick the most impactful missing feedback timebox. Start tracking the feedback from this timebox and discussing it before the next subsequent, related timebox. See if performance improves from loop to loop over time.

3 Ways to Get Control of Your Agile Teams

noreply@blogger.com (CyberusFaustus) — Tue, 21 Oct 2014 12:06:00 +0000

Your company rolled out its Agile initiative 6+ months ago. A team or teams under your leadership received the green light to manage itself in the approved flavor of Agile (usually Scrum). Team members have attended training. The team meets every day or at least once an iteration. There has been a Scrum Master appointed to the group. Yet the project(s) worked on seem to be slipping as each week goes by. You are assured that stories are being completed, yet you have no idea whether you are going to meet the deadline, let alone what is being developed toward that goal. Your situation feels less Agile and more like Anarchy.

Dire as this example seems, it is not far off the mark for many teams and companies who have given this 'Agile thing' a go. Here are three tips used by successful Agile teams you can use to regain control of your team, its delivery of value and avoid the brink of chaos.

1) Agile Metrics.
Despite the impression of some that Agile is just a free wheeling approach to development, the best Agile teams know that the built in feedback loops provide the perfect means to collecting tons of accurate data. Even if your teams have not implemented the tightest loops, like those around automated testing and pairing, they should be able to give you a fairly current Velocity measurement of the team's progress toward completion of the project as it is currently defined. This measurement is much more than a simple ratio of work not done to work remaining, but rather provides insight into numerous impacts on a team, including partial allocations, tech debt, changes in requirements, or lack of experience. Get your scrum master/project manager/team lead to track the team's velocity iteration after iteration. Trouble spots will become readily apparent and it will give you and the team something concrete to figure out how to improve. The team should start measuring its progress and those metrics should be openly communicated.

Here are some other metrics that you can glean from the Agile feedback loops available to you:

If your team is having trouble coming up with a velocity or their velocity is varying wildly, it might be time to empower the team to develop some:

2) Agile Discipline.
While this may seem contradictory at first, the most effective Agile teams seem to hum like a fine tuned German sports car. This is similarly achieved via a seemingly Teutonic application of discipline. However, you do not achieve this simply by broad mandate. The discipline should be applied as the team becomes aware of deficiencies. If the team is having difficulty meeting its sprint commitments and as such cannot provide a healthy, consistent velocity, the reaction is often to let up the slack... just extend the iteration. Soon iterations go from two to three then to four weeks. In the mean time evidence of progress toward the release goals is further delayed. Instead of lengthening the iteration, shorten it. Ask the team what it can commit to delivering in a week. You don't have to release every week, but the team should have produced release ready solutions in that time. Run four-to-six one week iterations and you and your team will very quickly develop a predictable velocity from which you can begin to more accurately extrapolate real progress toward completion or begin to see what other steps might be needed to improve.
Similarly, I often see teams that have a scrum/standup every day, perhaps multiple times a day, but they are still disorganized. Efforts may be duplicated, progress and blockers may go unseen, and no one may really know how close we are to 'done' let alone what the next highest priority item to work on is. This is often due to an overloading of the Scrum. Instead of quick team sync up, they become a catch all, rapidly devolving into half to hour long design discussions, requirements gathering, bug reports or ad hoc planning sessions. Again the time-box is your friend. Have the scrum master limit the scrums to fifteen minutes. Anything not relevant to the standard team sync questions of a stand-up or scrum should be tabled for discussion after. Not only does this allow team members to get back to work, it also quickly brings to light related issues which can actually greatly improve the delayed discussions.
Further more, set limited, regular meetings for grooming, planning, the showcase and your retro with clear outcomes for each. Grooming should produce estimable stories with clear acceptance criteria, planning should produce an estimated commitment of delivery of some of these well defined stories and the showcase should clearly demonstrate how the acceptance criteria of each have been delivered. Obviously, each requires the previous one starting with the grooming session(s). Therefore making a rule that a grooming session must conclude with a percentage of the considered stories defined is helpful. To facilitate this I have sometime even set time limits on discussion of stories. 5 minutes per story. If it is not enough the team can vote to extend 5 more minutes or to push the story back in the queue. At the very least this drives progress towards story definition. Even if a story cannot be fully defined due to externally missing information, the team is more aware of the scope and caveats of the story than they were before. Moreover, this is all documented in the even partially groomed story. The alternative is no documentation, no progress, a long discussion ending with a bunch of questions and looks of confusion on the team's faces. Worse yet, the team gets to planning and nothing is understood well enough to commit to and the planning session is postponed.

Here are further details on some Agile disciplines:

Of course, this means that all those Agile meetings are actually important. Once they are run in a disciplined manner, their value becomes quickly evident. However, the most important of the Agile meetings/rituals is routinely the first casualty in teams floundering with implementing Agile:

3) Regular Retrospectives.
For a team trying to get into a productive groove with Agile, the retrospective is the single most important Agile ceremony. Think of it like a professional sports team taking the time to regularly watch and critique their last game. If they don't do this, all they are doing is playing game after game and improvement is almost accidental. By taking even a short bit of time to call out one or two areas to improve upon for their next game or iteration, a team makes steady, focused improvements. This can be a powerful way to eliminate the most deleterious bad habits/practices and replace them with the habits which will make your teams deliver and succeed.

More on Retros:

Agile - O, be some other name!

noreply@blogger.com (CyberusFaustus) — Thu, 12 Jun 2014 16:36:00 +0000

Agile methodologies have been around for quite a while now. The Agile Manifesto was created way back in 2001. As such Agile - as a term - has gone from a curious status quo disruptor to a buzz-word to an entrenched term. It is similarly used and abused in the same fashion as most business terms. It also carries some significant baggage - not least of all the perception that it can only be applied to software development.

Shakespeare's Romeo & Juliet

Once one grows past the phase where one is simply trying to teach the practices of a particular flavor of Agile Methodologies, one is often faced with the decision of whether to drop using the moniker 'Agile'. If one sees the raw potential within the practices and principles to transform organizational culture to be healthier, happier and more efficient, how do you sell it beyond the IT department? Some leverage broader terms such as Organizational Development, others reframe the term to something like "Agility".

I see the appeal of these approaches and use them myself when appropriate. However, I am often struck that we are just superficially engaging in word play. We are left with Juliet's quandary (Act 2 Scene 2 Line 43): What's in a name? That which we call an Agile Team, by any other name would be as healthy.

Which brings us to the true focus of the more advanced Agile Coach - to transform the team's or organization's health so that they can be, among other things, agile. For this reason I like to embrace the baggage of the term Agile. It is a handle we can use to enter into a transformative conversation. This sort of conversation is usually had at the start of a relationship with a client or potential client. It might not even be had by an Agile Coach, but rather by a Salesperson. I am a firm believer that successful transformations begin at the very first engagement. Why not utilize the power of this handle to begin the transformation?

Rather than telling the person what Agile is, ask what they know about Agile; what they think it is. This can get into all manner of discussion quickly. Even misconceptions can be used as transformative opportunities. You can get quickly past lengthy and potentially boring or confusing explanations and start delving into where they are struggling and suggesting how you can assist them. If you use another name, you have to spend too much valuable conversation time just setting the stage.

... its like eating an apple

noreply@blogger.com (CyberusFaustus) — Fri, 09 May 2014 18:24:00 +0000

I was communicating with the owner of a client company where I am doing an Agile Transformation today when she said basically that to her Agile was all about reviewing, organizing and prioritizing things in bites... a bite at a time... I wanted to clarify to make sure that she understood much of the key is using the tight feedback loops available in iterative development; not just simply iterating. In response I continued the analogy - of eating an apple:

A bite at a time that gives you more info about the rest of the apple, is it juicy and sweet? Tart? Have worms? Rotten? But it is also taking that info and making decisions around the further eating of the apple... Should it be cut unto slices? Cooked in a pie? Can we cut out the bad part and keep the rest?

Or should we leave the apple at good enough and toss the rest in the compost pile while we go back to the fruit bowl for another piece of fruit?

Milwaukee Agile Meetup - Continuous Integration Panel Discussion Summary

noreply@blogger.com (CyberusFaustus) — Mon, 27 Jan 2014 20:52:00 +0000

Last week I had the pleasure of being on a panel discussion about Continuous Integration at the Milwaukee Agile Meetup. My fellow panelists were Carl Schrammel, Adam Black, Jason Duff, James Carpenter and Adam Johnson.

The discussion grew out of the group's growing interest in this area and challenges that individuals and teams attending the Meetup had expressed recently. I know several people who said they had wanted to attend, so I thought I would post a summary of the discussion.

Q: What does Continuous Integration mean to you? When were you first introduced to the concept and how did it change your perspective?
A: After the initial text book definition was given ("Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day." - Wikipedia), the group continued to express their take on CI.

CI is a powerful feedback tool for both engineering as well as project management - reduces surprises due to integration and deployments and makes the ones which occur more manageable.
The benefits of CI are best reaped via automation, but even manual CI can bring great improvement.
CI can help build confidence in the product and the team - one always knows the state of the code at the end of the day.

The distinction between Continuous Integration and Continuous Delivery (CD) was also drawn... with Continuous Delivery being an advanced practice where a team is able to deploy changes to production with high frequency.

Q: What tools are being used?

A: The standard list of revision repositories, build servers, testing frameworks, continuous integration servers, deployment automation were quickly run through - basically if you are interested in CI there is likely a set of tools readily available for your language/tech stack of choice. It was noticed, however, that there are software products and systems which are not necessarily easy to integrate automation or other CI tools too. Particularly these tend to be configuration based systems, or closed systems which perhaps try to manage revision control internally - preventing access or control by external ci/build/test setups. This led to the discussion that aside from tools, the other aspect of CI is the practices and behaviors. These behaviors are often harder to implement than the tools. However, if the behaviors are embraced by a team, even the most simplistic of CI tooling can become powerful. The further consensus of the panel was that it is best to start simply - implementing behaviors and supporting tooling gradually. In this way, each advancement should be lauded as a positive win in the right direction rather than disappointment at not having achieved an elusive ideal CI or CD state in one try.

Q: How to prove the value of CI to developers? to business/product owners? how soon can one expect to see value?

A: The answers to this echoed some of the previous discussion. The general feeling was to take each instance in a case-by-case basis and look for opportunities to introduce behaviors and tools pragmatically to provide greater insight into specific areas where a team may be experiencing pain. Examples: recurring bugs, painful massive code integrations, lost code, misunderstood and mis-implemented user requirements, painful coordination/integration with other systems/teams on deployment, out of sync versions of code between environments (Prod getting ahead of Dev or Test), not being able to do certain types of testing. All of these were brought up as pain points which some degree of CI could have alleviated.

Since CI sits on the front lines of ever changing code, the positive impact of CI can be seen very quickly (as soon as one build cycle in some cases). The key is to communicate this to the parties who would most appreciate the benefit. This is where a CI server can help by providing dashboards and automated messaging, but even utilizing the low tech approach of holding stake-holder meetings before and after a CI 'event' can shed light on the benefit of the activity.

The panel agreed that in order to get the buy in to do more advanced levels of CI requires the building of trust, something familiar to Agile Coaches and leaders everywhere. However, even the most rudimentary CI practices can provide excellent data and feedback which, if communicated properly, can facilitate the building of that trust. Due to the fact that CI can generate data about the health of the deliverable every time a change is made, it can be one of the most powerful and prolific sources of data for information radiators. The trick as with all data is to mine it for the most relevant information to the message you wish to communicate.

Q: Who owns CI? Who sponsors/drives for CI?
A: The panelists seemed to generally agree that in most large project teams, there was a dedicated Operations team in charge of maintaining the CI tools (build servers, repositories, deployments and deployment automation, ci servers, etc). However, many expressed how this could lead to the negative effects of siloing. Panelists shared their experiences with teams who used various means to combat this. Rotation of members into the ops team, rotation of ops duties among teams, the use of embedded testers to serve as the bridge between dev and ops, and use of separate scrum-of-scrums to coordinate dev teams with ops.

We did not get too much into the sponsors for CI. I would suggest, as with trying to get buy in, any individuals or groups who feel they would get a benefit out of the use of CI could sponsor it. I have seen CI started as an engineering initiative and I have also seen it sponsored by Project Management and even Product Owners.

Q: Of those doing CD, has anyone automated the signoff process in tightly controlled organization/industry?
A: No one had seen full blown automated sign-off at the level of the typical product owner, upper management deployment committee. However, several had seen groups who had automated sign-off at lower levels, such as moderate UAT sign-off. This can help pave the way to make the manual sign-off at the executive level easier once trust has been established through a record of reliable test/dev/pass/sign-off cycles.

The evening flew by and that was about all we were able to get to. Prior to the discussion, David Neuman from Redpoint Technologies sent out a list of questions to the panelists that would serve to seed the discussion. I think we hit most of the items tangentially. If anyone would like to discuss them further, leave a comment or email me.

The last question on the list asked if there was one recommendation we have for successfully implementing and leveraging CI.

Build Automated Tests. Even if you still have to trigger your builds or other steps manually, having automated test suites that you can run against your built code is one of the most powerful tools to start getting an ROI on your CI efforts. Furthermore, many of the behaviors and practices of CI which are difficult to master initially revolve around disciplines of TDD. If you have automated tests, it is easier to start to instill these disciplines in your team. As some of the other panelists and I were discussion after the meetup, automated TDD is fun... it makes coding like a game with almost instant gratification to our problem/puzzle solving efforts. Making something fun can really make developing good habits to support it easier.

Agile Martial Arts to Prevent The Project K.O.

noreply@blogger.com (CyberusFaustus) — Wed, 22 Jan 2014 17:30:00 +0000

Recently, I have been having a number of discussions with colleagues about our projects and I noticed that there was a similarity in how we were discussing using the communication and feedback tools of Agile Development with concepts of Martial Arts.

When training in martial arts, one comes to learn that one of the most powerful tools one has in dealing with an adversary is to control distance and time. One can either increase distance in order to give one more time to analyze and deal with an adversary's attack or one can reduce distance and/or time to limit your adversary's options and ability to develop an attack.

Similarly on a project, surprises are seldom a good thing. However one cannot know everything which is going to happen. Change management strategies have evolved to try to deal with this, however, traditionally these have been largely reactive... like a novice at Close distance flailing wildly to block each attack. So how do you control time/distance to problem fallout on a project?

This is done via the use of feedback loops of varying frequency. Feedback loops of lower frequency can give you the time to "step back" and assess changes before rushing in. This can called a Distance/Time giving action since they give you more time to assess a developing problem, but that means they may also allow the problem to develop further. Feedback loops of higher frequency allow you to address issues immediately, preventing them from ballooning into larger problems or even implementing proactive solutions to stop the problem in its tracks. These are Distance/Time takers, since they shorten the time allowed for a problem to develop. The number and roles of individuals who receive and deal with the feedback initially also plays a part in whether a technique is more giving or taking in nature. Generally, lower number of individuals whose roles are closer to the source of the problem will be a taking action, while the converse is more of a giving action.

Some Distance/Time giving techniques (lower frequency loops):

Daily scrums/standups
Iterative development/delivery
Regular demos
Regular retrospectives
Pairing

Taking a step back to gain distance and time is one of the simplest defense moves. Appreciation of the different threat levels each distance presents and the ability to back-out to a safe distance is something which is taught to beginners. Similarly, the techniques for gaining Distance/Time in Agile Development are simple enough that any team can begin to use them right away.

For example, a team is following a more traditional project management approach with few touch points across the team. Perhaps the PM only reports to a board the status of things as dates for delivery of milestones occur. At this point, it is usually too late and if a problem has occurred, the PM is delivering the surprise bad-news. Even worse, a milestone is delivered only to have the product owners report all the surprise deficiencies. Obviously, both of these are uncomfortable for all involved and panic may often set in. By establishing short, frequent and regular meetings with the team including all stakeholders, a team can get the opportunity to see the problems coming and properly deal with them. They are effectively giving themselves the time to deal with the problems rather than rushing headlong into the fray.

A recent client complained that historically, every time they tried to do a deployment the tools the development team used seemed to create surprise errors. Moreover, items were added to the new environment without the client's prior knowledge. These would not necessarily break anything, but they did add confusion and wasted time. In order to address this we established regular meetings with tech leads, the deployment lead, the clients and testing. At the meeting, proposed pending changes were discussed. A draft-deploy was reviewed for surprise dependencies which might be pulled in by the deploy tool. Team members who previously never spoke to one another now communicate regularly around a problem they were all experiencing from different perspectives. Out of this, the team has developed a much more rigorous deployment practice, surprises have been eliminated, and all parties have gained a much better understanding of their deployment tools and an appreciation for each others difficulties in deployment. It is not as much a typical Change Management/Approval board as it is an opportunity for the interested parties to communicate. Meetings never last more than half an hour and usually are about fifteen minutes, but we meet at least once a week. While not one of the traditional Agile Team Meetings, it follows the spirit of frequent cross-communication which is so vital to the Agile Approach.

Some Distance/Time taking techniques (high frequency loops):

Automated Testing
Test Driven Development
Acceptance Criteria/Tests
Test First Development
Pairing
Continuous Integration
Daily scrums/standups

Distance/Time taking techniques are usually more advanced in Martial arts as they require a bit more training and discipline to do safely. So too in software development. Most of the ones I listed here obviously employ development practices which software engineers will need to hone if they have not done them before. However, once proficiency is obtained in them, these practices can be life savers and even keep at bay the dreaded pager. Utilized well, these operate as a means of clinching problems by stopping them well before they develop into much larger problems in a more devastating environment. Use of Automated testing with a modern CI server can not only identify problems in with new code early on, it can also help make sure that once a bug is fixed, it stays fixed. One of the mantras on a team I was on was "Find a Bug, Write a Test." Basically, whenever a bug was reported, the developer wrote a test for how the solution was supposed to behave. Of course, until the bug was fixed, the test would continue to break. However, the bug would likely never resurface again since our test coverage of our code had now improved. This of course requires the developer discipline of running tests before committing, as well as a Test-First mentality. Without those disciplines the team is back to reactive flailing.

These techniques do not always stop problems from occurring, but like the clinch, they can prevent the problem from developing into a more lethal one. A team with good CI, TDD, and Acceptance testing practices for instance, can deal with and minimize the impact of major architectural changes or even severe problems introduced by changes in third-party software. They can tackle the severe problem in manageable chunks at a level of relative safety.

I listed daily scrums and pairing under both because depending upon the situation these techniques can act both to give time to deal with potentially developing problems as well as to stop problems from starting in the first place through open communication. The latter is usually the case where someone either states what they are working on or having difficulty with and someone else on the team either gives them a solution or can help them avoid pitfalls they have experienced before. As it happens at such a frequent schedule, the time for problems to develop is often completely eliminated - effectively clinching the problems. Scrums and Pairing as time gaining operate in the sense that by approaching problems as a team tends to prevent us from making rash decisions.

A final note: you will notice that I do not say Pair Programming... while I like Pair Programming, many of my colleagues have noted that pairing is a powerful tool for all team members. I have seen PM's, BA's testers, even Product Owners pair on deliverables with great results.

Do good communication skills improve your code?

noreply@blogger.com (CyberusFaustus) — Thu, 19 Sep 2013 14:49:00 +0000

What is one of the primary design flaws we see over and over in software?

Tight coupling.

I started wondering why this might be. I thought about the mental process I go through when attempting to write cohesive, decoupled code. I realized that is was very similar to the mental process I go through when composing paragraphs and sentences.

Could there be a correlation between composition skills in one's native language and their skill at designing and developing loosely coupled software solutions?

Certainly Uncle Bob Martin's book "Clean Code" suggests something of the sort. In it he approaches method writing much like telling a story. If one is better at communicating written stories, perhaps they would be better at this form of design.

UPDATE: I found this link after posting that I thought I would share: http://www.theatlantic.com/national/archive/2012/10/the-real-reason-silicon-valley-coders-write-bad-software/263377/

What do you think?

Put your Adwords fears to REST (managing lead consumption)

noreply@blogger.com (CyberusFaustus) — Mon, 16 Sep 2013 20:33:00 +0000

I just finished up a neat little project where I got to help a client streamline how they were managing the monitoring of lead consumption among their sales members and making sure that they were not overspending in a particular region at another's detriment.

The Problem: The client is using Adwords to garner leads in various states. Their sales team members are selling in multiple states. They have a third party system for distribution of leads to the team members. This system has a toggle the team members are to use to indicate whether they are actively taking leads at a given time. When this flag is on for a team member, leads are sent to them. However, many team members were failing to turn their flag off. These members where then getting leads assigned to them when they might not even be in the office. This was to the detriment of other members who then would get none. Due to the agreement to provide members leads, this meant a lot of work tracking consumption, establishing quotas and trying to modify behavior. It also meant that sometimes the Adwords budget for particular campaigns would have to be increased in order to meet the internal agreements with the sales team. The cross region nature of sales member activity also meant that it could effect campaigns in multiple regions. They were tracking all this using a spreadsheet and a lot of leg work.

The Solution: Redpoint was asked to build a small tool to automate what the lead manager was having to spend too much of his time fretting about with the spreadsheet. In a short 6 weeks from Inception to Production Deploy we came up with a system to do this. The tool ultimately consisted of a web app and a poller service. The components pulled data from various web services including the distribution tool, internal employee data services, and Adwords. Most were RESTful - hence the title of this blog. The web app gave the lead manager a dashboard to monitor individual sales member consumption of leads against an assigned limit. This contributed to the calculation of potentially available leads for a given state. Once all members selling in a state had gotten all the leads they were supposed to, all campaigns in that state would be paused for the day. If the lead manager wanted to they could then increase a particularly good sales persons limit and allow them to sell more, or refocus a sales persons lead consumption to a particular state or states. The poller service was automated by a Windows Service using Quartz.net CronTriggers to assign various pollers to maintain the list of sales team members, the states they sold in, their lead consumption, the availability pool, and ultimately to pause or restart the Adwords campaigns appropriately.

The Result: The final tool, as it had evolved, quickly gained admiration from the executive team and the project sponsors. Even in its testing state, it had shown that it was more than just an automated valve for Adwords. The dashboards and multiple levels of control built into the solution allowed them to quickly see patterns in sales member activity and behavior. The flexibility of the Cron Triggers allowed them to schedule polls and resets as well as to automatically turn off all campaigns during down times and prevent 'greedy' team members from stock pilling all the leads while others were out of the office. This also assuaged executive fears of blowing the Adwords budget as they soon saw how this was an excellent emergency shutoff valve to control their spending.

For phase two they are already talking about the ease of creating trending and tracking data from what is being pulled as well as adding in different lead types and groupings.

As an Agilist, I appreciated this project as it had become an interesting, dynamic and interactive information radiator that would allow them to manage and change how they do business to meet the changing needs of their market and their staff.

Other Applications: Of course this sort of tool could be very useful to any growing company who is struggling to wrangle their Adwords budgets and their sales team's quotas. A similar tool could be useful to Scrum Masters, PMs or Team Leads to monitor and control story assignment within a team, by using the information from their story management system and assigning categories or even complexity sections to various members ("you did three '1 point' stories this iteration so far, you have to take a '2' or a '3' next" OR "You have been only taking stories involving that piece you created, for the rest of this iteration you need to take stories from other areas of the application"). While not so helpful with a small team, it could be very handy with a large distributed team - like the sale team in this case.

The language you code in could say a lot about your developer culture

noreply@blogger.com (CyberusFaustus) — Mon, 16 Sep 2013 20:03:00 +0000

In his essay on "Race and Language" Edward Augustus Freeman suggested that language, rather than race (genetic similarities), might be a surer indicator of cultural kinship among peoples. Professor Freeman wrote: "Every word that a man speaks is the result of a real, though doubtless unconscious, act of his free will. ... A man cannot, under any circumstances, choose his own skull; he may, under some circumstances, choose his own language. He must keep the skull which has been given him by his parents; he cannot, by any process of taking thought, determine what kind of skull he will hand on to his own children. But he may give up the use of the language which he has learned from his parents, and he may determine what language he will teach to his children." As such, the choice of language made individually and in the aggregate points to attributes of the language and those already speaking it which the new speaker of the language appreciates.

Perhaps this is why few topics among developers can start a 'religious war' faster than that of programming language.

The use of language is in essence a tool to capture an idea and communicate it to another. In IT we largely look at languages as tools to efficiently solve a problem. By 'solve' we really mean 'communicate an algorithm to the machine'. Just as some spoken languages are better for communicating facts or legal concepts and others are better for poetry or philosophical abstractions, each programming language is more efficient at 'solving' some technical problems than others. One may be excellent for modeling relationships, another for solving concurrency problems.

Looking at language this way is a good pragmatic approach to choosing 'the right language for the job', but it also can subtly mask or neglect other factors in language choice. Namely, the speaker and prospective audience plays a large part in what language communication is captured and how that communication is shaped. The above take on the use of programming language measures its efficiency in processes or cycles or memory consumed. It is the run-time efficiency - or how easily the computer understands and can execute the message. However, when using high-level programming languages, that is really misleading. It is really more of a measure of how good the compiler or interpreter for a given language is at translating the solution into efficient byte-code. So, compilers and interpreters aside, the real audience to gauge the efficiency of the language used is - humans.

Looking at how we use language it seems to lie upon a continuum between communicating to a large or a small audience. On one hand we have the language of media news reports, which linguists will often point to as a general epitome of the common speech of a people. It is intended to be understandable and consumed by the largest portion of a population. Prose is also near this end of the spectrum. On the other end you may find cryptography, where the audience is narrowed to perhaps only one other person. Sonnets, slang and technical jargon are closer to this end. They use conventions and specialized vocabulary which only a smaller subset of the population would have exposure to. Obviously a particular communication can lie closer to one end or the other given both the speaker and the audience. For instance a technical treatise might use highly exclusive vocabulary, but be written utilizing the most standard academic grammar and style. This 'regulated language' ensures that the message, while directed to technical specialists, will be understandable to the widest range of that population. It could even be accessed by outsiders armed with a good dictionary. Slang, on the other hand, often develops out the needs of the speaker... Complex grammar and punctuation is boiled down sometimes to only partial words. This makes the message easier and faster to say. To achieve this, the complexities of grammar and punctuation are replaced by context. One must understand this context, often cultural, to unpack the meaning behind this terse communication. It is particularly difficult for outsiders to master, and often intentionally so.

The intended range of audience has certain temporal effects as well. A loose jargon may make it easy for a small, tight knit group to communicate efficiently. However, it will be much harder to decode the meaning years later, perhaps even by members of that same community. Formal and standardized grammars and punctuation provide a degree of staying power to a language, allowing it to be understood for generations.
On can see this in development shops where one language is the exclusive language of development. On one hand a particular language may have been chosen for its similarity to human language. This may make translating business requirements into code easier. The same language may make it extremely difficult to describe the algorithm for performing a particular calculation. With greater complexity, it makes it more likely that a mistake will be introduced. The wordiness of this complexity can also make it harder to find the mistake. It also assumes that those writing code write well in their natural language. Run-on sentences can exist in code. A run-on piece of code is recognizable by loose cohesion, which is well known to negatively impact its brittleness and maintainability.

As a developer it is important to know the ways of expressing algorithms that is most natural to the language being developed in. This is called Idiomatic Code. However, a more seasoned Lead or Architect may realize that the eventual human viewers of the code may dictate different ways of expressing the same algorithm. This may even call for a completely different language to be used. Perhaps it makes sense to write code in one language, but to write the tests against that code in another. Testing in another language is often a good way to extend the audience of code written to an otherwise smaller audience.

This also brings up the impact of language on developer culture. Code written in a language which is only understood by one group within a team can lead to a closed culture. Think of those developers who almost have a visceral reaction to coding in anything but their favorite language. This language is usually that of a closed culture. Entire myths can be constructed about the superiority of both the language and those who code in it. Ultimately, this can threaten to alienate the developers from others on their team (testers, BA's, designers) as well as the development community at large. Uncle Bob Martin touched upon this in his 2009 talk "What killed Smalltalk could kill Ruby too".

Then there is the case where the language is chosen for you. The experience in this scenario can range from mind-expanding to dehumanizing. Surely the Junior Developer can learn much from being made to code in another language, particularly if that language has been chosen by a group of seasoned veterans across an appropriate section of the team. In this case the developer may be introduced into whole new ways of communicating and problem solving. On the other hand, there is the case where the language is imposed by a corporate decision made years earlier which may have little bearing upon the way your team works today or the problems it is being asked to solve now. This is the fallacy of simply 'buying' a language as a tool among equal tools. When that happens, the choice can become less about the effectiveness of the 'tool' and more about the packaging/marketing (e.g. - IDE's, plugins, reporting tools, support plans, name recognition, third-party-apis and frameworks, etc.). In these cases, the developers may often feel as if they are being forced to use an inferior tool for what seems like an arbitrary reason.

Finding pragmatic ways to introduce other languages where appropriate can mitigate some of the negative cultural impacts and keep communication flowing. It also can make the developers 'smarter' by making them look at and 'speak' about the problem using different paradigms than they are used to. This is much the same as people who speak multiple languages. Becoming a polyglot programmer may just make you a better designer or architect as your pool of ways of looking at a problem will be larger.

What other ways does the language you code in impact your team or say something about you?

Clade Taxonomy with Solr 4.4 without getting burnt

noreply@blogger.com (CyberusFaustus) — Thu, 05 Sep 2013 15:56:00 +0000

I have recently had the opportunity to do some work with the Apache Solr search platform. It is a phenomenal tool and I am still discovering much of its power. I had the need to apply taxonomy to documents and essentially search using the taxonomy. Now, I understand that via the underlying Lucene search engine the newest versions of Solr have faceted searching and taxonomy support available, but there is also a nice open source tool called Clade which looked like it would fit the bill for working out some proof of concept ideas I had. The current version of Clade only supports Solr 3.6 out of the box. For my solution however, I was interested in utilizing the multicore support of the latest Solr version. Unfortunately, in addition to multicore support, there have been a number of other changes since Solr3.6. An added benefit is that the Solr UI has improved substantially since 3.6 so digging into the created index is now easier. Clade is also written largely in python, a language which I dive into only every other blue moon. This all led to bit of digging and as I could not find any documentation on how to get Clade working with the latest Solr version, here are the steps I took:

Updated the schema config so that the uniqueKey definition for doc_id included the multiValued="false" attribute
Changed the solr_url in the settings.py to conform to the multicore standard (e.g.: solr_url = ‘http://localhost:8983/solr/collection1/’ for the default example install of solr)
Similarly, when copying the conf files over, you have to make sure that you put them into the proper cores folder… (i.e. solr/collection1/conf NOT solr/conf)
Replaced the included sunburnt with the latest sunburnt (just named the included sunburnt folder sunburntOLD and copied the new sunburnt folder in) I did this just for good measure as so much of Solr had changed and I wanted to make sure that the sunburnt I was using to interface with it was a current as possible
In the Clade lib/taxonomy.py the sunburnt query in get_docs_for_category is being executed with the fields limited to just score=true… this causes a KeyError as doc_id and title are not returned. There is a note in the code to “FIX” it… this may have worked against the old versions of Solr and sunburnt as perhaps that limiting was not working properly in those versions, however, now you need to change the line to: results = query.field_limit(["doc_id", "title"], score=True).paginate(rows=10).execute() (see: the Sunburnt Docs)
That brings back all three expected fields to create the desired tuple for return.

Doing the above got Clade to work the same with Solr4.4 on my Mac as I had got it working with Solr 3.6 on my Linux machine.

One other thing I have noted with Clade: re-running the classify script on the same data appears to create dupes in the index. This is not the expected indexing behavior so I will have to look a little closer as to what Clade is doing in that script. Fortunately, just blowing away the solr data directory clears it out - but that feels a bit heavy handed.

I hope this helps someone. Now to start wiring up my Solr instance via Spring Data...

"Our team is too _____ to be Agile"

noreply@blogger.com (CyberusFaustus) — Fri, 07 Sep 2012 19:42:00 +0000

Team dynamics is a common challenge with teams new to Agile Development. This often presents itself in comments such as "Our team is too big/small/remote/unfocused/unavailable/divided to use Agile". Teams which have not thrown in the towel and who are still earnestly struggling with this might ask questions such as:

"What is the 'right size' for an Agile team?"
"Can we use Agile Management with remote teams or remote members"
"Does an Agile team need to be co-located?"
"Do agile team members need to be 100% allocated?"
"How do you manage multiple Agile teams (with the same backlog)?"

With the exception of the last one, these are all assuming the definition of a team as being a group of people representing the various involved skills and interests necessary to produce a solution against a shared backlog.

In essence, these are really all the same problem: how to synchronize the team members effectively. Not surprisingly, the first pain point that often comes up is around the Agile ceremony focused on team synchronization: the daily Stand-up or Scrum.

How does this look in the previously mentioned scenarios:

Small team: "We (I mean 'I') know what we are (ahem... 'I am') working on, so having a stand-up feels like a waste of time."
Sometimes a team can be small enough that it really does keep itself in synch almost naturally. So maybe you don't need to have a formal daily synchronization. This is however where the other important output of a daily stand-up might need special attention: updating information radiators. Just because the core team are all in alignment, make sure that you keep any other stake holders in the loop so that they can help or give guidance/direction where needed. Some sort of regular check point is also good to make sure that you have everyone's attention in case an impediment comes up that needs to be dealt with higher up.
Advanced Tip: try synchronizing not just on tasks, but on your common goals. A short daily re-dedication to team goals or values (perhaps those noted in a retrospective) can be very powerful even for small teams which are otherwise synchronized on the work being done.

Big Team: "We have so many people involved that our stand-ups take an hour and we never really feel that we have a common direction on the work. Also, because the team is so big not everyone shows up all the time"
First, ask yourself whether all these people are really needed for the work being done in the given Sprint. If not, can their needs/interests be met by less frequent special meetings to synchronize around their particular interests? Some teams get bogged down with extra BA's, Testers and even Scrum Masters and Developers who are really there to collect information for a tangential team. While communication and integration across teams is important, they are really more like additional Customers to your team and as such, should not be involved directly with the core teams synchronization. Once synching is completed, information they need can be more appropriately communicated in a separate meeting.
If everyone at your Stand-up is really part of the Sprint work, can the project backlog be carved up and the group split into two teams? This may take a little planning and grooming as well as coordination against the program/product backlog or release plan.
Agile can be done with large numbers of people. Team size however often tends to become less efficient beyond 6-8 developers plus the rest of the team. Usually this means 1-2 of each of the following: BA, QA/Tester, Scrum Master/Team Lead. The trick is to carve up the work and coordinate the effort such that value is still being regularly delivered. This is often done around value 'themes'. If team size starts to interfere with delivery of your commitment every sprint, think about splitting up the team.
Advanced Tip: Splitting up the backlog and team is simply an organizational abstraction. Given you have decided to split the backlog into two teams for efficiency, if you hit a sprint where it makes more sense to work as one team to deliver value go ahead and recombine for that Sprint.

Team with remote/non-co-located members: "It is hard to find a time to meet. Communicating with remotes loses much of the context so stand-ups become more of a status report. Not everyone shows up regularly. Not really sure what team members are working on. We feel like we are constantly fighting for synchronization, not just at stand-up."
I include non-co-located members in this category because one can feel like a remote if one is across the building, the street, or the city. The problems can be the same. When people work in close proximity to each other, there is a lot of high-bandwidth communication which goes on around discussions about tasks being done. Much of this can be lost by team members who are too far away to hear these conversations. This dis-synchronization seems to dilate with the difference in time zones. A team with members in the same time-zone +/-1 hour may be able to make up via various communication tools such as IM, Skype, Google Hangouts, Google Docs, and remote pairing techniques. Beyond that, the difference in time zones begins to add overhead to delivery. The benefit of the remote members has to be weighed against the cost to productivity on the Project as whole. One way to counter this friction is to gather remotes into teams which are closer in time zone (ideally co-located) to each other than the main team and carve off some of the backlog just for them, similar to dealing with a too Big Team. Similar also to the Small Team, regular and frequent updating of information radiators and publishing them so that remote members can see them is crucial. If it can be partially automated (such as build status, or even Sprint burn downs/ups) all the better. This is the type of team where having a digital Story/task board or at least duplicating the Story/task board digitally may be highly beneficial. Advanced Tips: 1) If you can afford to bring the remotes "to the mothership" at least for a sprint or two in the beginning, that can help solidify the team. Some teams bring the remotes "home" at regular intervals. 2) Remote pairing is a viable technique. If you have high bandwidth internet, you can pair using tmux or google docs (depending on what you are working on) while speaking over skype or IM. Particularly for paired programming, however, you should make sure that the remotes are good at pairing in person, so they know what to adjust to when pairing remotely. An excellent presentation on pairing done right can be watched here. Joe Moore also has lots of good tips on his remote pairing blog.

Team with partially allocated members: Similar to the Team with remote members with the added "Partially allocated members are going to A LOT of daily meetings."
Sure, it is nicer to have 100% allocated teams, however, team members do not need to be 100% allocated to an Agile project for it to work. They do need to be 100% dedicated to working on the Agile project for the percent of time they are allocated. If a team has a member who is allocated 50% to a project, for 20 hours of their 40 hr week, they should be working solely on helping the team meet its Sprint commitment. This is one of the reasons I do not like assigning tasks during planning but rather prefer to just place the stories in priority order and allow the team members to pick up tasks as they finish other ones. In the partial allocation scenario this avoids someone who finishes their task early, realizing their other tasks are blocked, and then just spending their time trying to get ahead on their other project. What should happen is they should be empowered to pick up something else in the Sprint backlog to help the team deliver the value it has promised.
Another guideline I have found is that allocation beyond two projects usually lowers productivity. The scheduling of time to synchronize with the team can actually become harder with too many allocations than it is with remote members. If they are allocated across 3+ other Agile projects, think about how much time they are in meetings rather than working for any of the projects' deliverables. Furthermore, the context switching demanded on them is a huge drag on their ability to engage meaningfully.
Due to the caveats and the hidden losses of productivity, this is why many teams just decide to have 100% allocation and avoid the potentially costly headaches.
Advanced Tip: If you have a program with many projects who share many of the same resources, consider consolidating those shared resources and carving the backlog such that they are only spread across one or two projects. With two projects, someone can work in the morning on one project, go to lunch and finish the day on the other project. This is much more manageable for all involved.

Multiple teams (or remote teams) with the same backlog: Similar to the Big Team with the added "We often feel we are stepping on each others toes. We find it hard to break up/hand-off/work on stories and deliver meaningful value." Add the complexities of the Team with remote members if one or more of the teams is remote.
The difficulty here seems to come from the fact that being too closely joined around a backlog, the teams are not independent enough to self-organize and drive value. Try to carve up the backlog into functional chunks. This may be almost a sub-backlog. If the teams are working this way, then also ask whether synchronization with both teams fully present is of real value or whether each team should synchronize internally and simply report out at a separate meeting regarding integration points. This can keep the team focused on the work it has in front of it, instead of the work the other team is doing which it really cannot impact anyhow. Having good testing between these teams can provide an excellent interface or bridge. This can allow teams to develop independent of the deliverable of the other team. When the other team completes its portion the testing framework is already there to verify the final integration. You may want to up your compliment of testers to at least one per team with possibly an additional one to facilitate cross team testing. Updated information radiators are also key to helping each team stay on top of what the other is doing without being over intrusive.
Advanced Tip: One of the dangers of carving up a backlog and using multiple teams is a siloing effect. To combat this rotation of team focus can be a powerful tool. If teams are all local, they can implement this via rotation of members. This provides a gradual change that has less impact on each teams velocity. If one or more teams are remote, this can also be done, but you will likely wish to switch the whole teams focus. In this case, you can mitigate the drag of context change by setting up a temporary exchange program. Let us say Team A is working on widgets and Team B is working on sprockets. When they switch a member of Team A continues to work as a remote pair for Team B for a Sprint or two. In this way Team B can get some of the expertise of Team A's prior work on widgets. Similarly someone from Team B would remote pair with Team A for a sprint or two sharing knowledge about sprockets. Again, well written tests also can greatly facilitate this transfer of knowledge and responsibility.

9th century incremental development

noreply@blogger.com (CyberusFaustus) — Wed, 30 May 2012 02:47:00 +0000

Appreciation of the incremental, pragmatic approach used in Agile Development from the annals of history:

"The just man builds on a modest foundation and gradually proceeds to greater things." - unknown circa 9th century.

This saying is used by Asser to describe the philosophy of learning of King Alfred the Great.

* 'just man' may also be more modernly translated as 'man with sound judgement'.

Homer on Pairing

noreply@blogger.com (CyberusFaustus) — Fri, 24 Feb 2012 03:15:00 +0000

(From the Iliad, Diomed in giving his counsel on how the Achaeans might capture some intelligence on the Trojan plans):

"When two men are together, one of them may see some opportunity which the other has not caught sight of; if a man is alone he is less full of resource, and his wit is weaker."

Prezi on Writing Communicative Tests using BDD, Cucumber and Rails

noreply@blogger.com (CyberusFaustus) — Fri, 10 Feb 2012 15:55:00 +0000

Writing Communicative Tests with BDD, Cucumber & Rails on Prezi

When the Cucumber is mightier than the Pen - Fixing Rails 3 in Action

noreply@blogger.com (CyberusFaustus) — Tue, 25 Oct 2011 16:10:00 +0000

Aslak Hellesoy and the good people behind Cucumber have realized there was a problem in the Cucumber community. People were using web_steps to build, rather than just as a set of examples. So, the latest versions of cucumber have removed the creation of web_steps and the attendant files when you run the generator.

As Mr. Hellesoy noted in his blog entry, this is going to break a lot of Rails training materials which have come to rely on these files being available.

Having recently gotten a copy of Rails 3 in Action and noticing that an edit seems to be a ways off, I offer my humble solution for getting through chapter 3 of this book using the latest Cucumber. You can and probably should still read through the text that I skip in the book, as it does give you insights, but my path below will allow you to use Cucumber the 'right' way.

The trouble starts on Page 55

I have rewritten the Gherkin for the feature file. Notice the more domain language syntax?

Listing 3.8

Feature: Create projects
In order to have projects to assign tickets to
As a user
I want to create them easily

Scenario: Creating a project
Given I am on the homepage
When I navigate to the new project creation page
And I create a new project
Then I should be shown the project created verification message

Then skip to the portion about doing the db migration

rake db:migrate (page 56)
rake cucumber:ok

You can implement step definitions for undefined steps with these snippets:

Given /^I am on the homepage$/ do
pending # express the regexp above with the code you wish you had
end
When /^I navigate to the new project creation page$/ do
pending # express the regexp above with the code you wish you had
end
When /^I create a new project$/ do
pending # express the regexp above with the code you wish you had
end
Then /^I should be shown the project created verification$/ do
pending # express the regexp above with the code you wish you had
end

stick this in features/step_definitions/project_steps.rb

Now edit this using Capybara API (http://rubydoc.info/github/jnicklas/capybara/master/file/README.rdoc)

Given /^I am on the homepage$/ do
visit('/')
end
When /^I navigate to the new project creation page$/ do
click_link('New Project')
end
When /^I create a new project$/ do
fill_in('Name', :with => 'TextMate 2')
click_button('Create Project')
end
Then /^I should be shown the project created verification$/ do
page.should have_content("Project has been created.")
end

The visit('/') is substituted for the path_to stuff from web_steps that is mentioned on page 56

basically we are sending it to the root

continue to page 57-58

when editing the routes.rb file

…
# You can have the root of your site routed with "root"
# just remember to delete public/index.html.
# root :to => 'welcome#index'
root :to => "projects#index"
...

continue through to page 61

when editing routes.rb

…
# Sample resource route (maps HTTP verbs to controller actions automatically):
# resources :products
resources :projects
…

You can continue on till Page 73 adding a title:

add the following line to the creating_projects.feature

And I should be on the project page for the new project

running rake cucumber:ok will give you the following step def to add to your file (project_steps.rb)

Then /^I should be on the project page for the new project$/ do
pending # express the regexp above with the code you wish you had
end

Then edit this to:

Then /^I should be on the project page for the new project$/ do
current_path.should == project_path(Project.find_by_name!('TextMate 2'))
page.should have_content("TextMate 2 - Projects - Ticketee")
end

This replaces the paths.rb stuff and the other web_steps.rb stuff referred to on page 73

do read the stuff on the bottom of page 73-74 about the dynamic method invocation

rake cucumber:ok

will give following error:

…
"expected there to be content "TextMate 2 - Projects - Ticketee" in "Ticketee\n\n \n Project has been created.\n \nTextMate 2\n\n\n" (RSpec::Expectations::ExpectationNotMetError)"
…

(instead of the expected #has_content? error mentioned in the text)

both are Rspec type errors though

continue on with page 74-77

New Cucumber Feature for Page 77

make your features/creating_projects.feature look like:

Feature: Creating projects
In order to have projects to assign tickets to
As a user
I want to create them easily

Background:
Given I am on the homepage
When I navigate to the new project creation page

Scenario: Creating a project
And I create a new project
Then I should be shown the project created verification
And I should be on the project page for the new project

Scenario: Creating a project without a name
   And I try to create a project without a name
   Then I should be informed that the project has not been created
   And I should be told that the name is required

rake cucumber:ok

Add these to the project_steps.rb

When /^I try to create a project without a name$/ do
pending # express the regexp above with the code you wish you had
end
Then /^I should be informed that the project has not been created$/ do
pending # express the regexp above with the code you wish you had
end
Then /^I should be told that the name is required\.$/ do
pending # express the regexp above with the code you wish you had
end

Implement them as per below:

When /^I try to create a project without a name$/ do
click_button('Create Project')
end
Then /^I should be informed that the project has not been created$/ do
page.should have_content("Project has not been created.")
end
Then /^I should be told that the name is required\.$/ do
page.should have_content("Name can't be blank")
end

This should get you on track to continue at the bottom of page 77.

I realize my Tests are not ideal. For instance the second scenario could also check that on a failed verification you stay on the New Project page. I was simply trying to mirror what the book had and put it into a more business domain syntax. It also nicely shows the division between the Gherkin and the step definitions.

Hopefully this is helpful to others. As I have time to go through the book, I will try to add further updates, but this should give most a good start in the right direction.

I am eager to see other modifications others make to this.

Eat your meat AND your vegetables!

noreply@blogger.com (CyberusFaustus) — Tue, 25 Oct 2011 15:48:00 +0000

No matter what dietary fads are upon us, most of us realize that we need to eat a balanced diet.*

Why is it in the software industry that I so often see teams binging on just one thing when it comes to testing?

What I mean specifically, is testing frameworks. In the Rails world the frameworks du jour are RSpec, Cucumber and Steak - with a little Capybara sauce. Mmmm sounds tasty!

RSpec has become a solid core for all layers of testing with the other parts. Cucumber was initially created to make the BDD'ness more domain specific... so that customers/project owners/BA's could read and even write them. Unfortunately, many Gherkin examples in the wild tend to be anything but non-coder friendly. This seems to have occurred due to many factors which may come to play on a team.

One of these factors seems to be part of what has attracted many people to Ruby on Rails in the first place. You often hear Rails coders talking about how they love it that RoR 'gets out of the way' and 'just lets them code'. This is definitely laudable when what is getting in the way is technical scaffolding or other technical encumbrances. It is like using a power tool over the old manual ones. Where it becomes a problem is when what is being 'gotten out of the way' is meaningful communication and collaboration across the team. Teams who have tried to get their 'business folks' involved through BDD have run into the problem that either the business loses interest or can't write the tests well enough. I believe at least a good portion of this is due to how the tests are being written. They are often too low level - basically a series of instructions for filling out web forms, clicking buttons, etc. The folks behind Cucumber have tried to force a solution by removing one of the culprits they provided early on: web_steps.rb. Web_steps was a sample collection of step definitions around many boring day to day activities. Instead of serving simply as an example, teams baked their tests around it - creating many awful to read, low level test suites.

While I applaud the removal of web_steps on the part of Cucumber, it is having the oposite effect. Teams are just moving away from Cucumber to Steak (because it is more 'coder' friendly) for their Acceptance/Integration tests. What lies at the core is a one-two punch of badness that I am seeing more and more: Teams are confusing Integration Testing with Acceptance testing AND coders are writing those tests.

Acceptance tests should be written in the domain language of the business. They are meant to be high level - to provide the coders a place to start.

Integration tests should be written to test across 'the stack' and as such are more focused on the implementation details. These are therefore in the more code like language of the implementation domain.

Unit Tests test the code directly... everyone knows the coders are supposed to write these.

Cucumber is good for BDD acceptance tests. Steak is great for integration tests. Eat your meat and vegetables!

Let me suggest a possible test driven development team and their roles in testing under the above approach.

"The Business" (Story owners/Customers/BAs/etc) - write high level Acceptance Test Scenarios - These could even be part of the story. In essence they write the Cucumber Tests. (Someone else might actually cut and paste the tests into a 'feature' file, but you get the idea).

"Coders" (Software Engineers/Programmers/etc) - write the step definitions behind the Gherkin. Step definitions can be more implementation specific. Writing these can be the first time the coders actually start doing Test Driven Design. Then the coders write their Unit tests however they see fit.

"Testing Team" (Test Engineers/QAs) - write Integration tests using Steak, etc.

The above scenarios are based upon testing using Cucumber and Steak. It could just as well apply to the myriad other testing frameworks (Fitnesse, JBehave, Selenium, etc). The essence is that integration and acceptance testing should be separate endeavors.

I can hear the objections already.

"We don't have a testing team.": Fine, there are projects which are small enough, simple enough where the coders serve this role. The point is, integration testing is a different animal from acceptance testing.

"Our business people are not involved/would never write tests)": Really? Who writes your stories? Maybe they were driven away by previous bad cukes. Listen to your business people describe the functionality they want. "So I am on the home page and when I go to create a new account then I should get some indication that the account creation succeeded". That is practically Gherkin already! If they won't physically write the tests, then the BA's could certainly be following them around 'taking dictation' as the acceptance tests drip from the Owners' lips.

On the other hand, if your business people are not involved, it is likely you have a deeper issue than what testing framework you are using. Do you have an Agility Coach?

"That is totally going to slow us down!": Again, fine! Speed by itself is not a good thing. Without direction it simply means you are going to drive off the cliff faster. If you are a business owner and the 'acceptance tests' you are being fed make you want to cry with their technical level of detail - ask why they are being written this way - or better yet - write them yourself. If the testing is being taken away from you completely, don't let them. Don't let anyone tell you that integration tests are the same thing as acceptance tests. Coders, if you want your business owners to engage more, realize that giving them pages of cryptic, coder centric tests to review does not really encourage their engagement. As stated before, the idea of BDD was to simplify acceptance gathering so that coders would know where to start. It supports Agility by giving a venue for business people and programers to collaborate frequently. In the end, the slight reduction in speed along a more prudently chosen path will likely save you way more time than a radical course change much later in the project.

All of this raises a number of other questions I have been pondering for years - such as having different types of 'Customers', What sufficiency of testing levels for a given project, optimizing testing approaches...

It also makes me hungry...

* - my wife wanted me to note that our household is vegetarian and that I am only promoting Steak in the 'software' sense. Thank you dear! :-)

Shortcuts don't always get you where you set out for

noreply@blogger.com (CyberusFaustus) — Fri, 02 Sep 2011 16:46:00 +0000

I have lived in a lot of places over the years. I have learned with each new place to get out and drive (or better - bike or walk) to start to get a feel for the place. Assess the lay of the land if you will. A number of years ago I was enrolled in a three year program at St. Louis University. So, I got myself a map which I kept posted near the door to my then apartment. Every weekend I would pick a place or a route to explore. I would mark it on the map either upon my departure or return. One day I had decided to check out some places downtown. On my way out, I asked one of my friends who was a native if they could tell me the best way to get to the first building. I thanked him and started on my drive. About three-quarters of the way there, battling down-town traffic I caught a glimpse of the building I was trying to get to down a quiet side street. This is nuts I thought. Why am I dealing with all this traffic when the building is RIGHT there! Needless to say, I promptly turned down the almost empty side street and headed merrily towards my intended destination. I remember congratulating myself that I had found a faster way than my native friend and making plans to tell him about the route when I got back. Then I made it over the first hill. While I could clearly see the building I had set out for from the initial route, what I had not seen was that this street ended with a very solid structure between my car and the streets beyond which would actually take me there. Of course, it then took even longer to go back, re-merge with traffic and get to my final destination. It turns out that St. Louis has a lot of these deceitful "shortcuts". My friend had a good laugh when I got back. I realized that I had momentarily put the real goal at risk by prioritizing speed to the goal. In the end it took a lot longer.

Let me tell you another story. One of my passions for many years is the traditional martial art of fencing. After a while, I wound up teaching it myself and running a salle. Now, most people who start fencing have at best, romantic ideas about what they are getting into. Few have any other martial arts background when they come to the salle. Fewer still, if any, have trained their bodies in the way that is necessary to move in our art. Our job is to train them how not to be hit by someone intent on hitting them. One of the first lessons a student learns with sword in hand is their guards and their parries. Inevitably, their first parries are all really wide (a natural reaction when someone is coming at you with a sword!). We calmly explain that while they may have avoided that attack, what if it had been a feint? Their large motion has left them wide open in all the other lines of attack. We work with them over and over through drills to parry a precise way and precise distance. We also repeat over and over that the goal is not to be hit and that part of that is to be efficient enough to be able to deal with multiple changing attacks.

Now one of the unique parts of our salle is that we believe in having all levels train together. There is value for the novices to work with fencers who already have the basics down. Additionally, the advanced students never stop drilling the basics. Eventually, therefore, we will get the question: why do you make us parry that way when the more advanced fencers parry differently when they fence?
Of course when starting out, focusing on form is important because it is really teaching you something deeper: how to be efficient so as not to get hit. More advanced students know more ways to do this by manipulating time, distance, blade engagement, etc. A novice does not. But all the novice sees is that the advanced fencer is using a smaller parry or a larger parry than what they have been taught. The thing is, it takes a minimum of two years of training and constantly being told the mantras about not being hit, and fencing is about trained, logical, efficiency rather than reactive reflexes before it sinks in.

So what does all of this have to do with Agile Development?

A friend and fellow Agilist, Chris Bartling, tweeted about a post he saw the other day: http://t.co/jq531Il

The blogger, Jay Fields, makes an interesting point. The goal of paired programming seems to address this principle of the Agile Manifesto: "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation." It also hits some of the other principles such as attention to technical excellence and of course the primary one of continually delivering valuable software (by quality checks and reduction of truck numbers). The thing is, no-where in the manifesto does it say that to achieve these goals you have to pair. Rather it trusts motivated, self reflective and self organizing teams to come up with good ways to meet these goals themselves. All of this I agree with. yet something still troubled me about the post.

Jay is like the native St. Louis resident, or the advanced fencer. He understands the deeper goals and has enough experience to be able to draw upon a myriad of techniques to achieve them. The thing is, it takes a minimum of two years of fencing training and constantly being told the mantras about not being hit, and fencing is about trained, logical, efficiency rather than reactive reflexes before it sinks in. Even after living years in a city, my knowledge of the most efficient routes will pale in comparison with someone who has spent their life there. Similarly, the principles of Agile development look great on paper... a team reads it and there is much head nodding and affirmative mumbling. Yet, they don't really sink in until you practice them. Jay mentions that he has been pairing for years before this recent project/position. Advanced fencers fence, novices practice techniques. Advanced Agilists are agile, teams new to Agility practice techniques to become more agile.

My concern with the post is that like so many novice fencers, teams new to Agility will just see Jay not doing a technique which they may not agree with or find particularly painful initially. They will ask, why should we do it if he isn't... can't we just ask each other questions when stuff comes up? Unfortunately, too often, new 'Agile' teams will fail to ask those questions. The rigor of stand-ups, TDD, pairing, even collocation, gets the team members into the habit of asking the right questions. It helps create a culture of communication and collaboration where it may not have existed before. Of course, simply doing these practices is not enough, and should never be dogmatically considered the 'only way to Agility' - a good coach will work with the team as individuals to nurture the new culture in subtle ways relevant to the team and individuals... but discarding a new practice without understanding the underlying goal or reason can wind up being as wasteful as my 'shortcut' in St. Louis - and probably a lot more costly.

Presenting on Agility in Second Life!

noreply@blogger.com (CyberusFaustus) — Wed, 20 Apr 2011 20:05:00 +0000

On Wed April 20th at 7pm PDT, I will be presenting in Second Life. The topic is "Developing Healthy Teams through Agility".

Second Life is a thriving Virtual Reality community on the internet. There is a group of Agilists who meet there regularly.

Follow this link to join us in Second Life for the presentation: http://maps.secondlife.com/secondlife/Agile%203D/60/123/25

Prezi of a collaborative presentation on Virtue and Agility Patterns

noreply@blogger.com (CyberusFaustus) — Thu, 10 Feb 2011 21:00:00 +0000

I gave a dynamic, collaborative presentation on Virtue and Agility Patterns. Rather than using a slide based presentation, I used a tablet to mark up documents on the screen. This gave more of a feeling of using a whiteboard/chalkboard and allowed me to interact more with the audience.

Afterward, I compiled the artifacts into a Prezi.

All in all a neat experiment in making various presentation technologies work for us, rather than the other way around.

Sex, Lies & Agility Patterns on Prezi

Be sure to view it in full screen (under the More menu which appears at the bottom right corner of the Prezi).

The Grammar of Agile Software Development

noreply@blogger.com (CyberusFaustus) — Sun, 06 Feb 2011 20:30:00 +0000

This is in response to David Hussman's Tweet : "+5 RT @jbrains: "Please please stop using 'agile' as a noun" - @pragdave (YES!!!!) #magicruby

Agile: (adjective)
Agilely: (adverb)
Agility: (noun)

Examples:

"We are striving to develop our software in a more agile manner." - (Agile describes the manner [noun: object of a preposition] in which they are trying to develop [verb] the software [direct object].)
"We are striving to develop our software more agilely" - (Agilely describes how they develop [verb])
"We seek greater agility in our software development" - (Agility is the object of their seeking - and is as such a noun/direct object)

I think that people have become lazy and started truncating the descriptive title "Agile Software Development" to just "Agile". I am as guilty of this as anyone. Unfortunately this has done more harm than good and has led to a lot of confusion. Particularly, it may have contributed to the misguided notion that Agility is a binary state - using "Agile" as a noun gives the impression that it either exists or does not exist ("Are you doing Agile?")whereas the term agility used in its common sense is well understood to be manifest in degrees.

"The young gymnast's agility was only enough to perform the most basic routine".
"All things which move of their own volition are agile, however they possess this agility to varying degrees. Hence a deer can move more agilely than a rhinoceros."

I hereafter shall endeavor to respect the proper terms of speech when discussing Agile Development.

Any other suggested corrections of the use of the term "Agile" and its variances?

Aristotelian Software Development = Agile Software Development?

noreply@blogger.com (CyberusFaustus) — Thu, 27 Jan 2011 00:08:00 +0000

I admit it. I am an Aristotelian. What I mean by that is that of the three predominant ethical theories, deontology, utilitarianism and virtue oriented ethics, the latter makes the most sense to me. It is the ethical theory which was promulgated by Aristotle. Jon Dahl gave an excellent introduction to ethical theory and comparison to software development. Initially, he alluded to the fact that determining what makes a good programmer has less to do with external signs, and more to do with the programmers habits. He briefly makes ethical comparisons of software development methodologies. In this sense, he starts to look at the human element of software development. The rest of the talk, however, makes comparisons between ethical theories and specific programming languages. Perhaps this is because he was at a Ruby conference. Ultimately, I think that a closer analogy can be drawn between specific programming languages and philosophical principles from Metaphysics or Ontology rather than Ethics. This is because languages are tools for expressing concepts. Ethics however, deals with humans and human society. The goal of Virtue Oriented Ethics is for the individual to achieve a state of Eudaemonia: human flourishing. Applied to software development we could say we wish the team or project to reach a state of flourishing, or Software Development Eudaimonia. As such, I think that comparison to ethical theories is more appropriate to things such as Software Development Methodologies or Project Management.

I am an Agilist (no surprise there). If you have ever had the good fortune (Eudaimonia is also sometimes translated as good fortune) to be on a highly Agile team, you probably have a sense of this flourishing. Notice how I used the comparative when describing the Agile team. Agility is not a binary state as much as it is a spectrum. Similarly, Eudaemonia can be achieved to varying degrees. The recognition of underlying rules or laws which help a society achieve such a state is codified as Natural Law. In so far that Agile methodologies work because of their recognition of humans at the core of software development, I think Agile comes close to describing a Natural Law of software development. Yet it is one thing to state a Law or set of Laws and another to develop a lawful society. Dahl brings this up when he talks about how the XP rules as they are promulgated are very deontological. Following the rules does not ensure that you will be a good programmer. This is interesting in that the people who came up with XP were good programmers. The generation of their rules probably was a long process of self reflection and attempts to make both their code and themselves better. So while the rules they promulgate may be used deontologically, the rules themselves were likely developed via a more virtue oriented or utilitarian analysis. A common concern is how to get a team or company to 'become Agile'. Downloading lists of rules is easy, transforming a culture is not. Approaching Agility as attainable by degree seems to lead to better transformations. However these transformations are not simply replacing one deontological set of rules for another. People sense a real change in how they interact.

Therefore I posit that Agility is the Eudaimonia of Software Development. To follow the Aristotelian thought that virtue is taught by role models, perhaps a better use of the XP rules is as good models to others; sort of a goal state for developers working on their own virtue. Yet this presupposes that there are virtues inherent to achieving Agility. What are these virtues? Where would we look to find these virtues?

In his blog, Todd Hoff compares Agile as Aristotelian with Waterfall which he deems Machiavellian. In this, the core element which he recognizes is a scale between Trust and Fear. He presents the Waterfall world as being fear driven, whereas the Agile world requires teams built on trust. I think this a good place to start with analyzing Agile from a Virtue Oriented perspective. In many lists of Virtues, the concept of Trust is apparent. It is usually listed as Faith. In our modern minds, this word often is equated with a naive, unfounded belief. The way we use the word faith throughout our language indicates trust born from an observation of habitual behavior. We can say a dog is faithful, or I have faith in my friends, team members or boss. We say we are faithful to our spouses. This is language of Trust. In many of the classical virtue lists, Trust/Faith is a core virtue. This is similar to what we see in Agile transformations. The failure of many teams to transform to a higher state of Agility is often tracked back to embedded distrust within the culture.

In order to place the virtue of Trust in an Aristotelian context, it should be on a spectrum, with the virtue found at the mean of two extremes. So let us say that on one side we have distrust breeding fear. On the other side we have naive or blind trust, where we have no reason to really trust. The mean then is a Trustful state - where the team members both work to be Trustworthy/Faithful and based upon the actions of other team members they Trust or have Faith in them.

Naive Trust

Trust/Trustworthiness

Distrust/Fear

What then are some other Virtues embedded in Agile which help to promote Software Development Eudaemonia? The XP rules mentioned by Dahl have been noted as being somewhat limited in scope. So,while they may be a good guide for some of the team, they may not be the best model for general Agility. In their broad approach, the Agile Manifesto and Agile Principles are a better suited for guiding general virtuous development toward Agility. In my next posting(s) on this topic, I will go through these documents and highlight traditional virtues which are embedded in the behavior they are trying to model. I will then discuss how these behaviors and particular techniques from various Agile methodologies can help a team or individual increase specific virtues and achieve greater Agility.

What makes Agile work

noreply@blogger.com (CyberusFaustus) — Tue, 04 Jan 2011 16:05:00 +0000

In my career, I have seen team after team struggle to 'succeed' at implementing Agile. I asked a recent Product Owner who felt that the team had been partially successful, but still had a way to go, what he felt made Agile successful on his team. His answer was enlightening. He said 'the ability to change.' In essence, this is a tautology. Agility is the ability to change. It is why one would chose to implement Agile. It is not however, what makes Agile work. This is unfortunately typical. A team will focus on one practice or outcome of Agile as being the be-all of Agile. In so doing their focus creates false expectations and at the same time prevents them from focusing on making the changes necessary to harness Agile most effectively.

So what is it that make Agile work? Let us look at the Agile Manifesto and the Principles behind it. Over and over again, one concept is referred to: People develop software. (Humanitatem Progressio Progressus) In other words, Agile is successful when it recognizes that Human Beings are the core component of the development process; not tools, not methodologies, not practices, not languages. Once we recognize that Humans, and not just Humans in the broad sense, but specific individuals, we are faced with individuals who each bring specific strengths and weaknesses to the development effort. From here on out, everything else in Agile is simply about helping teams enhance the strengths and mitigate the weaknesses of individuals on them. The primary tool which Agile uses for this is tight feedback loops. Agile fosters communication so that the dangers of weaknesses can be exposed and avoided and strengths can be shared or leveraged quickly.

This brings me to the failure point for every team I have seen struggle with Agile. Something gets in the way of the communication. While I intend to post on technical issues as well, my primary focus for this Blog is to explore the ways communication breaks down on Agile teams and what tools Agile methodologies have to solve this. Where the methodologies do not propose a direct solution, I hope to bring you thoughts from practice and professionals on tactics used to bring Agile teams back on course, keeping the principles and the humanity of software development in mind.