AgileIQ Blog

Agile Open Northwest 2012 Was a Fantastic Conference

Thu, 16 Feb 2012 19:04:00 GMT

by Emily VanDeWalker

Agile Open Northwest 2012 was my first Open Space conference, and it was just that – open. Walking through the front doors, people were scattered around the main gathering room and in the breakfast room mingling over fruit and pastries, talking about what they wanted to get out of the conference. As the conference introduction started, the group sat in a circle of chairs, leaving the center open for the speaker. The conference outline was explained: if you want to host a session, write the title on a sheet of paper, pick the time and location based on what is listed in the “Marketplace” on the wall, and give a 15-30 second description to the group. As the sessions were announced, I was pleasantly surprised that a wide range of topics were proposed. I was expecting a plethora of technical sessions that I wouldn’t be able to relate to in my operations position. Instead, I found that I wanted to attend more sessions than I had time for. The speaker anticipated this, however, and let the attendees know that we were welcome to jump from session to session and were not required to sit through a session from start to finish. I didn’t jump from session to session, though. I found that I was far too interested in the current session and didn’t want to leave!

The most interesting thing about the conference is something that I realized on the afternoon of the second day: many of the sessions that I attended were related to human behavior within Scrum and Agile teams, and everyone actively participated. These sessions included discussions about what makes greatness, resolving conflict to remove impediments, coaching “smells”, and self-organizing teams. During one particular session on the first day, someone mentioned that it would be nice for someone to host a session on a topic that was brought up; by the morning of the second day, someone took the initiative to host that session and post it on the wall. I later learned that the host was a first-time attendee to Open Space conferences. I think it speaks volumes that even as a first-timer, she was comfortable enough with the group that she would step up in that capacity.

At the end of day one, the group gathered back in the main room to share what they had learned. At closing, I heard someone say, “Does anyone that I don’t know want to go out for dinner?” I have to admit that is something that I didn’t think I would hear. I figured everyone would form their groups and stick with them. I’m glad I was wrong.

As the conference came to a close on the second day, the microphone was passed around so we could all express appreciation to whoever stood out in our experiences throughout the conference. This was followed by a friendly retrospective, where the organizers happily accepted all feedback for future conference planning.

During the introduction on the first day, I was expecting maybe 20-30 sessions to be proposed. Much to my surprise, there were 77 sessions over the span of just two days! I took away a lot of valuable information, and I am already looking forward to next year’s Agile Open Northwest.

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/78857/Agile-Open-Northwest-2012-Was-a-Fantastic-Conference

Agile Software Development: BDD With Spock and Selenium

Thu, 02 Feb 2012 16:42:00 GMT

by Tim Myer

Goal

To develop a web site specification-first using Spock and Selenium WebDriver.

tl;dr

The sample code for this project is available on github and will be a helpful resource for following this article. The interesting technologies showcased include Spock, Selenium WebDriver, Selenium PageObjects, Sauce OnDemand, Gradle and Grails 2.0.

The Project

Suppose our product owner would like to publish a website for teachers to schedule their classes and for students to register for those courses. At the first backlog grooming, the product owner has prioritized the stories that would constitute a minimum viable product.

User Stories

During the team's first sprint planning meeting, we sized the stories, accepted four into the sprint, and have added tasks for each. Our story board for the first sprint contains sticky notes for our user stories, the acceptance criteria and the individual tasks for each story.

As a teacher
I want to sign up
So I can add courses

Acceptance Criteria:

User is greeted with intro screen
User is able to register as a teacher or to login
After registration or login, a teacher sees account info

As a student
I want to sign up
So I can take courses

Acceptance Criteria:

User is able to register or to login as a student
After registration or login, a student sees account info

As a teacher
I want to add courses
So students can register for them

Acceptance Criteria:

A teacher can add a course
Courses occur in a semester
Courses occur in timeslots
Courses can have prerequisites

As a student
I want to register for courses
So I can get credits toward my degree

Acceptance Criteria:

A student can register for a course
A student must take any pre-reqs before registering for a course
A student cannot take 2 courses that occur in the same timeslot

Project Structure

We will create a top-level project course_registry and the two sub-projects web and specifications. We will also create a build.gradle file in the root along with settings.gradle and a build.gradle file in the specifications subproject for running automated acceptance tests (While this is not strictly necessary, it will help to keep our project boundaries clear).

course_registry/build.gradle

allprojects {
  apply plugin: 'eclipse'
  version = '1.0.0-SNAPSHOT'
  group = 'timezra.course_registry'
}

subprojects {
  sourceCompatibility = 1.6
}

course_registry/settings.gradle

include 'web', 'specifications'

course_registry/specifications/build.gradle

apply plugin: 'groovy'
 
 repositories {
   mavenCentral()
   mavenRepo url: "http://m2repo.spockframework.org/snapshots"
 }
 
 dependencies {
   groovy group: 'org.codehaus.groovy', name: 'groovy', version: '1.8.5'
   testCompile group: 'org.spockframework', name: 'spock-core', version: '0.6-groovy-1.8-SNAPSHOT'
   testCompile group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '2.16.1'
   testCompile group: 'junit', name: 'junit', version: '4.10'
 }

We will create a src/test/groovy folder in the specifications project. At this point we can generate Eclipse .project and .classpath files for all the projects from the project root in order to work within an IDE.

course_registry $> gradle eclipseProject
course_registry $> gradle eclipseClasspath

The Teacher Registration Spec

We are now able to write the first specification for our acceptance criteria.

timezra/course_registry/TeacherRegistrationSpec.groovy

package timezra.course_registry

import static java.util.concurrent.TimeUnit.SECONDS
import org.openqa.selenium.WebDriver
import org.openqa.selenium.firefox.FirefoxDriver
import spock.lang.Specification

class TeacherRegistrationSpec extends Specification {

  WebDriver driver

  def setup() {
    driver = new FirefoxDriver()
    driver.manage().timeouts().implicitlyWait 10, SECONDS
  }

  def cleanup() {
    driver.quit()
  }

  def "a user is greeted with an intro screen"() {
    when:
    driver.get "http://localhost:8080/course_registry"

    then:
    driver.title == "Course Registry Home"
  }
}

We can run this spec and watch it fail.

course_registry $> gradle test --info
 ....
 Test a user is greeted with an intro screen(timezra.course_registry.TeacherRegistrationSpec) FAILED: org.gradle.messaging.remote.internal.PlaceholderException: org.spockframework.runtime.SpockComparisonFailure: Condition not satisfied:
 
 driver.title == "Course Registry Home"
 |      |     |
 |      |     false
 |      |     17 differences (15% similarity)
 |      |     (Pr)o(bl-)e(m) (load)i(ng--) (pag)e
 |      |     (C-)o(urs)e(-) (Reg-)i(stry) (Hom)e
 |      Problem loading page
 org.openqa.selenium.firefox.FirefoxDriver@4532be10
 
 Test timezra.course_registry.TeacherRegistrationSpec FAILED
 Gradle Worker 1 finished executing tests.
 1 test completed, 1 failure
 
 FAILURE: Build failed with an exception.
 ....

Satisfying the Spec

Suppose the team decides to use Grails 2.0 to implement the specification. This is not a restriction based on our other technology choices, since gradle is a general purpose build tool and since our specification defines how a user will interact with our web project entirely through a browser. The decision is based on the convenience of the framework, community support, the plug-in ecosystem and the skills of the developers.

First, we need to create an empty web/grails-app directory to indicate to the grails bootstrap that the project will be a grails application.
Then, we will configure our web build for Grails 2.0 in a new web/build.gradle file.

web/build.gradle

buildscript {
   repositories {
     mavenCentral()
     mavenRepo url: 'https://repository.jboss.org/nexus/content/groups/public/'
     mavenRepo url: 'http://repo.grails.org/grails/repo'
   }
 
   dependencies {
     classpath group: 'org.grails', name: 'grails-gradle-plugin', version: '1.1.0'
   }
 }
 
 grailsVersion = '2.0.0'
 apply plugin: 'grails'
 
 dependencies {
   compile group: 'org.grails', name: 'grails-resources', version: grailsVersion
   compile group: 'org.grails', name: 'grails-crud', version: grailsVersion
   compile group: 'org.grails', name: 'grails-hibernate', version: grailsVersion
   compile group: 'org.grails', name: 'grails-plugin-datasource', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-log4j', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-url-mappings', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-gsp', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-filters', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-scaffolding', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-services', version: grailsVersion
   runtime group: 'org.grails', name: 'grails-plugin-servlets', version: grailsVersion
   runtime group: 'com.h2database', name: 'h2', version: '1.3.163'
   runtime group: 'net.sf.ehcache', name: 'ehcache-core', version: '2.4.6'
 }
 
 repositories {
   mavenCentral()
   mavenRepo url: 'https://repository.jboss.org/nexus/content/groups/public/'
   mavenRepo url: 'http://repo.grails.org/grails/repo'
 }

From the project root we will initialize the grails project.

course_registry $> gradle grails-init
 :web:grails-init
 The ResolvedArtifact.getResolvedDependency() method is deprecated and will be removed in the next version of Gradle.
 | Configuring classpath
 | Error log4j:WARN No appenders could be found for logger (org.springframework.core.io.support.PathMatchingResourcePatternResolver).
 | Error log4j:WARN Please initialize the log4j system properly.
 | Error log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
 | Environment set to development.....
 
 BUILD SUCCESSFUL
 
 Total time: 1 mins 36.328 secs

We can satisfy the spec simply by modifying application.properties with the expected web application deployment path and project version, the homepage to include the expected title, and messages.properties to contain the expected messages.

web/application.properties

app.grails.version=2.0.0
app.name=course_registry
app.servlet.version=2.5
app.version=1.0.0-SNAPSHOT

web/grails-app/views/index.gsp

<html>
  <head>
    <title><g:message code="home.title" /></title>
    <meta name="layout" content="main" />
    <style type="text/css" media="screen">
      #pageBody {
        margin-left: 280px;
        margin-right: 20px;
       }
    </style>
  </head>
  <body>
    <div id="pageBody" class="dialog">
     <p>
      <g:message code="home.welcome.message" />
     </p>
    </div>
  </body>
</html>

web/grails-app/i18n/messages.properties

....
home.title=Course Registry Home
home.welcome.message=Welcome to the course registry.

We can now add hooks to the specifications project to stop and start the web application before and after running the specifications, respectively.

specifications/build.gradle

....
test.dependsOn ':web:webStart'

gradle.taskGraph.afterTask { Task task, TaskState state ->
  if(':specifications:test' == task.path) {
    project(':web').tasks.getByPath('webStop').execute()
  }
}

These hooks depend on a specific interface in the web project, i.e., the existence of the tasks webStart and webStop. We can deploy the Grails 2.0 artifact to an embedded Jetty server through gradle to satisfy this interface.

web/build.gradle

....
 apply plugin: 'jetty'
 
 def stopPort = 8001
 def safeWord = 'banana'
 
 task webStart(dependsOn: 'grails-war') << {
   def jettyRunWar = tasks.getByPath('jettyRunWar')
   jettyRunWar.webApp = new File(projectDir, "target/course_registry-${version}.war")
   jettyRunWar.contextPath = 'course_registry'
   jettyRunWar.daemon = true
   jettyRunWar.stopPort = stopPort
   jettyRunWar.stopKey = safeWord
   jettyRunWar.execute()
 }
 
 task webStop << {
   def jettyStop = tasks.getByPath('jettyStop')
   jettyStop.stopPort = stopPort
   jettyStop.stopKey = safeWord
   jettyStop.execute()
 }
 ....

The first specification should now be satisfied.

course_registry $> gradle test --info
....
Started Jetty Server
Gradle Worker 1 executing tests.
Test a user is greeted with an intro screen(timezra.course_registry.TeacherRegistrationSpec) PASSED
Gradle Worker 1 finished executing tests.
....

NB: The Jetty plugin must be applied before the grails plugin, or else you will see an error similar to the following:

FAILURE: Build failed with an exception.
 
 * Where:
 Build file '/path/to/course_registry/web/build.gradle' line: 38
 
 * What went wrong:
 A problem occurred evaluating project ':web'.
 Cause: Cannot add task ':web:clean' as a task with that name already exists.
 
 * Try:
 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
 
 BUILD FAILED
 
 Total time: 7.624 secs

Adding another test for our second acceptance criterion should be straightforward now that our infrastructure is in place.

timezra/course_registry/TeacherRegistrationSpec.groovy

  ....
   def "a user can register as a teacher"() {
     when:
     driver.get "http://localhost:8080/course_registry"
     WebElement teacherLink = driver.findElement(By.id('teacher_link'))
     teacherLink.click()
     WebElement name = driver.findElement(By.id('name'))
     type name, 'John Doe'
     WebElement email = driver.findElement(By.id('email'))
     type email, "${UUID.randomUUID()}@rutgers.edu"
     WebElement password = driver.findElement(By.id('password'))
     type password, '1234567'
     WebElement create = driver.findElement(By.id('create'))
     create.click()
 
     then:
     driver.title == 'Show Teacher'
     driver.findElement(By.className('message')).text ==~ /Teacher \d+ created/
   }
 
   def type(field, text) {
     field.clear()
     field.sendKeys text
   }

We can satisfy this specification by generating a Grails Teacher domain object with fields for the name, email and password, along with a Grails controller that uses dynamic scaffolding to generate the view and actions available, and by adding a link to the scaffolded Create Teacher page on our web/grails-app/views/index.gsp. Since the point of this tutorial is not to cover basic Grails development, we do not need to go into those specifics here, but sample code can be found on the project site or in any Grails tutorial for further reference.

Page Objects

Before we get too far with automating our acceptance criteria, we should begin to look towards a more abstract representation of the testable components of our application. From the second acceptance test above, we can already see patterns emerging. For example, when we navigate to a particular page, it would be convenient to identify that we are on the correct page, perhaps by its title. It would also be convenient to represent each view in the application as its own object, and each view could encapsulate its specific WebElements. Finally, there is a small set of actions available for any WebElement on a page, and it would be helpful to build these actions into a testing DSL. Fortunately, by combining the Page Object pattern and Groovy's dynamic language features, we can achieve all these goals with a minimal amount of code.

As we are satisfying the specifications for all the acceptance criteria, a base Page Object emerges.

specifications/src/test/groovy/timezra/course_registry/pages/CourseRegistryPage.groovy

package timezra.course_registry.pages
 
 import static org.apache.commons.lang.StringUtils.splitByCharacterTypeCamelCase
 
 import org.openqa.selenium.WebDriver
 import org.openqa.selenium.WebElement
 import org.openqa.selenium.support.FindBy
 import org.openqa.selenium.support.PageFactory
 import org.openqa.selenium.support.ui.Select
 
 abstract class CourseRegistryPage {
 
   WebDriver driver
 
   @FindBy(className = "message")
   WebElement message
 
   static <T extends CourseRegistryPage> T goTo(String address, WebDriver driver, Class<T> page) {
     driver.get address
     PageFactory.initElements driver, page
   }
 
   CourseRegistryPage(WebDriver driver) {
     this.driver = driver
     String title = splitByCharacterTypeCamelCase(getClass().simpleName).join ' '
     if(!title.equals(driver.title)) {
       throw new IllegalStateException("Should be on page '${title}' but was on page '${driver.title}' instead")
     }
   }
 
   def methodMissing(String name, args) {
     def m
     if((m = name =~ /click_(\w+)/)) {
       def webElement = this."${m[0][1]}"
       webElement.click()
       PageFactory.initElements driver, args[0]
     }
     else if((m = name =~ /type_(\w+)/)) {
       def webElement = this."${m[0][1]}"
       webElement.clear()
       webElement.sendKeys args[0]
     }
     else if((m = name =~ /choose_(\w+)/)) {
       def webElement = this."${m[0][1]}"
       def select = new Select(webElement)
       select.selectByVisibleText args[0]
     }
     else {
       throw new MissingMethodException(name, getClass(), args)
     }
   }
 }

NB: The constructor verifies that the page is correct by comparing the class name to the page title; in addition, the static goTo method demonstrates how to use the PageFactory for initializing a Page Object; the methodMissing declaration also showcases Groovy's ability to call dynamic methods based on the combination of the click, type and choose actions with individual WebElement names. Such phrases become first-class elements of the testing DSL.

We can see the use of WebElement injection and dynamic methods in action with a CreateUser page that serves as the base for the CreateTeacher and CreateStudent pages.

specifications/src/test/groovy/timezra/course_registry/pages/CreateUser.groovy

package timezra.course_registry.pages

import groovy.transform.InheritConstructors

import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement

@InheritConstructors
abstract class CreateUser extends CourseRegistryPage {
  WebElement name
  WebElement email
  WebElement password
  WebElement create

  CreateUser(WebDriver driver) {
    super(driver)
  }

  protected <T> T register(name, email, password, Class<T> nextPage) {
    type_name name
    type_email email
    type_password password
    click_create nextPage
  }
}

Other Browsers

So far, we have done all our testing with a single WebDriver, i.e., the FirefoxDriver. For simple testing, a single driver works fine, but if we wish to ensure that our website works in multiple browsers, we will need to configure our tests to use multiple WebDrivers. Fortunately, since we are using Gradle to manage our dependencies, the InternetExplorerDriver and ChromeDriver should be available to us automatically, along with the AndroidDriver and IPhoneDriver for testing our application on mobile devices. In addition, an OperaDriver is also available.

NB: Some of these drivers require that additional platform-specific software be installed, so please read the project pages for documentation on additional requirements.

Configuring our specifications to use multiple WebDrivers should just be a matter of parameterizing each Spock feature.

NB: Unlike JUnit tests which are parameterized by fixture, Spock specifications are parameterized per feature.

We will modify our TeacherRegistrationSpec to use the Spock where: block for parameterization, we will configure multiple WebDrivers, and we will share the instance driver field among the parameterized features.

timezra/course_registry/TeacherRegistrationSpec.groovy

class TeacherRegistrationSpec extends Specification {
 
   @Shared
   WebDriver driver
 
   def setup() {
     driver.manage().timeouts().implicitlyWait 10, SECONDS
   }
 
   def cleanup() {
     driver.quit()
   }
 
   def "a user is greeted with an intro screen"() {
     when:
     ....
 
     then:
     ....
 
     where:
     browser << browsers()
   }
 
   def "a user can register as a teacher"() {
     when:
     ....
 
     then:
     ....
 
     where:
     browser << browsers()
   }
   
   ....
 
   protected def browsers() {
     System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver")
     def drivers = [
       new HtmlUnitDriver(),
       new FirefoxDriver(),
       new ChromeDriver()
     ]
     new Browsers(spec: this, delegate: drivers.iterator())
   }
 
   private static final class Browsers {
     @Delegate Iterator<WebDriver> delegate
     TeacherRegistrationSpec spec
 
     @Override WebDriver next() {
       spec.driver = delegate.next()
     }
   }
 }

Testing In The Cloud

Now that our specifications are running through multiple browsers on a single machine, we can move a step further towards testing multiple browser versions on multiple OSes with cloud-based services such as Sauce Labs. For such a powerful service, the configuration changes to our existing application are surprisingly simple. We will modify our TeacherRegistrationSpec to use a RemoteWebDriver for the specific browser and OS combination we would like to test along with our Sauce Labs username and API key as described in the Sauce OnDemand documentation. We will also need to add a Gradle plugin to start SauceConnect from the build, just as we start our web application before running acceptance tests. Finally, we should ensure that all references to localhost in our application are changed to the IP address of the machine where we will be running our tests; otherwise, we might see the SauceConnect proxy freeze (you might have a different experience, and this freezing might just be attributable to the Gremlins in my machine).

timezra/course_registry/TeacherRegistrationSpec.groovy

class TeacherRegistrationSpec extends Specification {
   ....
   def "a user is greeted with an intro screen"() {
     when:
     driver.get "http://<your.ip.address>:8080/course_registry"
     ....
   }
 
   def "a user can register as a teacher"() {
     when:
     driver.get "http://<your.ip.address>:8080/course_registry/"
     ....
   }
   ....
   protected def browsers() {
     def capabilities = DesiredCapabilities.internetExplorer()
     capabilities.setCapability("version", "7")
     capabilities.setCapability("platform", Platform.XP)
     capabilities.setCapability("name", "Testing Teacher Registration in Sauce")
 
     def sauceDriver = new RemoteWebDriver(
         new URL(
         "http://<username>:<apiKey>@ondemand.saucelabs.com:80/wd/hub"),
         capabilities)
     def drivers = [sauceDriver]
     new Browsers(spec: this, delegate: drivers.iterator())
   }
   ....
 }

course_registry/specifications/build.gradle

import java.util.concurrent.Executors
 import java.util.concurrent.ExecutorService
 ....
 repositories {
   ....
   mavenRepo url: "https://repository-saucelabs.forge.cloudbees.com/release"
 }
 
 dependencies {
   ....
   runtime group: 'com.saucelabs', name: 'sauce-connect', version: '3.0.18'
 }
 
 apply plugin: SauceConnect
 
 sauceConfig {
   username = '<username>'
   apiKey = '<apiKey>'
 }
 
 test.dependsOn ':web:webStart', 'sauceConnect'
 
 gradle.taskGraph.afterTask { Task task, TaskState state ->
   if(':specifications:test'.equals(task.path)) {
     tasks.getByPath('sauceDisconnect').execute()
     project(':web').tasks.getByPath('webStop').execute()
   }
 }
 
 class SauceConnect implements Plugin<Project> {
   
   ExecutorService executor = Executors.newFixedThreadPool(2)
   
   def void apply(Project project) {
     project.extensions.sauceConfig = new SauceConnectExtension()
     
     project.task('sauceConnect') << {
       
       def output = new PipedOutputStream()
       def input = new PipedInputStream(output)
       def reader = new BufferedReader(new InputStreamReader(input))
       
       executor.execute {
         println "Connecting to Sauce Labs as ${project.sauceConfig.username} with key ${project.sauceConfig.apiKey}...."
         try {
           project.javaexec {
             main = 'com.saucelabs.sauceconnect.SauceConnect'
             classpath = project.sourceSets.main.runtimeClasspath
             args = [project.sauceConfig.username, project.sauceConfig.apiKey]
             standardOutput = output
           } 
         } catch(Exception ignored) {
           // Executor has been shutdown
         }
       }
       boolean okToStart = false
       executor.execute {
         def nextLine
         try {
           while((nextLine = reader.readLine()) != null) {
             println nextLine
             if(!okToStart) {
               if(nextLine =~ /Please wait for "You may start your tests" to start your tests/) {
                 continue
               }
               else if(nextLine =~ /You may start your tests/) {
                 okToStart = true
               }
             }
           }
         } catch(Exception ignored) {
           // Executor has been shutdown
         }
       }
       while(!okToStart) {
         Thread.sleep 250
       }
     }
     
     project.task('sauceDisconnect') << {
       println "Disconnecting from Sauce Labs...."
       executor.shutdownNow()
     }
   }
 }
 
 class SauceConnectExtension {
   String username
   String apiKey
 }

Conclusion

This post took us through the definition of user stories for a small website, the setup of a Gradle project for automating the website build and the run of its corresponding acceptance tests, the creation of our first WebDriver-based Spock specification for our acceptance criteria and the fulfillment of that spec with a Grails 2.0 application, the use of Selenium PageObjects to create abstract representations of our web pages before those web pages are even written, all the way to the creation of a Gradle plugin for running our acceptance tests on various browser/OS combinations in Sauce Labs. Each of those items in itself is worthy of a tutorial series. The combination of all these elements demonstrates with a remarkably small amount of configuration and code the type of robust and scalable ATDD infrastructure that is possible and should be at the core of any enterprise web application.

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/78219/Agile-Software-Development-BDD-With-Spock-and-Selenium

When the bottleneck of an Agile team is the team itself

Tue, 24 Jan 2012 17:01:00 GMT

by Jayaprakash Puttaswamy

Most often, we have observed that agile implementations fail in spite of putting efforts towards educating the team about Agile principles, training the team on Agile practices, and getting the right people for Agile roles. Have you ever wondered why? There are chances that the culprit is the team itself. The team could be plagued with several dysfunctions (refer to “the five the dysfunctions of team” model from Patrick Lencioni). If you discover this in your teams, how would you go about dealing with it? How can we overcome the dysfunctions and thus help the team implement agile successfully?

Well, there are ways to do it. There are instruments and techniques to do it. But all these would fail if you don’t have good leadership skills. I would like to share my experience with similar situations, with success stories, challenges and the lessons learnt. Here is a case study which I would describe in three sections:

SITUATION: Problem statement
APPROACH: Key skills used
SOLUTION: Results obtained, challenges faced, lessons learnt

SITUATION

The situation was exactly as outlined below. It was a nine-member team, suffering from dysfunctions (mentioned inside the pyramid). As a result, the team was exhibiting the dysfunctional characteristics (mentioned on right hand side of the pyramid).

Problem statement

The team was slipping on deadlines, unable to fix critical issues and could not removeits high dependency on two key technical people.
There were also side effects:
- Decrease in the morale of one of the key technical people
- The other key person not being able to focus on continuous improvement

APPROACH

The team members were trying to follow Scrum as their way of implementing Agile, and there was a technical person who was struggling to play the ScrumMaster role. I had joined this team as a development manager. Irrespective of my role, I had to take charge of the situation to identify the bottleneck. The first thing I did was to take out the teams for a couple of lunch sessions (we decided not to wait for company sponsorship with the budget approval).

Those sessions, coupled with a team assessment instrument, helped us figure out as a team what the dysfunctions were that were affecting us. The rest was all about leadership skills (refer the inverted pyramid below).

Key skills used:

Situational leadership
Conflict management
One-on-one feedback
Time management

SOLUTION

As a responsible leader, I had to drive the team in a slow and steady manner on the following aspects:

Making retrospective meetings effective
Open evaluation of team dysfunctions
Consensus on tackling “trust” and “conflicts” (which were affecting the team most)
Enabling the team to share personal details (strengths and weaknesses)
Effective sprint planning
One-on-one discussions between me and other team members

Results obtained

After a period of 6 months of continuous experiments (I call them experiments since there was no single way to address “trust” and “conflict” issues), we achieved the following:

Completion of tasks on-time, with better quality
Significant reduction of technical dependency on individuals
Improved “self-organizing” capability of the team
Increased morale of the team, as well as the “key technical people”
More critical issues were identified and fixed
The other key technical person got opportunity to improve his “leadership” skills

Challenges faced

Convincing the people on abstract concepts like “dysfunctions”
Time management
Dealing with different personality styles

Lessons learnt

Most of the radical changes require “paradigm shifts”
“Going first” or “being vulnerable” is the first step
“Constant focus” is needed to bring real changes

Conclusion

Any successful Agile implementation requires a good leader who can drive the team by identifying dysfunctions, helping the team overcoming them, and facilitating the process of change management if needed. Any of the team members (especially management people) can do this and they should do ONLY this and nothing else. Rest, the team figures itself out. Ending with a quote from Henry Kissinger:

“The task of the leader is to get his people from where they are to where they have not been.”

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/77604/When-the-bottleneck-of-an-Agile-team-is-the-team-itself

Join us in February at Agile India 2012 in Bangalore

Thu, 19 Jan 2012 21:08:00 GMT

by Pam Dyer

Agile India 2012 is Asia's largest international conference on Agile and Lean software development methods. The conference will held 17th-19th February 2012 at Le Meridien, Bangalore, India. It offers many sessions from Agile experts, including SolutionsIQ CEO Charlie Rudd and SolutionsIQ Agile Consultant/Coach William Rowden:

Confronting Business Uncertainty

Charlie Rudd
Friday 17 February at 10:45am

We can never manage uncertainty. Yet in today’s business environment we confront it all the time. This talk explains why Agile principles are needed to make good management decisions in an uncertain business environment and why past practices no longer work. We will review what drives business uncertainty and why it’s here to stay. We will introduce Agile management principles and explain how they help us navigate uncertainty. We will learn how to use these principles to exploit new business opportunities, reduce business risk and increase returns, while decreasing investment requirements.

Lean Workflow: A Parable in Pictures

William Rowden
Friday 17 February at 10:45am

This talk is the tale of two Scrum teams — team Shu that watched utilization and team Ha that watched lead time — diagramed on their task boards. Watching work in progress on a task board is sufficient to explain throughput, local sub-optimization, constraints, utilization, efficiency, service time, lead time—and why even Scrum teams would benefit from a Kanban. Additional maps illustrate wait time, value streams, and wastes. The result is an introduction to Lean, its pillars and principles, primarily in pictures.

Distributed Scrum: Why Some Teams Make it Work Well and Others Don't

William Rowden
Saturday 18 February at 10:45am

How does a geographically-distributed team collaborate across distance, culture, and even language? What can an organization do to encourage successful teamwork? What technologies and practices increase collaboration? This workshop will explore the patterns that enable companies to successfully deliver software with a distributed team. It will be presented by an Agile coach that assisted with the transition of a company that adopted Agile for teams in three time zones with two languages and cultures, and (travel permitting) the VP Technology of that company.

Agile Governance

Charlie Rudd
Saturday 18 February at 3:00pm

Ironically, conflicts often arise between Agile practitioners and the governance policies of their organizations, even though Agile can provide an excellent (often superior) governance framework. This presentation will explain why these conflicts arise and how they can be resolved. Agile methods will be presented as a governance framework and compared against past practices. Special emphasis will be given to Agile practices as a governance framework for geographically distributed teams.

Don't miss this fantastic event!

At Agile India 2012, more than 500 participants from all over the world will share their Agile knowledge through presentations, workshops, and tutorials. Twenty-two international research papers from renowned institutes will be presented as well. The full schedule of events is available here.

Learn more and register today!

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/77657/Join-us-in-February-at-Agile-India-2012-in-Bangalore

Reflection Tools for .NET

Thu, 12 Jan 2012 13:05:00 GMT

by Robert Zormeir

For almost as long as there has been a .NET development environment there have been free developer tools that used Reflection to help developers understand the nature of the beast they were dealing with. The tool that almost everyone started with was .NET Reflector, which was originally written by Lutz Roeder. In 2008, Red Gate Software announced they were taking over future Reflector development, and that Reflector would remain free.

Early in 2011 Red Gate announced that .NET Reflector would become a paid-for product, and that the free version would no longer be available. They also announced that previously downloaded free versions would time out. After being on the receiving end of what had to be an amazing stream of bile and vitriol due to the idea of reneging on their original promise, they relented and announced that previously downloaded versions would be automatically updated to become permanent.

As we near the one year anniversary of Red Gate's questionable marketing decision, it probably didn't lead to many sales of their paid version, but sudden lack of an established free reflection tool did lead to the creation of several free alternatives:

JetBrains dotPeek
DevExtras CodeReflect
ILSpy
Telerik JustDecompile

A new version of JetBrains dotPeek was released on December 29th and is the cream of the replacement crop so far. It gives you some of the same functionality you'll find in their ReSharper product: The ability to navigate to a specific type, assembly, symbol, or type member; to move between declarations, implementations, derived, and base classes in inheritance chains. It has no installer, so once you download and unpack the ZIP file and you're good to go. DotPeek disassembled my test dll with very good fidelity, but didn't insert curly braces in all the places where I'd like to see them. Maybe a future version will give me the abiltiy to specify code formatting rules the same way ReSharper does. In general, the functionality of dotPeek gives you the idea that you might be in a mini version of Visual Studio.

DevExtras CodeReflect is available as an MSI installer or a ZIP file. It was quick to download and unpack. It performed well, but didn't do the full colorization of the code the way dotPeek and JustDecompile did. One nice feature is that it allows you to quickly switch back and forth between MSIL, C#, and VB.NET code. The decompilation fidelity was good, but it did generate some intermediate variables that didn't exist in the original code.

LSpy is an open-source project that requires .NET Framework 4.0. You can download the binaries or the source code from SourceForge. The program will check for updates once a week if you prefer. The performance was good, and the decompilation fidelity was very good, but the code colorization was a little weird. But I can live with that. You can quickly switch back and forth between generating MSIL and C# code.

The installer for Telerik JustComplile asks for too much registration info and takes too long to do its work, IMHO. Telerik appears to be viewing their product as too much of a marketing opportunity... to the point that it ceases to be a marketing opportunity with me. You'll see "Just kick back and relax. Good things take time. Please be patient." as the installation proceeds. You also need to be patient as the the installed program goes through the process of "Loading assemblies." JustDecompile had a bit of trouble with decompiling a relatively simple class, but it did come close and the software is still in beta. Once everything was loaded the performance seemed very good.

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/77019/Reflection-Tools-for-NET

Mocking Frameworks for .NET

Wed, 04 Jan 2012 19:07:00 GMT

by Robert Zormeir

The four major competitors in the free/open source .NET mocking framework arena are NMock/NMock2, NMock3, Rhino Mocks and MOQ. NMock and NMock2 were actually built by different teams, but they kept the same design philosophy and are backwards compatible so they can be used almost interchangeably. If you're familiar with NMock/NMock2, you'll see that NMock3 is from the same gene pool, but it starts fresh and has somewhat different syntax from its siblings. All of the major mocking libraries are mature products with a decent-sized user base, so it's not tough to find good examples and help for all of them but NMock3.

Like many other .NET developers, I started using NMock, and then "upgraded' to nMock2, but both use "magic strings" which can't take advantage of Intellisense, and make tests brittle because they're not easily amenable to refactoring without the use of additional tools like ReSharper to replace name-similar text in strings. On the plus side, NMock doesn't rely on the explicit record/replay statements as Rhino Mocks does. The NMock2 codebase hasn't been updated since late in 2009, and should not be expected to see continued development.

On the plus side, Rhino Mocks has a syntax that supports code refactoring and compile-time checking. Unfortunately, I don't find the syntax to be particularly intuitive, and the variety of ways to construct and condition mocks can lead to some confusion when different people are writing tests. There is also the annoying need to write explicit record and replay statements. Another problem with Rhino Mocks is that the project appears to be running out of steam. Aside from a few bug-fix patches, not much has happened since early in 2009.

MOQ is one of the new kids, with an annoying name that needs to be spelled out so that people know what you're talking about. The latest release became available in April of 2011, and MOQ has active developers and community. It's different from the first two in several ways. When using NMock and Rhino you create mock objects of a specific type. When using MOQ you create mocks that contain an objects of a specific type, and that wrapper around the mock objects lends itself to simpler syntax in your tests. MOQ requires .Net 3.5 or greater due to its use of lambdas, so if you haven't learned to use lambda expressions, now is the time. I haven't used it extensively, but so far I like everything about MOQ... except for the name.

NMock3 is the other new kid, and the one who seems less well known. Maybe it isn't on the radar for most folks because NMock/NMock2 seem behind the times, and that may have kept expectations low. Versions of NMock3 are available for .NET 3.5 and 4.0, and the latest RTM became available in January of 2011 with a beta release in July of 2011. Like MOQ, you create mocks that contain objects of a specific type, and that wrapper around the mock objects leverages lambda expressions, so NMock/NMock2 users are in for some culture shock. Since the NMock3 examples are a little thin in the wild I've include sample code below. Note that in conditioning your mocks, the parameters in the lambda expressions are just placeholders, while the "real" parameters are specified in the "With" method.

using NUnit.Framework;
using NMock; /* NMock3 */

namespace Foo.Test
{
    [TestFixture]

    public class FundsTransferPresenterTest
    {
        private MockFactory _mocks;
        private Mock<IFundsTransferView> _viewMock;
        private Mock<IAccountService> _serviceMock;
        private FundsTransferPresenter _presenter;

        [SetUp]
        public void SetUp()
        {
            _mocks = new MockFactory();
            _viewMock = _mocks.CreateMock<IFundsTransferView>();
            _serviceMock = _mocks.CreateMock<IAccountService>();
            _presenter = new FundsTransferPresenter(_viewMock.MockObject, _serviceMock.MockObject);
        }

        [Test]
        public void CanQueryViewUseAccountServiceToFundsTransfer()
        {
            _viewMock.Expects.One.Method(v => v.GetSourceAccount()).WillReturn("1234");
            _viewMock.Expects.One.GetProperty(v => v.TargetAccount).WillReturn("9876");
            _viewMock.Expects.One.GetProperty(v => v.TransferAmount).WillReturn(200.00m);
            _serviceMock.Expects.Exactly(1).Method(s => s.TransferFunds(null, null, 0m)).With("1234", "9876", 200.00m);
            _presenter.Transfer_Clicked();
            _mocks.VerifyAllExpectationsHaveBeenMet();
        }
    }
}

/******************************* Fragment of System Under Test (SUT) *******************************/

    public interface IFundsTransferView
    {
        string GetSourceAccount();
        string TargetAccount { get; }
        decimal TransferAmount { get; }
    }

    public interface IAccountService
    {
        void TransferFunds(string source, string target, decimal amount);
    }

    public class FundsTransferPresenter
    {
        private IFundsTransferView _view;
        private IAccountService _service;

        public FundsTransferPresenter(IFundsTransferView view, IAccountService service)
        {
            _service = service;
            _view = view;
        }

        public void Transfer_Clicked()
        {
            _service.TransferFunds(_view.GetSourceAccount(), _view.TargetAccount, _view.TransferAmount);
        }
    }

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/76751/Mocking-Frameworks-for-NET

Use-It Testing in Agile Software Development

Thu, 15 Dec 2011 20:13:00 GMT

by Dan Ebert

Agile Coach: “What kind of testing do the developers do?”

Development Team Member: “First we create some test data if needed. Then we go to the web page and use the feature to make sure it works.”

I call this anti-pattern “Use-It Testing.” I’m always surprised, and disappointed, when I encounter a team that relies solely on Use-It Testing when writing their code.

I’d bet everyone who’s written a line of code has done it at least once. You write a bit of code … then use it by running the program, going to the web page, etc. to see if it works. This *might* be OK for prototyping (I’m sure there are some who would argue that Use-It Testing is not sufficient even when prototyping) but it is definitely not sufficient for production work.

There are three main types of testing typically done by an Agile Team: Unit Testing, Integration Testing, and Functional Testing. All should be automated so they can be run frequently with little effort.

Unit Tests are the finest grained. The units under test should be very small, just a few lines of code, and are often written the same time as the production code. They isolate the code under test from all other components by using Test Doubles if necessary. There should be more Unit Tests than any other kind of test because they cover all paths through the code. Due to the quantity, these tests must execute quickly so they can be run frequently to provide feedback during development. They also provide the safety net when refactoring and changing functionality.

Integration Tests test the interaction between two components: testing a DAO’s (Data Access Object) interaction with the database, testing a client against a web service, etc. Fewer Integration Tests are needed because they don’t need to cover every path (Unit Tests are responsible for that); testing a few paths is sufficient to assert the integration works. Integration Tests usually take longer to run, but this is OK since there are fewer tests to run.

Functional Tests can be broken down into many categories: Acceptance Tests, UI Tests, End-to-End Tests, etc. These test the whole system, are usually the slowest running, and should be the fewest in number. They are good at detecting environment issues and testing the interaction of multiple components.

In addition to these automated tests, most teams do some Exploratory Testing as well. This *is* using the application … but with the main goals of assessing usability and finding features/behaviors which were missed. These findings often result in new stories for the backlog. Of course, bugs can be found during this testing. The appropriate (failing) test should be written, then code fixed, then the test run and the featured exercised to verify the fix.

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/75796/Use-It-Testing-in-Agile-Software-Development

Developing Eclipse Plug-ins: Program to Publish

Thu, 08 Dec 2011 13:00:00 GMT

by Tim Myer

Goal

The purpose of this blog entry is to demonstrate one workflow for taking an idea for an Eclipse plug-in from concept to product downloadable from the Eclipse marketplace.

tl;dr

If you want to look right away at working code for a simple Eclipse plug-in, feature, update site and their Tycho configurations, a sample project based on this tutorial is available. The plug-in can also be installed in Eclipse through the Marketplace (search for "save actions") or directly from an update site. In addition, the conventions for project setup contained in this tutorial have been extracted into a Maven archetype. Since this blog entry does not provide an in-depth introduction to the nuances of configuring and writing Eclipse plug-ins, having the project source as a reference, either locally or in a web browser, will be helpful for following the examples.

The Idea

Suppose we are starting a new project on an existing codebase and, as part of our Working Agreement, the team has decided to automate certain coding standards with Eclipse Java editor save actions. It would be convenient to bring all the existing code up to our standards before development even begins. Currently, formatting and import organization can be performed in bulk from the source submenu in the workbench. Eclipse users can also apply clean-up conventions from this submenu. Unfortunately, even though the clean-up and save participants have similar configurations, there is a disconnect between the two.

Exposing save actions for the bulk processing of Java/Groovy files through this source submenu is a simple enough feature to add to Eclipse and should provide the opportunity to experience one full cycle of Eclipse plug-in development, from the creation of a simple plug-in and its integration tests, to the addition of a feature to contain the plug-in, to the packaging of an update site, to the distribution of the product through the Eclipse Marketplace.

The Plug-in

We will begin by creating a new plug-in project named timezra.eclipse.apply_save_actions in our Eclipse workspace. Since we will eventually generate a Tycho configuration in order to automate the compilation, testing and packaging of our product, we will modify a few of the default settings for the plug-in. Our plug-in and fragment projects will be contained in a plugins subdirectory, here /path/to/workspace/plugins/timezra.eclipse.apply_save_actions. Our source folder will be src/main/java and output folder will be target/classes to follow the Maven convention.

We can either use the New Plug-in Project Wizard Hello, World Command Template or we can contribute a command to a menu with manual configuration. There are already quite a few resources for contributing commands through an extension point, so you can get more insight into specific configuration settings there.
For this example, the configuration is boilerplate. The internals of the ApplySaveActions command handler are somewhat interesting. There may be a more direct way to invoke the save participant, but this method works on both un-opened files and source buffers modified in memory for Java and Groovy, so in the interest of DTSTTCPW, we can use this simple implementation until it is no longer sufficient.

plugins/timezra.eclipse.apply_save_actions/src/main/java/timezra/eclipse/apply_save_actions/handlers/ApplySaveActions.java

package timezra.eclipse.apply_save_actions.handlers;

import static java.util.Arrays.asList;
import java.lang.reflect.InvocationTargetException;
import java.util.ArrayList;
import java.util.Collection;
import org.eclipse.core.commands.AbstractHandler;
import org.eclipse.core.commands.ExecutionEvent;
import org.eclipse.core.commands.ExecutionException;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IWorkspace;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IAdapterManager;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.OperationCanceledException;
import org.eclipse.core.runtime.Platform;
import org.eclipse.jdt.core.ICompilationUnit;
import org.eclipse.jdt.core.IJavaElement;
import org.eclipse.jdt.core.IJavaProject;
import org.eclipse.jdt.core.IPackageFragment;
import org.eclipse.jdt.core.IPackageFragmentRoot;
import org.eclipse.jdt.core.JavaModelException;
import org.eclipse.jface.operation.IRunnableWithProgress;
import org.eclipse.jface.viewers.ISelection;
import org.eclipse.jface.viewers.IStructuredSelection;
import org.eclipse.ui.IWorkbench;
import org.eclipse.ui.PlatformUI;
import org.eclipse.ui.actions.WorkspaceModifyOperation;
import org.eclipse.ui.handlers.HandlerUtil;
import org.eclipse.ui.part.FileEditorInput;
import org.eclipse.ui.texteditor.IDocumentProvider;

public class ApplySaveActions extends AbstractHandler {

  private final IAdapterManager adapterManager;
  private final IWorkspace workspace;
  private final IWorkbench workbench;

  public ApplySaveActions() {
    this(Platform.getAdapterManager(), ResourcesPlugin.getWorkspace(), PlatformUI.getWorkbench());
  }

  ApplySaveActions(final IAdapterManager adapterManager, final IWorkspace workspace, final IWorkbench workbench) {
    this.adapterManager = adapterManager;
    this.workspace = workspace;
    this.workbench = workbench;
  }

  @Override
  public Object execute(final ExecutionEvent event) throws ExecutionException {
    final ISelection currentSelection = HandlerUtil.getCurrentSelectionChecked(event);
    if (currentSelection instanceof IStructuredSelection) {
      final IStructuredSelection selections = (IStructuredSelection) currentSelection;
      try {
        applyTo(selections);
      } catch (final JavaModelException e) {
        throw new ExecutionException(Messages.APPLY_SAVE_ACTIONS_UNEXPECTED_ERROR, e);
      } catch (final InvocationTargetException e) {
        throw new ExecutionException(Messages.APPLY_SAVE_ACTIONS_UNEXPECTED_ERROR, e.getTargetException());
      }
    }
    return null;
  }

  private void applyTo(final IStructuredSelection selections) throws JavaModelException, InvocationTargetException {
    for (final Object o : selections.toList()) {
      final IJavaProject javaProject = getAdapter(o, IJavaProject.class);
      if (javaProject != null) {
        applyTo(javaProject.getPackageFragments());
        continue;
      }
      final IPackageFragmentRoot packageFragmentRoot = getAdapter(o, IPackageFragmentRoot.class);
      if (packageFragmentRoot != null) {
        applyTo(packageFragmentRoot);
        continue;
      }
      final IPackageFragment packageFragment = getAdapter(o, IPackageFragment.class);
      if (packageFragment != null) {
        applyTo(packageFragment);
        continue;
      }
      final ICompilationUnit compilationUnit = getAdapter(o, ICompilationUnit.class);
      if (compilationUnit != null) {
        applyTo(compilationUnit);
        continue;
      }
    }
  }

  private void applyTo(final IPackageFragmentRoot packageFragmentRoot) throws JavaModelException,
      InvocationTargetException {
    final IJavaElement[] children = packageFragmentRoot.getChildren();
    final IPackageFragment[] fragments = new IPackageFragment[children.length];
    System.arraycopy(children, 0, fragments, 0, children.length);
    applyTo(fragments);
  }

  private void applyTo(final IPackageFragment... packageFragments) throws JavaModelException,
      InvocationTargetException {
    final Collection<ICompilationUnit> compilationUnits = new ArrayList<ICompilationUnit>();
    for (final IPackageFragment f : packageFragments) {
      compilationUnits.addAll(asList(f.getCompilationUnits()));
    }
    applyTo(compilationUnits.toArray(new ICompilationUnit[compilationUnits.size()]));
  }

  private void applyTo(final ICompilationUnit... compilationUnits) throws InvocationTargetException {
    final IRunnableWithProgress delegate = new ApplySaveActionsOperation(compilationUnits);
    try {
      workbench.getProgressService().run(false, true, delegate);
    } catch (final InterruptedException e) {
      // cancellation is fine
    }
  }

  @SuppressWarnings("restriction")
  private IDocumentProvider createDocumentProvider() {
    return new org.eclipse.jdt.internal.ui.javaeditor.CompilationUnitDocumentProvider();
  }

  @SuppressWarnings("unchecked")
  private <T> T getAdapter(final Object o, final Class<T> c) {
    return (T) adapterManager.getAdapter(o, c);
  }

  private final class ApplySaveActionsOperation extends WorkspaceModifyOperation {
    private final ICompilationUnit[] compilationUnits;

    ApplySaveActionsOperation(final ICompilationUnit... compilationUnits) {
      this.compilationUnits = compilationUnits;
    }

    @Override
    public void execute(final IProgressMonitor pm) throws CoreException {
      pm.beginTask(Messages.APPLY_SAVE_ACTIONS_BEGIN_TASK, compilationUnits.length);
      try {
        for (final ICompilationUnit unit : compilationUnits) {
          applyTo(workspace.getRoot().getFile(unit.getPath()), pm);
        }
      } finally {
        pm.done();
      }
    }

    void applyTo(final IFile f, final IProgressMonitor pm) throws CoreException {
      report(f.getName(), pm);
      final IDocumentProvider provider = createDocumentProvider();
      final FileEditorInput editorInput = new FileEditorInput(f);
      try {
        provider.connect(editorInput);
        provider.aboutToChange(editorInput);
        provider.saveDocument(pm, editorInput, provider.getDocument(editorInput), true);
      } finally {
        provider.changed(editorInput);
        provider.disconnect(editorInput);
      }
    }

    void report(final String task, final IProgressMonitor pm) {
      if (pm.isCanceled()) {
        throw new OperationCanceledException();
      }
      pm.setTaskName(task);
      pm.worked(1);
    }
  }
}

The Test Fragment

We will similarly create a new integration test fragment alongside this plug-in, overriding the default configuration to store the fragment into the plugins subdirectory and to use Maven conventions for the source and output directories.

There are various approaches for testing Eclipse plug-ins, but the fragment approach has been embraced by the Tycho community, so we will use it here.
Again, the project configuration can be boilerplate for now. Of particular interest is the handler test case.

plugins/timezra.eclipse.apply_save_actions.tests/src/test/java/timezra/eclipse/apply_save_actions/handlers/ApplySaveActions.java

package timezra.eclipse.apply_save_actions.handlers;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Scanner;
import org.eclipse.core.commands.ExecutionEvent;
import org.eclipse.core.commands.ExecutionException;
import org.eclipse.core.expressions.EvaluationContext;
import org.eclipse.core.resources.IContainer;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IProject;
import org.eclipse.core.resources.IResource;
import org.eclipse.core.resources.IWorkspaceRoot;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.core.runtime.Path;
import org.eclipse.core.runtime.SubProgressMonitor;
import org.eclipse.core.runtime.preferences.InstanceScope;
import org.eclipse.jdt.core.IClasspathEntry;
import org.eclipse.jdt.core.IJavaProject;
import org.eclipse.jdt.core.JavaCore;
import org.eclipse.jdt.launching.JavaRuntime;
import org.eclipse.jdt.ui.JavaUI;
import org.eclipse.jdt.ui.cleanup.CleanUpOptions;
import org.eclipse.jface.text.TextSelection;
import org.eclipse.jface.viewers.StructuredSelection;
import org.eclipse.ui.ISources;
import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.MethodRule;
import timezra.eclipse.apply_save_actions.Constants;
import timezra.eclipse.apply_save_actions.tests.ModifiesSaveActionsPreferences;
import timezra.eclipse.apply_save_actions.tests.ModifiesSaveActionsPreferencesRule;

public class ApplySaveActionsPluginTest {

  private static final String SOURCE_FOLDER = "src/test/java";

  private static final String EOL = System.getProperty("line.separator");

  private static final IProgressMonitor NULL_PROGRESS_MONITOR = new NullProgressMonitor();

  private static final String TEST_CLASS = "TestClass";
  private static final String TEST_PACKAGE = "timezra.eclipse.apply_save_actions";

  private static final String TEST_CLASS_BEFORE_SAVE_ACTIONS = "package " + TEST_PACKAGE
      + ";import java.util.*;class " + TEST_CLASS + "{private List<" + TEST_CLASS + "> l;" + TEST_CLASS
      + "(List<" + TEST_CLASS + "> l){this.l=l;}}";

  private static final String TEST_CLASS_AFTER_SAVE_ACTIONS = "package " + TEST_PACKAGE + ";" + EOL + //
      EOL + //
      "import java.util.List;" + EOL + //
      EOL + //
      "class " + TEST_CLASS + " {" + EOL + //
      "  private final List<" + TEST_CLASS + "> l;" + EOL + //
      EOL + //
      "  " + TEST_CLASS + "(List<" + TEST_CLASS + "> l) {" + EOL + //
      "    this.l = l;" + EOL + //
      "  }" + EOL + //
      "}";

  @Rule
  public final MethodRule rule = new ModifiesSaveActionsPreferencesRule();

  private IProject aJavaProject;
  private IFolder aJavaPackage;
  private IFile aJavaFile;

  private IFolder aJavaSourceFolder;

  @Before
  public void setUp() throws CoreException {
    aJavaProject = createAJavaProject("a_java_project");
    aJavaSourceFolder = createASourceFolder(SOURCE_FOLDER);
    aJavaPackage = createAPackage(aJavaSourceFolder, TEST_PACKAGE.replaceAll("\\.", "/"));
    aJavaFile = createAJavaFile(aJavaPackage, TEST_CLASS + ".java");
  }

  @After
  public void tearDown() throws CoreException {
    aJavaProject.delete(true, NULL_PROGRESS_MONITOR);
  }

  @Test
  public void theCurrentSelectionMustBeStructured() throws ExecutionException {
    final ApplySaveActions command = new ApplySaveActions();
    final EvaluationContext context = new EvaluationContext(null, new Object());
    context.addVariable(ISources.ACTIVE_CURRENT_SELECTION_NAME, new TextSelection(0, 100));
    final ExecutionEvent event = new ExecutionEvent(null, Collections.emptyMap(), null, context);
    assertNull(command.execute(event));
  }

  @Test
  @ModifiesSaveActionsPreferences
  public void aJavaFileCanBeReformatted() throws ExecutionException, CoreException, IOException {
    enableJavaSaveActions();

    applySaveActions(JavaCore.create(aJavaFile));

    verifyThatSaveActionsHaveBeenApplied(aJavaFile);
  }

  @Test
  @ModifiesSaveActionsPreferences
  public void aJavaPackageCanBeReformatted() throws ExecutionException, CoreException, IOException {
    enableJavaSaveActions();

    applySaveActions(JavaCore.create(aJavaPackage));

    verifyThatSaveActionsHaveBeenApplied(aJavaFile);
  }

  @Test
  @ModifiesSaveActionsPreferences
  public void aJavaSourceFolderCanBeReformatted() throws ExecutionException, CoreException, IOException {
    enableJavaSaveActions();

    applySaveActions(JavaCore.create(aJavaSourceFolder));

    verifyThatSaveActionsHaveBeenApplied(aJavaFile);
  }

  @Test
  @ModifiesSaveActionsPreferences
  public void aJavaProjectCanBeReformatted() throws ExecutionException, CoreException {
    enableJavaSaveActions();

    applySaveActions(JavaCore.create(aJavaProject));

    verifyThatSaveActionsHaveBeenApplied(aJavaFile);
  }

  private void applySaveActions(final Object selection) throws ExecutionException {
    final ApplySaveActions command = new ApplySaveActions();
    final EvaluationContext context = new EvaluationContext(null, new Object());
    context.addVariable(ISources.ACTIVE_CURRENT_SELECTION_NAME, new StructuredSelection(selection));
    final ExecutionEvent event = new ExecutionEvent(null, Collections.emptyMap(), null, context);
    command.execute(event);
  }

  // contains a beaut that turns a stream into a String without using IoUtils:
  // http://stackoverflow.com/questions/309424/in-java-how-do-a-read-convert-an-inputstream-in-to-a-string
  private void verifyThatSaveActionsHaveBeenApplied(final IFile aJavaFile) throws CoreException {
    final String actualContents;
    final Scanner scanner = new Scanner(aJavaFile.getContents());
    try {
      actualContents = scanner.useDelimiter("\\A").next();
    } finally {
      scanner.close();
    }
    assertEquals(TEST_CLASS_AFTER_SAVE_ACTIONS, actualContents);
  }

  @SuppressWarnings("restriction")
  private void enableJavaSaveActions() {
    InstanceScope.INSTANCE.getNode(JavaUI.ID_PLUGIN).putBoolean(Constants.PERFORM_SAVE_ACTIONS_PREFERENCE, true);

    final Map<String, String> cleanupPreferences = new HashMap<String, String>(
        org.eclipse.jdt.internal.ui.JavaPlugin
            .getDefault()
            .getCleanUpRegistry()
            .getDefaultOptions(
                org.eclipse.jdt.internal.corext.fix.CleanUpConstants.DEFAULT_SAVE_ACTION_OPTIONS)
            .getMap());

    cleanupPreferences.put(org.eclipse.jdt.internal.corext.fix.CleanUpConstants.FORMAT_SOURCE_CODE,
        CleanUpOptions.TRUE);
    cleanupPreferences.put(org.eclipse.jdt.internal.corext.fix.CleanUpConstants.ORGANIZE_IMPORTS,
        CleanUpOptions.TRUE);
    cleanupPreferences.put(org.eclipse.jdt.internal.corext.fix.CleanUpConstants.CLEANUP_ON_SAVE_ADDITIONAL_OPTIONS,
        CleanUpOptions.TRUE);
    org.eclipse.jdt.internal.corext.fix.CleanUpPreferenceUtil.saveSaveParticipantOptions(InstanceScope.INSTANCE,
        cleanupPreferences);
  }

  private IFolder createASourceFolder(final String name) throws CoreException {
    final IFolder aJavaSourceFolder = aJavaProject.getFolder(Path.fromPortableString(name));
    create(aJavaSourceFolder);
    return aJavaSourceFolder;
  }

  private IFolder createAPackage(final IFolder aJavaSourceFolder, final String name) throws CoreException {
    final IFolder aJavaPackage = aJavaSourceFolder.getFolder(Path.fromPortableString(name));
    create(aJavaPackage);
    return aJavaPackage;
  }

  private void create(final IFolder folder) throws CoreException {
    final IContainer parent = folder.getParent();
    if (parent.getType() == IResource.FOLDER && !parent.exists()) {
      create((IFolder) parent);
    }
    folder.create(true, true, NULL_PROGRESS_MONITOR);
  }

  private IFile createAJavaFile(final IFolder aJavaPackage, final String name) throws CoreException {
    final IFile aJavaFile = aJavaPackage.getFile(Path.fromPortableString(name));
    aJavaFile.create(new ByteArrayInputStream(TEST_CLASS_BEFORE_SAVE_ACTIONS.getBytes()), true,
        NULL_PROGRESS_MONITOR);
    return aJavaFile;
  }

  // based on http://www.stateofflow.com/journal/66/creating-java-projects-programmatically
  @SuppressWarnings("restriction")
  private IProject createAJavaProject(final String name) throws CoreException {
    final IWorkspaceRoot root = ResourcesPlugin.getWorkspace().getRoot();
    final IProject project = root.getProject(name);
    project.create(NULL_PROGRESS_MONITOR);
    project.open(NULL_PROGRESS_MONITOR);
    org.eclipse.jdt.internal.ui.wizards.buildpaths.BuildPathsBlock.addJavaNature(project, new SubProgressMonitor(
        NULL_PROGRESS_MONITOR, 1));
    final IJavaProject javaProject = JavaCore.create(project);

    final List<IClasspathEntry> entries = new ArrayList<IClasspathEntry>();
    for (final IClasspathEntry entry : javaProject.getRawClasspath()) {
      if (entry.getEntryKind() == IClasspathEntry.CPE_SOURCE) {
        ((org.eclipse.jdt.internal.core.ClasspathEntry) entry).path = Path.fromPortableString(SOURCE_FOLDER);
      }
      entries.add(entry);
    }
    entries.add(JavaRuntime.getDefaultJREContainerEntry());
    javaProject.setRawClasspath(entries.toArray(new IClasspathEntry[entries.size()]), NULL_PROGRESS_MONITOR);

    return project;
  }
}

The Feature

Now that we have a tested plug-in, we will create an Eclipse feature to contain it for distribution. We can do practically all our configuration through the New Feature Project Wizard, except we want to put the feature project into a features subdirectory in the same way that we put our plug-in and fragment into the plugins subdirectory.

The Update Site

Now that we have a tested plug-in and a feature to contain it, we will create an Eclipse update site for publishing the feature. In the New Update Site Project Wizard, we will again override the default location so that our update site project is in an update-site subdirectory, just as we separated our plug-in, fragment and feature into plugins and features subdirectories.

NB: Whereas we may have multiple features or plug-ins for our project, we will have a single update site; thus the update-site project is not contained in a subfolder of update-site, but in this folder directly.

Tycho

Compiling, running integration tests and packaging an application entirely within an IDE does not scale to even a single programmer over time, let alone a team of programmers working on multiple plug-in projects. So far, the amount of ceremony for creating our menu contribution has been high, but the IDE has reduced a significant amount of the boilerplate. Before Tycho, the amount of ceremony and hackery required to get plugins, features and update sites packaged and unit and integration test suites running on a CI server had been prohibitively high. Tycho removes a significant amount of that pain. Generating meaningful poms for our projects is as trivial as going into each of the features, plugins and update-site directories and running a Tycho goal.

mvn org.eclipse.tycho:tycho-pomgenerator-plugin:generate-poms -DgroupId=timezra.eclipse

We can combine some of the boilerplate in each of the subproject poms in an über-parent pom at the root of our workspace. We will also add the Indigo p2 repository and a target platform configuration resolver since we are developing our Eclipse components Manifest-first.

pom.xml

We will also configure Tycho to use the UI test runner for our integration test suite, as well as add any platform-specific runtime plug-in dependencies or configuration to the test fragment pom.

plugins/timezra.eclipse.apply_save_actions.tests/pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
  xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>timezra.eclipse</groupId>
    <artifactId>apply-save-actions-plugins-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
  <groupId>timezra.eclipse</groupId>
  <artifactId>timezra.eclipse.apply_save_actions.tests</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <packaging>eclipse-test-plugin</packaging>
  
  <build>
    <outputDirectory>target/test-classes</outputDirectory>
    <plugins>
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-surefire-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <useUIHarness>true</useUIHarness>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <profiles>
    <profile>
      <id>osx</id>
      <activation>
        <property>
          <name>java.vendor.url</name>
          <value>http://www.apple.com/</value>
        </property>
      </activation>
      <build>
        <pluginManagement>
          <plugins>
            <plugin>
              <groupId>org.eclipse.tycho</groupId>
              <artifactId>tycho-surefire-plugin</artifactId>
              <version>${tycho-version}</version>
              <configuration>
                <argLine>-XstartOnFirstThread</argLine>
                <dependencies>
                  <dependency>
                    <type>p2-installable-unit</type>
                    <artifactId>org.eclipse.jdt.launching.macosx</artifactId>
                  </dependency>
                </dependencies>
              </configuration>
            </plugin>
          </plugins>
        </pluginManagement>
      </build>
    </profile>
  </profiles>
</project>

NB: we would use a different configuration if our fragment contained unit-tests instead of integration tests.

JAR signing

If you do not have access to a certificate from a trusted authority, you can generate a self-signed certificate with 1-year validity by a command such as

keytool -genkey -alias _keystore_alias_ -keystore /path/to/keystore -validity 365

In order to sign the jars deployed to our update-site before release, we will add a new profile with a plug-in management configuration to the über-parent pom.

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project ....>
  ....
  <profiles>
    <profile>
      <id>sign</id>
      
      <build>
        <pluginManagement>
          <plugins>
            <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-jarsigner-plugin</artifactId>
              <version>1.2</version>
              <executions>
                <execution>
                  <goals>
                    <id>sign</id>
                    <goal>sign</goal>
                  </goals>
                </execution>
                <execution>
                  <goals>
                    <id>verify</id>
                    <goal>verify</goal>
                  </goals>
                </execution>
              </executions>
            </plugin>
          </plugins>
        </pluginManagement>
      </build>
    </profile>
  </profiles>
  ....
</project>

Similarly, we will configure the plug-ins and features parent poms for jar signing.

plugins/pom.xml and features/pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project ....>
  ....
  <profiles>
    <profile>
      <id>sign</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jarsigner-plugin</artifactId>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
</project>

Publishing

We are now ready to build and test our signed plug-ins and features and to package them in an update site for deployment.

mvn -Psign -Djarsigner.keystore=http://www.solutionsiq.com/path/to/keystore -Djarsigner.storepass=_keystore_password_ -Djarsigner.alias=_keystore_alias_ clean package integration-test

There will now be a fully deployable update site in update-site/target/site.
For this particular project, I distribute the contents of this directory to the gh-pages branch of the project on github.
Since the update site for the project is publicly-available and since I have an Eclipse Bugzilla login, I can simply add a new solution listing to the Eclipse Marketplace to make the menu contribution even more discoverable by and accessible to Eclipse users.

Conclusion

This tutorial has provided the outline for the workflow of taking an idea for a single-featured Eclipse contribution from inception to delivery in a very short amount of time. While Eclipse tooling has for years provided the means to perform the tasks of plug-in, fragment, feature and update site publishing entirely within the IDE, it is the Tycho project that lowers the barrier to entry for scaling out plug-in development by making the build and test process far simpler to automate and to configure than other PDE-based build systems. Along the way, we have explored JAR signing and uploading of content to github and to the Eclipse Marketplace, and we have hopefully developed a menu contribution that others will find useful in their own projects.

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/75478/Developing-Eclipse-Plug-ins-Program-to-Publish

Agile Coaching: A Kanban Exercise

Wed, 30 Nov 2011 19:30:00 GMT

by Dhaval Panchal

Kanban at its essence is a signaling device that instructs the moving or creating in a “pull” based system. Kanban when applied within a system enables synchronization between the rhythm of demand (customer) and rhythm of production (producer). The pace or beat, of customer demand is takt time. In other words to enable just in time production of goods it is best for producers to produce goods at the same rate as the customer demands. A faster rate of production results in finished goods waiting to be consumed (inventory) and a slower rate of production results in missed opportunity costs or dissatisfied customers. This aspect of continuously tuning the producer’s rate of production to customers demand requires a continuous improvement (kaizen) mindset. My favorite article on Kanban applied to software development is by Kenji Hiranabe. Please read that article before you dive into the details of setting up and facilitating this exercise.

Time:

60 – 90 minutes

Input:

20 Coins (any denomination)
3x5 Index cards – 100 pieces
Table length to accomodate 4 to 5 people, preferably a U-shaped table
3 Stopwatches
Color paper – legal size, two different colors (say Purple & Brown) – 10 sheets for each color
Facilitator
3 timekeepers
Flip chart or whiteboard
Whiteboard markers

Setup:

Create a table as shown below on flip chart or whiteboard

	Round 1	Round 2	Round 3
1st Coin
1st Card
All Coins
All Cards

There are two processing flows that are happening in opposite directions. One for the cards and another for the coins
Processing steps for cards

Steps to process a card are intentionally complicated and one can invent any scheme do-able in a short time.

Processing steps for coins is rather simple, however dependent on chance. At step n-1, the instruction is to flip a coin until it turns to ‘heads’. In step, participants flip all coins until the coin turns to ‘tails’. Repeat this loop until the last person in chain is done.
For each round, you need less than 20 coins and the same number of blank index cards
Setup and people arrangement (click images to view larger versions):

The motivation behind creating opposite flows is to amplify the bottleneck effect and demonstrate the viability of the Kanban signaling mechanism in intersecting workflows, the same as in reality where people have to switch context and work on a different stream.

Exercise Context:

I set up the context for this exercise by making myself (facilitator) the owner/CEO of this operation. My company produces a magic set which requires a mathematically valid index card setup (see card processing steps) and a coin that has been flipped appropriately. The magic product set must have one card and one coin that make it possible for customers to do magic tricks. (All made up )

There are five team members who are producing the magic set of card + coin and three timekeepers to record production time - one for recoding 1st coin processing time and last coin processing time, second for 1st card processing time and last card processing time, and third for backup and tabulating results during each round.

Round 1

In this round, all coins must be flipped to ‘heads’ by a person on one end of the table before (s)he can pass the entire set of 20 coins to the next person. On the other end of the table, the person who is starting to process cards must process all cards by executing only step 1 (demarcate quadrants) before all cards are passed onto next person for step 2.

Soon enough a situation similar to the one below will arise:

This causes a sever bottleneck mid-stream through the round and lot of wait periods for people when it happens.

Timekeepers tabulate the time taken for 1st coin that completes the flow, the 1st card that completes the flow, the last coin that completes the flow, and the last card that completes the flow.

Round 2

After Round 1, remove the constraint of processing all coins and all cards by a person before they can be passed on the next person in processing chain. A short debrief and retrospective on Round 1 participants will show that they naturally gravitate to select small batch size, in this case the participants chose to pass a coin and card as soon as it was processed by a person.

This is enabling reduced batch size in a PUSH based system since the participants move their processed item to next step as soon as they are done.

In Round 2, the problem of ‘waiting’ is resolved by reducing batch size, however pushing artifacts to next step causes unpredictable bottlenecks to pop up throughout the process. Don’t be surprised if this round feels chaotic with participants egging each other on to complete their steps.

As timekeepers tabulate time taken, it is clear that Round 2 processed the 1st coin and 1st card faster than round 1. Improved time to market!

But as an owner/producer, you know that just being first to market is not good enough - you have to continuously supply the market with your product so that you can keep the market advantage gained by faster time to market. (For those fixing production issues using the Kanban system, simply fixing the first production issue really quickly and demonstrating no predictable rhythm thereafter does not help your customers.)

The stage is now set for Round 3.

Round 3

As a facilitator/owner in this context, there is some new setup steps required prior to executing Round 3.

Setup and Rules:

Each person has two different colored papers next to them, except for the first and last person since they only need one (see diagram above). Each color is marked either for card or for coin. A work in progress (WIP) limit is enforced by ensuring that only one coin or only one card can be placed in on their respective color paper. Only when a participant PULLS an item from the board can the downstream person work to fill the empty color space.

Execution:

As this round begins, the following dynamic emerges:

Reduced batch size eliminates wasteful wait periods as in Round 2. Establishing Kanban cards to enable PULL based system normalizes flow, preventing bottlenecks from building up. In the exercises that I have facilitated, the time for the 1st coin and 1st card processed are extremely close (almost within seconds) and the total time to process all cards and all coins is also very close (again almost within seconds).

I was fortunate to have a video recording for one of these exercises. Following is a selected transcript from the debrief. (Full transcript is 5 pages long and peoples name have been replaced by random one letters, except my name is in full.)

Debrief

Y: We were producing cards and coins at the same pace, evenly in 3rd round. we got the production flow of cards and coins..

Z: We were making money sooner and continually having product available, so we got product sooner, a more consistent delivery..

X: Versus a lot of overhead of constantly keep stuff on shelf. This avoids the problem that we gave you a big batch in January but we can’t get anything for you until march

…….

Dhaval: You said something about predictability almost 3 times, what did you mean by that?

X: Inventory was predictable, it was consistent, we had a process that was going and even though one of them was a random process you still ended up with a consistent output, because the batching system. The Kanban system allowed you to control flow and really no one was idle. Work was evenly distributed

Dhaval: So, ummm.. I liked what you said. This (flips a coin) is sort of 50% chance, it is not wildly unpredictable as some software stuff could get. And you are saying that we could control something even without knowing how much flips you are going to need.

X: Yup!

A: Not everybody is overloaded, you have only two buckets to watch for; it's not like there are hundreds. It isn’t that you don’t have work at sometimes and you are overloaded at other times.

……….

Y: In Round 3, if there is a bottleneck, if somebody is not processing well then this .. this .. signal cascades down the line and makes it apparant, right? (looks around for acknowledgement and gets some) as opposed to let’s say, if I’m the bottleneck and J is tossing over and cards are queuing up it does not become apparent down the production line.

Dhaval: very interesting, so if I restate what you said was in Round 2 the output would have been a stack of queue in front of R but in Round 3 you (1st guy to process cards) could feel the effect of R not being able to process. All this with out having R to explicitly communicate that I’m blocked!

all: Yeah! Yeah!

Y: We could feel the flow, we were all interconnected. Builds cohesion and in the same vein build that dependency between us too.

Summary

There are two things very sacred about using Kanban:

Limit Work in progress: WIP could be 1 item or 2 items or at most 5-7 items but definitely not 100.
Have a signaling system that implicitly communicates upstream bottlenecks downstream. Never exceed your WIP limit even if this means that you have to idle (wait) for upstream process steps to free up their kanban bucket.

Most Kanban implementations either blatantly disregard their WIP limit or invent new workflows for exception handling which soon becomes the norm. Instead it would serve the organization better to plain and simply idle, twiddle thumbs. Of course this would make the organization optimizers concerned for fear of underutilization. A better alternative is to utilize unused capacity to find continuous improvement experiments that will prevent similar disruption of flow in future (a Kaizen event!)

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/75047/Agile-Coaching-A-Kanban-Exercise

Programming with Groovy: Trampoline and Memoize

Tue, 22 Nov 2011 19:20:00 GMT

by Tim Myer

Groovy 1.8 introduces two new closure functions. Memoization is the automatic caching of closure results. Trampolining permits a form of declarative tail-call optimization. This article introduces the two concepts and demonstrates how to combine them in order to create cached, tail-recursive closures.

tl;dr

The example code from this article is available on github.

Simple Memoization

Creating a closure that caches the result of some calculation is as easy as appending .memoize() or one of the alternate memoize...(...) methods that allows more fine-grained control over cache sizes to a closure declaration. One benefit of memoization includes the caching of results of long-running calculations that have no side effects. Memory leaks are a potential pitfall, which is why a maximum cache size should generally be prefered.
The specification below contains a closure with a side-effect. This side effect happens just once, despite the closure being invoked twice.

SimpleMemoizationSpec.groovy

package timezra.groovy.trampoline_memoize

class SimpleMemoizationSpec extends spock.lang.Specification {

  int count

  def identity = {
    count++
    it
  }.memoize()

  def "each call should be cached"() {
    when:
    def first = identity 0
    def second = identity 0

    then:
    count == 1
    first == second
    second == 0
  }
}

Recursive Memoization

Suppose we want to memoize the results of a recursive closure call. For example, we can unroll the call tree of this naive implementation of the fibonacci function.

  def fib = { n ->
    if(n == 0) 0
    else if(n == 1) 1
    else fib(n-1) + fib(n-2)
  }

The call trace for the fourth fibonacci number looks like this.

                       ___________fib 4___________
                      /                           \
               fib 3 + fib 2                 fib 2 + fib 1
             /               \                 /
      fib 2 + fib 1     fib 2 + fib 1     fib 1 + fib 0
       /
fib 1 + fib 0

NB: The closure here is entered nine times but we can see that it only needs to be entered 5 times because 4 calls are repeated. If we cache the results of the fibonacci calls, the complexity of even a naive implementation such as this will increase linearly, rather than exponentially.

Unfortunately, because, in Groovy 1.8, a closure cannot invoke another closure directly, memoizing this closure is not entirely straightforward. The example directly below does not work.

RecursiveMemoizationSpec.groovy

package timezra.groovy.trampoline_memoize

class RecursiveMemoizationSpec extends spock.lang.Specification {

  int count

  def fib = { n ->
    count++
    if(n == 0) 0
    else if(n == 1) 1
    else fib(n-1) + fib(n-2)
  }.memoize()

  def "calls should be cached"() {
    when:
    def actual = fib 10

    then:
    actual == 55
    count == 11
  }
}

The stack trace when a closure calls itself.

groovy.lang.MissingMethodException: No signature of method: org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.doCall() is applicable for argument types: (java.lang.Integer) values: [9]
Possible solutions: call(), call([Ljava.lang.Object;), call(java.lang.Object), call([Ljava.lang.Object;), findAll(), equals(java.lang.Object)
  at timezra.groovy.trampoline_memoize.RecursiveMemoizationSpec.$spock_initializeFields_closure1(RecursiveMemoizationSpec.groovy:11)
  at groovy.lang.Closure.call(Closure.java:410)
  at groovy.lang.Closure.call(Closure.java:423)
  at timezra.groovy.trampoline_memoize.RecursiveMemoizationSpec.calls should be cached(RecursiveMemoizationSpec.groovy:16)

Since methods can invoke memoized closures, the solution is to invoke the call method on the closure.

RecursiveMemoizationSpec.groovy

class RecursiveMemoizationSpec extends spock.lang.Specification {
....
  def fib = { n ->
    count++
    if(n == 0) 0
    else if(n == 1) 1
    else fib.call(n-1) + fib.call(n-2)
  }.memoize()
....
}

NB: The un-memoized version enters the closure 177 times, but the memoized version enters just 11.

Trampoline

Declarative tail-call optimization is as simple as adding .trampoline() to a closure declaration and ensuring that recursive calls to the closure invoke the trampoline method on the closure instead of the closure itself. Some problems are more clearly solved with recursive solutions, but without automatic tail-call optimization in the JVM, recursion can quickly lead to an explosion in the size of the call stack. Trampolining is one work-around for this design tradeoff (or defect).

A tail-recursive fibonacci closure:

  def fib = { n, a = 0, b = 1 ->
    if(n == 0) a
    else fib n - 1, b, a + b
  }

By tracing the call stack, we can see its linear growth without memoization.

fib 4
  |
fib 3, 1, 1
  |
fib 2, 1, 2
  |
fib 1, 2, 3
  |
fib 0, 3, 5

In order to avoid a java.lang.StackOverflowError for sufficiently large inputs, the tail-recursive closure must be explicitly trampolined.

TrampolineSpec.groovy

package timezra.groovy.trampoline_memoize

class TrampolineSpec extends spock.lang.Specification {

  int count

  def fib = { n, a = 0, b = 1 ->
    count++
    if(n == 0) a
    else fib.trampoline n - 1, b, a + b
  }.trampoline()

  def "tail calls chould be optimized"() {
    when:
    def actual = fib 1000

    then:
    actual == 1556111435
    count == 1001
  }
}

In this example, the trampolined closure is called 1001 times. If a method in Java were to call itself 1001 times, the stack would be overwhelmed. By trampolining, Groovy turns this recursion into iteration.

Memoizing a Trampolined Closure

Suppose we want to cache the results of a computationally expensive tail-recursive closure with no side effects. A trampolined closure that calls itself can easily be memoized in a separate closure declaration.

OneTimeTrampolineMemoizationSpec.groovy

package timezra.groovy.trampoline_memoize

class OneTimeTrampolineMemoizationSpec extends spock.lang.Specification {

  int count

  def fib_aux = { n, a = 0, b = 1 ->
    count++
    if(n == 0) a
    else fib_aux.trampoline n - 1, b, a + b
  }.trampoline()

  def fib = fib_aux.memoize()

  def "top-level trampolined calls should be cached"() {
    when:
    def first = fib 1000
    def second = fib 1000

    then:
    count == 1001
    first == second
    second == 1556111435
  }
}

NB: This solution caches the top-level trampolined closure, not the results of the intermediate calls.

Trampolining a Memoized Closure

Suppose we want to cache the intermediate results of a trampolined function call. For example, in our trace above, suppose we want to cache the results of fib 4 and fib 3, 1, 1 and fib 2, 1, 2 and fib 1, 2, 3 and fib 0, 3, 5. Again, this situation is not straightforward because no closure can call a memoized closure directly, so in this case, the trampolined closure cannot call a memoized version of itself directly. Again, we can use the call function on the memoized closure.

FullTrampolineMemoizationSpec.groovy

package timezra.groovy.trampoline_memoize

class FullTrampolineMemoizationSpec extends spock.lang.Specification {

  int count

  def fib_aux = { n, a, b ->
    count++
    if(n == 0) a
    else fib.trampoline n - 1, b, a + b
  }.memoize()

  def fib = { n, a = 0, b = 1 ->
    fib_aux.call n, a, b
  }.trampoline()

  def "all trampolined calls should be cached"() {
    when:
    def first = fib 1000
    def second = fib 500, 315178285, -1898383934

    then:
    count == 1001
    first == second
    second == 1556111435
  }
}

Conclusion

Trampolining and memoization are two powerful new features in Groovy 1.8, but the combination of the two is not always straightforward. This tutorial has presented strategies for combining the two and for working around some of the limitations of their combination.

View this original blog post here: http://www.solutionsiq.com/resources/agileiq-blog/bid/72880/Programming-with-Groovy-Trampoline-and-Memoize