Social IT outbursts

Writing requirements: assembly manual or design manual? Don't do both, it just confuses me.

2018-02-21T16:18:00.000-08:00

Requirements should be written two different ways. Of course, some people try to get away with not writing them at all. And some people try to write them both ways at the same time. But, let's focus.
You can write requirements for project management completion or business process value creation. This is the difference between an assembly manual and a design manual. Just because you can build a soap box derby car doesn't mean you know how the brake works let alone how to make a better one.

My main complaint these days is that projects manage "requirements" but are really just collecting expectations users have. Here's a few random wants, needs, or expectations, but not requirements.

Joey wants parts that optimize speed like Z-glas wheels.
Phil likes rear ballast weights for steering control and speed. The result of the combination of the wheels and weights is not known.
Mom is worried the brakes could fail.
Dad wants to make sure we win at least one race.

Now go build that.

Soap box derby management

Agile project management styles promote project requirement writing in user stories. The stories remind us of what people want. Joey's story says "As a racer, I want to go really fast so that I can become the state champ." To translate that into something useful, projects need a map of activities and tasks to organize the moving parts so the stories provide useful input:

Car assembly (activity)

Put fast wheels on the body spindles (Joey).
Stabilize steering with weights in rear of the body (Phil).

The other stories get put somewhere and maybe in a new category. Nobody has a standard process. Nobody really defines them well. Somebody will likely want to have the wheels in the shape of a kitten, etc.

Programmers and especially software testers want scenarios that describe the process well. They abstract out the system and sub-systems and provide details in a use case that describe the steps of the process.

You can't assemble a car from a use case diagram but you can test it after you figure out what the design should be. And you see that the brake system is probably separate from the steering system. And you know how to add to the functions if you want.

Both types of requirements should be done so that the project is implemented successfully and the operation of that process continues to show good results. But with pressure to deliver quickly, the project style is always a bright light that management moths are attracted to. The old-school requirements document that instructed the software developers how to organize and build the software has become more and more the responsibility of the software developer due to increasing complexity.

Project requirements

From the project schedules in a Gantt or PERT format to the process maps today, we still just want to keep our work progress under control. Requirements prepared for project management include:

chunks of project activity ordered by completion time and priority
associated documents with that chunk that help to create it
and are in the context of project building only

The target audience of these requirements are the project managers those interested in project progress. So the document creators will write these documents to provide

good metrics and
progress summaries

In an Agile project, elicitation happens at the user level with the stakeholders providing their user stories from their point of view. The user story is a marker for the project manager to take care of their request and for more conversation later when appropriate. The user provides the priority and the value as they see it. The user story provides the raw materials for the chunks of project activities and tasks (or goals and journeys) that are then mapped to show project progress.

Because there's no real unit for measuring the scope of the sticky note task for what to do, the application of different names and styles are not easily understood. Sometimes there's a goal that seems like a higher level activity but it could be a feature. Maybe there's a theme but that could be just a descriptor to group various chunks. Even user stories are re-purposed to a minimal viable sentence style to fit on a sticky note that just look like a smaller activity.

The mapping style process that generates hundreds of sticky notes is not mapping a project as much as it's trying to outline the entire process for development. It looks impressive but misses the goal of providing good metrics and summaries and documenting too many of the details. Mapping should be about understanding how to deliver simple and important features so you get feedback quickly. Remember the Agile principles! Here I agree with Mike Cohn's point about the story as a conversation point.

Projects are about people that get stuff done. Short-term successes of project oriented requirements documentation can be measured by

end products for completing one or a group of chunks
satisfying the person the activity was elicited from

At the end of the project, you take a breath, see if you did a good job, and then go on to the next project. Goals for project requirements are to

complete the project under budget and on-time
minimize risk during implementation

Process requirements

Requirements for software development or any process decomposition that seeks to uncover better value involves

chunks of process descriptions ordered by size and trigger
descriptions of data, rules, and process qualities
context of system's SLA, other systems, and management systems

Here the target audience for the requirements are the developers or implementers of the process. Notes about how the system might print a report or the way that it should work with other systems are useful. The documentation provides a high level of

communication and
understanding as a result

In a more traditional project, the elicitation starts at the user level with the stakeholders providing what they expect from their point of view. That's not too much different from the Agile style. User expectations are collected through workshops, interviews, and document inspection. Analysts create testable requirement statements from these.

What should happen next is that the project should be decomposed by use case processes to provide visibility to the business for what processes are requested and how they function. Often this is skipped and the code and its class diagrams provide enough functional visibility to technical people because of excellent tools. The unfortunate result of passing the buck here is to let the technical staff make business decisions. This also removes quick decision making from the management because it's been encased in code.

At the end of the requirements structuring, you can test whether you have done a decent job or not. Short-term successes of process requirements are measured by

verifying all requirements are testable
grouping process chunks to the same granularity
having one chunk size end with business value being realized
separating functional from non-functional requirement types

When you complete the development process, you end up with a product you hope will last for many years through many changes. Goals for process requirements are to maximize

adaptation to change through structures with low coupling and high cohesion
realized business value

Combinations

If you try to merge the two types of documents or skip either one, you'll end up creating less quality in your software and less value in your results. The purposes of each type of document should be clear. Project documents tell you when and how much you can build at one time. Process documents tell you where the value is and let you understand how to manage the automated decisions.

Keep the roles and documents focused so you don't confuse me. If you need to, the process document can be split up into project friendly pieces so it can be used like a work breakdown structure. I'm of the belief that a goal-driven use case is equivalent to a work package myself. Arrange pieces into iterations with IDs from the use cases based on priority and size or complexity. Track the user stories back to the functional or non-functional pieces.

If you want to win a soap box derby, ask what the requirements are. Then build your process documents to deliver that value from the high level on down to the design level if you need it. The last step is to build your project document to get your soap box car out the door. Now, go win that race.

Understanding three use case types in process modeling - not just system or business

2017-11-26T12:48:00.003-08:00

Use cases give us a detailed understanding of a process to optimize or build into software. When use cases go bad, and they do, they take up space and no one uses them. I've been finding out as I teach them more and more, the main reason people don't use use cases is for lack of detail. So I've built more and more useful detail into the templates that I use for my classes.

Details, details

Let's take just one small entry in the metadata of the use case template: the type of the use case. The type should tell you about the context of the use case. If your analysts understand the context, there's no need to include this in the documentation. But analysts are usually given the context of a software system to be built and the context of the project that will be driving that software to be delivered and requirements come from all angles.

I was told there were two types of use cases: system and business. (I think there's a different set but that's coming.) Not enough examples of detailed business use cases with sequential tasks listed in the course of events are being produced. Maybe it's because we don't understand how to do it.

One solution to making your job more tolerable and eliminating those confusing and bothersome requirements is to ignore the complex non-system use cases. Another simplifying solution is to just grab the large scope names of the business use cases and not worry about the details. Then there's the solution that says if it's important enough to know the details of the software workflow, the business process we're planning to automate, why isn't it important to know the details of the business workflows also? Let's do that. The devil is in the details.

Two traditional types

The first and major type of use case traditionally is the system type when we design interactions around software. The trigger to the process is when someone or something chooses something from the main menu or a system hits our API. Sometimes, that main menu is a log-in part of the process that protects the value of the use case by keeping unwanted people out and those that we give access to the rights to initiate the use case.

Then there's the business type, which from a lack of detailed examples, no one does. I think the best work on doing enterprise analysis to show how UML can be used to draw diagrams that the business doesn't want to use is Eriksson and Penker's Business Modeling with UML - Business Patterns at Work (2000). These Swedes have a beautiful system that technical people can admire for completeness, details, and symbols that look just like software diagrams. But it's the business people that need to be drawing, updating, and using these diagrams and they sure don't want to spend time with UML to do that.

One of my pet peeves is the UML way to draw a business use case. The use case symbol is the ellipse, also known as an oval for the non-technical crew out there. You put the name of the use case in there leaving little space for anything else. Then you draw a diagonal line inside the oval making it look like you had a masking problem with another symbol. And if you followed UML to put the extension points in there as well, you'd have an unreadable ellipse. Let's just use a stereotype to say <<business>> so that the category is clear when there's no system boundary box in sight.

Even Erikkson and Penker don't get down to the level of detail that is important to start with on a business process. But they do a bang up job on high-level structures. The level of detail that is necessary for a system use case to be useful to build for the developer is fine-grained down to a task level that needs no other explanation.

Maybe we don't think like Swedes here in America and don't take the time at a high-level to plan our goals. We think the market is moving much too fast for that and like to do shorter-term planning. Do we see the Swedes competing well in any American market other than build-it-yourself furniture? Like how many people do you know that own a Volvo?

A pure start

Moving on to how I see the process types being divided up, they start with a set of tasks performed by a human. That is what I would call a pure business use case. This business use case

starts when a trigger sets it off,
moves through a set of sequential tasks,
ends with significant value for the business and
probably has documents created to support it.

There's value in promoting merchandise, there's value in making a sale, there's value in fulfilling a sale, and there's value in getting payment for a sale. There's even value in accepting a return for a purchased item.

There's no talk of automation in the pure business use case and if someone wants to use a tool, then that is part of the design for fulfilling the process tasks. That's what increases efficiency. The task sequence described in the pure business use case is what increases effectiveness. Both can increase value for the business.

System use cases made easy

I encourage my students to write their system use cases as pure business use cases that will be automated by the programmers. That clarifies what types of assumptions they have made about what they will be building. What if we eliminated the design constraints and changed all those "system displays a main menu" types of tasks into a business pure "system prompts for what to do"? Then the system use case turns into a pure business use case and the system role becomes a business role of some sort. I think it's better to think about how a person might handle the system functionality when writing use cases.

But this next type of use case pops up from every single student that I've ever taught and one that lets design elements creep into the use case tasks. It's expected in many industries because of what they already have accepted before the project began or in other words, elicited as a constraint. So this is called a system with constraints type.

Students always want to "submit" the data that is collected. They also want to use buttons, keyboards, mice, screens, menus and every other standard GUI option they think is normal. But in doing so, the tasks are now explicitly describing a design that must be used. When a web site is being designed, the design constraint of using a set of web languages and a web server has already been decided. Why should we follow that architecture if the requirements don't really point us in that direction?

So, not to fight the common usage of this type of use case in business, it's just easier to put a field at the top of the form to capture what the design constraints are that have already been decided to use before we capture the requirements. The requirements then have the freedom to talk about all those clicks and hovers they want. But let's call it a system process with constraints.

Distributed tasks

The last type of use case is the distributed use case, a mix of business and system tasks or different systems, and one that I've seen no documentation on. There's no UML approved designs for it and no discussion about it that I know of. But as soon as I started to do business use cases for a real scenario, it stuck its oddly shaped head up and stopped me from using the other two. I could have changed the business process to manage, but the sequence of activity is so common, it has to be dealt with by changing the way we do the use case.

Another use for the distributed use case is when you have real distributed processes that are seen as a complete goal and within the scope of development. An example might be when microservices or API calls provide less than a goal value but are integrated in working with a web page to deliver the end result. I do have a problem with creating a use case just to validate an address on the server because that is less than a business goal that a use case should end with.

The distributed use case does a step or two in one system, existing or to be developed, and then does a business activity or a different system's task, like talking to a person to acquire information. Then it cycles. The process is distributed over two activity flows. Think of ordering at a fast-food restaurant. There the actor initiates the food order and uses an automated system for the customer as an agent or proxy because it is too complex. The process goes like this: the actor-agent

greets the customer (business task)
opens a new order (system task)
asks the customer for their item to order (business task)
selects the item, enters the quantity, checks for additional questions or restrictions (system task)
gets needed data from the customer (business task)
enters extra information (system task)
informs the customer of information (business task)
confirms end of order with customer (business task)
sends order to be processed (system task)
tells customer the total of the order, how to pick up their order and says goodbye

An extension point for optional multiple item order which is clumsily handled by trying to show a loop is better handled by adding the extension point at #8 and sending them back to #3 as an option for the extension "Add another item to order."

Now the problem is how to diagram that. I propose using two ellipses with <<system>> and <<business>> stereotypes and showing the dependency of the system to the business use case with an <<include>>. The business use case with acknowledge its dependency with a use case name that has a system usage in it. Here's how it would look:

The diagram also shows that it is possible to do an entirely business use case to take an order and then later, do an entirely system use case to create the order.

Course of events layout

You might think this distribution would cause some confusion in the course of events. There are some issues. But the first thing to do is to label the actor correctly, the one whose only job is to initiate the use case, and then add those other supporting roles, aka "supporting actors", as the customer and the system.

Maybe a two column layout would be better to separate what the business use case has different from the steps in the system use case. That way you wouldn't have to duplicate the system use case as a distinct use case when it copies the steps from the business use case. Or just make sure you put what use cases are included at the top in the metadata and manually keep them in sync.

But for now, this reorganization of use case types massively helps my ability to model real life business use cases and I hope helps those who would like to document their business processes. Having an understanding that the use case starts with pure business functionality clarifies what a basic use case is about. Then as we automate some portions or all of it, the type of the use case changes and clarifies how to model it.

A static new web site - Zurb's Foundation with Panini

2017-05-25T15:11:00.000-07:00

I've been meaning to update my personal web site over the last ten years but just never got around to it. The web evolved. My site has been through basic html, custom php, custom JSP and Java filters, and one attempt at using a CSS framework called Semantic UI. The defining moment of need for change was when my web host was attacked by hackers in a DDOS attack lasting over a week.

Analysis

It wasn't something I could spend a lot of time maintaining so spinning up Spring, WordPress or another unreasonably complex framework that I didn't need wasn't the answer. I had several basic requirements:

Very low and easy maintenance
High data use for link pages
Cutting edge technology I can teach

CSS transpiler
JavaScript transpiler if possible
CSS/JS framework

A few requirements were worth noting as negative requirements:

No need for security
No need for tracking state
No need to collect data

Design

The client-side frameworks getting all the buzz like Google's Angular and Facebook's React were slanted towards developing mobile sites and that was not where I was going since it adds so much complexity to the site. I had thought more along the lines of a serverless architecture but it looks like that is more bleeding edge. But it was simple. Or it will be at least in a few years.

I love that the WebAssembly team is moving along with a new game demo and no need to use Google Canary, but will have to face committees and meetings for probably another five to fifty years. But the end product will be a compiled client for browsers based on client-side frameworks just beginning to explore that territory.

The other framework style I decided to investigate was the static site generator. I could have static data files, static pages in template form and generate the pages as I had updates. No server side language. It's the perfect development workflow when you have no need for complex forms which is typical of a marketing site or a blog.

Jekyll was the big contender but I didn't want to deal with Ruby. Hexo seemed like a possibility but it uses ejs templating which seems so dated. I liked mustache styled templates after teaching and trying all sorts of templating styles. But I was running in circles investigating the various popular frameworks at StaticGen which ranks them and allows me to compare them easily, No solution yet.

So the next strategy was to ignore that and look at my CSS framework. It was going to be either Twitter Bootstrap or Zurb Foundation. I've had enough problems in class with getting around Bootstrap's site and using a simple template that it wasn't my first choice. And the fact that it's harder to read and has fewer JavaScript components than Foundation was why I decided on Zurb.

Development

The thing I never noticed was that Zurb's developers use their own static site generator (they say flat file generator) called Panini with Foundation to build all their web sites. I like anyone who eats their own dog food. It's simple and uses Handlebars templating, SCSS, and either JSON or YAML formats for data. I installed it with Foundation's CLI and after two days of learning Handlebars and YAML, I was hooked. The Foundation CSS and JavaScript isn't that important to structure the site and will be added more in the future.

My pages are just simple combinations of cards for now. On a partial (included) page component, I have two Handlebars loops producing multiple cards with multiple links on them. Logic in Handlebars lets me work with data optionally.

The responsive flow of the HTML5 multiple columns is simple and not based on any grid. I'll be refactoring the style attributes out of there eventually so I can use better SCSS style management and using much more of the CSS from Foundation but for now the default styling is adequate.

The partial page that holds the card component is only just a shell that declares the right data file and itself is included in a basic page layout template. The data is inserted when I build the file. Panini spins up a development server with BrowserSync to update the pages as I change the html (but not the data so far, I think I can tweak that though. Update - Kevin Ball of Zurb made the change to the Gulp file which takes care of this. Thanks for being very responsive!)

And the YAML data is easily understood, updated, and the schema customized just like a NoSQL style database. It has minimal delimiters and uses indentation for nesting. If you are off by one space, though, you will get some odd error messages when you try to view the page.

The final Gulp build combines all the CSS in one file, and all the JavaScript in one file so there will be minimal downloads and also does image compression.

It's a slick set of tools made as easy as possible for the flexibility that you get. I might be tacking on some JavaScript code to log sites that don't respond or collect other data especially from forms. I'd have to combine this with my local site and regenerate it again. But for now, it's fun again. Hopefully, this will last another ten years.

Lunch Seminar: Thu Nov 3 - Is Angular the future of web development? WebAssembly is coming.

2016-10-31T11:39:00.002-07:00

If you are interested in my last blog, Is Angular the future of web development? WebAssembly is coming, then you might be interested in my presentation of the architectural topic with more detail at my school here in Kansas City, Centriq Training. Sign up at the event site.

The talk lasts about 2 hours and a free lunch is provided. I think strategic technical planners will get lots of value out of the overview.

Is Angular the future of web development? WebAssembly is coming.

2016-09-23T10:52:00.002-07:00

Web development has passed through three major architecture patterns. And a fourth is on the way which will add more choices for designers and keep developers busy as they blend the types both by design and confusion.

Google's Angular framework is an example of the latest style of architecture and with the release of Angular 2.0 has demonstrated that the Single Page Application is getting easier to develop. But it's not just another framework. It's on the architectural path that the web is taking in general. The question I want to address is whether or not Angular will be a long-term solution for the web. In doing so, the choices will become clearer how other styles are appropriate or not based on business needs.

The early server side web site

The 1.0 web, a term used after the web had evolved some, was the basic setup where the developer created pages in a text editor, loaded them on the web server, and a browser requested them for viewing. That's it at a high level. There were millions of documents that were available but no search engine like we have today. Directories indexed files like librarians cataloged books. Most of the cutting edge activity occurred through the 1990's but business and other people learned how to make static web pages well into the 2000's.

Web 1.0 - Development workflow for static web pages

As web servers evolved to handle more and more processing besides just delivering pages to viewers, form processing, state management, and templating systems increased in complexity. Eventually, the server became the point where pages were built using a favored language injecting data stored in a data base. .NET, Java, and especially php were the dominant processing languages for creating a dynamic web site refreshing the page for every viewer and every request.

Web 2.0 - Dynamic web site with server side page generation and client side JavaScript UI

This style I associate with the Web 2.0 name, has been the driving enterprise architecture since the 2000's and continues to dominate page generation for most corporate web sites. The web server has enlarged its scope of functionality to add features that caused the name change from web server to application server on the Java side. The Microsoft side just adds features. And now nginx is becoming the dominant web server, taking over Apache's httpd server lead as the top dog for handling web requests, due to its speed and low or no cost.

The web 1.0 returns

The architecture where the web server does no web page processing may seem like a historical starting point of design but it's really a better solution for achieving the fastest possible speed for a page request. Other use cases emerged in the last few years that put typical Web 2.0 server processing operations in the development workflow instead of the on-request process occurring during the page request.

For instance, if you have no or little updating to do on your web site, say once a month like many blogs need to do, you don't need to provide an up to the minute current state of the web page provided by processing out the page on every request. The hack that added complexity to the design to overcome that problem was caching but it has issues, so much so, that the recent HTML5 caching proposal has been shelved.

The easier solution is to create the pages in the development phase once or as needed by the business, upload them to the server and not do any page building on the server. This is called static site generation and can be achieved will any language. Templates, JavaScript, CSS, and any kind of technology can be used and the user will only get the fastest possible delivery.

Web 1.0 - Development workflow for static site generation

Other types of non-server processing workflows in the development cycle can be cobbled together with tools to advance the poor state of affairs with the old technologies of CSS and JavaScript. Recent advances to help build better, faster, more scalable code have resulted in multiple languages to transpile, or convert into another language, new languages like CoffeeScript, Babel, and especially TypeScript to JavaScript. LESS, SCSS and SASS are the leading languages to transpile into CSS providing an easier set of style rules to update and manage.

Web 1.0 - Development workflow for CSS

Web 1.0 - Development workflow for JavaScript

My personal favorite tool I use in my development classes for these types of enhancements is Microsoft's Visual Studio Code due to its great support for TypeScript and extensions that keep me from making a patchwork of grunt script calls to JavaScript packages. But JetBrains' WebStorm might be my pick if I were a paid programmer because of the amazing feature set it has.

The mobile 3.0 web app

So as the browser expanded to handle the needs of providing the user with a better experience, the renderer sped up, and the JavaScript engine sped up immensely, making game developers much happier. JavaScript's ability to process so much so fast enabled the UI of a web site to support a full application and add more features like the web server did. Mobile phones and tablets were trying to provide the same experience but with less bandwidth so the round trip to the server to pick up resources was not a good idea.

Web 3.0 - Web app with client side execution using scripts

The transfer of resources and the processing of them on the browser was the ideal and then managing all the operations of an application except for supplemental data access and security became the goal. Even those services are outsourced leaving the application with the ability to be wrapped in a hybrid application shell like Ionic so it could be deployed using a marketplace service from Apple or Google. Google recognized the need for outsourcing the basic non-client services and has been putting a lot of effort into their Firebase project.

The UI operations handled previously by a proprietary technology like a Java applet, Silverlight, or Flash, now with the demise of those non-stardard add-ons, have to be managed with the only possible solution on the browser of JavaScript. But our ability to do full fledged client-based applications is increasing as we are able to produce better code for application building libraries.

Angular is that new kind of application framework that has finally figured out a blended solution for how to take bundled resources of HTML, CSS, and JavaScript and manage the user interaction on the browser so that no server side page building needs to occur. It uses a library of components to interact with for the app that you build into a browser's web page. Now it's a web app and not a web site. The routing, controller logic, data binding, and templating are all features of its client side framework that creates the single page application.

Execution environment maturity

An execution environment is anywhere where code runs under a controlled context. for example, the database server has stored procedures, the mainframe has COBOL apps, the midrange computers have RPG programs, PCs have Windows and Macintosh apps, and phones and tablets have proprietary apps. Even embedded systems are under the control of a type of small to large chassis that takes care of the computing such as an ATM machine, a cash register, or a flight recorder. We're seeing the flourishing of familiar development environments Linux and Windows for Arduino and Raspberry Pi single board processors.

But the systems evolved as did other execution environments. Code structures all started with a basic script. Then the code became structured with reusable functions. And then libraries evolved to package up groups of cohesive operations. And then those scripts became faster by compiling and building the code into a machine readable format. This is the maturity cycle that has happened and is happening with web languages.

The browser is an embedded execution environment. It runs under another execution environment but by a set of standards that allows for the same results to be expected from any OS and any brand of browser. The standards allow us to see the browser as a viable execution environment to code to. It's even possible to provide the browser execution environment without an OS as we've seen with the introduction of the Chromebook.

The final stage of web programming

So with the emergence of client side programming on the web, we have begun the full acceptance of the web browser as an execution environment which supports the mobile crowd well. We will see the maturity of the Web Components ideas both supported by and combined with JavaScript frameworks for many years to come. Polymer is an excellent example of this. The Angular framework now looks like a jQuery UI plug in framework for browser apps and will continue to grow with more "plug-in" libraries like the current Angular Material project.

But the final 4.0 step will be when we convert our JavaScript code into compiled code. This is now on the blackboard. It's called WebAssembly. We will keep our browser JavaScript APIs only they will be available to us in every major language like C#, Java, VB, php, and of course, JavaScript. Then we will compile our code down to the WebAssembly format and load it on the browser. All major browsers have committed to supporting this standard so they will add an engine to execute the code replacing the JavaScript engine's functionality increasing the speed of our apps around 10 times.

Web 4.0 - Web app with client side execution using compiled code

The structures of client-side development that we are designing will all be stable and useful but the code will be done in Visual Studio or Eclipse and new IDEs. I am guessing that the ability to start doing this kind of work will be in another four years or about 2020.

Angular's future

So, does this mean that Angular will fade away? Not likely since it's been the trailblazer for a successful framework in JavaScript. I expect to see clones in most languages for WebAssembly like Jangular and Nangular as they might be called. But the dominance of JavaScript as a primary language, the need for TypeScript, and the requirement to know the JavaScript set of development tools are all going to be less important in the near future.

I'm excited for the future of the web. It's not the web as we knew it. It's a new development environment that will influence other environments such as television and other places where we might see an embedded browser execution environment. Maybe we'll see new types of hardware environments emerging that tailor to VR instead of just putting the browser to poor use on our web-enabled refrigerator.

Whatever it is going to be, we can be assured that the browser has blossomed fully as a development environment and the rate of design change and churn of projects related to that will slow considerably. The design choices are stable, the languages are looking stable and it's time to mature our processes surrounding the use of the web.

Complete testability - analysis quality essentials for the confused

2016-08-29T14:53:00.002-07:00

Requirements are usually described as having an intertwined bunch of qualities we aspire to have. In reality, it ends up like a person admonishing you to lead a good life. Engineers took a lot of time listing these qualities out in documents like the IEEE Standard 830-1998 superseded by the more impressive 29148-2011 defining qualities like

measurable
testable
consistent
correct
complete
unambiguous
ranked
modifiable
traceable
feasible
independent
necessary
non-redundant
terse
understandable

Most people don't want to take the time to assess a list of qualities like those on a set of a thousand requirement statements, nor do they think it has any value. They're right but they probably don't know why. They want their requirements to lead a good life, but there's so many options it's hard to focus. Here's some advice to help people focus.

The two most important qualities of requirements that are useful to keep in your head as you write documents for projects are easy. All the others are dependent most likely when these two are followed. All the important documents for modeling requirements are covered when these two are followed. The ability for the documents to be understandable and communicate well will be enforced when the two qualities are paid attention to.

The qualities we seek to promote to the top of our analysis minds are testability and completeness. Let's call it complete testability for short. They have the most value for peer reviews because they are easily remembered. And they handle the different levels of granularity well addressing the needs of the single requirement task as well as the larger requirements in scenarios and entire projects.

Testability per requirement

To be testable we must be able to write a test for a requirement. If we say we are going to arrive at work, we can't test that because we don't have a metric to measure by. Business people have seen that kind of task irresponsibility led by vague promises and the need for follow-up at a specific time or day. An assigned task requires a test to see if it was completed. Those metrics have to be known ahead of time.

If we say we are going to arrive at work and our only metric is that of time we have to give some sort of unit and measurement to the test. That unit is in minutes and that measurement will be at 8 am. Now I can send someone to my office and have them check to see if within one minute I show up at the office at 8 am. Test done.

Tests can not be formed from a requirement that is negative unless there is a constraint involved. If you said you will not be at work today, you have a negative requirement to test but limited by a time constraint. Still, a person would have to spend the entire day waiting for the event to happen from 8 am to 5 pm just in case you showed up for one minute to invalidate the test. There would be no end to the test if there were no constraint so no final test result would occur.

So a test requires both a positive event and a way to measure it with a metric. The metric involves measuring something with a unit. Writing the requirement this way creates a formal approach that eliminates most bad requirement writing.

Completeness per project

The other type of requirement quality that works to create great requirements is really not about the individual requirement. The completeness check is about traceability and dependencies throughout the project. If you know you have to be at work today, you might want to think about all the other days you want to be at work. This is looking at the requirement as a specific and making a generality out of it. You can find more requirements for days you should be at work this way.

The other type of completeness check involves the going from generality to specific. In this case of going to work, when you go to work, should you stay at work all day? Can you leave for lunch? How long should you be gone before it's not seen as being at work a full day? If you drill down into just the lunch period for more detail, then you can ask if taking a longer lunch but talking about work counts as working?

Another type of completeness check might involve switching the role or actor of that scenario or looking for alternate scenarios. Is the work day the same for all job titles? How about if we talk about a different type of work activity like being on the road? Now being at work today takes on a different approach. Also in the scenario of a use case, we can ask if we captured all of the sequential tasks of achieving our goal.

Even looking at one specific quality for completeness can be rolled up into asking whether we have captured all the qualities that should be checked for all of our requirements. Now we are asking as an analyst about what we have forgotten. Sometimes, it an implicit requirement that no one has said but everyone expects. Or maybe it was just forgotten. Don't let this lack of completeness analysis turn into scope creep you blame on the users.

Complete testability

I do like completely understanding and testing the requirements so much that the documents I write can be used in their entirety of content and structure as test documents for successful test cases. It's also in my mind the reason that programmers have taken to Test Driven Development which is just making requirements testable in code editors because they didn't get good requirements in the first place and feeling that a giant methodology is an administrative waste of time.

If the developers can change how they write code, surely the analysts can change to a more structured approach that keeps emphasizing the testability and completeness of the requirements document. It does mean that an analyst now can't just document what they hear. They will have to apply a judgment using some rules about what is complete and testable for their environment.

But the end result will be that the business or individual will end up with a better and cleaner implementation in code or business process, that will be easily tested and create more feedback to improve the quality of the end product.

Ranking rain and risky business events with confidence in strategy and testing

2016-07-28T18:57:00.002-07:00

Risk isn't measured easily. It's better viewed as a personal understanding of probability applied to avoiding unpleasant situations. It can also measure positive chances but we'd rather know what the chance of rain is instead of the chance of sunshine. The point of risk management is to find out where risks are and how to mitigate them if possible. Why are we obsessed with measuring it when it's really a useless number?

I am amused by meteorologists who say they measure the probability of rain by saying they measure the area that will get rain as a percentage of the total area and multiply that by their confidence. Nobody really understands what 40% means but that's math. We like percentages for rain and risk because in repeated forecasts, we have something to compare whether it really rains or not for that one time. But the weather guys spend more time than project managers do on that so we'll go with their best guess on risk measurement.

Probability of events

Probability, and therefore risk, starts with measuring outcomes of similar events. To measure anything in a business metric so you can compare results, you need a quantity of a standard unit. Math is funny. When you divide the number of useful past events by the total past events you lose your units of measurement. So you don't have any probability units. There is nothing that assigns an IT project to have 4 RUs (risk units) and another 15 RUs. So probabilities of different kinds of events can't be compared. That's our first problem.

Risk has to be related to a concrete event. Rain forecasts assume a probability of occurrence from a large number of the same types of events and as a stand-in we use similar historical events. A 50% chance of getting heads on a coin toss doesn't mean anything after one coin toss because it's one event. Your only two outcomes are heads and tails. There's no 50% tails. But a history of 500 tosses shows that heads is 50.3%. Now we have some confidence.

Confidence in chance

Confidence is what gets degraded though when we don't have similar events. In projects, we never have the same events so, like the thousands of influences on a storm system, if more of them are about the same, the confidence level of the forecast increases, which increases the chance of rain. Confidence is easier to do when the events are less reality based and can be easily counted. Coin tosses dominate the probability world but even tossing a coin has been shown to be not that easy.

Most business people don't have a backlog of 50 similar sized projects of the same type run by the same project manager with the same budget. The more events with the same influences, the better the confidence value. So confidence is easily eroded making our probability of an event less likely making our calculations less helpful. That's our second problem. We can make an educated guess of how likely the event is based on past events of the same type but the necessary confidence reality check will destroy a risk result.

How priority affects risk

Priority is not risk. Priorities are best assigned to scope units called work packages or use cases that estimate that total value. I like ITIL's breakdown of priority into impact and urgency and adapt it from the incident or trouble ticket to a use case. ITIL puts the value in terms of numbers of people that are affected (impact) and the value perceived (urgency). . For internal projects it might be expressed as a high, medium or low number of employees and high to low level of visibility to executives or high-level managers. For external projects it might be expressed as a high to low market share, and a high to low price point compared to competitive products.

In fact, priority does not influence risk at all. Risk can be multiplied times a priority ranking to show how value is reduced but the end result of a subjective number times another subjective number isn't useful except to compare to each other. Priority is good for ranking order of development of use cases. It's also good for ranking test case execution effort and thoroughness after factoring in development complexity and unfamiliarity of the unit under test.

Project risk

There are two easily confused types of risk in a project. There is project risk which occurs during the project. And there's organizational risk which is taken as a bet when the project is chartered and realized when the value is gained.

PMI defines project risk as "an uncertain event or condition that, if it occurs, has a positive or negative effect on a project’s objectives." But using that definition, since all future events are uncertain, means that all project activities are risks. Even if we declare risk to be the measure of the likelihood of a project failing to meet objectives, we're still in the same boat because any event could conspire to reduce the full obtainment of the objectives.

Project risk bets are taken mostly in design. The bet on the scope to be delivered starts when you spend money towards a design and ends when the product/code is delivered using that design. Strategy and analysis phases can influence design but design commits you to a path. These are all known risks and estimated up front. But there's always a component of unknown risks like developers changing jobs or emergencies but that's what project managers have 'flex time' for.

I have seen risk associated with failure to do adequate analysis called scope definition risk. That's the same baloney as when people call bad requirements gathering down the line, scope creep. This isn't project risk, it's systemic and therefore organizational risk. Check your process maturity and start there with a project constraint. Since you know you don't have good enough requirements management, you should expect some sort of decay around the elicited scope. It's a given, not a risk.

Organizational risk

The estimated value or dollars associated with the project which guides the project manager are broken up into three critical success factors. These three things that make a project succeed are

finishing on time
coming in on budget
delivering total decided scope

The value measurement units that we can use for each of these are standardized into

number of days
amount of dollars
use cases

The first two can be reduced to similar units of dollars. Days is just a measure of cost of overhead, labor, and other consumed resources. Scope is harder. If you are using good requirements estimation units as recommended in my blogs, you should choose a scope measurement unit that delivers those units of value known as use cases and user stories in the analysis world or work packages in the project management world.

Then you can use a standard conversion of like sized use cases to the approximation of PMI for work packages of up to ten days depending on complexity. So, input from the developers who gauge the difficulty of a use case is critical. Scope estimation is a similar process to test case effort estimation. It might break down to

low - 2-4 days
medium - 5-7 days
high - 8-10 days

The budget component should be constrained at the point at which the value can be assessed before the decision for approval is made. At that point, if the time component is constrained, that creates no flexibility on the scope component and it means that only a certain unknown amount of value can be delivered creating high anxiety for the project manager who hopes that the scope is minimal, the complexity is low, or the developers are wizards. Once the requirements have been analyzed, it may be the that the solution to the project is too big and the scope can't be delivered successfully leading only to a project cancellation and sanity preservation for the enterprise.

If the time component is flexible, then there's still some angst about the outcome but now with some adjustment ability. Using employees over time will still accumulate project cost so knowing the amount of time necessary to deliver a solution may push the expenditures over what the project value is. So where's the risk here if the value is estimated, the time is estimated and the scope determines whether the project is green lighted or not?

The bet on known risks is taken at chartering and cashed in when in production. But unknown risk is in the changing influences on the solution which is a strategy responsibility. A Pokemon Go clone is only good if the market has not produced any other competitors or people feel like they have found enough ‎Bulbasaurs or Meowths.

Strategic constraints subject to influence now become sources of risk. Events expected to happen that influence the planned time for the project or the budget must be listed. How do you judge a probability of occurrence of a one-time future event? You don't. You just put a number on it and rank the items by that. It's all subjective. And the next person puts a different rank on it and then you compromise. Instead of risk, let's call it a proposed ranking of possible scenarios that affect value.

It's all a guess

But because the infinite number of influences can never be known, it's not going to be that useful to collect historical information on what is recorded since the past may not accurately reflect the future. And it takes quite a bit of time to collect, enter, write up those reports, and present. Maybe it's better to just go with your gut when you need to rank risky events. Or see if your arthritis pain is flaring up like it does for weather changes.

The new web with AngularJS 2 & TypeScript - a browser marriage, not a date

2016-04-25T18:42:00.000-07:00

The world of web architecture is undergoing its third major overhaul and there’s still more to come. This third version of the web development architecture is centered on the browser as a stand-alone engine so that mobile apps and desktop apps have little differences, developers can write an app and then easily use it on a desktop or mobile device or even embed it in a proprietary mobile app. This is what folks are calling a Single Page Application (SPA).

As web developers, we've discovered the benefits of having static pages, then having our entire web site be pre-processed by the server, and now, since jQuery gave us the confidence to move some of that UX processing to the browser, we’ve been slowly moving more and more to the client. But a client-side web app is a marriage to the browser and not just a date.

Angular's popularity

In the last few years, the client-side processing world has matured, although you may think it’s still in its adolescent phase, but it’s showing enough promise and enthusiasm to garner attention from the big players out there. The buzz is concentrated around Google’s AngularJS web framework which manages a client-side set of views and handles some I/O. Microsoft thought well enough of it partner with Google to provide a much needed improvement for JavaScript called TypeScript. Google dropped development of a similar language and supported TypeScript as a primary language for AngularJS 2, just released in beta as of this January.

Angular also matured through five years of work to find a way to reuse web components like they do in other types of software. But the web is stuck with an amalgam of HTML, CSS, and JavaScript that requires the assistance of the browser to make it all work. And contending with the offline nature of mobile apps put a real wrench into traditional coding. We have to have our own database storage, routing features between pages, and communication with other servers. And we can’t hide behind those protective walls of the secure server.

Google has done a masterful job of keeping a framework both powerful enough to satisfy hungry developers for lots of complexity and customization but making it simple enough to be used by both designers and jQuery level developers. It’s got all the magic of RxJS observables (really? you’re still working on Promises?) but we see patterns to help us code easily with AJAX. AngularJS 1 had this problem of too much high-level architecture and a brick wall that kept them from adding some great features that we now get to have in Angular 2.

The only other major competition to Angular has come out of Facebook’s React framework. It has less buzz and more complexity than most developers need. Unless of course, you’re developing large systems like Facebook. But it looks like it's still morphing into better versions and a few years will give us a better understanding of how to use it.

Working with status quo and my class

In business, there’s a need to keep moving ahead but still use the resources that have been paid for. The frameworks of .NET and Java still rule the enterprise web sites and in order to make Angular play well with these guys, it will take some thoughtful redesign. Remember that these server side frameworks do all the rendering of the web page before the page is sent to the browser. There on the browser is where Angular expects to do its stuff but it makes more sense to do secure data access back on the server and generate a page there for the most part. What we are looking for in a client-side application is to improve that and find the best of both worlds.

The improvements that we are able to get from Angular include the ability to improve the user experience through a faster page rendering. We also make gains when we are able to design a great reusable library of web components that designers just drop in to pages. Those can be excellent advantages if we learn how to trim back the features we've been using on the server and now concentrate on the asynchronous data access and microservices challenges before us.

For the last four months, I’ve been developing a class for my employer, Centriq Training, in AngularJS 2 and TypeScript that I think will meet the need of the business user from the component developer on desktop and mobile to the designer who thinks about the future of web components possibly using Google’s Angular Material. It's been the most difficult course I've written.

It's been a challenge since minimal documentation was available to start, other published material was poor except from the Angular developer community (God love you Pascal Precht and Victor Savkin), and documentation continues to be updated by the user community. The AngularJS 2 API docs have a lot of "WHAT IT DOES - Not yet documented"in them. But, hey it's still beta. For the most part, it's midway between the massive headaches of the patchwork packages without a framework I see with JavaScript solutions and the tried and true .NET frameworks.

I'm always seeing improvements and industry support which indicates that TypeScript/AngularJS 2 is going to be a leader for years to come. The editors I use are a delight. For free use and my class, it's Microsoft Visual Studio Code, which has nothing to do with the Visual Studio editors for C# and other languages. (Do they know they are following in the footsteps of confusing Java and JavaScript?). That and JetBrains' WebStorm are the best tools available so far.

If the transition to responsive design and mobile apps didn't overwhelm you and you want to see how the next wave is going to improve things, it's definitely worth a look to look at these mature products that will continue to have an effect on the web architecture for the next five years easily. Hope to see you in class!

Assertions & assumptions, eels & heels - Clear terms create understanding in analysis

2016-01-05T18:28:00.001-08:00

There's several terms that cause confusion in analysis and sometimes they come in pairs. The two are preconditions / post-conditions and assertions / assumptions.

It's not that people get them mixed up, it 's just that they use them however they like. It's so much better when you know what you are saying and know why you are using the term appropriately. A clear term used properly is the beginning of analysis understanding.

It's not that I blame any particular reason for not using terms well because I know I had no clue when I was suppose to write up a project document and the people I was learning from had bad habits. It's like when my wife regretting doing something said she "felt like an eel." It's an obvious thing if you know the real use of the term but can cause confusion in those that don't have that experience.

Preconditions prevent functionality

In a use case, there is a particular section of preconditions which never is fully explained and ends up being a convenient way to start a use case from the middle of the scenario. My favorite example of this is when a use case requires the credentials of a user to be checked as part of the process. A log-on process is always part of a secure use case. It should be modeled as a part of a goal-driven use case.

Many people shortcut writing the full scenario by avoiding the repetitious log-in tasks and declaring that the user must be logged in as their precondition. So if logging on has changed the state of the system to a different state, that use case must be a complete use case that leads to value for that changed state. The log-in, no matter how many people log in, will not provide the business any value whatsoever. It's not a separate value-driven use case. Ever.

The precondition is a state of the system that must be true in order for the use case to progress to the first step that is unique to this use case. In the classic ATM use case example, you are prevented from doing a Withdraw Cash use case when the machine has no money and the menu option for withdrawing cash is disabled. That means that the precondition for the Withdraw Cash use case is that the ATM can pass the rule of allowing a user to withdraw up to their maximum daily limit or some other rule. The state was changed by a previous Withdraw Cash use case which did have value.

In a business use case where the user first drives to the bank's outside ATM to get cash, the precondition has to change some. The scope of the business system must include the full bank property and building. The system precondition is that of the machine having enough cash but business way to prevent a person from using the ATM is to put a traffic cone in the ATM lane or turn the system off. It's anything a manager would do that doesn't involve the software itself.

Post-conditions repeat important tasks

Use case post-conditions are usually defined as important things that need to be affirmed at the end of the use case tasks. That usually means repeating the important changes noted in the use case narrative. For instance, in the classic ATM example, a post-condition will be to make sure that the bank got the message of a withdrawal transaction at the end. But it doesn't have to be a state change. In fact, the state of the machine should return back to the way it was before the use case started if we are to follow the rule of being able to run a use case multiple times.

The post-condition of the bank transaction is that a message was confirmed to be sent to the bank by some means. In the design stage, this could be a callback function to the ATM or a log provided by the bank. Either way, it should be a task that could be audited.

Assertions are conditions too

Assertions are more of a programmatic style of pre or post-condition when the condition involves only that code. You assert that inputs matched your data rules, a technical precondition. You assert that outputs match your data rules or the state of something changes, a technical post-condition. The code for assertions is removed when deployed.

Assumptions occur after analysis

Assumptions however are not preconditions. You don't get to assume that your users are logged in. They are important decisions taken at design time when we need to guess and take a bet on the chances that something will occur. You assume that a health care system you manage will be affected by a possible repeal or replacement of ObamaCare.

You can understand what your stakeholders want in a sign-up site for your insurance and you know that they want something that follows and takes advantage of the current laws. But there's an uncertainty that is measured through a risk assessment process of whether or not the bill will pass. The project manager will be the one to assess the risk and make that bet on the value that could be gained or lost.

If the choice to take the risk is a good one, then the design team starts working on the system and while they are designing screens and systems, the assumption must be made that there will be a replacement for ObamaCare. Not all the risks that you assume in your design will happen but the choices you make may allow for you to reduce your time to market increasing the value of the system on the ones that do pan out.

Create understanding through clear terms

Terms, although deposited squarely in your beloved template, do not mean that they are correct. The best way to gain some understanding is to start questioning what the value of each piece of information is in your template. Corroborate that with the other team members. Update it, move it, refactor it, or delete it. Make it useful. Add an appendix if you have to. Then it will start being more understandable.

Friend or foe robot? Designing automated processes for business

2015-12-31T09:40:00.000-08:00

Programmers are guys who write your business processes with a kind of invisible ink they call code. The better you tell them what your processes are, the better business results you will have. They don't really model your world because they have no clue how the business works.

Programmers will come up with a well-engineered version of the reality they perceive if left alone and that will have to be the way you run your business if you don't tell them how the business works. It will likely more resemble a massive robot rather than a enabling friend.

As a programming instructor, I talk about the declaration and reuse of a set of statements that become methods (or functions or procedures).

As a business instructor, I teach the same thing concerning recognizing a useful process and promoting the larger scale use of it into the business. As your process maturity level grows, the process expands from your own use, to people around you, to even more people until it's a company-wide standard.

Architecture

Those same maturity steps are used to develop large scale architectures that provide guidance to the necessary design of the infrastructure that has implemented it. In fact, it's the principles of good design that produce a workable architecture. Sometimes business doesn't know which architecture to choose or isn't able to strategically point the direction and then takes the low road of design by infrastructure. This is the choice to buy instead of build and results are good as long as you stay compatible with the other components that are added on. Again what you end up with is a well-engineered solution without regard to your specific business needs.

The business processes existing within a coordinated automated system or an infrastructure are essentially the executable programs. They have been structured to achieve a specific objective. They take inputs and turn them into outputs. They have a specific trigger. They are layered into automated roles and responsibilities to work well when they have to be maintained. Just as if it were an employee, you can fire an architecture layer and hire another version of that layer that fits the bill when managing a cohesive set of responsibilities in a business.

As far as I know there's not as much business writing going on for how to design a job role with few dependencies to other job roles but one of the common axioms of being promoted in business is that you must make yourself replaceable or else you'll never get out of your job. That's the same kind of thinking as when programmers say they must make their programming units reusable or maintenance will become more of a headache.

The process

If processes equal computer programs then they must have similar features. They can be broken down into smaller processes. They are either ongoing processes or processes that have well defined starts and stops. Processes can update policies, standards, guidelines, activities, other processes, procedures and work instructions if needed just like computer code can generate other code, update rules, and modify data that change how the program operates. A program is made up of activities, procedures, and work instructions or as the programmers know it, the class, the methods, and the program statements.

Metrics for a process are built in from the start and can be injected into computer code or built in to the abstract base of the code. In business, the best place to think about your metrics for success is during the planning stage so you can always build with them in mind or else you'll have trouble retrofitting them later. Roles of who should be involved in the process are important. In code, the types of relationships that you maintain with other code is important and successful relationships are spelled out in recipe books called design patterns. I admire Peter Coad who tried to assign roles to classes and discover some reusable patterns for interactivity and not just static recipes.

Process improvements are always a part of all processes and can be found in wrap-up sessions or project post mortems. In code we get a closed feedback loop from the folks who are using the programs who give us reason to enhance the program and get more financing for the next maintenance project. Feedback comes in the form of new or improved features and it gets added to the system as the value becomes known.

Processes should also be well defined in understandable text just like we always do to our fellow programmers when recommending comments in their code. And just like the norm in programming, most documentation happens after the fact because people don't have enough time to document the program until it has stopped changing at the end of the development cycle.

Process owners are the people who understand the process and can help educate others on how to use the process. I help ITIL students who think process owners are not in charge of implementing the process. This always gets me on my soapbox. An implementation of a process, e.g. an ITIL key process, is not going to happen because you read the book. It must be digested, customized by deleting the right parts, and supplemented with all of the current business structure in order to be understood. Then the process activities are noted and people check them off as they need to when they need to encouraged by success and trimmed down by reality.

In programming, it would be the project sponsor who got the ball rolling for the new software and would own the software when complete. They also get new funding when the project is a success and get project funding reductions when the process is too cumbersome.

How and when the process is used is a matter of policy and the ways that you can use these process steps have limits. Just like a computer program, the steps can be run for certain types of tasks and shouldn't be run for others. You don't close the books of an accounting process except for once a month. You don't ship more inventory than you have in stock. Process objectives are probably determined by what planted the seed in the ground for using this process in the first place. In a software project, the business objectives drive the funding and usually can be found in a business case document or a sponsor's mind.

I was reminded of the input and output nature of a process when looking at the method declaration and saw that it defined a general form of a resource just like all the other types of resources. But until functional programming came along to supplement object oriented programming, we never had a way to send the process information from one place to another internally even though the values of any data could be sent to and fro.

Process testing

Software testing is a way of thinking about your code from the norm or from the original business requirements. This validation is a quality aspect of coding. Test driven design drives home the point that the test is a way of thinking about how your code should be called and used before the implementation is written. ITIL states that

"Even before starting, it is important to think about what the process outcomes should look like." (Service Design, p. 44)

We also work with requirements in a hierarchical way so that the lower level requirements of the code can roll up to the next level of activity and eventually be mapped to the business goals that drove the project. Then when the system is complete, we have a measurable way to compare what was originally planned and implement improvements if necessary.

Process enablers

The enablers of a process are combined assets of the resources and the capabilities of the business. These are the raw materials and the smarts of the folks that are getting the job done. On the computer programming side are the programming language capabilities and the hardware and software's raw power to get a calculation performed or a sequence of code executed. We measure performance here in FLOPS (floating point operations per second) and computers are moving through the petaflop (1,000,000,000,000,000) ranges now.

Business processes depend on the business assets just like code depends on the infrastructure it runs on. Unfortunately, business is both enabled and limited by humans and computers both enable and constrain business.

Get in front and drive

Processes are not much different whether you do them manually or if you have a robot do them for you. The programmer acts as a plumber in a business process. They know the mechanics and apply the right torque to the right nuts and bolts to hold the system together. The business analyst is going to map out the system so that it works for the business the best.

Sometimes the programmer has no choice but to plan it out and do the best they can. But the business should always get involved at the first to direct the code so that is best represents the business. That's always the best strategy to take for any new project. Don't let the programmer be a back seat driver and transform your software project into something you didn't expect.

Building your dream home - knowing web site roles and skill sets

2015-12-21T09:11:00.000-08:00

As a teacher I'm interested in getting the right information to the right student. A major part of my training is web site building and students often want to know what skill sets to learn for their job or career. Building a house is one of my go-to metaphors I use when talking about developing a web site. Getting a home built from the ground up takes skills from

a general contractor to get all the people collaborating,
an architect / designer to figure out how it should be lived in, and
the carpenters, plumbers, electricians, and other specialized contractors to build it.

You can separate the skill sets of building a web site into those same basic categories or roles:

admin people to set, build, and deploy the code
GUI / UX designers to understand the work flow and usability and
the developers and database people who make it work.

These roles of the web admin, designer, and developer have been implicit in many job titles and much training can be fuzzy about who should be learning what for the best results. Here's my simple guide to what skills each role could need in a prioritized list:

Web admin - OS, network, server configuration, server-side language, JavaScript, architecture, HTML, CSS, graphic arts
Web designer - graphic arts, CSS, HTML, JavaScript, server-side language, OS, network
Web developer - server-side language, JavaScript, database, architecture, OS, network, HTML, CSS, graphic arts

My classes at Centriq (including some I no longer teach) and the roles that they target in groups by primary role would then look like:

Admin

WD-519 - JavaScript tooling - admin, developer, designer

The JavaScript Tooling class is a two day class highlighting the admin skills for JavaScript based web sites and was taught as part of the JavaScript Powered Web Apps but is now offered separately.

Designer

WD- 400 - HTML/CSS - designer, developer
WD- 500 - HTML5/CSS3 - designer, developer
WD- 505 - Intro to jQuery - designer, developer
WD- 510 - Mobile Web App Design - designer
WD- 515 - Mobile Web App Development - designer, developer

Developer

xxx - any Java, C#, or php class - developer, admin
JV-394 - Intro to XML - - developer, admin
WD- 405 - JavaScript - developer, admin
WD- 520 - JavaScript Powered Web Apps - developer, admin, designer
WD- 525 - TypeScript - developer, admin
WD- 530 - AngularJS 2 - developer, designer
SOA-102 - SOA Intro - developer, architect

The JavaScript Powered Web Apps doesn't get you hardly any design because it's about setting up a web site using nothing but JavaScript. That means that the web site isn't much to look at but it's very cool under the covers.

I hope that you are better able to guide your educational path towards a team of skills so you can build your own personal dream home with some of these role explanations.

Don't be a LeBron and write user stories for you. Create team value with these examples.

2015-12-13T17:29:00.000-08:00

I'm still not happy with Agile user stories. The user story should set a goal for the team. Most perspectives of story writing I see follow the LeBron James school of thought. The typical user story I read finds value in winning for the individual player and then the projects follow those rules running into conflicts. I think it's time to do some coaching examples from stories I found on the internet.

The easy expected format

The traditional user story format:

As a (role) I want (something) so that (benefit)

is supposed to benefit the person/business paying for the work. Or at least that's what I want if I'm in charge. You don't want to watch the project goals slip away and then everybody and their dog starts wanting something different. Without a guiding set of project value propositions, the user story will look like

As a <stakeholder> I want <something for me> so that <several excuses>.

Just making an excuse for a personal need does not make a user story a requirement nor makes it easy for a developer to code it. When it's in any kind of formatted statement, the analyst can put it neatly into a document and make it passable as a project document. It's now user stories. Nobody questions it because there are no well understood rules about a good user story.

So stop already. Here's my rules (I get to make the rules if I'm paying for it):

The role/stakeholder triggers the activity. It's like a security group description instead of a real type of person at this point. Make it broad.
The something is for the business. Make it the focus.
The benefit is for the business. Make it have value. No excuses.
Use testable functional requirement writing. Any kind of limitation of data or process is a rule.

The hard part

Let's apply this several different ways to sets of requirements I found. In my style of writing requirements, the business rules are extracted out and placed under the requirement (they are given an ID if reusable and placed in another document). The information about the data entities are extracted out and placed in a data dictionary with the business details. The design elements are noted. And any project information is noted as well for inclusion in the project plans. Only the rules stay with the requirements while the rest get sorted and filed away in their own documents.

Desktop backup app

Here's an example of an epic with two smaller user stories:

As a user, I can backup my entire hard drive.
As a power user, I can specify files or folders to backup based on file size, date created and date modified.
As a user, I can indicate folders not to backup so that my backup drive isn't filled up with things I don't need saved.

It's interesting to me that this first choice extracted from my Google search didn't bother with the benefit part. So here's my take on them collapsed into one story:

As a user, I want to backup my hard drive so I can restore files.

RULE - select files by size, date created, data modified, user choice.
RULE - select folders by size, date created, data modified
DATA - File: name, size in Kb, date created, date modified
DESIGN - option to select an entire disk backup is the default?

The distinction between the two roles fades away as the trigger is the same for the same result. The ability to use rule-based choices for file selection is what the user story author thinks is important. But the importance might come if there is a functional difference. The distinction between the user and power user will appear as design personas for testing.

Most of the meat of these requirements is in the rules which corresponds to my experience of seeing a document fluff up by including the rules as variations of requirements. It's just bologna.

Agile project management app

Here's another example of four related user stories with other information included from the three other stories that were deemed too broad (vague) or too detailed (mostly data info):

A team member can view the iteration's stories and their status with main fields.
A team member can view the current burndown chart on the status page, and can click it for a larger view.
A team member can view or hide the tasks under the stories.
A team member can edit a task from the iteration status page.

This one was interesting to me because the template for writing a user story looked very promising and included the goal of expressing business value. I just didn't see it in the stories. Here's my version:

As a team member, I want to view project user stories to be aware of project scope.

RULE - select stories by iteration
DATA - User Story: status, tasks, main fields(?), rank, name, size, package, owner
DESIGN - tasks are able to be hidden and shown under story

As a team member, I want to view project status to be aware of project management metrics.

DATA - User Story: is in scope, hours estimated, hours worked
DESIGN - project status is best viewed as a burndown chart
DESIGN - the burndown chart should be able to be viewed larger

As an analyst, I want to manage user story tasks so that user stories have details.

DESIGN - CRUD should start from the iteration status page

I know there are four basic operations on entities which occur. When they all occur together, I find it useful to use a higher level requirement using the word 'manage.' The two operations mentioned in the original versions, edit and view, would be paired with create and delete in a full set of user stories. Data base people know these basic CRUD functions. The entity discovered in the requirements is just the User Story. The project status is a report of all in scope project user stories.

Credit card payment

Another story with additional details from the same Google search result above really did use some value so here's what it looks like before:

As a purchaser on the website, I want the ability to pay with a credit card so that I may immediately confirm my purchase.
Acceptance criteria

accept Discover, Visa, MC
Validate CC# when entered
Validate expiration data and CVV
Validate billing address
Generate success and failure messages after processing

Definition of Done

passes all regression tests
passes testing per acceptance items
approved by UI team
able to show feature in company demo

mockup.png

and after using real world documentation:

As a customer, I want to purchase a product with a credit card so that transactions are convenient.

RULE - credit cards must pass basic local validation before submitting to credit card processor

credit card type - Discover, Visa and MC
number - passes Luhn algorithm
expiration date - after or equal to date of transaction

RULE - credit cards submitted to credit card processor must validate with the following information (or create separate DD item)

credit card number
expiration date
card security code
billing address

RULE - credit card validation should be done within 2 seconds
DATA - Credit card: type, number, card security code (not stored), expiration date
DATA - Message: body, time/date, ID
PROJECT - Testing, exit criteria: All regression tests must pass.
PROJECT - Testing, exit criteria: Entire UI team approves screen forms.
PROJECT - Testing, user acceptance: prepare mock data for company demo
PROJECT - Project plan, stakeholders: Credit card processor, customer
DESIGN - suggested screen layout of entry form - mockup.png
DESIGN - self-service customer purchases are best made on a web site

I really don't know why I need to confirm my own transaction since I can't confirm with anything. The confirmations are the validations which should be made quickly. The validations are really in two phases since it's a system dependent on a credit card processor.

Summary of value driven analysis

I believe that people don't do requirements well. It's why user stories were advanced as a replacement for bad requirements. In some ways I agree with Cowan+'s throwing the baby out with the bathwater statement, "User stories are not requirements. They were meant to replace requirements, which suck." But I think that they just help do a better job at user needs elicitation. And then the rest of the analysis should be about writing the business value user stories/use cases.

By following a format of zeroing in on the functional requirement and extracting out all the distracting and fluffing-up text into their own documents, I believe it's much easier to create usable requirements. The thing that drives the requirements the best is value. But it's value to the business, not value to the individual stakeholder. Do it for the team. Don't be LeBron.

Related blogs

Re-inventing usable requirements: Value-driven projects & the failure of the Agile user story

2015-11-04T21:28:00.000-08:00

I'm fascinated by the attraction of the Agile user story with requirements writers and the effort to make it be a final usable requirement. Stakeholders format their needs from their view using the simplistic Agile user story. But they shouldn't. That list of stories then becomes the requirements document or backlog with little analysis. It can reel in any amount of confusing scope easily much like a black hole will attract anything it gets close to.

I wrote about rewriting user stories into business' goal targeted requirements in my last blog and my best words to describe it is as a value-driven approach. After writing that I found that Elizabeth Keogh (@lunivore) suggested that business value is more important than the user role back in 2008. But I want to go deeper. I find that the detail level of the user story is unusable as a coder except to start making informed guesses.

Project value

Needs elicitation is the common starting point of a project. This is normal. Needs are selfish. But needs can't be met until everyone decides what the business goal is that the needs address. The goal should be to solve a business problem. The agreement on the problem statement becomes the starting point for gathering focused needs.

Once there is a problem statement, the needs can be framed in that context instead of the individual project stakeholder or customer. I don't think the stakeholder is going to give you any amount of money to write your code so why tailor the project to each one of them? It's the business that gains the value; it's the business that pays the bills; it's the business that should be the context of the requirements.

User story versions

Scott Ambler, a prolific Agile writer and process thinker, outlines the user story on his Agile Modeling site:

pre-2008: user story begins as an informal simple "I want a ..." kind of statement
2008+: Mike Cohn adds the party benefiting from the value In User Stories Applied with the format of "As a (role) I want (something) so that (benefit)."

Scott has a list of user stories on his site in the informal style that include:

Students can purchase monthly parking passes online.
Parking passes can be paid via credit cards.
Parking passes can be paid via PayPal.

The Mike Cohn user story of the first bullet point above according to Scott would be:

As a Student I want to purchase a parking pass so that I can drive to school.

The Elizabeth Keogh user story of the same story would probably look like

In order to drive to school, as a Student, I want to purchase a parking pass.

Both of those examples have the Student getting the value. If I'm the starving student, I want to park free. Did you people ask me? None of those are real benefits. After I get done changing this need into a value-driven requirement it will look completely different:

The school sells permits to park vehicles on school property.

You'll see the benefit as a large scope requirement that traces down (is a parent item) to this requirement. The user story is a needs elicitation tool but with a process flaw. The user stories are passed on the the programmer who think they have structure. The reductionist user story rearranges the words of the "requirements" you find in an Excel spreadsheets and passes them on to development.

Needs vs requirements

A need is a statement of any scope granularity, any priority, with or without a responsible party, and either functional or not. It could even be a design as envisioned by the stakeholder trying to grasp how to convey what they think will get the job done. Needs have so many shapes and forms that they can't be processed easily. They have no structure which is why "requirements" meetings last so long. They are really needs elicitation brainstorming sessions.

Needs have to be rewritten into organized requirements. A good requirement has structure. That structure is testable. A good set of requirements is complete. That completeness is validated by a model and the stakeholders. Structure and models take training and time to do well. Stakeholders don't have that time. Analysts do.

The biggest problem for my students in my Business Analysis class seems to be stopping at a good enough stage where the needs are all lined up neatly with enough words so it looks passable. The critical thinking skills should kick in to test the requirements but the tests are not well understood about what the goal is for a good requirement. Programmers don't communicate and use their tool-driven BDD/TDD/XDD style processes to wrangle the statements into some code.

Value-driven requirements

My value-driven approach for the first parking pass requirement would be much more detailed and focused on the school that is paying for the project. A goal accomplishes business value in a functional requirement statement and can be rolled up to a goal group. A goal group expresses larger scoped business values. For the goal, rules express how the functional requirement is to be executed.

Goal group: The school offsets direct costs for parking.

Goal: The school sells permission to park vehicles on school property.

RULE: parking permissions are valid in one month increments
RULE: permissions are only offered online
RULE: permissions can be paid for with credit cards (Visa, MasterCard, AmEx) or PayPal.

Goal group: The school provides a secure environment.

Goal: The school controls vehicular access to school property.

RULE: Drivers access school parking lots through entrances requiring permission.

Design recommendation: parking passes should be used as permission evidence.

The Student of the user story is a valuable piece of requirements information but not for the requirements statements. It should become a persona as long as descriptive details are added to make them more real. Personas should be used as a design step in the project and for testing but not for requirements. You'll also need an Instructor or Employee persona.

A parking pass is a permission mechanism that can be replaced with another solution and not affect the requirements. That makes it a design choice and unless there is a constraint on the project for doing it, it is not worded in the requirements statements. It should be captured as a stakeholder idea for a solution in a category of design recommendations.

My goal follows a similar structure as Alistair Cockburn's goal driven use case except for adding the rules directly into that scenario if the rules are not reusable in other use cases. Many goals should be able to be found that roll up to a goal group. And goal groups can be drilled down into to find other goals. Business value becomes more strategic as you roll up your goals.

It's the focus on solving our own needs as we see them instead of creating value out of solving a business problem that has us creating confusion in requirements. Encourage the stakeholders to start thinking as a team from the business viewpoint and find the value that is in your project.

As an Agile user story writer, I want a kitten - Rewriting user stories for business value.

2015-10-22T11:04:00.000-07:00

source: Flickr

Teaching yet another iteration of my Business Analysis class here in Kansas City, I was struck by the two versions of Agile user story techniques I had somehow collected over the years. They were different and would yield two different results. And I didn't like that.

Project phases

There's a basic flow to managing a project:

Figure out the problem (strategy)
Break the problem down into chunks (analysis)
Plan what we have to do (design)
Do/build each of the chunks (development)
Make sure we did it right (testing)
Use the solution (production)

But I think we should add a step that will influence how we write our documentation. It's the step where someone pays for the project. That someone is the project sponsor and they want to see some sort of value returned for the investment of time and money on the project.

The User's Story

The common form of a user story in the Agile world goes something like this:

As a <type of user>, I want <some goal> so that <some reason>

The <type of user> is important so that we can wrap our security access controls around them in the design stage. And the type is also useful when we are eliciting other types of user stories during a brainstorming session of some sort so that we can come up with related user stories from the same sort of user. A use case would combine similar types of users into one actor so that the interface is simplified when moving into design.

The <some goal> is important because it provides a similar sized chunk of scope that allows a project manager to create a work breakdown structure with work packages. I also find it useful to provide an architectural sized component based on the goal in software.

But there is trouble here. We can ask for the goal from the perspective of the user or from the business. It's been my inflexible rule that you never ask a stakeholder the priority of a requirement. The expected answer from any manager worth their salt is that they gotta have it. Ask a room full of managers what the priority is and you'll make your meeting much longer than you wanted to.

One of the lessons I learned from Incident Management in ITIL is that you measure priority of incidents as a combination of impact and urgency giving you some hard numbers to work with. Impact is the number of people affected or marketplace and urgency focuses on the perceived value often backed up by a level of management to manage that process or measured by how much you want that new 4k monitor.

But this priority should be from the view of the business and not from the user of the product or service. If we asked the user what they wanted (akin to asking a 7-year old what college they want to attend) they will tell us all sorts of useless goals. Maybe it really is an important goal of that stakeholder to own a kitten or draw seven red perpendicular lines but the business is not going to find value in it.

The <some reason> then expands on the trouble that started with the user's goal. They'll tell you a wonderful reason why it's important to them. But we're looking for business justification here and in my mind, I'd want that justification to be in the form of a higher level business goal. When things go right, the business goal is in mind, but it's not a constraint of the format.

I know that in Extreme Programming, where the user story came from, user stories were seen as an estimation tool. They also were written from the perspective of the system. That makes sense to me. I need a metric to measure my project where all the units are of the same size. That way I can make that PERT chart make sense in terms of scope.

The System's Story

And why is a basic functional requirements statement written from the perspective of the system? Because the requirements have to be executed by the system and tests confirm that. So we write it that way. But why not write from the perspective of the user? Let's try a test. A user story to run a report for monthly sales goes like this:

As a sales manager, I want to see a sales report, so I can identify the sales reps who are under-performing.

What's the value there? Aren't the under-performing sales reps making money for the business? Is this a personal agenda or a goal that isn't well thought out? How about we try the story from a business perspective?

As a sales manager, the system shall create a sales report, so that sales trends can predicted.

Or another version from a different user might be:

As a marketing manager, the system shall create a sales report, so that marketing can budget for advertising.

I can see the business value in sales forecasting and marketing budgeting. What I see in the other story is a sales manager who creates churn in their sales division which is not value to the business. The requirement is the same.

User story to requirements

So I'm not going to try and change the user story because it's a good elicitation tool. It does not have enough detail to be a good way to deliver requirements to anyone downstream in design or programming. But I did want to raise the issue of the two-faced nature of the statement. In the statement, we can direct the value to the stakeholder or better, we should direct the value to the business.

My recommendation is when using user stories, that we take those stories as a user's statement of what they want and then rewrite them as a good requirements statement. In a use case format we would have

Stakeholders (interests): sales manager (wants forecasting), marketing manager (wants budgeting)
Actor: report reader
Use case: create sales report

As it was, I figured out that all the business value reinforcement I had been getting from other sources over the years caused me to rewrite the user story in a different format when adding it to my lecture. So in the end, I made a mistake about the real format of the user story and learned from my mistakes. No, I didn't get a kitten.

Related blogs

Blame chain or value chain? Value drives a project from strategy to test.

2015-05-15T08:28:00.000-07:00

This series of blogs is based on a talk entitled "Is Your Programmer a Better Business Analyst Than You Are" given by the author to the Kansas City IIBA user group meeting in April of 2015.

It's always been a problem for me that programming students have no clue about working with existing project requirements. I've never seen any teaching nor any methodology come from the Agile movement let alone any other technical cultural phenomenon. If no one is concerned that the project moves from the business analyst role to the programming role, that must mean that the project really starts with the programmer because that's where the project deliverables really begin. And so, programmers have reverse-engineered analysis and design skill sets to cope by themselves,

Of course, real world projects are the practical training ground for understanding how to work with project teams, but couldn't we make this a part of the skill set for a programmer before they get thrown into the war room?

Web programmers are used to this. They are a department of one. They gather information on understanding customers' needs, write copy, design layouts, put the code together, and deploy and maintain a site. As the web application world expands, this puts immense pressure on the technical web employee to know about any new development and keep up their own skills tied to the legacy apps they maintain.

Programmers I talk to have an implicit distrust of the project. They find a purpose but no real detail in it and muddle through the development as best as they can. In fact, most businesses without a solid project management process don't have much to work with. The management will throw stuff out into the market and claim that the market is crazy. The analysts are documenting whatever anyone says and then complaining that the users don't know what they want. Coders code whatever they think is right and try to make the project manager's deadlines.

And testers. Standing at the end of the line with a deadline rapidly approaching doesn't make anyone try to establish any standards knowing that they are holding up the deployment. They just try to find as many bugs as possible in the time they have.

The programming students I see have not been trained in business so it's reasonable they don't know how good goals can drive their behavior. A couple of questions have always intrigued me when working with a software project:

What if all the people on the project have the same goal?
What if they all had documents that built on the previous documents and were traceable back to the original value-driven goals?
What if the project was measured on how it realized that goal?

A project proposal should be estimated on value. Not only just a return on investment but understanding what the priority is for the enterprise. Value for internal projects can be seen as basically two parts but more can be added if justified:

The number of people in the company that will use the project in their processes
The visibility of that project in the organizational chart of the business so that if that project fails, the level of management involved determines the severity ticket level.

Value for external projects is estimated the same way. There are the same two metrics involved:

The number of people in the market that will find value in using the product/service.
The amount of money that the market is willing to spend to get that product/service

The two measures are both based on quantity of value. The project then can look back to this value of opportunity and see how to organize the project pieces. For instance, business goals, the basic business driven goals that begin the requirements process, are based on what value the business gets. Requirements meetings should never ask the question "What do you want the project to do for you?" because it's not about the person. The better question is "What should the project do to produce the best value for the business?"

Then, the business can see that the customers have needs where they should adapt their product/service to achieve the best solution to a large market segment. The question for the strategists and enterprise analysts would be "What can I do to help our customers/employees solve their problems?"

Business analysts would then understand that the value perceived at the strategy level should be communicated downstream to the programmers clearly. That communication should be able to move directly into testing of that code because the target of the project development effort is to communicate business value to the technical staff. The analysts' question then is "How does what the customer/staff do right now help them get tasks done?"

The designers have the task of taking the current processes and possible new processes and shoehorning them into a fit with the business assets so that it improves the value. They have to think about improving life in general. The question for them is "How can we translate the requirements into code/real life? The trick is to make it look like it's simple.

The final stage in the value transition from idea to final service/product is that someone double-checks that the value has actually arrived. That's the job of the tester who assures that the quality will be there though verifying minor things throughout the project and validating that the expectations were met at the end. Their questions are "How can I guarantee quality?"and "How can I let everyone know how well they did?

In making the move to measuring by value, the chain of FUD will change to the chain of quality. But there are lots more questions that I've asked and I'll continue this series by taking on those questions. Here's a sample of some:

How should scope be measured by value?
How should scope be organized by value?
What is the role of analysis?
What does a programmer need requirements to look like?
What are good business quality metrics for code?
Is TDD dead or alive?

New ThoughtWorks Technology Radar out today

2014-07-08T06:11:00.004-07:00

Four trends are identified in the Thoughtworks strategic IT report called Technology Radar published twice a year. The most important one for me is the one concerning JavaScript technologies and the challenge it brings for understanding. That is the reason that I'm developing my new course, JavaScript Powered Web Apps focusing on the language's application to building client-side logic for web pages and developing more with the browser. And it's been challenging without the guidance of book authors and coordinating corporations.

Here's the full excerpt on JavaScript:

Churn in the JavaScript World — We thought the rate of change in the Ruby open source space was rapid until the full rush of JavaScript frameworks arrived. JavaScript used to be a condiment technology, always used to augment other technologies. It has kept that role but expanded into its own platform with a staggering rate of change. Trying to understand the breadth of this space is daunting, and innovation is rampant. Like the Java and Ruby open source spaces, we hope it will eventually calm to at least a deluge.

The other three trends were

Microservices and the Rise of the API (also somewhat aligned with the JavaScript trends),
Conway's Law ("organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations") and
Re-decentralization

JavaScript Powered Web Apps - new programming course

2014-04-21T13:37:00.000-07:00

The world of web programming is moving ahead into what should be called Web 3.0 (the semantic web is just a pipe dream). It's using JavaScript as a unifying layer and re-imagining how the web can do what it does without the benefit of large back end frameworks. Web 1 was delivering files to a client. Web 2.0 was letting clients think they were in control by faking a desktop application over HTTP. And now we can have a true application that has been enhanced with networking to services in 3.0.

For the next several months I will be developing a new course to complement the classes that I wrote a few years back on HTML5, CSS, jQuery, jQuery Mobile, and JavaScript that bring these components together. I see tooling on web applications starting to mature and it is the right time to start to promote a new style of web application development. Except that there's no one way to do anything yet. Adobe is getting close to providing another great IDE with Edge Code but I think we're a ways off yet. Even Google might become a player here with their IDE code named Spark built with Dart and Polymer.

The new course, JavaScript Powered Web Apps, will walk students through building a "site" using combinations of node.js, nginx, SASS, Mongo, Bootstrap, Github, jQuery, jQuery Mobile, Grunt, AngularJS, Knockout, Express, etc. I'll probably do four days of sample sites and then show a web work flow and let students choose their own tools.

The course will assume some programming or design experience with a web site but most of the exercises will be scripted so that anyone can follow them. The students that likely will be most interested are those that don't have the back end skill set and see a tremendous advantage in learning only one language to do both front and back end coding. That means that anyone from high school that has learned the basics of HTML and CSS can enroll.

A lot of the training is on the administration and workflow of the tools which is harder to learn from books. I'll try to capture what I can for the exercises but if anyone has suggestions, I'm willing to listen. And as always, the class will continually be updated as the tools rev and newer tools emerge.

For instance, I'm still waiting to see how Famo.us is going to impact the animation tools. It looks incredible but won't be completely public until May 19th when HTML5 Dev Conference in San Francisco starts. And I want to spin up a partial.js site as well I think. But I'll never know until it's finished. After all, a project's requirements are never finished until the project is over.

Documentation as a control mechanism - not! Think communication.

2014-01-23T09:10:00.000-08:00

People associate documentation in Agile with waste. In fact, the Agile Manifesto prefers "working software over comprehensive documentation" but does that mean that the purpose of documentation is secondary to the output of the project? Agile set us up with a poor dichotomy. I mean I prefer getting a paycheck over driving to work. Maybe we're asking the wrong question.

I don't see documentation as an optional part of the project. It's definitely an output and can be measured and that is the allure of the artifact. The traditional usage of documentation in a project is as a control mechanism when I look at the process outputs. It's often managers and specifically project managers who finalize a phase with the documentation. It's what they do. They design their work package by understanding the scope of the effort and if there is something measurable at the end it becomes a good work package.

Back in the darker ages of procedural coding, there was a movement to measure code by what it did. That entailed putting an estimate on the smallest granular operations of the computer in the code itself. That worked for awhile when the code was consistent in its granularity. But code has changed and what we can hide in a line of code has become enormous. The function point estimation methods died.

Now we estimate, if at all, with a measurement of work package time completion and complexity (from the programmer's perspective) or confidence (from the project manager's perspective). The dependency is on the work package design. If we don't have a good description of what it is we're going to do, we can't plan it. Many projects flounder from a lack of analytic description.

So where does this good description come from? Well, your requirements are the place to find these descriptions. Of course, the best requirements are ideally the needs and wants of the stakeholders massaged in to testable work packages detailed down to repeatable tasks so that no business questions have to be asked constrained by project limitations. But in reality, they are more of a garbage dump of what people said in excruciating long meetings.

The requirement document gets written and then we're on to the next phase. Follow along on your Gantt chart please. As a programmer, my input should be the output of the analyst. But with all the hubbub in programming circles about how to organize and manage testability, it looks like the analyst isn't doing a very good job. I don't see much evidence of usable requirements from my point of view either.

So, the Agile people are right. Processes without consumers are pure waste. Let's right-size this documentation by eliminating it. No one used it anyway. But what are we losing? We're losing the ability to record a decision and the to think about the design of the business. Of course, if it wasn't good for the programmer, then it was useless.

But let's consider that the analyst can benefit the programmer. Then the documentation becomes a stepping stone to better code. Then there is communication of the needs of the business. Then there is less thinking on the role of the programmer who gets to focus on writing well-structured and reusable code.

The role of the documentation is really that of communication when the project scales. If you are the sole stakeholder and programmer, you probably have all the requirements circling in your head at any one time. No documentation is necessary. If two people know exactly what has to be done after a good agreeable meeting, no documentation has to be created. But if there is a memory loss, a sick day, a new member to the team, you will need some documentation. The need for documentation increases as the need to communicate increases.

The management of documentation needs a metric instead of lazily setting the goal to that of creating a greater return on investment (ROI). Just how do you measure that? My standards for measuring are in the more subjective realm whereby you produce the documentation, ask the user of that documentation if they understand, and then see if they are able to do their tasks without any further questions. The quality score descends as the need for answers or the amount of perceivable confusion increases. Get feedback.

Documentation is not a gate to the next phase and to be signed off. I'll take the stance a little further than the traditional "living document" style of writing. Since it is to be a communication mechanism, it has to always communicate the current understanding of what the project is about. Anyone and everyone can be a contributor but the use cases / user stories / work packages should be maintained by the analyst / technical writer role so that they achieve the best level of testability and detail. Wikis are good.

The trend of "barely good enough" documentation, I think, is allowing the programmer to use their analytic skills in place of poor business analysis skills in the workplace, which is sad but the best workable solution to getting the job done. Stop producing unusable business documents and let the programmer get on with the code. Why are programmers strongly in favor of commenting their code and following Test Driven Development? Because those are the tools that get the documentation done a better way.

So, lets eschew the notion of controlling the project by requiring the project members to produce a result that isn't used in the next phase. Control the project by understanding the work package completions. The artifact that completes the work package is the code or the pseudo-code (the use case) in some form or another, not a project document.

So, is documentation is secondary to the output of the project? Getting a paycheck over driving to work is comparing the result with a task for getting that result. It depends whether you have to go to the office or not. Documentation is not secondary. It's just a question of whether you need to communicate more or not.

Thoughtworks Technology Radar

2013-05-22T09:13:00.001-07:00

Free software, free books and Technology Radar. If you've sat any of my classes, you've heard me talk about those three things every time.

I have been reading Martin Fowler's writing for most of my IT training career and have found him practical, in-depth, and current. He also satisfies that craving for a little higher level analysis thinking that developer blogs don't always cover well enough exposing their lack of experience with other technologies and limiting their authority on changing my opinion.

The new edition of TR is out today. It's taken me years to work through some of the recommendations that they've put together in their think tank, ThoughtWorks, that are considered to be the cutting edge worth keeping abreast on. They also don't mind telling you when a technology is not worth your time. Both are worth my time to read and understand.

The trends highlighted in this issue are:

Falling boundaries - cloud development, co-location, perimeterless enterprise
Proven practices to areas that missed them - CSS frameworks, database migrations for NoSQL, etc.
Lightweight analytics
Infrastructure as code

The four major areas that are reviewed are

Techniques
Platforms
Tools
Languages & Frameworks

Techniques

For me, being mostly a web developer, I was happy to see that the Adopt recommendations in the Techniques section were for mobile testing on mobile networks moving away from the fake simulators as well as using promises for asynchronous programming, giving assurance of feedback for us JavaScript/AJAX coders.

The next level of recommendation down from the Adopt level is the Trial level which should be approached with a little more thought. You see HTML5 storage replacing cookies and Mobile First here as well as responsive web design. I agree with all those because they're not total solutions to a problem. What's interesting is their lack of concern for exhaustive browser based testing in the Hold level which means don't worry about it.

For Windows and Powershell people, you'll be glad to know that Chef, Puppet and Octopus (automated deployment of ASP.NET apps without PowerShell) to support infrastructure tasks has made Windows automation a much better choice.

Platforms

Martin is following the NoSQL movement very closely and has put MongoDB as the choice for his Adopt level. CouchBase, Hadoop and BigQuery are down one level in Trial. Node.js is down there too probably as a technology too green to make it worth our while yet. But I'm waiting for their take on Polymer and Meteor. Also interesting in the next level up from the basement, Assess, is PhoneGap (Apache Cordova) and Zepto.js, the smaller relative to jQuery.

I wasn't surprised to see WS-* holding the bottom place in the platforms with REST taking over web services slowly but surely.

Tools

I'm using NuGet for .NET development and was happy to see it in the top level. Check out Chocolatey NuGet as well if you do Windows administration. Maven is on Hold.

I've been searching for that right observer component that Google's Angular.js, Knockout.js or Ember.js cover but also includes the whole MV* framework thing which I have covered with either a Java or .NET web framework. Reactive extensions for .NET (RxJS) didn't fare too well in the Assess level. My choice here for a solution to the observer pattern could be ReactJS or RxJS.

The one tool that surprised me was D3. I have been recommending Raphael for JavaScript charting and watching D3 some but it shows up now in the Adopt level due to better complementary libraries such as Rickshaw and Crossfilter.

Languages and Frameworks

CSS frameworks like SASS/SCSS and Compass were staying in the Adopt level. The web apps that are moving away from traditional client/server architecture have much to learn yet but many frameworks are beginning to have business value so that they show up on the Trial level. These are HTML5 for offline applications, JavaScript as a platform and JavaScript MV* frameworks. Twitter Bootstrap also shows up as an Assess. But Backbone.js and handwritten CSS are as good as last year in the Hold level.

A surprising observation was that Team Foundation Server caused productivity problems as a version control system. ThoughtWorks recommends Git, Perforce, or Subversion instead. It's a good thing that Visual Studio works with Git.

And just when you thought analytics couldn't get any better than Google Analytics, they see great promise in the data set aggregation and AWS/Hadoop management of your billions of web hits with Snowplow Analytics.

I'm sure I've missed some recommendations and packages people are using such as that new fangled language Mel Tillis and Kenny Rogers started about a paralyzed vet's wife going in to town for the evening without him. But read through the assessment and mine the results for some great improvements to your technology stack.

HTML5 and future of the universe

2012-07-10T06:16:00.000-07:00

Web site people are always catching up. You get involved in an extended project and by the time you're done, they've changed the rules again and it's time to learn a new technology. I used to think this was a problem. Now I see it as a responsibility to manage the information of an ever expanding set of hardware devices that are becoming digital.

As these digital devices become of age, they mature into a web access point because of the value it adds to the device. Some devices like tablets are born with the ability to talk to the web at birth. Others, like DVD players, had to wait until they grew up.

HTML5 and CSS3 are expanding to fill the needs of the digital appetites of these new devices. As we innovate to market test a device at every screen size possible from phone to tablet to TV, the older development technologies will continually be updated until they break. So far, HTML and CSS are holding their own. CSS not so much. Even the old dog, JavaScript, is doing well but has some sturdy jQuery and CoffeeScript crutches.

The raft of programming languages not originally focused on the web that try to manage that environment push the simple text retrieval process to be more app like. In their desire to improve the unique asynchronous request-response communication model, they, in my mind, will eventually destroy it. Face it, it's slow. You can do faster communication with AJAX which is why you see Google using so much of it on their web apps. It's why jQuery Mobile designed their GUI library for speed with AJAX so much that it's the primary communication model that tolerates the web request model.

Now a newer model of finer grained I/O control in JavaScript is starting to appear called the WebSocket API. Did you not see this coming? It's not the end until each language has a way to use a simple library of functions/methods to talk to any device of your choice.

Languages are shifting little in popularity. A recent TIOBE trend announcement showing that the iOS platform language of choice, Objective-C, has been the faster gainer over the last two years to take the #3 spot on the chart, is the only major change. With Java trending downward and losing the lead to C, a venerable stalwart, I'm not seeing where anybody is picking up the slack. C# is on a slow trend upwards but not that much. Even php is losing ground.

My guess is that JavaScript is picking up the slack from reading language popularity articles. You can run a survey of projects on StackExchange and Github and find out that JavaScript has the top ranking there. Hacker News put JavaScript in the top three as well with Python and Ruby taking the top spots. Even job rankings on Dice.com show JavaScript in the top three to get a better sense of real world usage. Book sales from O'Reilly put JavaScript at #2 showing a mix of business and hobby use.

The next step for JavaScript would be to revive the server side version of the language and make it as available as php is for Apache. Then the major languages would write APIs to talk to JavaScript and we would have a programming interface for the web. Oh, yeah, and somebody do something in JavaScript to make CSS easier to work with. LESS is a good start.

So, if you are a web person, the future of the web looks like it centers on web application development and JavaScript is taking center stage. APIs will be promulgating and JavaScript/jQuery/minor language support plug-ins will be promoted. One of the APIs that is next on my bucket list is YQL. I think with the first book out on YQL in a few months from O'Reilly, we'll see an interest in mining the data of the web from JavaScript languages.

Excuse me now, I have to get back to the future and read about what's coming so when I start working again I won't be too far behind.

Impressive Shadow debuts at SXSW

2012-03-09T12:47:00.001-08:00

Adobe is showing off a great new product for mobile developers at the SXSW conference in Austin this weekend. I caught the announcement and have been using it to achieve a better workflow. It seems the more devices you need to test, the better it works. But I'm just working with an iPad and and Android phone. It's great to sit back and watch all the screens update in real time all at once. and it's still a 1.0 product with many features to come in the future.

Some of the things you start realizing when you leave Shadow open while you browse are that some sites don't do a good mobile design, some require constant authentication, some use AJAX to fake a new page request and which ones have a great sense of adaptive design.

The main feature is a Chrome plug-in that talks to your iOS or Android app so that when you launch your Windows or Mac Shadow application with the apps talking to your local network, the apps will "shadow" what you do on your desktop. The more you like Developer Tools in Chrome the more you will like the product because it shows a webkit Developer Tools based window for your remote device.

The product was a very well timed conjunction of talent from the weinre open source code that was acquired through Adobe's purchase of Nitobi, the maker of PhoneGap and Adobe's BrowserLab. Adobe Shadow is free also and looks to get only better as they add support for Firefox and localhost development environments.

The 1.0 version is posted at the Adobe Labs.This will definitely a product to work with in my new Mobile Web Application Development using jQuery Mobile course here at Centriq.

The web’s four hats & an overview of HTML5

2011-09-28T17:38:00.000-07:00

I recently returned from speaking at a conference in Kentucky where I ended up putting on four hats and seeing HTML5 and the future of mobile computing differently each time. I was struggling with trying to figure out why standards committees take so long to come to an agreement. After all, the vision of the web is clear, right?

Why did we get off the XHTML superhighway and jump on the scenic route where we ignore the bumps and potholes of bad syntax and limited metadata? What kind of folks are driving the bus right now and are they really going the right direction? Should we stay on the bus while we head into Mobileville? I've decided that the questions you pose are at least if not more important than the answers you think are right. Because that tells you what your vision is and if you've followed a little of this blog, you know that strategic vision is important.

The four hats are:

The Engineer - who likes all of the functionality logically thought out
The DBA - who likes the the data well defined and stored
The Designer - who likes to deliver an emotional message to the customer
The Businessman - who makes sure the customer is happy and they make a profit

Those four hats have led me to not watch the standards groups as much as watch Google and Amazon for what to use in the future that people want. The real vision of the web is to make money by giving computer users what they want. Waiting for standards groups won't get you there. After all, the current prediction is that CSS3 will be completely standardized by 2022. Here's a rundown on the thinking under each hat:

Engineer

As a software engineer and instructor of many students who become programmers, I know that the web has become populated with these folks that want to turn it into an application environment. That way they can program for it and make it do their will. The back end servers are full of Java, C#, and php and that same force is heading to the browser with JavaScript.

I see the HTML world being altered so that these variable manipulators can have better code and read each other's functions easier. The push in HTML5 towards meaningful naming of the loosely typed datatypes that were divs and spans is going to give programmers a better handle on what it is they are talking to. The code will become less dependent on ids and more dependent on setting up the HTML with the proper element names like <mark>, <time>, <progress>, <meter>, <details> and some of the more argued over names like <style scoped>.

I can see where the types of activities previously handled by JavaScript are being pushed into HTML. This reduces the language down to one, slightly more complex, language. Do we want a merger here of interests? Is the MVC principle being corrupted and should we really mix our text markup with programatic data validation in the same code module? The data and its validation are View Model pieces that are tightly coupled and should be handled together in my mind.

So the future lies with the ultimate merger of HTML and JavaScript and we'll have a Razor style view engine in all browsers eventually. My opinion is not to follow the same syntax and find a better way to merge the two. But the merger will eventually come. Simplified syntax for CSS3 and JavaScript have started to reinvent these stuffy languages and if jQuery can simplify CSS with a replacement like EZ-CSS, sass, or less, then we're on our way. I'm hoping it's not like node.js and stylus, but I need to try those more.

DBA

I also love to understand data and its meaningful use. Having the right field name and the right multiplicity gives me a warm glow. But I also know that too much data analysis can torture those poor programming souls who have to recombine the data for the page they work on.

Here we're not talking about the fields where programmers hand over their data after the output is available but where the data can be semantically meaningful as text. These are the domain entities and at a high level they are represented by <header>, <nav>, <footer>, <section>, <aside> and <article>.

My biggest hurdle here is knowing that the data already comes from a semantically proper framework called the database. If all the work has been done there, then who benefits by rethinking the meaningful structure one more time for the web? The only answer is that the DBA traditionally send the data to the programmers who use that "data markup" for their benefit. The field names become bundled together and as I put back on my Engineer hat, I am happy to create complex View Model types for my View, the web page.

But it's reinventing the wheel when there's a database and it's better for team coding experiences to have more readable code. But wait, is there a web service that might be looking for my article elements? Not yet in the HTML5 way. If you are looking for a current semantic framework of entities, you have to look no further than Google's recommendations. The microformats they recommend carry much more weight for reasons to mark up with them than just a standards board. People are using the schemas for reviews, events, organizations, places, restaurants, recipes, etc. without even knowing about it but you are losing out on having Google process your data and provide search results with it if you don't.

The XHTML push that we unceremoniously dumped was a DBA's dream for a perfect interface language between data source and data consumer. The problem was that it's not a homogenous set of data. Our web DBA's don't work under the same domain and the complexity made it uncomfortable to work with so many needs and choices that we found had already been solved with another data model. Or two (relational and objects). We didn't need a third (XML).

Designer

The designer in me wants to have my great user interface so well integrated into the data and processes that you don't notice. And I want it to be beautiful. That hat wasn't the one the standards committees were wearing when they proposed the gradients and transforms. Misguided blink spans and easy marquees have been put to death as many times as Freddie and Jason but keep coming back for more. Now we have them again, only this time a little more well dressed. I really don't even like the drop shadows but some of these will look much better to the high-res tablet crowd.

I am happy with SVG support and the graphics we can do, but where is my visual SVG graphics designer integrated with HTML and CSS? Android was the last to jump on the SVG graphics bus and most of the rest of the cheesy effects have partial support so I know I will be trying out Inkscape and SVG Edit when I need. Or I'll just sit back and let the widgets take over for me like the SVG based Chart control in Visual Studio and jQuery plugins.

I find the most pleasure with web fonts, especially Google's public accessible fonts and having Web Squirrel give me everything I need to do it on my server. I can talk to the device to get info I need to deliver a pleasing layout with the @media queries. And when I need that old-timey newspaper look, there's multi-columns, but I fear it's a retro feature that will be better left alone due to the complexity it creates for the layout.

Surprisingly, I think the designer ended up with some of the best improvements. Colors are better defined both with more names and flexible opacity. The video and audio elements will have to wait for the patents to become less profitable while we find a common standard to work with. The dominance of the iPad will force me to encode a different way but I don't like it.

Businessman

If you want to know who is going to win the HTML5 battle, you just have to follow the money. And the hat in charge of the money is the businessman. This hat is the hardest to wear since it requires you to not think technically and focus on the customer. Does the customer really want what we think is geeky cool warez? Probably not.

I liked the idea of doing a survey of the currently used names for divs and classes in the wild in order to understand how people used HTML. But what were we supposed to do with that? Did I really want a better element because I was really happy with just calling my div a wrapper in the id and not a section. You want me to recode it? When?

What helps me make money? Coding faster. Reusing other people's code. If all you do is rename things, then it's going to get in my way. Now if you make it easier to find elements in CSS by adding better selectors, I'm not going to complain but it is looking more and more like a grep or regex language these days.

I can reuse other people's code if they've marked up what I want to use and now we're back to microformats because what I want is not generic text groupings that are or aren't related. What I want are finer granulation on the most common business entities which is exactly what the microformats do.

Other types of useful abstraction that will be faster or give me critical information are the new communcations APIs like Web Workers or Web Sockets and the ability to let me work offline with session or local storage when paired with the page caching. Geolocation is just a one-trick pony but had to happen when you start moving yourself around instead of sitting at the same IP day after day. And the database storage on the client will improve the user experience for offline as well.

Conclusion

People want a desktop application experience on the web and want it whenever they want it there. People have to have it easy and that means the developers as well as the users. So will I use the semantic markup? Probably. It will help me as a programmer. Will I use the <canvas> and other graphic elements? Probably not. I like SVG and JavaScript generated graphics as long as I don't have to write it. I can use Adobe Edge or other visual programs for that. Will I use the features that aren't well supported like drag & drop and datepickers where I can use jQuery and jQuery mobile to get standardization now? Obviously not.

The 2-D and 3-D graphics are incredible and not in my realm as a web developer but if you are a game developer that wants to leverage the e-commerce platforms of the future like the Apple App Store, Android Marketplace or possibly the Amazon juggernaut commerce system, then you must develop using <canvas> because it's the fastest and standardized.

As to why standards groups like the W3C and WHATWG don't move too fast, it's because you have four different groups of people all lobbying for their view. The DBA had the upper hand in the last version and now the designer and the engineer seem to be dominating. But like a real business, it's the dialog that is important to crafting a best solution for the whole. Let one company or one hat take over and it would be an impending fail.

The businessman is going to take you to that application experience that you so want. And he will charge you for it and pay you to develop great apps for it. The standards committees don't think like businessmen so you'll need to see what people are really using, really need, and get paid for. My best bet is to watch Google for how they use search data and to watch Amazon for how they will use their e-commerce platform. Of course, you'll also be watching Apple and Facebook as app delivery leaders. Then code for it and enjoy.

KCMO is added to Google fiber network project!

2011-05-17T16:40:00.000-07:00

google A Googler

by KC_next

Crossing the river from Kansas City, KS to Kansas City, MO: ultra high-speed broadband available in both KCs in 2012http://goo.gl/hCyYR

5 hours ago

KC_next KCnext

Google will extend ultra-fast Internet project to Kansas City, Mo.bizjournals.com/kansascity/new… via @KCBizjournal

6 hours ago

KC_next KCnext

The official video on today's exciting news that Google Fiber is expanding into Kansas City, Missorui http://youtu.be/TmSuNyOpsXU

6 hours ago Favorite Retweet Reply

Kansas City gets a gigabit heart from Google. The Wizard is back.

2011-03-31T08:26:00.000-07:00

Image via Wikipedia

Yesterday, Google announced that Kansas City, Kansas was the winner of the high speed fiber network deployment that took a year to decide. It's as if the Wizard of Oz came to town and got a job as CIO with the KCK government. Certainly Topeka was a noisy contender getting Google's attention last April 1st with a Google Doodle to suit the occasion. But it was the solid community behind Mayor Joe Reardon and the business support groups that swayed Google to bring gigabit speed to our prairie town.

Some of the groups that have participated in the decision, I'm sure have been the Marion Ewing Kauffman Foundation, KCNext, and the KU Med Center. The Kauffman Foundation in particular has been the leading supporter nationally for the education of entrepreneurs and now gets to take their passion for community building into the grey streets of KCK.

Google is also interested in supporting the Kansas City, KS community through non-profit groups. It's important to raise your awareness if you are helping to benefit our area through technology to get on their list by registering with them on their site.

The first thing that KCK will benefit from will be the build out of the network where local folks will get jobs necessary for running cable and building infrastructure. It will take some time before the users of the network get to benefit from any kind of high speed surfing. But it won't be the browser and texting users that will see an increase in speed as much as we'll see other benefits. Sure we can knock on the doors faster asking for web pages but most of the world can't respond at the same speed so most of the responses will be just as fast.

There are large consumers and producers of internet information here around Kansas City and for the people that access that information, they will be seeing a better response time. Maybe the people like Infegy, a social monitoring service, or AdKnowledge, the largest privately owned advertising network, will move some of their operations over to KCK to take advantage of the speed and produce better and advanced services.

One thing I have to answer for myself is just how the fastest fiber network that is currently available in the area will play into Google's plans. The Sprint backbone of fiber supports telephone communication and for personal use, hardly anyone gets to benefit except for those that pay to use it to support their phone systems. The military certainly is a major consumer of the fiber backbone with Ft. Leavenworth being a quiet powerhouse on the digital frontier. Sprint has been longing for a partner which they might just have found in Google. They were ignored as a suitor once more for the merger of T-Mobile recently. This could have big implications for the future of Sprint.

I'm certainly excited about all the future possibilities that Google has honored our area with. We're a great community that was waiting for that perfect storm which I think is finally here. The parts we needed were the technological know-how that Sprint attracted in the last decade, the entrepreneurial impetus that the Kauffman Foundation is continuing to supply, and now finally the infrastructure from Google that will generate decades more business energy to a community that deserves the best.

Test first – don’t hire an analyst? Project management reorged.

2010-11-17T15:03:00.000-08:00

Coders are being asked to write tests before coding. Testing first sounds cutting edge. But let's get real. You don't have anything to test so it's not testing. It has to be something else.

I’m not objecting to the practice, just the naming. That’s because I’m thinking like an analyst instead of a programmer. Naming determines the direction of the thoughts from that point on. We shouldn’t tell people to test something they don’t have because it’s not right.

Ask any Denver bound passenger on a plane to navigate where they are going. The chances of getting to Denver are about as good as having a project complete successfully without analysis. The analysis is your plan, your map, and your radar. The destination is your business goal.

Testing in project workflow
There are several concurrent workflows during a project. Each basic workflow takes up a majority of the current activity during the project at some point can occur concurrently with most other workflows. Here’s my version of the project workflow without any particular allegiance to another methodology. It’s what I’ve found to be a cohesive set of activities that require unique skill sets.

strategy - coming up with a good business idea
project initiation – committing assets to making the idea real
needs – talking to folks about what they want based on that idea
requirements – understanding and communicating those needs in testable terms
analysis - organizing business data and workflows based on those requirements
design - figuring out how it's all going to work with what we have for the technical folks
implementation - the construction
testing - the double check of validation and verification
deployment - getting it launched
maintenance - keeping it working and updating it

What do you do when you test?
Testing is about two things. It's about validation and verification. Validation is checking stuff that has been planned and making sure that the plan was carried out. Verification is about seeing happy faces on your stakeholders no matter what they said and what you wrote down in your plan. Valid product + verified happiness = project success.

Testing can start as early as the requirements phase. Since the project is being expressed as testable statements, it follows that testing can start early. The information that is extracted at that point is useful for planning and setup. Then more planning is done throughout analysis and design for the large scale types of tests that match up with the large granular data definitions and workflows.

When the granularity of design comes down to chunks of code and individual statements, you still have testing that goes on. It is this drill down into the finer details of design that the test first advocates are talking about. Here the design is put forth as a supposition and as it gets implemented it is pushed back into the validity loop of testing. Hopefully, there’s enough high-level analysis and design to make this low level design and testing stress free for the coder.

Test-Driven Development in isolation
Testing is not what is done before you write code. It doesn't matter how much you like Kent Beck. Kent says that Test Driven Development (TDD) is a style of development. So, it's not a style of testing? What's the first step? It's assert. What do you assert? Now you have to go to your data and workflow models to figure that out. What? No analysis and design models? But if you did have some analysis, then you could do technical design and get the rough sketch of the code.

The TDD coder is supposed to come up with types of tests to run (analysis level testing), rename identifiers (analysis), assert first (analysis), test data (design level testing), and practice this iteratively. If the analysis and design work had been done previously, the coder could use it.

And if you don’t have any of that, you do the best you can and try to figure it out. Some programmers use pseudocode and activity models called flowcharts for their technical design. They were trained to take exercises in programming class and wrangle them into code without any supporting analysis or design. Management got trained into thinking that the real world was the same and programmers could pseudocode the entire business project given enough bright students.

So, as an analyst, I was interested to figure out what programmers were being told to do if they only had a listing of needs from the stakeholders. That list is usually so hard to understand that without proper editing and rewriting, the resulting code is flying without radar. Testing first in that case could be about several things including

testing stuff that you make up on the fly
doing something else and calling it testing

Most of the principles used in TDD are based on doing what hasn't been done before which leads to higher stress in the programmer because they have a greater responsibility. With the TDD process in a less than perfect design world, the programmer has a way to manage this chaotic development cycle from their chair. Management is more satisfied when the coder does the analysis and design on the computer because a business analyst position would cost money. Productive work on computer is good, work on paper – not so good.

Make it work first, then make it better.
The mantra of TDD is red/green/refactor. When I personally use it in practice it turns out to be, guess/make it work/make it better. I was told to do that by senior programmers. I've been doing that as a programmer for too many years as a way to manage when the analysis wasn't done. Guess what the stakeholders want and try to get it into words that you can manage. Then turn that best guess into workable code that won't break. And if you have any time after that, give it some thought on how to reduce the code or make it more reusable.

I know that Beck is more interested in getting the gap controlled between decision (design) and feedback (test) so any thinking is good when it shortcuts the path to better code. But analysis isn't good at giving feedback so to a programmer, it's better to have some concrete metric of success rather than a bunch of papers. Write another automated test. One more green light.

Maybe that's partially the fault of revamping our programming curriculum in school and leaving out structured programming. The dearth of analysis after object oriented programming took over had the programmers clamoring for some sort of control that worked.

What's the most important quality of a requirement? It's testability. So if you gather some related needs and can conceive of a test for them, then it usually becomes a good requirement. The remaining tests are easy to extrude from the requirement.

I think coders flipped this on its head. They say that they should be able to conceive of a test and see if it fits the requirement. From their perspective, it makes sense because they have to see the linkage back into the stakeholders' requirements from where they sit in their 8x8' cube. Without guidance they have to draw the map.

But coders aren't looking at the big picture so I'm against TDD being a stand-in for analysis. It works for low-level design though. And it makes sense. So as long as the analysis is sufficient, TDD works well as a design phase validation of the requirements in analysis to the working code put to the fire by automated tests. If activity is anything before the unit tests, the lowest level test possible, then it's not about the test, it's about the requirements.